I know...
but the truth is that if I'm only using the converted AVI's why take up the additional space? It's a lot of wasted drive space.

Man, I've been struggling with this one as well with the 7D.

I'm erring on the side of keeping the originals because Cineform is a proprietary codec and if it some how "goes away" I'm screwed. I'm not saying that it WILL go away, I just like knowing I still have the original files.

It is kind of a pain to store all this stuff though. I'm having to work on being more disciplined with my data handling/storage.

Invest in Hard Drives and a hard drive caddie? No?

Grazie

I've been doing this for a while but I've run into 2 problems:

1) All the connections/formats change - I've got a bunch of PATA drives and caddies but everything is going SATA.

2) All my drives are dying. I figured they would be good just sitting on a shelf but they are starting to croak. Some of these guys are 12 years old so I guess it was going to happen sooner or later. I've invested in a 4TB NAS but it is disappointingly slow and too expensive. I like the raid 5 redundancy though.

What I've been doing is just picking up big, cheap USB hard drives and dumping stuff to them. Portable, pretty quick and work with anything. I guess ultimately all this stuff is just disposable so cheap, quick and easy seems to be where I'm at right now.

Would love to hear what others are doing.

Everyone is struggling with archiving. Both in codec to use and in media to store it on. My choice is MJpeg2000 on BluRay.

Jpeg2000 is quite standard, it's the underlying codec to Cineform, REDCode, and several other commercial codecs. It's extremely high quality, and carries it's own proxy built in.

BluRay Drives are fairly inexpensive now and unlike hard drives are not susceptible to mechanical failure. They are also easy to copy, transport, and store. They are NOT a permanent solution. Nothing digital is at this point.

Those files are your OCN, did you ever throw that away.

Vegas doesn't do the metadata thing, you'd have to ask Cineform if they even copy that data across during the transcode. There's a reason already to archive the original files.
I'd strongly urge you to get into the habit of backing up cards, not shots. I've had more than one frantic call from people who've transferred shots and then cleared a card. Days later they go looking for a shot and....
Moral to stick to is, "the shot doesn't exist until it's backed up in three places". CF cards are reportedly strong enough to survive washing machines. Still best not to rely on this. Back them up before you leave the set. Thers's several devices that make doing this automatic. Pocket sized USB drives cost peanuts.
Grazie's suggestion is excellent for back at your place, for high value shoots I'd be thinking to have all data backed up and verified on location. At the very least get one or more of the Pelican CF card cases. Think like a swab nurse in an OR. 12 cards came to the shoot and 12 cards must leave the shoot.

I saw thesein the latest ASC mag, at under $2K / lens that could be quite a bargain and should get lots of love from your focus puller.


To see some artifacts you need to view footage at full raster and frame rate.
A lot of nasties can lurk in the blacks, oftenly bumping the levels up will make them easier to see. Even though you cannot see them easily downstream encoders do. Noise isn't grain. If you're shooting at low ASA you should be fairly safe.

Bob.

CF cards do degrade even with no use.

"o see some artifacts you need to view footage at full raster and frame rate.
A lot of nasties can lurk in the blacks, oftenly bumping the levels up will make them easier to see. Even though you cannot see them easily downstream encoders do. Noise isn't grain. If you're shooting at low ASA you should be fairly safe." Tho.. this is true for a lot of poorly light or just poorly designed scenes without considering the equipment limitations. Vic does not fit within this group of users unless he's had a major stroke or brain injury in the last couple of hours.

I like BrDisk for storage, Sony has a nice 400 disk BD unit that can be converted to storage archive with a little work. ;-)

But cheap HDD disks adds a nice combo.... which I think Vic stores complete projects on their individual drives since he been using NLE's.

Update...
MetaData proved to be an important part of my current explorations with the Canon 5D2.
Who the heck can remember all the details of shooting with an entirely different paradigm? Not me, for sure.
With my trusty Sony gear (PD170>Z1) the essentials were pretty clear. White balance - a single click. Shutter speed - limited to "normal" or "slow shutters". No variable ISO - just the hated (but totally used) "Gain boost". One lens. Depth-of-field a familiar, if basically missing item. Auto-focus...
Life was easier in those olden days (a couple of month ago).

So metadata suddenly becomes an important item.
I haven't found anything in either Cineform (which is what I'm using to transcode for use in Vegas) or in Vegas itself. As an aside, I'm sorry (happy) to say that Final Cut imports the Canon mov's w/o issue. I'm certainly not ready to jump ship, but....

Anyway - Canon has kindly included a free video viewer which contains all the pertinent metadata - lens used, f stop, ISO, White balance, shutter speed - everything you need to figure out a new working methodology. So I'm looking at my tests with that, and then editing with nothing more than end useage in mind (which is what I have always done).

It's not easy to be on the bleeding edge of technology, but then again, it never has been.

Meanwhile, I'm keeping all elements - original MOV's and Cineform transcoded AVI's. And I'm thanking "technology" for making it all possible with ever cheaper Hard drive space. I could never have afforded this storage space, even a few years ago.

Also - you are right. My workflow has traditionally been a new hard drive for each individual client. It certainly made revisions easier. Everything in one place - yum. But with the new advent of 1.5TB drives, I've begun to archive everything.

You see, one of the giant fears of video storage in Florida, is the ever-looming hurricane and flooding issue. It's not a joke. I've got two giant waterproof cases, filled with hard drives. But now practically everything I really care about is on about four of these 1.5's. It make the thought of escape a lot simpler. Plus, I will actually be making one more dupe of all the old stuff and sending it to my daughter in NY. Off site storage is simply the safest bet - especially when you've got hundreds of thousands of dollars worth of media.

The saddest part of all this, is how much of my work (much of it highly awarded and most of it very expensive) cannot even be viewed anymore, as technology races onward.

Funny story (if you're not too bored).
Since the '70s I've tried to keep film dupes of all my stuff (all shot in 35mm but reduced to 16mm screening prints). Then when we started to edit on 1" tape (shoot film, transcode, edit) I also kept the 1" masters. So I had an air conditioned storage situation with about 40 of those white boxes, filled with 1" masters. In 1998 I began to source in ditial media and began to transfer everything to hard drives. The "old stuff" was put to the side and forgotten. Recently, storage has become so expensive (real storage, not hard drives) that I thought it would be a good time to upgrade everything to HD.

Could not find a 1" facility that I could afford (or hardly one that even existed anymore).
So I pulled out all these 16mm films for transfer.
I had always heard that the film would hold it's color WAY better an longer than the tape.
Not true! All the film looks like poo-poo. Reds are completely gone. Mold on some stuff. Nothing left that I could ever show.

Through a wonderful happenstance I met a teriffic guy (Don Broadhead - are you out there??) who had 1" machinery and volunteered to help me transfer my stuff.
Well - the 1" looks like we shot it last week - all this from the 80's and 90's. No shedding. Full color and sharpness. No problems at all.

So - Farss, Grazie and everyone else. You are right. Try to keep everything you possibly can - and don't trust ANY single format or medium.

End of rant.

v

v

Cineform has a relatively new app called First Light, which displays a small subset of the passive metadata (time code, date, etc.) from within the video file in a non-destructive fashion. First Light is included with the mid- to higher end Cineform products:

http://techblog.cineform.com/?cat=102

Further, you can also utilize active metadata, such that you can color grade Cineform clips without physically changing the clip and re-rendering it in an NLE.

The passive metadata component is presently somewhat limited and the Cineform encoder does not presently preserve frame-specific passive metadata, such as f-stop, shutter speed and ISO. But I suspect that feature will eventually show up.

In the next release ALL the canon 5D (and 7D, 1DS, etc.) metadata will be embedded in the CineForm master. There are 200-300 fields you can access and display as needed.

David Newman
CTO, CineForm

Whooppeee...

Thanks Dan... and Cineform programmers, whoever you are.

v

Nice! Thanks Cineform!

From my twitter feed (http://twitter.com/David_Newman)

Sneak peak: Exploring Canon 5D/7D DSLR video metadata in CineForm First Light. http://twitpic.com/wtpj7/full

David

Looks good, Dan. Is that frame per frame metadata-- I hope!

Do Firstlight adjustments work within Vegas?

Of course, wherever possible. However the Canon 5D/7D, doesn't have per frame metadata for the really cool stuff like lens focal length (for tracking zooms.) These are primarily stills cameras. CineForm supports per frame for all metadata types.

David
http://twitter.com/David_Newman