Agree on defining a RAID's purpose. That's part of the question I have.

Do I even need a RAID?

The goal is first and foremost to edit HD footage in real time. This could be AVCHD or HDV or DVCPROHD via Raylight Ultra or P2, QuickTime, uncompressed YUV or whatever else gets thrown at me.

One of Vegas's strong suits is its ability to play so many formats. Very often I'm asked to fix video that all our various Macs can't ingest.

I'm going to be driving the system with an Intel Core i7 Six Core 980X 3.33GHz 12MB 130W (Socket 1366 32nm).

If a 6Gb/s SATA3 drive can handle HD in real time, that's good enough for me.

I know RAIDs are tricky and can really destroy data, so if I built one, it would be one that has a redundant drive for recoverability (RAID 5, I think). We use a now-discontinued Apple Xserve RAID to edit uncompressed 1080i on our Media 100s.

Drives fail all the time and need to be replaced and the data rebuilt.

Then there's that little problem of Apple having dropped our Xserve line like a hot rock. (Thanks, Steve...you've done this to me before and I don't appreciate it!)

RAID 5 will actually slow you down, but offers the redundancy and safety of recovering from a failed drive.

RAID 0 is the fasted configuration, but offers no redundancy and if one of the drives fails you lose both drives (all the data).

So, you could do a RAID 0 and do nightly backups and have the best of both worlds.

Most RAID 5 hardware controllers are not known for their speed. You trade off speed for data security. RAID 0 will gain you speed at the expense of data security.

Except for uncompressed HD YUV, which requires a data transfer rate of about 120 megabytes/second, most modern drives will handle all the other formats you mentioned just fine. 6Gb/s is just the speed of the interface, no hard drive can actually reach anywhere near that speed. I am not aware of any hard drive that can yet maintain 120 MB/s across the entire drive. Solid State drives certainly can.

Pardon the obvious oversimplification and dated point of view, but RAID is useful with NLE essentially for two things:
1) Redundancy; and,
2) Bragging Rights

Why not try to setup a Vegas Render Farm for speed....

oh wait, does anyone even really do this?

We use a now-discontinued Apple Xserve RAID to edit uncompressed 1080i on our Media 100s.

Okay, sounds like you're coming from the same place we are where I work. We've long since left media100 due to "Thanks Steve" issues, but were running Rorke Galaxy arrays over a fibre SAN to feed several edit stations.

You might start by looking at some of the system recommendations given by AJA, or BMD. They usually recommend RAID for HD capture from their cards.

Most of the media you've listed wouldn't warrant an array. AVC and HD don't require all that much disc throughput. Some flavors of quicktime and AVI would benefit from higher disc throughput. Uncompressed HD definitely needs some help but you might be able to get by with single 10k RPM drives.

A *good* RAID setup would feature a high-end controller card with all it's own logic and processing on the card. You don't want your CPU to work to control an array, save those cycles for rendering.

The array itself really needs to be RAID5 for data security with *some* speed. I'm not sure what the metrics are on the sheer number of drives but you probably get diminishing returns as you ramp up the members in the array. Ours have 16 drives but might be divided into 2 sets of 8. I know they serve up two volumes but don't know how they're laid out. If you're just serving a single edit station you could easily go with something much simpler.

You might be able to get an enclosure with it's own RAID controller that could run off a single eSATA port. a single SATA2 channel should still be enough to service a 4-member RAID-5 array.

Truthfully, a single 10k drive might cover your needs (with a lot less expense and hassle), but you need to do a bit of research elsewhere to get a sense of what you really need.

Rob

This might be more informative than most of us here but you should consider *why* BMD thinks you need a RAID array,

http://www.blackmagic-design.com/support/detail.asp?techID=62

Sidecar may be under the misapprehension that because he will be editing HD that he will need super fast disks. In fact HDV has the same data rate as SD while AVCHD at full quality has a little over half the data rate. So to edit AVCHD, the "hard" one to edit, you can get by with slower discs. I am currently rendering SD and my disk light flickers on and off but is off more often than on. With AVCHD, the light would be off for an even longer tiime, especially when you consider that the rendering would be slower in any case.

Thanks for all the input. I suppose the DVCPROHD at 100mb/s is the highest bitrate, unless I'm working with uncompressed files.

I assume the 1/3 fps fps playback speeds I'm getting with my old P4 are simply because the processor can't decode the AVCHD fast enough, and not drive throughput? Thus the RAID isn't really necessary?

If so, that's good. I'd rather avoid RAID problems altogether.

Change "old" to "crumbling into the dust ancient" and you've made a very good point. HD in general, and AVCHD in particular are still challenges for even the latest CPUs and multiprocessor systems.

There's 2 RAID arrays on my system and a single disk. A Raid 5 array is set up on the motherboard ICH10R and a Raid 10 is set up on a 3Ware 9650 Raid card. The Raid 10 will deliver up to 900mb/sec, the single disk will run about 85mb/sec, and the Raid 5 will run about 120mb/sec. So as you can see, Raid 10 provides the best speed and still has redundancy compared to the others on my system, the Raid 5 is still much faster than the single disc. The single disk is a WD 1TB Caviar Black. Both of the Raid Arrays are 4 disk arrays.

I've experimented all combinations of where to put the OS, Data files, and where I render to with long AVCHD projects and no configuration made a note worthy difference in render time or performance in preview etc. Neither of the Raid arrays use more than 2% CPU for parity calculations. I have also done some test with AVI files and still haven't seen noticable difference, although I admit my tests haven't been extensive with AVI.

I guess if a person was working with large uncompressed files as an input or output it drive configuration might make a difference in performance but I think it would be minimal based on my observations.

My OS is now on the RAID 10 on a separate partition from the data area and it does boot faster and load programs faster than other configurations on my system and it's noticable.

Like others have said, I don't think you will see any performance differences with RAID in Vegas, you may in your overall system performance and you will have redundancy to protect against drive failure. Today's good HDDs just aren't a bottle neck in Vegas.

Alright, I am not a professional yet I have tons of digital photos and terabytes of video (mostly DV). What is the best solution for data backup preferably automatic. I am constantly faced with the frustration of "media disorganization".I am thinking of a network drive or something like that. Any suggestions. Thanks

Yes, unless you talk to someone who works where there are thousands of servers and dozens of enterprise-class storage systems.

How many NLE users do you know of that have that?

Pixar!!!

Well, they already have their bragging rights.
;?)

I've gone to 60 fps HD for editing, and - if the project is long/complex - I was using Cineform for all clips. Unfortunately, most 7200 rpm disks cannot deliver the data fast enough to allow smooth motion during editing, due the the large file sizes and consequent data rate requirements. I tried various RAID options, but found them frustrating.

The RAID hardware controller on my motherboard was picky about which drives it would work with - Seagate OK, WD and others wouldn't work. But Seagate drives proved to be unreliable.

Software RAID had its own problems. For example, each time I booted to WinXP-32bit (required by some of y older programs) instead of Win7-64bit, the RAID drive would not appear in WinXP. And, as I recall, sometimes the data would be garbled when I went back to Win7. Software RAID also requires its own share of CPU cycles, thereby increasing rendering times slightly. Eventually, I had to give up on software RAID.

At one time, I had a dedicated RAID controller, but when it died, I couldn't access my data, nor could I find a substitute controller that would read the data (the company that had made that controller had moved on). Fortunately, I already had learned the hard way to back up each day's work to USB-connected external drives, each with separate power supply - so not much work was lost; but I lost confidence in hardware RAID controllers.

In the end, I decided to use the PicVideo codec for shorter projects, for the data-rate demands are more easily met (with the quality slider set to 19). After editing, I compress the entire TL to a single Cineform file that serves as a master for compressing to MPEG2 for DVD, H.264 for web, etc. Since I'm not editing it, there is no reason to use RAID to hold it.

I'm also experimenting with PicVideo for longer projects, even though I don't like mixing codecs on the TL and almost inevitably have a few Cineform clips (from previous projects, etc.) that get included on the TL. I'm editing in Vegas 8c and rendering (usually) in 9e-64, and so far have found the PicVideo/Cineform mix to be a pretty good compromise for 60 fps editing - so far, everything is rock-solid stable, and editing is smooth with the PicVideo 60 fps clips.

Get fast 10k single drives for the DVCProHD and you'll probably be okay. One thing about RAID is it might help if you're trying to pull multiple streams off the drive at the same time.

AVCHD is low bitrate but it's CPU-bound. You don't need fast disks until you transcode it to something much fatter and more easily decoded. (and you should do that with AVCHD, especially for Vegas)

Rob

rmack350,
You say, "...transcode [AVCHD] to something much fatter and more easily decoded."

What is the best format to transcode AVCHD to within Vegas 9e 64bit for editing?

Also, I see "Cineform" being touted. This needs to be purchased and somehow incorporated into Vegas? Or is it a standalone program/codec?

> What is the best format to transcode AVCHD to within Vegas 9e 64bit for editing?

You will get a variety of opinions on this but for my money, Cineform is the obvious choice. This is what I use. You could use free lossless codecs like Lagarith or Huffyuv but they create much larger files. The objective is to use an intermediary that is lossless or near lossless and still plays smoothly, but some people convert AVCHD into HDV or MXF and it works for them (both are lossy but they do play smoothly in Vegas). It depends on your needs and how much post processing (i.e., color correcting, chroma keying, etc.) you plan to do.

> Also, I see "Cineform" being touted. This needs to be purchased and somehow incorporated into Vegas? Or is it a standalone program/codec?

There was a version of the Cineform codec that shipped free with Vegas 6, 7 & 8. Unfortunately, Sony stopped shipping it with version 9. You can purchase Cineform Neo Scene for $99 USD at Videoguys.com and it will also batch transcode all of of your AVCHD files for you. You can also render to Cineform from within Vegas like any other codec. It's well worth the investment if you ask me.

BTW, on the RAID question...

I've been using a RAID 0 for my video editing drive for several years now. I have a 1TB RAID 0 from 2 x 500GB Western Digital enterprise class drives. I back it up to a single 1TB external drive. That gives me speed and redundancy. I don't see what big deal about reliability with RAID 0 is. If I bought a single 1TB drive and it failed I'd loose all my data anyway. Yea, I know the 2 500GB drives have twice the chance of one of them failing but a failure is a failure. You still need a backup at the end of the day either way.

~jr

JohnnyRoy,
What controls your RAID 0? A dedicated RAID card or the built-in motherboard ability? Any preferences?

And thanks for the Cineform info.

I use the built-in motherboard RAID controller on my Intel D975XBX2. There are two RAID controllers on that mobo, one is connected directly to the ICH7-R I/O Controller and the other is a 3rd party that goes through the PCI bus. I use the Intel RAID connected to the ICH7-R.

~jr

JR answered everything more clearly than I would have but I'll emphasize a couple of things.

Cineform NeoScene can batch transcode. Drop a pile of files on it and it'll convert them all. That's much handier than using Vegas to do it.

One thing I don't know (and it's worth finding out) is whether NeoScene will preserve your file's timecode. This is an important consideration, assuming you've got timecode to preserve.

Once you've transcoded to a better editing codec you shouldn't need to go back to your AVC files. You can just archive the stuff.

JR is totally right about Raid0 reliability. You need to do backups no matter what. The reliability argument really comes from the distinction between Raid0, 1, 5, and 10. The last three offer more data security to keep you from being completely out of business until the array is rebuilt because all of these will at least limp along if one member of the array fails. With RAID0 you're dead until you can restore the array or swap in a new storage solution.

People often take this argument and extend it to say why a single drive is more reliable than RAID0. I'd say that a single drive is just simpler to manage and cheaper.

As for controllers, usually onboard RAID uses the CPU much more than a good controller card...but RAID0 and RAID1 aren't very CPU intensive. Controller cards come into play more for RAID5.

Rob