To RAID or not to RAID?

Al Min wrote on 4/6/2010, 7:56 PM
I have three hard drives with OS on one, source on another and destination on the third as has been recommended on this forum before. However, I was just told that if I set up HDs two and three in RAID 0 I will significantly improve the performance when reading and writing video with VP9. RAID 0 combines the two discs and writes to both and reads from both at the same time thereby cutting down the time to do both. Is this good advice or not? Should I do it, and what are the pitfalls if I do do it?

Comments

farss wrote on 4/6/2010, 8:45 PM
It might be good advice or not.
If the data I/O requirements are creating a bottleneck then yes RAID 0 will help. This is unlikely to be the case unless using uncompressed video.
On the downside if either drive in a RAID 0 array fails you have a problemo.

I have two small (250GB) drives in RAID 0 on my system. Nothing vital is kept on them and I also have several 1TB drives not in RAID 0.

Also keep in mind that RAID 0 can require more CPU overhead and could have a negative impact on video processing. If you do want to run RAID 0 for video generally best to use good hardware controllers. The newer MoBos may have reasonable RAID controller chips on them.

Bob.
srode wrote on 4/7/2010, 3:39 AM
Like Bob said, results will depend on what files you are working wtih - uncompressed mightl benefit and others probably won't unless you have some seriously slow drive. I wouldn't use RAID 0 for anything but temp files because you are increasing the risk of losing data in the event of a disk failure by the number of disks you are using. Raid 10 is a better solution than RAID0 in my opinion - you will read from all 4 disks at the same time and so read/write performance is similar to a 4 disk raid 0 array but you have redundancy. I use RAID10 on one array, raid 5 on another array and have a single disk that is used for back up of data. The RAID 10 is one a 3ware 9650SE card, the 5 is on the ICH10R chip on the MB. CPU overhead is 2% or less on my benchmarking tests on either array. My tests with AVCHD and MPEG files showed zero difference in rendering times from putting, OS, read, and write files on separate disks / arrays compared to putting them all on the RAID10 array which is quite fast. If you have room for 5 disks, I would use 1 RAID 10 array and 1 single disk with the single disk being for back up and put everything else on the RAID10 array.
Al Min wrote on 4/7/2010, 3:58 AM
Thanks for that Bob and Stephen. I think I'll stick with what I have for the moment.
Earl_J wrote on 4/7/2010, 6:19 AM
Hello Al,
if I'm not mistaken, RAID 0 is the every-HD-for-itself sort of thing... they are not securiing any data from any other drive. So, each HD works independently of every other drive... good speed without any security. If a drive fails, you lose everything that was on it.
RAID 5, requiring at least 3 HDs, spreads the data across the drives in such a fashion that if one drive fails, it can be replaced, and the data is not lost.
RAID 10, requiring at least 5 HDs, spreads the data across the drives in such a fashion that two drives may fail at the same time, and when they are replaced, no data is lost.

So, depending on the demands of your situation for security of your files and data, you can choose which one you need and implement it for your purposes. From your description, it appears you're in a RAID 0 configuration now... three independent drives.

I apologize for insulting your intelligence if you knew this ... I'm just trying to help... really.
If I'm in error, I'm certain we'll hear about it . . . (grin) which is good for both of us; I just hate living under a misconception ... almost as bad as under a bridge in winter... you get cold stares as people come by but no one stops to help very often... (sigh)

I hope that helps...

Until that time. . . Earl J.
farss wrote on 4/7/2010, 6:56 AM
The problem with RAID 5 is the write speed suffers. This is fine in say a datacentre where most of the action is reading data. Not that datacentres use anything as crude as RAID 5 though.

RAID 0 is not "every-HD-for itself". Data is split or stiped across two or more drives with no redundancy. If any drive fails or has an URE then the data is just gone. Three independant drives is Just a Bunch of Ordinary Drives (JBOD).

Bob.

PerroneFord wrote on 4/7/2010, 7:22 AM
I use RAID 5 in my datacenter, but that is only after modeling our usage patterns and making an informed choice. I am also RAID 5 across 5 spindles, so it makes a difference. I can't see using this configuration for data though unless it was an archive configuration.

RAID 0 has no place in a video configuration unless it's for scratch disk.
baysidebas wrote on 4/7/2010, 7:33 AM
One question about Raid5, do the replacement drives have to be identical to the failed drive? If so, that would mean multiple drive purchases with replacements warehoused in case of drive failure?
Earl_J wrote on 4/7/2010, 10:49 AM
Bob,
exactly . . . dhoh!... RAID means striped ... and the number indicates the level of redundancy; which, in turn, determines the level of security as well as the read/write access speeds...

I love this forum... If I don't learn something new each day, at least I can get reminded of something I have misunderstood or convoluted in my scrambled-eggs, pea brain. . . (grin)

Until that time. . . Earl J.
Byron K wrote on 4/7/2010, 12:07 PM
Reply by: baysidebas, Date: 4/7/2010 4:33:16 AM
One question about Raid5, do the replacement drives have to be identical to the failed drive? If so, that would mean multiple drive purchases with replacements warehoused in case of drive failure?

It is best practice to have identical replacement drive, but if you cant find an exact model a larger better drive should work.

I'm a firm believer in RAID10 and/or RAID5. I've had 2 IBM Deathstar drives die at different times on my RAID10 machine. One was replaced w/ an exact model and the other was a larger faster drive because the original 40Gb was to expensive. I was still able to function in both situations w/ out any down time except to rebuild the drive.

NOTE ABOUT RAID:
You still need to backup important data. Needless to say, you can loose the whole array in one fatal swoop if you don't know what you're doing or if your RAID card goes. USE QUALITY CONTROLLERS! Believe me, a lost array is disastrous. I'd recommend a RAID controller from a reputable company like 3WARE because you can replace it if necessary. I've built many servers using 3WARE cards and none of them have died (so far). (;

I mention this because I was beta testing a RAID 3 card from a new manufacturer (I forgot the name). RAID3 is like RAID5 but has a dedicated parity drive rather than spread the parity over multiple drives. It is supposed to give RAID 0 performance w/ the economical efficiency of 5. During my testing of about 6 months, the card died and all my data was lost not able to recover anything. Needless to say I'm not going with this manufacturer. Good thing I'm a backup hound and had multiple backups of my files and data...

Also, I'm always nervous about on-board RAID controllers because I've seen many, many IDE controllers fail but since these were single drive machines the user just switches the drive jumper to slave and can read the data on their new machine. This is not the case if the motherboard RAID controller dies.
srode wrote on 4/7/2010, 6:40 PM
Earl,

Actually Raid 10 is built on 4 drives - it's like having Raid 0 and Raid 1 combined, you can lose a single drive and be fine, not 2 drives. Raid 6 will let you lose 2 drives and still be fine, it's somewhere inbetween raid 5 and 10 for speed, and better than either for security as you can lose 2 drives and be fine. Raid 5 can be built on 3 or more drives - My Raid 5 array is built on 4 drives.

Another consideration is that Raid 10 is less efficient for drive space than 5 but is the same as raid 1, with the speed of raid 0. With raid 10 using 4 identical drives, you get storage space of 2 combined. With Raid 5 and 3 identical drives, you get the storage space of 2 combined drives.

For speed comparisons, Raid 5 is faster than a single drive or Raid 1 generally for read and writing, raid 10 is faster than Single drives, Raid1 or Raid 5. In some situations I can get speeds over 700mbps on my Raid 10 array for reads and writes, but you have to consider each drive has 32mb of cache and the Raid card as another 256mb of cache. I use the cache for read and write ahead but only because I have a battery back up on the Raid card. Another option for a MB Raid card like ICH10R to achieve max speeds with Cache enabled is an external battery back up for you computer. Either will enable you to retain the cache in memory or finish the writing before powering down in the event of a power failure and you array's won't be corrupted.

My OS is on the RAID 10 array because it's the fastest, my data used for videos is read from my Raid 5 array (although it's backed up on the raid 10 as well as the single drive too, and so are my project files). Because the single drive isn't in the computer but once a month to do back ups, about the only thing that would cause me to lose everything is a home fire that burned long enough to destroy the single drive in my fire proof safe. It could happen, but not likely.
jabloomf1230 wrote on 4/7/2010, 8:24 PM
"I have two small (250GB) drives in RAID 0 on my system. Nothing vital is kept on them and I also have several 1TB drives not in RAID 0."

That's a good rule to follow. Let me add that you can also "soft RAID" two physical drives using Windows. All you need are identically sized partitions on each drive. It's not that much slower (if at all) than using either a separate RAID card or the RAID controller on your mobo. And it's more flexible:

http://www.tomshardware.com/forum/43125-32-raid
Steve Mann wrote on 4/7/2010, 10:47 PM
Unless you're running a smokin' fast system, the processor is most likely going to be your bottleneck.
richard-courtney wrote on 4/8/2010, 8:16 AM
Others have indicated using 3ware controllers and I agree good product.
Hardware raid is far better than software raid control.

As far as your servers depends on how they are used. Using them to store
video for shared editing or CG frames you want to decide if you are grabbing
a copy to edit locally keeping priority on redundancy in case of a drive failure.
Or speed because you are using the desktop computer for NLE or CG rendering
and the shared drive is your storage.

I'd recommend speed on your editing computers and have a common
server with redundancy as primary. Local computes with multicore processors
and plenty of memory seem to me to be the best investment over the drive speed.
TimTyler wrote on 4/8/2010, 10:04 AM
FWIW,

I just put together a fast new box after being inspired by the Sony Vegas Suite announcement. Using the Asus mb's controller I setup two 1GB drives in a RAID-0 for the system drive and then realized that was probably a mistake.

So before and after converting to a RAID-5 I ran some disk tests and discovered that read throughput was about 25-30% faster with the RAID-0 than with RAID-5. A little disappointing, but at least I don't need to worry about lost data.

As an aside, I used the Win7 built-in System Image backup and restore utility to move from the 2-disk to 3-disk RAID (copying the image to an external USB disk) and it worked perfectly. Nice to know we don't need third-party apps for that sort of thing anymore.

HD Tune Pro: Intel Raid 5 Volume Benchmark

Test capacity: full

Read transfer rate
Transfer Rate Minimum : 36.1 MB/s
Transfer Rate Maximum : 157.4 MB/s
Transfer Rate Average : 119.8 MB/s
Access Time : 14.3 ms
Burst Rate : 179.4 MB/s
CPU Usage : -1.0%
richard-courtney wrote on 4/8/2010, 6:50 PM
Glad I was not the only one having multiple posts.

No Tim and I don't stutter.... Some network issue causing connections
to reset.
Earl_J wrote on 4/9/2010, 12:49 PM
Just as I said... I learn something from this forum everyday!... whether I want to or not ... lol
I do visit with every expectation of learning something new... in addition to the tidbit here; I also learned from Chienworks that we don't need no stinkin' DVD printing software to print to printable DVDs... gee whiz...

Thanks for all that you pass along. . . we're all doing better because of the sharing attitude here by many ...

Until that time... Earl J.