What HDDs for RAID??

RalphM wrote on 11/11/2009, 11:56 AM
I'm considering a MoBo (GIGABYTE GA-P55A-UD4P) that has RAID 1 capability. I'd like to configure the RAID with at least 1TB discs.

One of the commenters on newegg claims that WD does not recommend their PC class drives for RAID usage "According to WD it is because the drive enters a deep recovery cycle and the RAID controller times it out."

What are others using for on-board RAID Drives?

also, any comments on this MM appreciated.

Thanks,
RalphM

Comments

vtxrocketeer wrote on 11/11/2009, 12:49 PM
Ralph,
I use both onboard RAID and a separate (dedicated) RAID controller card (an Areca):

1. Onboard RAID 5 with Western Digital 1TB (Gigabyte GA-X48-DQ6): 5 x http://www.newegg.com/Product/Product.aspx?Item=N82E16822136284

2. RAID card with Hitachi 2TB: 4 of these bad boys: http://www.newegg.com/Product/Product.aspx?Item=N82E16822145276

It looks like you're considering something like my (1) above. NewEgg reviewers say they work great for RAID whilst other say exactly the opposite. I fall into the former category: never had a problem in over a year. When I ran Vista 64 Ultimate, I'd sometimes see my onboard RAID array "verifying", which I chalked up to a hiccup from hard rebooting my machine (b/c of Vista issues). So far in Windows 7, like most of the time in Vista, I haven't had a single problem.

In case you're going my route (2), the 2TB Hitachi drives work great. Took A LONG TIME to format and build the array, obviously, but once done it works flawlessly.

$0.02,
Steve
RalphM wrote on 11/11/2009, 7:28 PM
Thanks Steve,
Interesting - the WD model you noted is one that WD says is not guaranteed to work in a RAID system. I guess it depends a little on the RAID controller on the MB.

I did later find a link to a more RAID-friendly 1TB WD at $160.

Thanks again,
RalphM
farss wrote on 11/11/2009, 10:37 PM
From the little I know about RAID it would seem that RAID 5 is not a good choice for video. You'll need to do your own research as I'm overwhealmed by the complexity of RAID. Why are you wanting to run RAID 5, it might not be as secure as you think, especially with 1TB drives.

Bob.
Steve Mann wrote on 11/12/2009, 12:10 AM
Newer consumer disk drives are "green" and power down after some time of no I/O activity. This may make then unacceptable for some RAID controllers. A sleeping disk may be seen as dead by the RAID controller.
farss wrote on 11/12/2009, 2:30 AM
I have 4 "green" drives and they do not power down. They spin at 5400 RPM instead of 7200 RPM which means they run much cooler and use less power. Excellent choice for backup drives.
I checked several reviews of green drives and found no mention of them spinning down. What may account for the confusion is WD originally said their green drives used variable spindle speed, a claim they later refuted.

The disks that do spin down are the ones like the WD MyBook. Even without RAID they can be a pain as for some reason Windoz dozes waiting for them to come back online.

Bob.
vtxrocketeer wrote on 11/12/2009, 4:49 AM
@Bob: when I first built my system, I chose RAID 5 only for the array onto which I write rendered projects. It is PLENTY fast for that purpose. I chose that level of RAID for a slightly increased peace of mind should one drive fail (3 drives = 1 drive used for parity). Another reason why I wanted RAID was simply to have just one very large logical drive for all of my rendered material from now to eternity...well, at least the foreseeable future.

I built the 4x2TB RAID 5 array (6TB effective) to backup all data on (1) my rendered RAID5 array, (2) yet another RAID array that stores all of my media, and (3) my system drive (OS, applications). Thus, I have a few 'layers' of redundancy and backup throughout my system.

I was pretty naive in selecting HDDs for my RAIDs and simply had no idea at that time of the deep cycling that some 1TB (and others) HDDs undergo, making them unsuitable for RAIDs, according to some. I can't dispute the reasoning against choosing such HDDs, but I made a choice based on factors including reviews who reported fine operation in RAID. I'm just a hobbyist, so a drive dropping out, however briefly, is not a show stopperl to me. I can say that, as I stated above, I haven't experienced any RAID-HDD problems other than what I think was attributed to Vista issues.

Steve

EDIT: I started a separate and lively thread some time ago on backup software, which was motivated by my sudden conviction that I REALLY needed a true backup solution for all of my RAIDs.
RalphM wrote on 11/12/2009, 5:47 AM
Thanks to all for the inputs - very thought provoking.

RalphM
JohnnyRoy wrote on 11/12/2009, 6:28 AM
> One of the commenters on newegg claims that WD does not recommend their PC class drives for RAID usage "According to WD it is because the drive enters a deep recovery cycle and the RAID controller times it out."

All this really means is that you should not use desktop drives for a server application. Pretty simple really. Buy any one of the Western Digital Enterprise Class drives and sleep well at night knowing that they were designed for RAID use.

> What are others using for on-board RAID Drives?

My current on-board RAID is made up of 1TB (2 x WD 500GB WD5000ABYS) SATA2 in a Raid 0 configuration. The WD5000ABYS is an enterprise class drive. According to WD "...these drives are designed with enhanced reliability in a 24x7, high duty cycle environment. " We're talking a few extra bucks here for piece for mind.

I currently want to build a RAID 5 array for archive. I'm thinking of using the WD green drives. I already have two 1TB green drives that I use for backup. I'm just waiting for NewEgg to have a big hard drive sale and then I'll pick 5 drives up and configure them for RAID 5 in my external enclosure.

RAID is not all that difficult to understand. Here is the "elevator pitch" for the most popular configurations:

RAID 0 is built for speed with no redundancy. It stripes the data across two or more discs for increased speed. Unfortunately, if you loose one drive you loose all of your data. The capacity of the drives are added together. Two 1TB discs yields a 2TB RAID 0.

RAID 1 is total redundancy but slower because everything is written twice (once to each drive). If you loose a drive you can rebuild it from the other. This is called a "mirror". The capacity of the drives is 1/2 the total. Two 1TB discs yields a 1TB RAID 1.

RAID 5 is a nice hybrid. It is slower than RAID 0 but faster than RAID 1. It stripes the data for speed but uses one drive for parity so that you can loose a single drive and rebuild it from the others. It requires at least 3 drives. (this is like the Drobo that you hear about). Three 1TB discs yields a 2TB RAID 5. The good news is that 5x 1TB disc yields a 4TB RAID 5 so you only loose the capacity of a single disc. This is what makes this so popular.

RAID 10 is actually RAID 1 + RAID 0 (where 1 + 0 = 10) It redundantly writes data to a mirrored pair like RAID 1 and then stripes the data across two discs like RAID 0. Think of it as a striped mirrored pair. If you loose a drive it can be rebuilt very efficiently from the redundant array (more efficient than RAID 5). The downside is that this configuration uses the most discs for the least amount of space. Four 1TB discs yields a 2TB RAID 10.

RAID 10 is the most robust and expensive and RAID 5 is somewhat robust and is the cheapest dollar-for-dollar. For example, a 4TB capacity RAID build from RAID 10 requires 8 discs while a 4TB RAID 5 only requires 5 disc. So you save the cost of 3 discs by using RAID 5. Some will argue that hard drives are so cheap that the argument for RAID 5 is no longer worth it and RAID 10 should be preferred. I guess it depends on your tolerance for data loss vs cost to implement.

Hopefully that will help someone understand the differences.

~jr
ingvarai wrote on 11/12/2009, 6:42 AM
I have used Raid 0 (Striping) with Western Digital Raptors (10 000 RPM) for almost 4 years now.
I do take complete image backups (Acronis) of all drives every night and additionally take incremental file backup stored on USB drives.

I run the risk of losing data, yes. On the other hand, I have super-fast harddisks. One of the computers runs 24/7/365.
On my multi media machine I have one Raid 0 2x70 Gg = 140 Gb for Windows and installed applications, and another Raid 0 2x150 Gb = 300 Gb for my work in progress.

After 4 years with two machines using this setup, and with no problems whatsoever (knock on wood), I find it hard not to recommend this.
Ingvar
Cliff Etzel wrote on 11/12/2009, 9:07 AM
This discussion has me somewhat confused since past discussions on RAID and editing in Vegas seemed to indicate that it wasn't needed. As I read this thread, I'm getting the impression that RAID IS useful for editing in Vegas Pro.

In what situations is RAID beneficial and when is it not?

Cliff Etzel
Videographer : Producer : Web Designer
bluprojekt
vtxrocketeer wrote on 11/12/2009, 9:44 AM
Clif, others more knowledgeable than I can chime in about performance of RAID when it comes to editing video and whether it is needed. I've described my system, which gives crisp performance. Here is my situation in which I think RAID is beneficial (to me):

My main concern in implementing RAID for editing was two-fold, I guess. First, I didn't want to spread media, rendered projects, and other stuff across a variety of single 1TB (or whatever size you like) logical drives (i.e., what Window will "see" as the G: or R: drive, for instance). That is, I knew that I would be generating terabytes of data and I didn't want to wonder, "Now, which drive was that on?" I dump all raw media on one RAID, and I render everything to a separate RAID. Done. No question of what went where.

Second, but relatedly, RAID 5 (especially) is an inexpensive way to accomplish my first goal and have SOME reassurance in data integrity because when/if a HDD craps out, I can just swap it out for a new HDD. Done.

So, it's 1 & 2 working in concert, not screaming performance, that motivated me.

Oh, and as to backup, a few words: RAID is not backup, RAID is not backup, RAID is not backup. I was just plain dumb. I mangled the master boot record which resulted in my RAID not "seeing" one of the drives. It so happened that this particular RAID contained my media and was in RAID 0 and, hence, I "lost" all of my media. It was a "come to Jesus" moment. In a panic, I figured how to and actually fixed the MBR, got my RAID 0 back in a snap, and then promptly dropped some coin on a backup RAID and Acronis. Now I feel better.

Steve
ingvarai wrote on 11/12/2009, 11:20 AM
Cliff:
>In what situations is RAID beneficial and when is it not?

I use RAID 0 because it (allegedly) gives fast harddisks reading (and writing). T what extent it its the RAID setup or my fast spinning disks (10 k) that does it, I cannot tell. Probably a combination.
Working with compressed media, like AVCHD video files, I think it is not important. But when using uncompressed media, image sequences in particular, where 30 seconds of video can fill up several gigabytes, I believe fast harddisk reading is very beneficial.

I have started to use image sequences more and more, for several reasons. One is that it is compatible with almost any video application I know about. The other reason is that I can write a script in Photoshop and run a batch automation on my image sequence and then import the sequence back into Vegas. In Photoshop I can do things with the video (image sequence) than I only can dream of doing in Vegas.

And image sequences are demanding a lot from the harddisk I think.
Ingvar
farss wrote on 11/12/2009, 12:37 PM
JR,
the basics of RAID is easy enought ro grasp. The complications of the various ways say RAID 5 is implemented in the controllers and the impact on access times and read and write performance is not so simple.
Much of what I've read comes from people running data centres. In that scenario maximising average throughput is the goal. Latency is not so much an issue. The opposite applies (sort of) for our needs.

Bob.
JohnnyRoy wrote on 11/12/2009, 3:00 PM
> the basics of RAID is easy enought ro grasp. The complications of the various ways say RAID 5 is implemented in the controllers and the impact on access times and read and write performance is not so simple.

I'm with you there Bob. I was just trying to simplify the explanation for our needs of speed and storage for video editing. If you're trying to maximize database throughput that's a different story.

~jr
ECB wrote on 11/13/2009, 7:20 AM
I have Promise SX 4000 RAID 5 with 4 WD drives that has been running 8+ years and never lost a byte or a HD. I also have 2 RAID 1's populated with WD drives running several years with no problems. My first choice for a RAID would be an WD enterprise drive. I recently upgraded a RAID O using Hitachi 500G drives. NewEgg now ships the Hitachi drives with each drive wrapped in bubble pack in box and wrapped again in mutilayers of bubble wrap instead of each drive in a layer of bubble. Time will tell. So far they are running great.

Ed
RalphM wrote on 11/13/2009, 7:43 AM
Am I understanding correctly, that a drive used in a RAID 1 configuration could not be removed, put in an external enclosure and read like any other drive?

..that recovering the data is dependent on having the RAID controller that was used to write it available?

Thanks,
RalphM
ECB wrote on 11/13/2009, 8:00 AM
I have had a RAID 1 fail because of a defective power cable connector (Molex) to a drive in the RAID. The RAID posted an warning message and continued to operate but with no redundancy. I powered off, replaced the cable. When I powered on the RAID rebuilt the drive with the bad connector. Not a byte was lost. As far as the drive being readable in an external enclosure I have no experience. I have one RAID 1 which will allow you to assign another drive to the RAID to replace the defective drive. Be prepared if you have a 1TB drives to rebuild a drive will take a while. :) Bottom line, you are not dead if you lose a drive.

Ed
RalphM wrote on 11/13/2009, 8:39 AM
Thanks Ed,

Losing a drive is not my concern. A fried controller is another matter if that indeed is necessary to read the data already written.

Since I'm horrible at backing up data, I was seeing RAID as a way to do this seamlessly. It seems, however, that RAID is a real-time backup, but not a long term backup.

With the film transfers I do, it is not unusual for me to have Terabytes of data at any given time. Capturing it all over again would be unpleasant.

RalphM
UlfLaursen wrote on 11/13/2009, 9:49 AM
Hi

I have a netgear ready nas, with 4 500 GB discs in, and I use the WD raid RE2 discs. They run 24/7 and have done that for 1½ year without any problems. It is configured so that one can break, and I have a spare on the shelf in that case, so that I can send the faulty one in for repair.

/Ulf
LReavis wrote on 11/13/2009, 11:18 AM
comments and a question . . .

Question: Is there any reason not to use Win's own software raid instead of hardware raid? I do that even when using a RAID card so that I'm not dependent upon that particular card in case the card fails - I can still get to my data by putting all the drives on any other controller. Years ago, software raid was a bad idea because it ate up CPU cycles, but CPUs today are so robust that it's hard to imagine that, say, rendering performance would be significantly affected.

Comment: Why RAID 0? I'm editing mostly 1920@60fps Cineform. Preview was a bit jerky until I started putting those .AVIs on one of my RAID 0 arrays. Now the AVIs preview smooth as silk.

Security: I'm fanatical. I have most of my work disks bolted to a shear wall (thick plywood-reinforced wall to protect against tremors). However, I'd never used the same power supply to power a backup drive, for I've read that some PS failure modes can ruin all the drives at once. So I use cheap Chinese power brick/USB combos (I just bought another one on ebay for less than $7, free shipping!).

When they arrive, I always plug them into an old 1997 drive that is defunct but nevertheless spins up so that I can check voltage (it should be 5v & 12v, 5% max error). If all OK (and it's been years since I had out-of-bounds voltage readings from one of these), I put the new backup drive into a cheap firesafe from Walmart, bolted to a tight-fitting wooden frame.

I try to get WD green drives for these backups, for they run cold - especially with the fan that I put into the safe (I keep the power bricks & USB electronic device outside of the safe so that their heat won't add to the heat produced by the drives).

I think it was Spot who once argued that firesafes are designed to protect paper, which can tolerate much higher temps than hard drives. True, but the thermal mass of a hard drive is so much greater than that of paper that it probably would take a good bit more time for a fire to heat up the interior of the safe so much that the drives are destroyed. In any case, I put the safe on the floor, with the lid almost closed. I'm trusting that this arrangement would provide protection in the typical fire.