Using multiple drives, need advice about expansio

Ros wrote on 2/21/2010, 10:31 AM
For several years I have been using 1TB sata drives on 2 external docking station (and manually backing them up onto separate cloned drives). My work is evolving and my main problem is accessing several drives at once for a given editing session. My footage is now spreaded over 4 to 6 drives and I have to get most of these drives accessible at once for editing projects.

I have been looking at a 4 Bay SATA to eSATA (Port Multiplier) from Rosewill (RSV-S4-X). Using 2 units connected to a provided adapter card will provide 8 drives capacity which should be great for several years and it remains cost effective. I have also looked at Drobo, but much more expensive.

I have also built myself a server from an older computer (P4 2.4ghz 1.5mb ram) with Windows Home Server. It's great for backing up my main computer OS and my other computers. I have not used it for backing up my HD footage yet, still want to see how reliable it can be.

So I would appreciate your input on how you deal with expansion?
Is my WHS adapted for editing right off the server?

Thanks,

Rob

Comments

Rob Franks wrote on 2/21/2010, 12:53 PM
I don't know what you have for a mobo but there are some out there with secondary on-board sata controllers.

I have the xbx2 mobo which has 4 sata heads on the primary controller and 4 on the secondary. I have seven 1.5TB drives (and a sata BD burner). There's also the IDE connection for 2 additional (old style) drives if need be. I back up my OS drive (disk imaging) to an external USB.

It's always seemed (to me anyway) to be better when the drives (if possible) are handled internally
Ros wrote on 2/23/2010, 3:51 PM
Only have 6 sata heads on mobo: 4 are being used and 2 spares but won't be enough, so looks like I will get that port multipler from Rosewill.

Thanks for your input Rob!

Rob
Ros wrote on 3/7/2010, 6:02 AM
Update:

Bought the Rosewill (RSV-S4-X) 4 Bay SATA to eSATA (Port Multiplier), installed the pci-e control card in my main computer and windows 7 installed the drivers.

So far, got three drives in there and it just works great! This unit is really amazing and well built and I can easily put it away for safe storage.

Rendered a 40 minutes project that ran across all 3 drives with multiples fx and all went fine. Did a lot of editing on it and no issues.

Will buy another unit and eventually might try to expand my server with these.

Rob
LarsHD wrote on 3/7/2010, 12:00 PM
.
Ros wrote on 3/10/2010, 4:30 PM
Just received my second unit and all I can say is that I haven't seen any difference in performance from a drive hooked up directly to the mobo or these units.
I have been editing on one for over a week and I am more than happy, all renders when fine as well.

I like to dump all my footage on the timeline prior to editing, I view it and cut it on the timeline, never using the trimmer and therefore can have over 18 hours of video on the timeline and it handles it very well, no hiccups.

I was looking for a solution but couldn't come across anything so I had to figure it out myself. I like to keep it simple and wasn't looking forward to installing a controller card, but hey, it works and it's cost effective!

Rob
BudWzr wrote on 3/10/2010, 4:42 PM
If you get a NAS and connect it via USB, you can make all the drives one giant drive, that has Raid 5 and automatic online backup.

You don't need to "access several drives".

Take a look at ReadyNAS Pro. The best out there.

John_Cline wrote on 3/10/2010, 5:17 PM
Curious that BudWzr is recommending an NAS when just today in another thread he was saying that he has issues with it temporarily freezing.

Also, USB 2.0 is quite slow, it has a maximum usable transfer rate of 30 megabytes/second. (A modern single hard drive on an eSATA port can easily exceed a 100 megabytes/sec sustained transfer rate.) Also, unlike Firewire which is bus mastering and does not use much in the way of CPU resources, USB requires a fair amount of CPU horsepower to manage USB transfers.

USB 3.0 is available and it's fast, though not much in the way of available hardware yet.

USB-based NAS is OK for offline storage, but in my opinion, not for online editing.
BudWzr wrote on 3/10/2010, 7:11 PM


Well, that's because it's on a network, not USB, and it has to do other things too. That's why I specifically mentioned USB, because that would be a dedicated connection, so no need to be curious anymore.

Wait! Hold the fort, I meant an adhoc connection for the OP.

This is a prime example of how I sometimes say something different than what I mean. I didn't even notice it even after getting a lecture from you-know-who.

P.S. If your network card supports gigabit (CAT6), you can (theoretically) get up to 1000 MB per second, faster than all of the above scenarios offered by Mr. Cline. All of the ReadyNAS line supports GB networking, even the cheaper units.


Yeah, that's Leisure Suit Larry
John_Cline wrote on 3/10/2010, 8:58 PM
Get you facts straight or just quit answering questions here on the forum. People come here looking for real answers by knowledgable people and your overall accuracy rating on the forum is less than 1%. You don't know any audio or video terminology (and amazingly, you're proud of that) and you don't know the facts about the most basic computer science and you have the nerve to argue about it. Please be quiet.

A Gigibit network supports a speed of 1000Mb/s (note the small "b", it means BITS, a large "B" means BYTES) that is 1,000 megabits per second. A gigabit network can support up to 125 megabytes per second, but in practice the actual data transfer rate is slower because of network protocol overhead. eSATA is about THREE TIMES FASTER than Gigabit.
Rob Franks wrote on 3/10/2010, 11:16 PM
"People come here looking for real answers by knowledgable people"

Yes. But they also come here for mature and rational interaction

I see the suspension has done little good for either one of you. Keep it up... although it may interest you to know (I'm reasonably sure anyway) that a suspension is counted as a strike 2 and the next step is a permanent banning.

I think Forum Admin did a good job at stating:
a) Sony are watching these forums and aren't too bothered by the idea of shutting it all down
b) They're busy people down there and haven't got time to baby sit those who can't handle themselves.

It should be abundantly clear by now that you two mix like oil/water and should not be in the same thread. So please, for the good of us all, both of you take a grow-up pill and walk away before this spirals out of control again.
John_Cline wrote on 3/11/2010, 1:41 AM
Rob,

First of all, your second forum post EVER was last December and was in response to a problem Sebaz was having. You said, and I quote, "Oh please... Does anybody really care about this person, his words, what he thinks of vegas... or ANYTHING else he has to say." Looks to me like you were trying to start a fight. (Just don't call him an idiot.)

My problem with BudWzr started and has continued to be that his answers to technical questions have almost always been flat-out wrong. It isn't a matter of ignoring his opinions, I simply can't ignore that his facts are wrong. Less knowledgable users should be made aware that they should not listen to what he has to say. I care about the accuracy of the information presented here, don't you? In his post above, he took a shot at me and then said something that was 100% wrong. I corrected him in a subsequent post and once again, asked him to either be certain of his facts or stop posting. Was this an unreasonable request?

Sony should be bothered that BudWzr is here providing technical support to their customers. They sure as heck wouldn't hire him for their technical support staff, so why do they allow him to provide front-line technical support here on the forum?

SCS doesn't monitor the forum, WE DO. WE check the accuracy of the information. There is a core group of maybe a dozen or two users that provide the bulk of the technical support around here and I happen to be one of them. I think it's pretty obvious that I will passionately defend the integrity of the information presented here. I said I was leaving the forum but I got a lot of e-mail asking me to stay and continue to stand up to BudWzr and his nonsense.

You've only been here a couple of months yourself but anyone that's been here for years knows that BudWzr isn't the only user with whom I have had a conflict based on the accuracy of their flood of posts. Every year or two one of them shows up and a number of users butt heads with them and they either leave or get banned. I've been here for eight years and have been a very mellow, well-behaved contributor when there isn't someone like BudWzr threatening the integrity of the forum.

So let me get this straight, you want me to shut up and let BudWzr or the bozo that follows him to pass out faulty technical advice? Would that be fair to users that may know less than you? What's the difference between me passing out technically accurate answers and me trying to prevent someone else from dispensing technically inaccurate information? Am I not doing a service to the forum users either way?
PeterWright wrote on 3/11/2010, 1:49 AM
Great post John - I hope all digest it.

... but - since your now celebrated suspension, I don't think I've read anything from you that wasn't in some way connected with BW - I look forward to reading a "perfectly normal" epistle from you!
BudWzr wrote on 3/11/2010, 1:59 AM
The OP stated he couldn't find a solution to deal with multiple drives, so he invested in more of the same.

Getting a 4-disk NAS and using his existing disks as a single volume in Raid 5 mode is an ideal solution.

Unfortunately, the way JC misquoted me and threw out a bunch of numbers that sound "technical" to the uninformed, the great idea has become another BudWzr ridiculous claim, and that's bad for the forum.

Minor technicalities aside, if you consider the concept of a NAS and how many benefits it provides, surely it's got to be high on the list of possible solutions, and that's why I rebutted Mr. Cline.

His "counter-rebuttal" just to stomp me in the ground regardless of the merits stifles new ideas and progressive thinking, that's why I put the "5 Star" animation to highlight my post.

And because of Mr. Cline's stature here, some poor putz out there will probably spend a fortune on a rack of single drives and controllers and continue to compound their headaches because John Cline wanted to win an argument with "an idiot".

I think THIS is what Sony is talking about. The suppression of new thinking by bullying from the "pros".

Rob, I respect you AND John Cline, I said so in the "John Cline Suspended?" thread, and I tried to make peace with him, yet he continues to stalk me and cross reference my posts in a mean way.

I don't know what to do.

EDIT: If anyone wants to shut me up, just start talking about Vegas, and some of the newer features like gradient transitions, 3D track motion, even network rendering, and let me learn something from you. Dazzle me with diamonds. I want to learn something from a real Pro, not engage in endless debates over minutia.
John_Cline wrote on 3/11/2010, 2:04 AM
I don't know, correcting BudWzr posts has been a full time job.

OK, here's something to look forward to and it has nothing to do with BW... I have created an updated version of RENDERTEST-HDV to release to the forum. The exciting part of this is that Steve Mann has created a really cool online database where people can post their results along with their hardware configuration. The problem with the last RENDERTEST-HDV was that the thread had over 400 posts and I never had the time to compile the data into anything useful. Steve Mann has elegantly solved that problem. Also, the results of this new RENDERTEST-HDV should take exactly four times longer to render than the old test so the results can still be compared to the old RENDERTEST-HDV. Stay tuned... unless I get suspended or outright banned.
PeterWright wrote on 3/11/2010, 2:06 AM
> "I don't know what to do."

BW, I often read questions from people and feel inclined to try and help, but then if it's something I'm not ABSOLUTELY SURE about, I restrain myself rather than post a misleading or half-answer, and leave it to someone who actually has the knowledge required.
Maybe if you did that too .......
BudWzr wrote on 3/11/2010, 2:22 AM
Peter, you're right. The best way to get along here is to lurk.

JC is basically a "Policy Wonk" (Not said offensively John), and I'm an "Ad Lib" shoot-from-the-hip type.

Why can't both types contribute here?

The NAS solution is not a half answer or something I'm unsure of. I HAVE a NAS myself, I use it for lots of things. NAS is a proven solution that works well.

I went through two cheapee NAS's before getting a ReadyNAS NV+. The ReadyNAS is a VERY robust machine and has great popularity among the oldtime gurus that KNOW a thing or two.

I'm NOT making this stuff up, reading it on WiKi, or pretending or faking in any way.

I can never be as technically astute as JC, on technicalities he has me beat, hands down. Does that mean I can't express any comment or opinion or idea?

I'll bet lots of people out there use a NAS but don't mention it, and won't dare get caught publicly agreeing with me, and that's fine, I'm not here for glory.

I read a quote some time ago that stated "If you keep doing the same things, you'll keep getting the same results", and sometimes we get stuck in a mental block and you think "more horsepower", a faster computer, etc. is the answer, but many times a change in "ThinkFlow" is what's needed.
John_Cline wrote on 3/11/2010, 2:44 AM
Bud, you were dead wrong about the speed of a gigabit network (by a factor of about 8) and I corrected you. Deal with it. I didn't want some newbie saying to a friend tomorrow, "Dude, Gigabit networks can do a 1,000 megabytes a second. Word up! I read it on the Sony forum so it must be true."

Where did I say that NAS boxes couldn't be a consideration? Point that out to me. I run an NAS here and it works great for certain things. I sure as heck don't edit on it. Your suggestion was a USB NAS box and USB 2.0 and gigabit NAS boxes are considerably slower than SATA which is what the original poster was already considering. No reasonably priced solution is particularly fast using RAID-5. RAID-5 is pretty much the slowest of the common multiple drive configurations, but it does provide excellent data integrity in case of a drive failure.

BudWzr, this has nothing to do with my ego or wanting to win an argument with "an idiot." This is NOT personal, the ONLY thing I want is for you to make absolutely, 100% certain that the information you are posting is factually accurate. Your other fallacy is that your thinking is somehow "new and progressive", it isn't, you might think so because you have no experience whatsoever and you really don't know what's new and what's old. I've been on the cutting edge for 40+ years, you aren't there. I also didn't just throw out a bunch of numbers to "sound technical to the uninformed," the numbers were 100% correct and they actually mean something to the educated. Look, I don't know as much as the true heavyweights in the business, but I'm a lot further down the road and I know a LOT more than you. I was involved in the invention of the personal computer in 1974, I just might know a little more than you about computers, too. There are a bunch of seriously knowledgeable people here and they're here to help. Why don't you just sit back, lurk and learn something from all of the talented people here and contribute ONLY when you are absolutely sure you know what you're talking about. That's it, "Lurk and Learn." Shooting from the hip is a style call, go right ahead but you better make sure that you're correct.
farss wrote on 3/11/2010, 2:47 AM
"Minor technicalities aside, if you consider the concept of a NAS and how many benefits it provides, surely it's got to be high on the list of possible solutions, and that's why I rebutted Mr. Cline."

Well I have a 5 disk NAS and NO, it is not high on the list of solutions to serve data to a video editing system. It's OK for backups if you don't mind waiting and it's fine as a drop box for moving stuff around my edit systems but trying to pull files off it for editing is not a good solution at all and I'm running good HP Procurve GigE switches.

Now John presented reasons as to why you were wrong. These were not minor technicalities, they were gross errors in your understanding. If you want to challenge someone who makes a statement using facts then you have to be prepared to back up your challenge with facts.

On the other hand if you said something like "Wouldn't a NAS box be a good solution?" then some of us here might have taken the time to explain why it wasn't. Or you could go do some research to try to understand why people who use shared data storage over a network spend a very large amount of money on their systems.

Bob.
BudWzr wrote on 3/11/2010, 2:57 AM
John, that's a typo, and I left it there unedited because you caught it and it does need to be corrected.

Would you be willing to allow that my suggestion to hook up a NAS adhoc IS worthy of mention? And for many it can be a panacea?

If you can make the distinction that my post WAS relevent and timely, although some facts were wrong, and that I'm not an idiot for posting it, that would go a long way in resolving our differences and then I can consider your advice of lurking more and posting less.
BudWzr wrote on 3/11/2010, 2:59 AM
But is it hooked up adhoc? That's the key ingredient.

My NAS is slow too (20 MBs) on a router.

That's another thing, many of my posts here get a cursory read and then I get chastised because a caveat or qualifier was not noticed.

==========================================
Well I have a 5 disk NAS and NO, it is not high on the list of solutions to serve data to a video editing system. It's OK for backups if you don't mind waiting and it's fine as a drop box for moving stuff around my edit systems but trying to pull files off it for editing is not a good solution at all and I'm running good HP Procurve GigE switches.
farss wrote on 3/11/2010, 3:16 AM
I don't quite know EXACTLY what you mean by "ad hoc"

My NAS is a Thecus 5200 with 5 Samsung 7200 RPM 400GB disks in it. It connects directly to my main edit system and another of its four ports connect to the Procurve GigE switches that feed the other systems on my network. My router connects to the internet on one interface and the other port connects into the backbone, no local network traffic goes through the router. This is important, if you're getting poor performance and I used to, the fix, the only fix is good switches. All my problems vanished when I threw out the mixed switches I was using and invested in Procurve switches.

The reasons why these boxes perform poorly are well documented.

The CPUs are too slow, there's high overheads in writing to RAID 5 etc. Also when you look at the published performance figures keep in mind to look at the sustained data rate, not the average data rate.

I shall no doubt at sometime have to upgrade my tiny NAS. The best NAS boxes come from QNAP.

Bob.
BudWzr wrote on 3/11/2010, 3:58 AM
A direct, single CAT6 cable connection between the NAS and the network card with NOTHING running on the NAS except SMB, or whatever protocol you use.

No DHCP server, no backups (except at night maybe), with journaling turned off and optimized for maximum throughput.

===========================================
I don't quite know EXACTLY what you mean by "ad hoc"
BudWzr wrote on 3/11/2010, 4:03 AM
Another thing is if you let the NAS do backups at off-times, you can run it in JABOD mode, and expand the memory to optimize it even further for video. Yes?

Of course the "Office Server", or similar situation (like mine) will encounter seek delays.

And I'm sure that a dedicated "slave" type TCP/IP setup may never be as fast as a buss connection, but I doubt Vegas would suffer much. There are other bottlenecks too, like the CPU clock.

Oh, I re-read the first post and he's using Windows Home Server as a NAS. Better to run Apache, it's very light and fast. And a NAS uses a raw filesystem (fast), not NTFS (slow).

Another boost is to enable jumbo frames on your network if it's gigabit.
farss wrote on 3/11/2010, 5:00 AM
"But I think that a dedicated "slave" type TCP/IP setup may still never be as fast as a buss connection, but I doubt Vegas would suffer much."

You think. Vegas suffers a lot for me running off internal SATA drives in RAID 0 using a reasonable controller delivering over 100MB/sec.
I've given serious consideration to going to 15K SAS drives and I'm doing nothing spectacular. Plenty of Vegas users running cameras that record at 100Mb/sec too. Not to mention the guys using DPX.

Bob.