OT: Need advise on media archiving.

UlfLaursen wrote on 9/28/2008, 5:12 AM

I want to archive some of my media, mostly intros, outros, cutaways, etc. etc. for use on different PC's that I have.

My questions is, how would you recommend to do it. I se 2 different options that are obvious:

1. I have a NAS server with aprox. 1 TB free space I could use, and with Gb connection it would be pretty fast. Would I be able to import media directly from the shared drive, even AVCHD converted to 77 Mbit / sec. AVI's and run play a few streams back, or should I copy to local media drive first?
2. A USB harddrive with 1 TB space I could move arround between the PC's.

I sure would like option 1 the most, but if it means that I would have to move media first from the server to local disc, I might as well better want the external USB drive to import directly from.

Thanks for any input on this matter.



Chienworks wrote on 9/28/2008, 5:27 AM
Had you thought about the fact that the gigabit connection to your NAS server is over twice as fast as the USB connection? If you think the NAS is too slow why would you think the USB drive would be any faster?

For comparison, i do this sort of thing over a 100Mbps connection with 30Mbps files and it's fine. Your network is 10 times faster and your files are only 2.5 times the bitrate.
UlfLaursen wrote on 9/28/2008, 6:09 AM
I guess I just think of networking as slow in my mind, but you are right - GB is fast - I think I'll try it out and see how it goes. :-)


farss wrote on 9/28/2008, 6:25 AM
I'll sure second using a NAS box over anything else. I KNOW where the NAS is, it's on every PC mapped as a drive.

UlfLaursen wrote on 9/28/2008, 7:45 AM
Thanks for your input as well, Bob.

musicvid10 wrote on 9/28/2008, 9:04 AM
I tried my WD USB 2.0 drive for rendering, storage, and playback, and I just wasn't satisfied. Too many errors, hung files, once XP insisted on rebuilding the file system from scratch after a long render. I don't know if it's the drive, but I've had problems with it on two different machines.

I think NAS on a fast network is the way to go, for the reasons mentioned above.
UlfLaursen wrote on 9/28/2008, 11:26 AM
I did a test on a a 30 min. file, on my Netgear Ready NAS NV+ , and I was acutally very satisfied.
Playback was smooth, the only small thing is when I jump the timeline and start playback again, the first 1 or 2 sec. will stutter a tiny bid, probably because of the little longer access time for the server drive. After that it just runs very smooth again.

I will defenatly start using it more regulary for footage, and it runs something called XRAID (a little bid like RAID 5) with one spare discthat will give some security at failure of a disc.


John_Cline wrote on 9/28/2008, 11:53 AM
The only problem with NAS is that they are incredibly slow. Most modern single hard drives have transfer rates approaching 100MB/sec, even with a Gigabit interface, most NAS boxes can only muster between 7 and 25 MB/sec, depending on the box.

Anyone know of a fast one?
UlfLaursen wrote on 9/28/2008, 9:41 PM
Hi John

Did some test on my Netgear, and you are right - I get aprox. 21-23 MB / sec. depending on the mix of filesizes etc.

I don't think I will use it as main source for footage, but more like archiving stuff like I use a lot, so I don't have to copy between the PC's all the time like I do now.
That means that my main footgae so to speak will be on the PC I edit it on, and the rest will be from the NAS. I don't think I will do NAS only.

When it comes to speed of a NAS, I think someone here once wrote that you could get NAS drives with dual gigabit interface.
I was not sure whether the bottleneck is in the NAS's interface or the PC's, so I did a copy of aprox. equal filesizes to 2 different PC's starting exactly the same time, and the result was the same - aprox. 22 MB/sec. in total for both PC's.


fldave wrote on 9/28/2008, 10:47 PM
All of my USB 2.0 drives are very buggy, lots of bad files that "seem" to copy good at the time.

So for me, this is totally unreliable. I can't risk one file gone bad twice (main and backup).

Firewire or NAT would be the way to go.
Tinle wrote on 9/29/2008, 2:31 AM
Are Esata connections for external drives more appropriate than USB 2.0 for this type of application?

UlfLaursen wrote on 9/29/2008, 12:35 PM
Yes, could be, but not all PC's has eSATA, and if you like me have several PC's you still need to switch, and I'm not sure you can do it when already started up the PC. For me it would be ok for one PC, but for more I'm not so sure.

I think I'll for now follow this route: putting main footage from tape or card to local media drive on PC and have shared media on NAS.

musicvid10 wrote on 9/29/2008, 1:16 PM
This article http://www.lyberty.com/tech/terms/usb.html suggests a much higher maximum transfer rate over megabit, in the 47 - 60 Mbs range.

I don't know how they arrived at the figure, and I have no way to test it, but that's what I was going on when exploring alternatives to USB drives, since they never worked quite right for me either. I'll poke around and see if someone has done some charts on different drives / connection options.
RalphM wrote on 9/29/2008, 4:56 PM
I'm a little surprised to hear of problems with USB external drives. I transfer very large (25GByte) video files between drives and have never had a problem except for two drives that died completely - a different issue).
Coursedesign wrote on 9/29/2008, 11:06 PM
Transferring files to a USB drive is very different from having an advanced application generate a file on that drive.

With a USB interface, the CPU that is doing the rendering also has to do all the low level data shuffling to communicate with the USB drive.

If there is a small bug in the rendering app (or in some other app or process in the machine), the USB disk write can stop right in the middle of writing a block. Result = a borked disk.

Firewire drives have their own logic that doesn't bother the main CPU. This also means your renders run faster.
barleycorn wrote on 9/30/2008, 3:40 AM
I'd recommend a Windows Home Server:

Windows Home Server at microsoft.com
Windows Home Server at Wikipedia

We're about to expand the storage to 6TB (from 3.5TB), all of which will be available as a single pool (drive letters aren't necessary - the server handles the distribution of files). Configuring the server and clients is absolutely trivial (there's almost nothing to configure). Our data transfer rates (over Gigabit Ethernet) are typically around 40 MB per second (though it might be possible to improve this). We usually copy files from the server to a local drive for editing.

Machines are available off-the-shelf but we built our own. The hardware can be very modest though it's worth getting a motherboard with lots of SATA connectors and a case with room for lots of drives. No monitor, keyboard or mouse is required.

You also get automatic backups up of client machines, optional automatic file duplication (in case of drive failures), remote access to computers via the Internet (to clients running XP Professional, Vista Business and Ultimate) - this is occasionally a life saver, and various other goodies...

Definitely worth considering.