Using Vegas with file server?

rs170a wrote on 11/13/2007, 10:15 AM

This question below came up on another Vegas forum but the responses so far have been saying that it can't be done for a variety of reasons.
I'm looking into doing something similar at the community college I work for and would appreciate the wisdom of folks that know a lot more about these issues than I do to give me either a thumbs-up or a thumbs-down on this idea.

Mike

Original question:

We've used Vegas for a few years now at our small tv station. Due to increasing workload though, we're thinking about a way to streamline the workflow, but also have redundancy and backup as well for all files concerning Vegas projects (captured media, music library, pictures, veg files, etc.). Our IT specialist is highly recommending the use of a file server with at least 3TB of storage. All of our office computers could read/write to it, replacing the staff member's "my documents" folder on his or her computer. The real trick though comes with being able to work on Vegas projects, able to work on projects from any of our four workstations, AND be confident that their materials are always being backed up.

Right now we work from the local drives on the machine. If I open a project created and saved on edit station 1 on edit station 2, I have to go through the process of replacing missing files. A couple of producers store everything they work on to a USB external drive, but this can be unreliable unless the entire USB drive is backed up periodically. A high speed file server seems like a great way to go. So far, it seems Vegas will stream video on a timeline over a network just fine. It can play back video clips from another computer (such as a server or network drive), and it can render to a computer on the network pulling the project's media from a local or network drive. But... I've discovered that it can't capture and write video data to a network drive or server without constantly dropping frames. It appears to cache incoming video then write it to a drive in chunks. I haven't tried printing to tape a project (including captured video) opened and stored on a network drive or server.

SO the big question is... can a file server work with Vegas in every step of the editing process, including capture? In the near future, we may switch to using the new PMW-EX1 cameras soon, so that will simplify things. For now I'm open to suggestions right now. Again, we want to be able to keep it simple, open and work on projects from any edit workstation, and have a backup system in place. One suggestion was to capture video to the internal SATA drives, then move it to the server before working on it in a project. Obviously higher performance can be had if media comes from the local drives inside the computer vs a media server. Think of it like Avid Unity, only with Vegas. I need suggestions, so lemme know what you folks think. Thanks!

Comments

Chienworks wrote on 11/13/2007, 10:39 AM

Sure, it will work splendidly fine.

I wouldn't mess with the my documents folder; leave that on the individual users' machines. Just don't ever use that folder. On every computer map the same drive letter to the common storage directory on the server and then use that drive letter for all the media.

Keep in mind that you can share media files simultaneously between users, though you won't be able to save to the same file. Also keep in mind that it would be a BAD idea for multiple users to be editing the same project file simultaneously. What can happen is that user A opens the project, then user B opens the project, then A saves (or even autosaves) some changes, then user B saves some changes. Since user B opened the file before user A saved her changes, user A's changes are now gone.

If multiple users must work on different parts of the same project simultaneously then each user should work on a small independent .veg separately from the other users. These independent .veg files can then be nested into a master file for final rendering.

In short, there's no reason why this wouldn't work quite well.

rs170a wrote on 11/13/2007, 10:46 AM

Kelly, thanks for the quick response.
I agree about the My Documents folder as that's the first thing that gets changed on my home machine as well as all the Vegas machines for the students.

What do think about capturing across a network though?
Feasible or not?

Mike

Spot|DSE wrote on 11/13/2007, 12:14 PM

We do this at our place, media lives on a Ciprico RAID connected to a server.
My files are saved by project name with my initials appended. if it goes upstream, the next person appends theirs. We also help deal with confusion by opening the project properties/summary and making notes.

[edit] the server system is where we capture, but have successfully captured DV and HDV over the network without issue. I don't know if it would work if you had several students accessing the server/RAID at the same time someone else was capturing.

farss wrote on 11/13/2007, 12:33 PM

I think to really get video to work 100% transparently you need a system designed for the task, not an IT centric network. It can be built with low(ish) cost parts but the design is the trick.
Systems I've seen working run multiple GigE feeds from the servers to the switches to ensure enough bandwidth from each workstation to the server. From my own experience it is amazing how much you can squirt down GigE, jumbo packets help with video.

Bob.

rmack350 wrote on 11/13/2007, 2:25 PM

In fact, you (Bob) had provided links to a system designed specifically to do this over GigE.

First off, if you're going to map network shares as drive letters I'd recommend mapping them at the top of the alphabet...X, Y, Z. This is because most removables like thumb drives will automatically map to the lowest available drive letter. I find that if I have a network share mapped but I haven't browsed it then a thumb drive can grab that drive letter and trouble ensues. Just a helpful hint.

There are several ways to do central storage. Here at my shop we have a SAN that shares a RAID array to the edit stations over fiber cables. Throughput is very fast, enough for HD, and the SAN server locks files for you as you use them. This is an expensive solution though.

Bob is talking about running several network cables to a switch but it's actually more detailed than that. I think this is called gigabit aggregating or something along those lines. It requires a little more setup and forethought but you can get good speeds out of it. Other parts of the equation would be jumbo frames (to send large chunks of data) and some sort of Quality of Service management to guarantee delivery.

One thing about Ethernet is that it's not really designed to handle steady streams of data. It's designed so that everyone can get a little something as soon as it's available, as long as they can politely wait their turn. So you need to do some tuning to make Ethernet behave more like a direct connection to a hard drive.

Good luck.

Rob Mack

willlisub wrote on 11/13/2007, 6:52 PM

HDV and DV capture and edit via GB enet works just fine. The trick with a large number of users is the # of drives themselves and bandwidth to the server(s). more raids are better than 1 large raid.

A simple trick is to have multiple raid's, preferably Raid 5, on one large raid card or multiple cards. Have mulitple GB enet ports on your sever. You can also use multiple servers. Have two servers with 2-3 separate raid 5 subsystems on each. Have 2-3 subsystems for editing and then have another subsystem to back up the other server at night. The backup raid can be much larger than the editing raids subsystems.

For instance, you could take a 12 port SATA raid card, and make 3 - 4 drive raids from that. Or maybe 2- 6 drive raids.

I don't know the exact number of people that you can use per raid 5 subsystem, but it shouldn't be to expensive to start out and then add to it if you need more speed or capacity for your users.

I have captured two cameras via firewire live on one computer to network and local drives. I probably could have captured both to the server, but. I was testing long live captures on one computer, not really testing the network.

The type of editing and whether you are outputting real time to decks are also considerations. If you are rendering your output, then your capture and playback should be your determining factors. If you are editing with multiple tracks, remember that each video track gets pulled up during playback or rendering.

Keep your current jobs spread out amongst the different raid 5 storage modules and figure out a way to not have 6 machines capture to the same raid 5 storage subsystem.

I have been playing/testing a 4 port card which works with port multipliers which handles 5 drives each. I have 3 multipliers and 11 drives. . 4x5 = 20 drives. I could reconfigure the drives and make 3 raid 5 subsystems and see how many machines could capture simultaneously. I could test 2 hdv and 2 DV's at once.
Not sure I have the energy, but who knows.

Example.... 12 drives on 1 -12 port card. take drives 1-4 and make them raid 5 subsystems and call volume video 1. Drives 5-8 and call this raid 5 subsystem video 2. Drives 9-12 and and call this raid 5 subsystem video 3.

This would probably give you enough to handle 3-6 editors. Use 500GB drives because they are cost effective.

I'm assuming hdv or dv editing. If you are doing some other codec with higher data rates than you will need to spend some more money and get some bigger pipes, like fiber, higher speed drives, and more expensive raid cards.

For HDV or DV, SATA raid will work fine.

rs170a wrote on 11/13/2007, 8:07 PM

WOW!!
Gotta love this Vegas community :-)
Take a few hours off for dinner & night school and the excellent suggestions just pour in.
I'm going to save all of them (especially for my IT guys) for future reading as there's a lot of information to digest here with some of it currently over my head.
I've also let the original poster know about the responses and I'm sure he'll be very pleased as well.

Mike

rmack350 wrote on 11/13/2007, 10:08 PM

BTW, Link aggregation is the search term to use regarding multiple ethernet lines.

Rob

Terje wrote on 11/13/2007, 11:12 PM

I have previously recommended against this (on the other forum) but will expand a little. If you can get the bandwidth up you can absolutely do this. I would still not capture to anything but a local drive, but that's OK, you can either run the capture on the server it self, or on the workstation and then move the captured files onto the server.

Now, if you want to do this, you are going to need quite a bit of bandwidth. Multiple GigE channels to a switch is one solution. Fibre Channel at higher rates than 1G is another.

Check out
http://en.wikipedia.org/wiki/Storage_area_network
and
http://en.wikipedia.org/wiki/MetaSAN

Doing this with regular 100M Ethernet is not going to be pleasant, but if you put some thought into it, it can be done. It's going to cost a little bit of money, but probably manageable.