Editing large files - hardly works

Jay M wrote on 12/9/2007, 11:01 PM
I was working on a little project that consisted 4 cameras each 15 minutes. The video was origionally DVCpro50 in QT on a mac. I used compressor to create uncompressed SD QT that work in Vegas. The resulting files were about 25GB each.

When I drag them into Vegas the first one pops up as fast as any DV25 files would. But as I started to add the other 3 both Vegas and windows explorer would lock up as if they had crashed. but eventually it worked. Then if I go to another progarm and then back to Vegas, it takes a almost a minutes for Vegasd to start responding again.

The files were being streamed from the mac desktop to the PC. While playing back the multicam project the frame rate was at about 7fps, network was at 8%, and CPU was at 70% (dual opteron, win XP).

DV files are never a problem over the network, but these large ones usually cause problems.

Can Vegas work WELL with uncompressed SD files? I really don't want to have to learn FCP. For what I do audio is more imortant than video, so Vegas would be the prefered tool.

~Jay

Comments

rmack350 wrote on 12/10/2007, 12:47 AM
DV25 is a compressed format, so if what you have is truly "uncompressed" then it's not DV25.

Rather than try to calculate it, I just looked at an example DV clip on my system. It's 16 minutes and 40 seconds and the total file size is 3.53 GB. So your 15 minute file weighing in at 25GB is definitely not DV.

DV25 has a standard bit rate, by definition, of 25 Megabits/second. Your rendered QT files should be half the size of the DVCPro50 files if they're DV25.

So, you've got files that are about 5 times bigger than expected, you're trying to play them over ethernet, they're not in a format that's native to Vegas (AVI), and given the CPU load on your dual opterons, this is some sort of processor intensive quicktime format, not really even an uncompressed format. Gee, it's hard to guess what could be going wrong...maybe everything?

Regarding uncompressed SD. If it's uncompressed AVI and you have at least two disks in a striped array, Vegas will play it at full frame rate. If it's Quicktime, and at the other end of an ethernet cable, all bets are off.

The general windows behavior you're talking about is the system running out of memory. It's using the swap file like mad, which is why it takes so long to pull Vegas back up. So maybe your PC is memory poor, or maybe Vegas' RAM Preview setting is way too high. You could set it down at 16 MB and Vegas would get along well enough.

Rob Mack
Jay M wrote on 12/10/2007, 1:26 AM
"Gee, it's hard to guess what could be going wrong...maybe everything?"

That may indeed be true. The files are defined by Compressor as Uncompressed. I can't remember if they are 8 or 10 bit. I may have confused the issue by mentioning DV25. I only used that for a performance comparison.

My system has 2 gigs of ram, and two 2.39 ghz Opteron 250's

Gigabit ethernet should be able to sustain 60MB per second, as well as any modern HDD. I just found that the bit rate for 10-bit SD and it is 27MBps. So the first problem is that two streams would be the theoretical limit. Only one should be considered safe.

Next is the memory issue. I'll look into that.

Considering I am attempting the impossible I should give Vegas great credit! I was able to do a multicam edit, and render an MPEG2 for DVD.

Now- What must I do to edit 4 streams at DVCpro50 quality or better? Uncompressed would require a multidisk raid, any compressed format might require more CPU than is currently available.

Is an uncompressed QT that much different than an uncompressed AVI? Apple/FCP may be the king of codecs, but they seem to go to great lengths to limit compatibility with windows!

Maybe I should just learn FCP ;)

~Jay



deusx wrote on 12/10/2007, 1:55 AM
>>>Apple/FCP may be the king of codecs<<

More like court jesters.

That is the first thing that is wrong. You are using files contaminated by a Mac. It's like having unprotected sex with a $10 street walker.
farss wrote on 12/10/2007, 4:05 AM
Something I've neve quite got my head around is what happens with disk systems when we playout multiple streams. I think the constant repositioning of the heads as they jump between multiple files can really slow things down. Some multitrack HDD recorders interleave the data ro avoid this problem, BWF does the same thing.
I recall from decade ago when I worked with 8" SCSI drives something about elevated seek. Today I read about Native Command Queing. Seems possible this has some bearing on the situation here.
Another factor is that Vegas also creates another file (.sfk) for every piece of media we access. That might also place more strain on the disk system. It's perhaps a pit we cannot specify where those files are located. I'd figure having them on a local drive might ease the situation.

One final thought, why not copy the files to a local RAID.

Bob.
rmack350 wrote on 12/10/2007, 10:26 AM
Eeeek! Learn FCP! Noooooooooo. ;-)

Regardless whether Gigabit ethernet *should* support the throughput, it may not actually do it. For instance, I have tried playing 8 and 10-bit 422 AVI files created by our axio systems and they don't play well across our gigabit ethernet.

Many compressed formats strike a good balance between disk throughput and CPU load. Generally, I think if you can use a 422 codec that's a good start. DVCPro50 is 422 to you shouldn't need more, and wouldn't want less.

http://www.onerivermedia.com/codecs/ has a great comparison of quicktime codecs, so if you have to use QT in Vegas this is a good place to start looking. I'd make some 10 second render tests to figure out what codecs are good candidates.

Rob