Vegas can't handle lots of m2t clips at once!

Laurence wrote on 4/18/2006, 10:42 AM
If I wasn't already bald I would be pulling my hair out in frustration over this one:

Vegas can't seem to handle lots of small m2t clips on a timeline all at once.

Thus my approach of capturing with HDVSplit, generating proxies with Gearshift, editing these proxies, then "shifting gears" back for the final render simply won't work.

If I was capturing one or two large m2t files with Vegas rather than multiple scene separated m2t clips with HDVSplit this wouldn't be a problem so I can see why it hasn't been pointed out before.

Anyway, I'm not sure exactly what to do right now. My 3.06 P4 isn't really up to editing with Cineform and for a current project I have five or six hours worth of scene separated m2t footage and Gearshift proxies sitting on a hard drive ready to edit. It looks like the approach I planned on isn't going to work!

Comments

apit34356 wrote on 4/18/2006, 11:09 AM
Laurence, there are many approaches to this problem, but 1) break your veg into smaller parts, render out and put them together in a master veg. or 2) append the M2t file s together into large enought sections that will not too much work to replace on the timeline....etc...

You could try to remove all background and non-critical tasks from the system, memory and cpu requirements are high when m2t files start to generate video frames,
so if you stacked a lot of tracks of m2t on top of each other............ memory space and cpu usage thru the roof!
FrigidNDEditing wrote on 4/18/2006, 11:48 AM
what filters are you using? - if you're using the Aav6cc filter I get the red clips when working in HDV footage. But I think I actually linked it to the resolution or something, it wasn't just the m2t files, I didn't dig too much, I just saw that it didn't work, and stopped using it.

I'd say if you're using any 3rd Party filters, you may find that that is the problem.

Dave
Laurence wrote on 4/18/2006, 12:34 PM
No filters whatsoever. I didn't get that far. What I'm finding is that I can get about 90 m2t clips on a Vegas timeline. After that some kind of buffer overflows somewhere and Vegas crashes.
rmack350 wrote on 4/18/2006, 2:02 PM
Sound like the plan of breaking it up into segments would be the way to go.

Rob Mack
apit34356 wrote on 4/18/2006, 3:27 PM
This may help a little with resources, disable the "pics" in the Project Media. Media Manager consumes resources as well, tho, the new media manager is better written and operates with better management of resources.
Laurence wrote on 4/18/2006, 10:35 PM
How do you do this?
FrigidNDEditing wrote on 4/18/2006, 11:20 PM
CTRL+SHIFT+W will disable them in the T/L - but as for the project media window, or media manager window, I don't know.

Dave
Laurence wrote on 4/18/2006, 11:30 PM
Thanks, I had just figured that out. Anyway, it doesn't help. Vegas can still only handle less than a hundred m2t files on the timeline at once. This is a Gearshift dealbreaker for me. I guess I have to use the Cineform approach after all. I just wish my PC was just a little more powerful. Yes, I could work with just a couple of big clips, but that is just too confusing for me. When I'm editing and need a bit of video, I can find it if its in clip 97 for instance, but finding that same bit 20 minutes into clip 1... forget it. I just can't work that way.
PeterWright wrote on 4/19/2006, 12:49 AM
Laurence

In case you didn't know, any sequences contained within say a one hour single clip can be indexed and immediately accessed any time by carefully naming and saving regions in the trimmer, just as if they were separate clips.

I have always worked this way, and have got used to naming each region in such a way that they are listed in exactly the order I want, and so that I know exactly what is in each region.

The advantage with HDV, whether I use Cineform Intermediates or Gearshift Proxies, is that the average job will have maybe four or five clips only.
rmack350 wrote on 4/19/2006, 7:19 AM
You might try to get comfy with subclips then. Just give them a little extra head and tail when you make them. Of course, maybe there's no difference between 200 media clips and 200 subclips taken from one piece of media.

Rob Mack
Laurence wrote on 4/19/2006, 8:32 AM
Has anyone tried this, that is putting a couple of hundred m2t subclips on a timeline? It seems to me that it is likely that Vegas will be as likely to crash with the subclips as with the separate clips. It also seems likely to me that once I have my footage divided up into small sections on the timeline, it may crash even if all these little sections are based upon one or two large clips.
Laurence wrote on 4/19/2006, 8:38 AM
Another question: has anyone successfully done a large project with Gearshift?
johnmeyer wrote on 4/19/2006, 8:40 AM
Vegas can't handle lots of m2t clips at one time

This is, unfortunately, not a news flash. Sony is well aware of this. You need to use intermediates or proxies. This approach has many other advantages as well.

To be clear, I am not talking about the general slowness of even one m2t file compared to the proxies or intermediates, but rather the additional slowness that happens from having many dozens of m2t files on the timeline at once.

What version of Vegas are you using? I have not tested, but 6.0d may be better.
Spot|DSE wrote on 4/19/2006, 9:03 AM
Laurence, I guess it depends on what you call a "large" project. I've edited/assembled a couple of 120 minute + projects, derived from 8+ hours of originating tape. Used GearShift with the proxies and HDV files on my external hard drive. The only issue I had was one we've already updated for, and that was for audio files that I'd shipped over to Forge for editing as takes.
Laurence wrote on 4/19/2006, 9:03 AM
Even if you use proxies, this problem bites you when you switch back to the m2ts for final rendering. I'm not just talking slowness, I'm talking about clips not showing up on the timeline and renders crashing with less than 100 m2t clips on the timeline.
Laurence wrote on 4/19/2006, 9:07 AM
Spot, that is the size of project I am talking about. Did you use the original m2t files for rendering or did you switch out the Cinneform avis for the fnal render.
Spot|DSE wrote on 4/19/2006, 9:21 AM
I swapped back to the m2t files. Bear in mind, I'm using 8 tapes, all captured as full length pieces, which are trimmed either on the timeline or in the trimmer. They're not captured as smaller, segmented files.
rmack350 wrote on 4/19/2006, 9:29 AM
It sounds like, from Douglas' answer, that you can do this if you have fewer actual m2t files. It's kind of screwed for you becuase you're well down the wrong road already.

So how did breaking it up into smaller files work for you? I know you've had time to try it by now...

Rob Mack
rmack350 wrote on 4/19/2006, 9:58 AM
Aside from trying to break up the project into smaller files, is there a way to reconsolidate everything into single clips?

I think it'd be easier to break the project into bite sized chunks. I have no idea whether you could trick Vegas into using one big file after you'd started with a bunch of smaller clips. Hypothetically, the info is there in the project to do it - you know the tape name and you know where your clips start in each tape so the goal would be to somehow programatically convert all of your media pool objects to subclips or regions of a single master clip. Seems like it could be done and it'd be a worthwhile script.

Seems like Veggie Toolkit has a tool called Capture cutter that does the reverse of this. It allows you to capture a monolithic clip, break it up on the timeline, and then recapture it as individual clips. Maybe Randall built it to go the other way as well...

Rob Mack
Laurence wrote on 4/19/2006, 10:24 AM
Working from fewer larger clips would be really hard for me. I really like working with small scene separated clips. From what I'm reading and experiencing, it seems like my best option is just to forgoe Gearshift altogether and work with Cineform. I get a smoother preview from the proxies, but I'd rather live with a rougher preview than have to deal with finding the footage I want from the middle of large tape length clips.

I just wish I had known this before I got so far into this project. I am sitting with clips from about six hour long tapes on an external hard drive, all divided up into separate m2t clips and with Gearshift proxies already rendered for every single one of these clips. Boy was that a lot of rendering time to have to throw away and redo!

I'll bet there are a lot of other people in this forum who thought they could capture scene-split m2t files with HDVSplit and work with them using Gearshift proxies. anyway, for anyone wanting to try this approach: Don't do it. It doesn't work. It's not Gearshift's fault, it's Vegas's, but either way, you can't do it this way.
FrigidNDEditing wrote on 4/19/2006, 10:54 AM
why not just use the trimmer and just drag and drop the sections you want on the TL?

to me - that's the best workflow, but then again, that's me. I will also drop clips on and just trim, but Trimmer is my method of choice.

Dave
riredale wrote on 4/19/2006, 12:13 PM
From what I can tell, 90 m2t clips is about the limit.

I did a test just now, capturing 12 typical m2t clips from the camera, with average size of about 70MB each. I then copied them over and over in the PC until getting about 12GB of files, which is about what one miniDV tape would be able to hold.

I found that I could pull in 90 clips before getting the international "no" symbol when trying to drag additional clips to the timeline from the Vegas explorer window. I also found that I could only get 90 clips if I carefully pulled in just 10 or 5 clips at a time--more than that (when getting near the limit) and Vegas would crash.

I don't know what the limit is for DV clips, but I can play with that later today. I also don't know what the limit might be for Cineform clips.

Incidentally, 90 m2t clips in my tests works out to about 34 minutes of footage.

Incidentally, the limit has no bearing on clip size. Also, though my ram is 512MB, my swapfile (on a different drive) is 2GB. Finally, I found my PC would get bogged down in rendering if I tried to import clips with the thumbnail and audio render features turned on, so I left them off (the PC is currently just an Athlon XP running at 1.6GHz).
JohnnyRoy wrote on 4/19/2006, 2:37 PM
> forgoe Gearshift altogether and work with Cineform. I get a smoother preview from the proxies, but I'd rather live with a rougher preview than have to deal with finding the footage I want from the middle of large tape length clips.

Why would you forego GearShift? GearShift will make the CineForm files for you and swap between proxies and CineForm. (Did you think GearShift only worked with M2T’s?) So you can still edit at the speed of DV Proxies and swap to CineForm when you’re done. Beside you already own GearShift, why not continue to use it.

> I just wish I had known this before I got so far into this project. I am sitting with clips from about six hour long tapes on an external hard drive, all divided up into separate m2t clips and with Gearshift proxies already rendered for every single one of these clips. Boy was that a lot of rendering time to have to throw away and redo!

You don’t have to redo the proxies. Open GearShift and load all the m2t files in to the filesystem box. Select (none) for Proxies and HDV 1080-60i Intermediary under HD Media. Place the CineForm files in the same directory as the proxies. When you switch gears, GearShift will find the CineForm files and swap them in.

GearShift always looks for CineForm files before M2T files so you can always make the CineForm files later.

~jr
riredale wrote on 4/19/2006, 3:41 PM
I didn't know that, but the practical effect is that you will need to render in parallel to not only a DV format but also a Cineform format, which will take a lot of time and disk space. Still, it removes the 90-file limit, and that's a good thing.

Over lunch I made a bunch of Cineform clips and tried loading them onto the timeline. Sure enough, they behaved almost like DV clips, and loaded quickly. I loaded as many as 180 Cineform clips before getting bored and going on to something else.

So at this point, it look like working with m2t at any point creates certain limitations. For one thing, it takes forever for a slower system to show thumbnails and audio waveforms, so those features probably need to be turned off. Secondly, upon reaching about 90 m2t files your program will crash. Nonetheless, I view it as something like the old 4GB limitation of working with DV files--yeah, it was a pain in the neck sometimes, but you could work around it. For example, my workstyle involves making lots of 10-20 minute chapters of a documentary, so in most cases, I won't have 90 m2t clips (GearShifted DV clips) on the timeline anyway. So I can still do all my work in widescreen DV and get away with it when it comes time to do a final render back to tape or to a 4:2:2 format such as HuffYUV. I've never been a fan of putting EVERYTHING on a single timeline at one time--there's just too much opportunity for a single glitch to screw up the whole project. At least with chapters, I can put a piece at a time in the can and move on, knowing that the piece just finished is good to go.

Still, the Cineform/widescreenDV GearShift trick is a good one.


Incidentally, I tried opening two instances of Vegas and was able to load 60 m2t clips on each, so the limitation of 90 is apparently per each Vegas instance. Still, it really brings my 1.67Ghz Athlon XP to its knees to render thumbnails and an audio wafeform. (Anthropomorphic note: yes, processor chips really do have knees, but they're really tiny.)