At my wit's end with HDV in Vegas

NickHope wrote on 1/22/2008, 7:04 AM
I hate to be negative but I've got serious problems here and really don't know what to do next.

I've been doing some stability testing of HDV in Vegas Pro 8.0b versus 8.0a and 7.0e. This is on my Quad Core Q6600 with Windows XP x64 and 4Gb RAM. I'm simply trying to get a load of 5-25 second HDV files onto the timeline and playing back smoothly in order to do some editing. The HDV files come from 3 different sources:

1. HDV 1080i50 m2t HDV files captured with HDV Split (these are edited portions of longer scene-split files):

7.0e - will open hundreds with 25fps "best/full" playback (with very slight momentary stall between events)
8.0a - will open hundreds but with only 23-24fps "best/full" playback (i.e. not smooth)
8.0b - will open hundreds with 25fps totally smooth "best/full" playback

2. HDV 1080i50 m2t HDV files from my archives smart-rendered in Vegas Pro 8.0b from files captured in HDVSplit:

7.0e - opens 70 to 130 files then crashes with an exception error. Best/full playback is 25fps (with slight momentary stall between events) for files with audio (but only 16-20fps for files without audio which are handled by mcplug.dll instead of Sony M2TS).
8.0a - opens 30 to 40 files then crashes with an exception error. Best/full playback is 25fps
8.0b - opens 30 to 40 files then crashes with an exception error. Best/full playback is 25fps

3. HDV 1080i50 m2t HDV files from my archives smart-rendered in Womble MPEG Video Wizard:

7.0e - opens about 70 files then crashes with an exception error. Best/full playback is only 11-16 fps and there are minor playback problems at the start and end of files such as a reversed frame at the beginning and a repeat frame at the end.
8.0a - opens about 30 files then crashes with an exception error. Best/full playback is 25fps
8.0b - does not open these files at all! It just crashes with an exception error

So there is no problem in 7.0e or 8.0b working with raw HDV captured using HDVSplit. The problems in all versions are all related to HDV files that were previously smart-rendered as m2t HDV files to my archives using either MPEG Video Wizard or Vegas Pro 8.0b. I've never done long projects in HDV before now so it hasn't been a problem until now.

I really don't know what to do now to get my work done. I was considering biting the bullet in terms of disk space and workload by encoding all my archives to Cineform but I just tried rendering a clip to the Cineform v3.3.0 codec that comes with the latest Neo Player and playback on the timeline is very slow and jerky. I don't know what would cause that but it doesn't encourage me to buy the full Neo. Maybe I've got something wrong with it. Has anyone got any thoughts on my problems?

One thing I have noticed with 8.0b is that it is the first version to correctly smart-render HDV. 8.0a was rendering the files one frame shorter than the edited file on the timeline. 8.0b gets it frame-perfect.

Comments

Laurence wrote on 1/22/2008, 7:48 AM
Have you installed the Neo Player? If so, you'll find two options under Start/All Programs/Cineform/Tools: "Desktop Playback - Fast" and "Desktop Playback - Quality". Selecting "Desktop Playback - Fast" basically causes playback from Windows to drop one field which improves playback and gets rid of interlacing problems all in one go. Do this and you'll be fine with Cineform.
Laurence wrote on 1/22/2008, 7:52 AM
If you do go the Cineform route, you don't need to rerender all your clips into Cineform. Just rerender the clips that are giving you problems. Also, you may already have Gearshift. If so you can automate the conversion without buying Neo HDV.
NickHope wrote on 1/22/2008, 7:57 AM
Thanks Laurence. I have installed the very new Neo Player released January 16. No matter whether it's set to "quality" or "fast" I only get 14-15fps playback at best (full) over DVI to my Dell 24" display. I need 25fps at best (full).

Also it's not a case of certain clips giving Vegas problems. It's the number of m2t clips that crashes it, but that number is nowhere near enough for my projects.
Laurence wrote on 1/22/2008, 8:55 AM
Have you tried 7d? You won't get the efficient HDV preview of later versions, but you may be able to avoid crashing.
Udi wrote on 1/22/2008, 1:29 PM
Did you call vegas support?

You can try and change the RAM Preview size.

Are all the crashes the same?, same module etc.?
I assume that the crash is not on the same file in each test.

Even smart rendering requires, in some cases, the re-rendering of the first and last GOP. I suspects that Womble have an error in the first or last GOP - that is why you get frame errors in 7e. 8a and 8b are handling it differently.

Udi
blink3times wrote on 1/22/2008, 2:27 PM
"Have you tried 7d? You won't get the efficient HDV preview of later versions, but you may be able to avoid crashing."

I quite agree. While 7d proves to be a little less efficient, it's by far the most stable for HD work.
JJKizak wrote on 1/22/2008, 2:51 PM
I am presently upgrading to Q6600 with a Gigabyte board and my guru buddy is flogging this thing out with Gigabyte techs. One thing to check on your board is in the bios power control section which has settings for 32 bit and 64 bit. Of course I am assuming you have these called HPET settings. Make sure you are in 64 bit. There are a whole bunch of issues that relate to my board so I won't get into that.
JJK
ushere wrote on 1/22/2008, 10:29 PM
xp / 32bit, e6600, 3gig ram, gigabyte GA-965P-DS3P, and never a problem editing straight m2t from camera, captured with sony. whatever problems i had, (and there weren't many) appeared to have been 80% operator error (ok, i didn't check ALL my settings), and 20% hardware issues (or rather, bad installations).

i'm not a heavy cc or effects user (mostly doco's), and 8bit is fine for me, so i can't answer for power-users, and my timeline is usually, m2t's from camera, graphics always converted to .png, and if i have to bring in anything else, such as .mov, etc, i generally render them out beforehand to m2t in vegas, ditto with audio to .wav.

the one main problem i had (and i haven't tested it yet on 8b) was ptt having rendered project with 'norecompress' on. simply wouldn't. however, if i rendered project with 'norecomp' turned OFF, then ptt worked perfectly....

leslie

NickHope wrote on 2/1/2008, 2:55 AM
I read over 4eyes postings on this thread again and now I am pretty sure that my crashes are caused by the .m2t files I have being in PS (program stream) format, not in TS (transport stream) format.

PS format .m2t files seem to be handled by the Main Concept mcplug.dll codec whereas TS format .m2t files are handled by the Sony m2tsplug.dll codec (right click on the file in Vegas Explorer and scroll down to properties to see which is used).

The reason many of my .m2t files are in PS format is that either I smart-rendered them from Vegas Pro 8 without audio (because the audio is useless to me), or I smart-rendered them with audio using Womble MPEG Video Wizard (which just creates PS format files).

So, how can I now smart-render (batch convert) 3500 x .m2t PS files into TS files so that Vegas will play nice with them? I don't fancy much doing them one by one!

I was hoping to do this with Gearshift using a custom template that is a staight copy of the HDV 1080-50i template but unfortunately Gearshift only sees custom templates in the AVI category. Perhaps the process is scriptable?
farss wrote on 2/1/2008, 5:06 AM
I'd have a look at the Veggie Toolkit from Peachrock.

You can get it to take every file in a folder, render / encode it using a Vegas template out to another folder.

Bob.
NickHope wrote on 2/1/2008, 6:26 AM
I was just doing exactly that Bob, but unfortunately it won't run. I dont think it likes x64. Have emailed Peach Rock and I'll try it on my 32-bit box now.
4eyes wrote on 2/1/2008, 3:30 PM
Nick,
I'm not an expert on mpeg2 editing (just been there), I think there is a very good chance your going to have to turn off the "No Recompress Long GOP Structure" under Vegas Preferences to get the streams back in the correct mpeg format. If Vegas sees a GOP that matches the video parameters that does activate the smart render (when video properties match the source file) then it may pass that GOP along to the new file, that GOP may be one it couldn't work with to edit, causing a problem.
Like one of the posts above, for writing back to tape, the mpeg2 file needs to be 100% compliant:
the one main problem i had (and i haven't tested it yet on 8b) was ptt having rendered project with 'norecompress' on. simply wouldn't. however, if i rendered project with 'norecomp' turned OFF, then ptt worked perfectly....I also tried various methods using files that were smartrendered and tried to PTT for a full hour, to see how Vegas was smart rendering compared to other programs, if the file(s) were ever previously edited the stream to the tape would eventually have pauses somewheres in the process.
I haven't had any problems yet with any high def videos as files burnt to hd-dvd, blu-ray bdmv, or avchd disks where Vegas smart render was on.

Vegas with the latest updates does seem to do a nice job of smart encoding.
NickHope wrote on 2/1/2008, 9:26 PM
The print-to-tape thing is something I haven't had to fight with yet and hopefully never will. I'll stick with other delivery methods.

So, if I started today, then this production workflow works perfectly: Capture with HDVSplit > ripple edit on 8.0b timeline > smart-render 1080-50i to HDD (with audio thereby creating transport stream files) > drop back in 8.0b timeline > edit finished production. This keeps everything with the Sony codec and away from that Main Concept codec, and as far as I can tell, all is frame-perfect, rock solid and responsive.

But Vegas Pro 8.0b just cannot work with my archive of 3500 program stream .m2t files (that were smart-rendered in Womble MPEG Video Wizard) so I've been trying to convert them into transport stream .m2t files by smart-rendering them again from Vegas using the 1080-50i template. It works up to a point but if I smart-render them from Vegas 8.0a I get repeat or even sometimes reverse frames at the ends of many files and many files are one frame shorter than the original. And if I smart-render them from Vegas 8.0b I get a black frame near or at the end of the file.

If i can't think of another way I think I'm going to have to smart-render them in 8.0a and remember to trim the ends by a couple of frames in any finished projects I use them in.

Anyway I got my Veggie Toolkit working. Randall kindly sent me a beta version of 3.0c that fixed the 64-bit issue. But it is hanging/crashing after rendering a few of my files, so I'm looking into that.
4eyes wrote on 2/2/2008, 6:27 AM
As a test with Smart-Render off maybe take the worst Program Stream file, load it into Vegas & export in back out to 1080i-50 (HDV). Hopefully Vegas can still read the stream correctly. This will of course re-encode the file but at least you will know it they need to go through a complete re-encode or not.

Then try this in Vegas with smart-render on.

I pointed out the PTT only because that mode requires the files to be 100% compliant.
I do write back to tapes to archive projects, yes there is some loss if I edited in the mpeg2 format, but with my videos there isn't much loss at all. The videos are very acceptable, Vegas does a nice job of re-sampling which you can also increase higher than the default setting.

Full re-encodes do take time, not as much time working with corrupted mpeg's & projects.
I've never fixed corrupted mpeg2 videos without having to rebuild the complete streams again.

It's the nature of the beast being a compressed format, avc/h264 will be worse. Very few of the avc/h264 videos I've smart-rendered work correctly (other program), maybe 2 out of 20.
farss wrote on 2/2/2008, 6:36 AM
Seems to me that with this Smart Render feature an I frame indicator would be mighty handy. For what Nicks been doing making the trim cut on the start of a GOP would endure no problems down the track.
DVDA has had the feature for some time for placing chapter markers so it's obviously doable.

Bob.
4eyes wrote on 2/2/2008, 8:51 AM
Seems to me that with this Smart Render feature an I frame indicator would be mighty handy.Yes, being able to cut exactly on I-Frames and see them would be a big help.
I've had some corrupted m2t files. Vegas will play them but when it would get to the bad GOP or frames would then stall or generate an error.
To get around this I'd have 2 options, render to a new m2t file with smart off and re-encode the whole stream or perform a split right before the bad section and remove the bad section.
NickHope wrote on 2/4/2008, 7:10 AM
Right, well, I've finally got my problems sorted out now.

On my x64 machine which has Vegas Pro 8.0b I installed Veggie Tookit 3.0c beta version (needed to run on x64) and used its MultiRender feature to smart-render again all of the 1500 clips that I had recently rendered without audio and rendered them to the default HDV 1080-50i template with audio. I still had the original footage and projects but there was a lot of renaming and overwriting to do manually!

Then on my old XP 32-bit machine I installed Vegas Pro 8.0a and Veggie Toolkit and smart-rendered my 2000 older clips that had been smart-rendered as program stream files in Womble MPEG Video Wizard. Vegas Pro 8.0b is really unstable with them. It displays lots of black frames and crashes if I do a timeline zoom or if I load more than about 30 on the timeline. 8.0a does much better, although there are sometimes a repeat or even reversed frame at the ends of the files.

So now the whole lot is in TRANSPORT STREAM format and Vegas Pro 8.0b appears to be working well with them. Nice and responsive and I can load hundreds of clips on my timeline and get 25fps on my 24" Dell at Best (Full). So I'm a happy camper.

Huge kudos to Veggie Toolkit MultiRender for getting me through this. It churned through hundreds of folders containing 3500 HDV files in 24 hours and automatically put the new versions right in those folders alongside the older version. It did stall a few times but that was probably down to Vegas as opposed to the extension.

I should also mention Directory Opus, which has been absolutely invaluable for finding, moving, copying, bulk-renaming, sychronising and just about everything except making the tea during this rebuilding process.

One final thing... I did render (not smart) a couple of files with TMPEGenc Xpress 4.0 using the HDV template and the result was indistinguishable from the original. So it's a nice tool to have as well.

Moral of the story... smart render HDV in Vegas Pro 8.0b and keep the default HDV template and include the audio even if you don't need it.