New theory about HDV problems and tape capture.

Laurence wrote on 4/4/2008, 8:36 AM
This is just a theory at this point, but how does this sound to you all?:

I am suspicious that most of the problems that people are having with Vegas and HDV footage are a result of formatting errors that were introduced when the footage was captured from tape.

It could be that some PCs are underpowered. It could be that certain virus checkers or other programs are interfering with the capture.

These are the problems that I suspect might be related to mpeg formatting errors accumulated during capture:

1/ crashing on certain m2t clips in Vegas versions 7e through 8b when the audio waveform is being drawn.
2/ crashes when Vegas tries to render some hdv clips.
3/ two black frame problem.

If these indeed are errors that are introduced during capture, it might explain why some people have never even seen these problems and some people are plagued by them constantly.

It might also explain why people who have problems with HDV in Vegas are just as frustrated when they try to move to Premier Pro instead. Because it is the capture not the editing program that is at fault.

What do you guys think?

Comments

Spot|DSE wrote on 4/4/2008, 8:45 AM
It could be that certain virus checkers or other programs are interfering with the capture.

Why on earth would you run an antivirus or anything similar while capturing? That's an almost assured issue on any computer.
Laurence wrote on 4/4/2008, 9:30 AM
Because you only have one computer that you use for everything, from web browsing to video editing to doing your taxes, and you are scared to death of losing everything to a virus infection!

I always test everything for myself. When I started using AVG I did a bunch of checks to see if it affected my video preview performance or rendering time. It didn't so I started leaving it on. Same thing more recently with Avast.

When you do a capture, and the capture utility goes through the whole thing and tells you at the end that there were no dropped frames, you start to assume that the capture went well and you move on to the next challenge.

If things go wrong at the next stage, you look for some problem at that stage. It is only natural.

What I'm saying is that I am now suspicious that that earlier capture didn't go as well as the "captured with zero dropped frames" lead me to believe. I now think that there might have been errors introduced in capture that are leading to the crashes and black frames that many of us have experienced.

Having said that, I will now go back and examine my background processes more carefully. I not only have the anti-virus running, but I have Skype and some adware blocking going on as well. None of this seems to affect what I do negatively in any other way.

Maybe the solution might be as simple as adding a fresh login with no background processes that I could use when I capture video. It would be just like logging in as another user except that I would be logging in as "video capture guy" and have my preferences in that mode to be very clean.
rmack350 wrote on 4/4/2008, 9:36 AM
If it's an issue at ingest and related to having to do it in real time, then maybe you'd see the problem go away if you were using a direct to disk recorder, or solid state media like with the XDcam EX. I'm not making an assumption about what people do see, just a guess at something to look for.

The black frame issue is as old as the hills and I thought it went all the way back to DV. I've often wondered if this was an issue with Vegas's caching of frames combined with some sort of small hardware timing problem or other flaw that only some people have. This could explain why only some people experience it and why it never seems to get fixed. Perhaps SCS has never really been able to duplicate it or pin down a cause.

I've not had any problem ingesting DV with a virus checker running (AVG). I don't use HDV so I can't really speak to that but the data rate is comparable. The only thing with long GOP is that I'd expect a data error to have a much bigger effect than it does with DV.

Rob Mack
Bill Ravens wrote on 4/4/2008, 9:38 AM
Laurence,

I suffer, chronically, from black frames when I import m2t files from my JVC GY-HD110. The interesting thing I want to note is that I never use tape, so, I capture directly from the camera to a Firestore FS4HD or nNovia QCDeck. m2t's from both types of hard drives fail with black frames. I submitted a ticket to both Sony Vegas and nNovia tech support. nNovia told me the problem was on Sony's end and Sony told me the problem was on nNovia's end. As near as I can figure out, there's a problem with the "End of File" marker that Vegas reads from the clip. The embedded timecode has a huge discontinuity right where the black frames start. Curiously, Cineform's Neo always reads the same identical m2t's without a hiccup...ever! Since neither manufacturer wants to own up to the real problem, we, as users, are basically told to bugger off.

This is NOT an anti-virus problem. Not unless Sony is really a virus, in sheeps clothing.
rmack350 wrote on 4/4/2008, 9:43 AM
With my very first 1394 hardware, I'd see very visible errors in DV without getting dropped frames. I'd say my first ten 1394 cards and drives were all crap. Luckily I was getting stuff from places with good return policies.

Just to say that you can have errors without dropping frames.

Rob Mack
Laurence wrote on 4/4/2008, 9:46 AM
So maybe the two black frames problem isn't related to this.

Bill, since you don't use tape, I'll ask you if you ever get m2t clips that crash Vegas?
Bill Ravens wrote on 4/4/2008, 11:02 AM
nope...

but, since Vegas chokes on the m2t's, I, mostly, transcode to CFHD first. rarely import m2t into vegas, anymore.
Laurence wrote on 4/4/2008, 11:25 AM
I sometimes use AVCHD footage from my CX7 in Vegas projects. On the CX7, the AVCHD footage is on the card with the .mts extension, but if I make an AVCHD disc using the VRD-MC5, the AVCHD clips on the disc that you burn will have the .m2ts extension. These AVCHD .m2ts clips work fine in Vegas except that they preview at a low frame rate.

I get around this by using Gearshift proxies.

I have never been able to get HDLink to convert AVCHD.
John_Cline wrote on 4/4/2008, 12:36 PM
I have had very good luck using a freeware program called "MPEG2REPAIR" to fix problematic .M2T files. It virtually eliminates issues with files that have a tendency to cause Vegas to "hang up" when importing. This includes issues 1,2 & 3 mentioned by Laurence above. MPEG2REPAIR also generates a text file with information about problems it has encountered and fixed in the files. This report text file may provide some clues about why some files work and some don't.

http://www.videohelp.com/tools/MPEG2Repair

VideoReDo also has a transport stream "fix" utility that works well.

John
LReavis wrote on 4/4/2008, 1:04 PM
I always capture from my pair of HC-1s to .M2t with HDVsplit and NEVER have had any problem with the files unless I try to render those files later to .M2t - then I often get black frames (but I never got a faulty file that made crashes).

Nor have I ever seen any dropped frames from the HDVsplit captures - not even once in dozens of captures even with antivirus running. My only problem is the black frames that are seen in my renders to .M2t.

For that reason, I always render to an .AVI format, often using the Cineform intermediate option. Never once a crash or black frames, nor any other type of problem with those rendered .AVI files.

I'm using a Q6600 @ 3.05 gHz, with 4GB RAM on an Asus P5B MB.
riredale wrote on 4/4/2008, 2:26 PM
I edit HDV exclusively on my homebuilt PC, an utterly conventional AMD-based machine in pretty much all respects. It is used for every other task I can think of besides just video editing and is constantly loaded down with maybe 50 processes running in the background, including ZoneAlarm, AVGfree, Copernic, Thunderbird, LogMeIn, and a host of others. In the past year I've had exactly ONE black-frame pair issue, and the way I eventually was able to make it go away was to go back to tape and re-capture that particular clip. In both cases I used HDVSplit v.77.

Something very odd is happening here. From what I can tell (please correct me if I am wrong):

(1) Black frames can occur either via tape or tapeless;

(2) Black frames never occur with Cineform;

(3) Black frames never occur with DV material;

(4) Some people never see black frames. Ever.

(5) Some people get them frequently;

(6) So far, there is no specific indicator (motherboard, capture card, camera, operating system, antivirus) that points to the black frame phenomenon.


Personally, I suspect it's a combination of settings within Vegas, but if so, then the only way to test this theory is to have some sufferers reset their Vegas program to defaults and try to get the frames to appear then.

I think it would also be very helpful if one of the Vegas wizards could write a trivial little utility that could spider through captured m2t clips to verify that everything is within spec--no odd metadata, no odd audio/video sync issues, no errant IBP frames, and so forth. When I capture a clip and it breaks, then re-capture under identical circumstances and it works fine, I'm curious about just how those two clips differ, and how that possible difference could be confusing Vegas.
Nx wrote on 4/11/2008, 9:51 AM
I work with tapes and mt2 files in HD, togerther with jpg stills of various sizes, on my home pc. I get numerous black frames and sound dropouts on specific sections of a project, even in parts where only stills are used (project is 45 minutes, 1440x1080 HDV).

If I limit the size of the project to, let's say, 2 minutes, the black frames are gone. It seems to be related to hardware while rendering in my case (not shown in preview).

Nx
NickHope wrote on 4/11/2008, 1:33 PM
I think a lot of the trouble is hardware related. The other day I had identical m2t HDV projects and identical media on my desktop and laptop. The laptop was fine but the desktop had crashes, red preview screens etc. and could only be fixed by cutting the project in half.
Laurence wrote on 4/11/2008, 8:01 PM
Since I started turning off all my background processes during HDV capture, I haven't had any problems. Only four tapes so far, but hey, I never got that far before! ;-)

I just did a couple of tapeless interviews today with a $150 32GB Transcend compact flash in my Z7. Not one error in the whole session.
blink3times wrote on 4/11/2008, 9:08 PM
In all honesty Laurence, I am not having any major problems with 8b and m2t files. I still get the odd crash, but it's not NEAR as bad as from 7e to 8a.

8a was a mess and Sony asked me to send a clip into them. I did of course and when 8b came out, my crashing slipped down to a minimum. (I was having major crashing problems with both HDVsplit and Vegas capture involving scene splits.... vegs would hit a scene that it didn't like and die which was happening probably about every4 out of 10 scenes)

My theory which I believe I have told you already is the addition of avchd editing somehow screwed up vegas's ability to see straight so to speak. That was a pretty major change between 7d and 7e and I believe that problem was forwarded onto 8

Black frames are still quite the issue though, and capturing in Vegas with certain cams doesn't work too well (the HC3 for one... but not my HV20.... figure that one out?!).... my scene splits are a constant 3 frames late (which is why I use HDVsplit for the most part).
NickHope wrote on 4/11/2008, 10:32 PM
The one ongoing issue I am having with a Z1 > HDVSplit > 8.0b workflow is 2 repeat "frozen" frames at the start of every clip. I still need to run the TrimCapturedClips script after I put clips on the timeline.

Recently I tried shooting again with Quick Rec on, on my Z1. Many of the captured clips were a right old mess, with lost packets reported in HDVSplit, audio streams longer than the video stream etc.. But 8.0b took them without complaining and I could easily ripple-edit the errors out.

I must admit in the past I have done a lot of capturing with AVG and a few other things running in the background, and I don't [b]think[b] it caused me trouble. But these days I do tend to turn all that off for a capturing session.
blink3times wrote on 4/15/2008, 5:19 AM
Sorry Laurence.... I lied.... I'm have an AWFUL time with this latest M2T project. CRASH, CRASH... nothing but crash

I don't get it. My last M2T project went just fine... I had 2 crashes and that was it. With this one I can't even render out a file bigger than 800Kb without it crashing. It's so bad that I'm now having to put 7d and 8b beside eachother so that I can manually transfer everything over to 7d.

I've tried everything else but nothing works. It seems to want to crash at the splits and/or transitions.
jabloomf1230 wrote on 4/15/2008, 10:06 AM
This is not in response to the generic issues that have been raised, but rather to the issue of black frames & HDVSplit. I have never had this problem using Cineform Neo's HDLink (either to Cineform avi or to Elecard m2t). I have never had this problem using the built-in HD capture routine that is part of Vegas Pro 8b. I have only seen this problem with HDVSplit and then, the problem is not consistent.

Now, that in itself, is not enough evidence to find Vegas innocent of all charges, since previous versions seemed immune to these issues. But keep in mind a couple things about HDVSplit. It's a wonderful piece of freeware, but it hasn't been updated in more than a year. People on various message boards have asked the author for the source code, with the intent to keep the software current, but to my knowledge, this has not happened. My point is that if you use freeware and particularly, freeware that has not been kept current, you should be willing to accept any unexplained results that you encounter. My guess is that HDVSplit does something in encoding the m2t file that is giving the MainConcept codec a headache from within Vegas.

One other thing that worries me about HDVSplit is that it wants you to install ffdshow, so that the MPEG-1 audio can be properly encoded. Putting another freeware codec into the stew also adds to the uncertainty, when bugs/crashes appear.
bigrock wrote on 4/15/2008, 10:57 AM
This Mpeg2repair tool says there is errors in every captured file I've got whether from Vegas 7 or 8 yet I have has none of the reported probelms occur so I question it's usefullness.


BigRockies.com Your Home in the Rockies!
John_Cline wrote on 4/15/2008, 1:03 PM
Mpeg2repair will always report an error in the last frame of the video clip. If it reports other errors, then there is indeed something wrong.