MPEG-2 Main Concept multigeneration re-rendering test

Chienworks wrote on 12/27/2004, 9:11 PM
I figured it was about time to do a re-rendering quality loss test for MPEG-2. The topic of using MPEG-2 as source files for editing keeps popping up more and more often lately. I started with a basic uncompressed 4:4:4 image, added a bit of animation to it, and rendered a 1 second MPEG-2 file. This file was then used as the source for a second rendering, and so forth, through 99 generations.

I chose the standard DVD-Architect NTSC video stream template since this topic often comes up with respect to reusing video from DVDs. I left all settings for the Main Concept encoder set at the default with only two exceptions: field order was set to progressive and two-pass encoding was enabled. The list of settings is:
- Video type: DVD
- Frame rate: 29.970
- Width: 720
- Height: 480
- Aspect ratio: 4:3 Display
- I-frames: 15
- B-frames: 2
- Profile: Main Profile
- Level: Main Level
- Field order: Progressive only
- Video quality: 31 (High)
- Insert sequence header before every GOP: Yes
- Variable bit rate
- Two-pass: Yes
- Maximum (bps): 8,000,000
- Average (bps): 6,000,000
- Minimum (bps): 192,000

This should be pretty representative of most good quality MPEG-2 material.

The results are posted in http://www.vegasusers.com/testbench/files/generations/ and all begin with "mc-mpeg2-". I've included the actual .mpg files as well as .png stills of frame 15 of each clip. The stills may not be representative since the quality can vary greatly from frame to frame. They do show a steady progression of loss of quality though.

The file "mc-mpeg2-dvd-00-99.mpg" contains all the other .mpg files in one video clip. Please note that this is a re-rendering of the re-renders, so each clip is actually one generation higher than the number displayed. Clip "0" in this file is 4:4:4 uncompressed rendered to MPEG-2, so it is actually first generation. Clip "99" is 100th generation. This only applies to this one file. All the individual .mpg files are the generation specified by the file name.

I was impressed to see how well Main Concept's encoder holds up. Even the 9th generation apeared quite usable. Noticeable artifacts appeared at around generation 15. While i still don't condone using MPEG files as editing sources, it looks like they will survive a few generations unscathed.

Comments

NickHope wrote on 12/27/2004, 9:42 PM
Good test Chienworks, thanks!

Now all we need is the comparitive tests in Procoder, CCE, WMV, DivX etc etc.. ;-)
Spot|DSE wrote on 12/28/2004, 1:03 AM
B_JM did several tests with these, perhaps he'd be kind enough to repost the links.
Great information, Kelly, but while it holds up in an eye view, it's not so impressive on a scope. All the more reason that the format is terrible for editing, aside from all it's other warts. I know that wasn't your point, but I'm afraid some folks will think that it's OK to edit MPEG streams and claim they're held together well. And of course it goes without saying that MPEG is really nasty on the CPU.
Thanks for taking out the time to do this!
farss wrote on 12/28/2004, 2:13 AM
SPOT's right. Even static bars don't fare too well down one generation when you look at them on the scopes. Also I think testing any form of temporal compression is a fairly difficult task.
I suspect (and maybe someone with a lot more knowledge can correct this assumption) that you would see a kind of 'event horizon' coming into play. If the encoder was able to fully encode a scene with no residual (i.e. it didn;t have to throw anything away) then the generational loss maybe as low as nothing.
However if the scene contains more motion than the bitrate will permit to be encoded then the generational loss could be huge.
Let me try to put that another way. The results of the encoder not being able to encode all the data are themselves adding to the scene complexity which the next generation of encoding will make a further mess of.
I'd suspect that wavelet compression such as used by Cineform performs much better, the degredation is more harmonious, the loss in one generation doesn't make the scene any harder to encode for the next generation.
Hope this makes some kind of sense to someone.
Bob.
Chienworks wrote on 12/28/2004, 3:43 AM
Bob, yes, i agree wholeheartedly. You can see the loss buildup in the chocolate chips rather quickly and it snowballs from there. Also, the blue background behind the portrait broke up quickly and the artifacts rapidly induced more artifacts in sort of a "ping-pong" effect until it became a motion background rather than a static image. These effects were definately measurable in the 1st and 2nd generations, but not naked-eye visible until later. I'm sure everyone will have their own threshhold of tolerance. Those who's tolerance is rather low and expectations are high know not to use MPEG as a source anyway. Those who think a DVD handycam or MicroMV are the bees knees probably aren't concerned with professional quality results.
farss wrote on 12/28/2004, 4:17 AM
I don't think this means that mpeg-2 is flawed as a capture medium, even XDCAMs IMX is mpeg-2. What it does show is that you certainly want to get your footage into a lossless or less lossy format asap.
Bob.
Sullivan wrote on 12/28/2004, 7:13 AM
So this leads me to a different question.

What is the right format to use for "bounce mixing" of the video? If I have a Vegas timeline and I want to include it in another project, I have to render it out to a single file and then place that track in the new timeline. I knew better than to do this with MPEG and I've been using .AVI files (Windows Codec), thinking this was basically an uncompressed format. But I still think I can see a subtle bit of generation loss - not mosquito noise, but more like added grain.

Is AVI compressed somehow? What's the best fomat for an intermediate file like this?
riredale wrote on 12/28/2004, 7:28 AM
If you get the chance to do a search, this board will prove to be a rich trove of information about this topic.

Basically, DV avi is compressed roughly 5:1, which is very mild by some standards. BY FAR the best codec for encoding DV avi is the one in Vegas, which can survive many dozens of round-trips before showing any artifacts.

NTSC DV is 4:1:1 in the colorspace, which is a substantial hit, but it does remarkably well for things like greenscreen chromakey.
rs170a wrote on 12/28/2004, 7:41 AM
What is the right format to use for "bounce mixing" of the video

If you don't want to go uncompressed, try using the Sony YUV codec (which is 4:2:2) instead of the standard DV-AVI (which is 4:1:1). I'm not even close to my Vegas computer but, as I recall, It's in the Custom > Video options.

Mike
rmack350 wrote on 12/28/2004, 9:47 AM
The Microsoft DV codec is far worse than the Vegas DV codec
All DV codecs will crush text and graphic elements to smithereens so if you are going to move edited footage to a new project you'd be better served to render it to one DV-AVI file without the text and graphics. Move the text and graphics separately.

Rob Mack

BillyBoy wrote on 12/28/2004, 12:26 PM
Interesting, but so academic. Anybody that used even 3rd generation let along 20 or 99 raise your hand. For sure, The Vegas DV codec is pretty darn good. <wink>
Sullivan wrote on 12/28/2004, 2:09 PM
I've never used 3rd gen yet, but I can see it happening. I frequently need to nest the timeline 1 level (i.e. use a second generation file).