My MPEG 2 Encoder test: TMPGEnc, Vegas, MC, CCE Basic

prairiedogpics wrote on 4/4/2004, 5:49 PM
I've been reading a lot lately about using standalone encoders in tandem with editing with Vegas 4.0. So I decided to do my own tests this weekend.

Here were the contenders:
Vegas 4.0 (latest build)
Mainconcept 1.4.2 standalone encoder (demo)
CinemaCraft Encoder Basic (demo)
TMPGEnc (MPEG2 decoder with 30 day trial)

I used footage from my 5 year old Sony TRV45 Hi-8 camcorder. The footage was digitized using my Canopus ADVC 100 unit. (I'm a hobbyist. I have plans to buy the Canon Optura Xi in a month or so, and then I'll pop for a an HDV camcorder in 5 years or so.)

I put together a 2 1/2 minute clip in Vegas with footage of my son on a beach and indoors. Mind you, my old camcorder doesn't have manual white balance. I rendered the timeline to an uncompressed .AVI file.
Then I rendered that in each of the 4 apps, choosing VBR when possible (but not changing the bit rate presets at all). In TMPGEnc, I added the default noise reduction setting as well (seeing as how most of what I've read about that encoder leans toward recommending it). I also had to multiplex the render from TMPGEnc with an .ac3 file to get an mpg file. (All audio was rendered to .ac3 for all 4 situations.)

I then created one DVD in DVDA using the 4 mpeg files so I could quickly jump back and forth to view the footage.

My eyes conclude the following in order of visual quality:
1. TMPGEnc (worst, although I concede I know nothing about tweaking its settings). Serious blocking/pixelization visible.
2. Vegas 4 (licensed MC encoder). Compared to the others, footage is contrasty and colors are overly saturated (especially bleeding reds). some noticeable blocking.
3. CCE Basic. Excellent colors and contrast, minimal blocking, but more that MC 1.4.2, especially where beach grass is waving in front of my son's face in the footage. BIG Downside to this over MC 1.4.2: you can't vary the bit rate. Just straight render to MPEG2 (unless the purchased CCE Basic is different from the the demo). This is unfortunate if you're trying to cram 2.5 hours of footage on a DVD.
4. MC 1.4.2. Excellent results at default settings (with VBR set). The beach grass shows virtually no pixelization/blocking. Huge number of tweaks available. Nice interface.

We'll see what Vegas 5 has to offer; I don't need board control, so depending on what version of MC encoder they include, I may just (yikes!) skip the upgrade and pop for the standalone encoder.

If anybody has an FTP site, I'll upload the VIDEO_TS folder from work tomorrow so you can take a look yourself.

Dan

Comments

starixiom wrote on 4/4/2004, 6:09 PM
You could also try the Procoder demo encoder from canopus if your interested. I think it has a master quality setting.
prairiedogpics wrote on 4/4/2004, 7:14 PM
I left Procoder out of the mix because the pro version is $499 (out of the question for my budget) and Procoder Express ($58) does not have a demo version.
CCE Basic is $58 and MC 1.4.2 is $149. I can handle either one those. I didn't want to bother with something I couldn't afford.

Dan
4thorder wrote on 4/4/2004, 8:07 PM
Why would the stand alone MC be better than the one inside VV?
prairiedogpics wrote on 4/4/2004, 8:15 PM
The one inside Vegas is an older version than the newest MC 1.4.2. As I understand it, the one inside Vegas is frozen, if you will, at the version # that was current at the time when MC licensed it's encoder to Sonic Foundry/Sony for the Vegas 4.0 release.
johnmeyer wrote on 4/4/2004, 9:14 PM
The only surprise in your results was the "blockiness" in TMPGEnc, especially since you had some noise filtering.

Which TMPGEnc template did you load? I would recommend just using the standard DVD NSTC template.

Also, you said: "but not changing the bit rate presets at all". Does this mean that you took whatever bitrate happened to be the default? If so, your test is not valid. Each encoder should be set to the same average bitrate (for VBR) or contant bitrate (for CBR). If they are set to different bitrates, your comparisons are not valid.
JJKizak wrote on 4/5/2004, 5:38 AM
The only thing I have noticed for sure is that after rendering in Vegas to DVD settings that the color saturation is down just a hair. So I make sure that if I have flat looking stuff to "juice " it up a tad. But not too much.

JJK
prairiedogpics wrote on 4/5/2004, 6:02 AM
johnmeyer,

thanks for your suggestions. As you can tell, I am by no means a video expert...
As far as my settings in TMPGEnc: I did choose the DVD NTSC template, along with Automatic VBR (CQ_VBR), which I think tries to optimize the bitrate given the size of the file to be rendered and the available disc (DVD-R) space. For a 2.5 minute clip, I think that average bitrate was fairly high. I also added noise reduction with its corresponding default settings.

You can't alter the bit rate in CCE Basic, but I assumed (maybe wrongly) that its default settings would be a VBR. (Thus my decision for choosing VBR in the others.)

I used the default settings for the DVD NTSC in Vegas and MC Standalone (choosing "Best" inside Vegas), as well as VBR, and, to my recollection, the avg. bit rate in both Vegas and MC 1.4.2 was 6,000,000.

I agree my tests were not scientific. If anyone knows whether CCE Basic's bit rate is CBR or VBR, and what the value is, I will re-render everything.
ScottW wrote on 4/5/2004, 6:17 AM
CCE Basic's default settings are for CBR at 6Mb/s - you can alter this by double clicking on the file that you want to encode and a screen will come up that allows you to change things. If you use 2 pass VBR, you'll get an elementary stream, which you can't use directly in DVDA - you can remux this into a program/system stream using the MPEG tools in TMPGEnc (click on the multiplex tab and just don't specify an audio file).
prairiedogpics wrote on 4/5/2004, 6:48 AM
ScottW,

Thanks for the tip about CCE Basic. I feel kinda silly not discovering that. I will re-render all files tonight at CBR of 6 Mb/s tonight.

Dan
johnmeyer wrote on 4/5/2004, 10:02 AM
The key thing is to make the actual average rate -- whether CBR or VBR -- the same for each encode. If you are using 6,000, then use that for each encode.
AudioIvan wrote on 4/5/2004, 10:33 AM
For bff interlaced video (DV) Canopus ProCoder Express & ProCoder will give you best results.M.C 1.4.2 yes, it's good and far better than the one included in Vegas but not as good as CCE & ProCoder.Also every good encoder will give you reasonable quality @ avg 6,000.
Try min~1000, avg~2500, max~4000 & you'll see the difference in the quality between the encoders.
For all my DV Video I use Canopus ProCoder Express, I LOVE IT.
Also do a bit of a reading in the CCE manual.

AudioIvan
prairiedogpics wrote on 4/5/2004, 11:02 AM
My objective in doing this testing was to see if there was noticeable improvement in quality between the current version of the Vegas encoder and standalone encoders. I have already answered that question for myself.

The answer is an unequivocal and resounding YES. The newer standalone encoders make quite a difference (the quality is noticably BETTER). It will be interesting to see what's incorporated into Vegas 5.

I'm just a little astonished that for $60 or so (or $150 for MC), you can add so much more quality to your DVDs. It's not that much more effort to render to an .AVI and run it through one of these encoders.

AudioIvan,
Do you know where to find a demo version of Procoder Express?
AudioIvan wrote on 4/5/2004, 12:26 PM
No DEMO, you'll have to buy it.The best price would be CCE Basic, I have that one too, but somehow like I said I'm in love with Canopus Procoder Express.
Between M.C 1.4.2 & CCE Basic I'd go for the second (if you want quality),for ease of use M.C 1.4.2.

AudioIvan
farss wrote on 4/5/2004, 2:46 PM
The other BIG issue is how did you view the results?
Burning a DVD and popping that into a player isn't a valid test. In many cases higher quality encodes produce far worse looking results in many players. I'd suggest taking the encoded mpeg file back into Vegas and look at it frame by frame.
At higher bit rates many of the decoders in DVD players cannot keep up and/or the TVs cannot cope with high res video. I've done a few tests with footage encoded on a very high end hardware encoder. On many DVD players it looks horrid, I mean really horrible. On a pro DVD player it looks stunning. Footage originated in HD, very high chroma frequency stuff with lots of motion. Its all there in the mpeg stream and its all within spec, just that most decoders cannot handle it even though they should.
johnmeyer wrote on 4/5/2004, 3:49 PM
On many DVD players it looks horrid, I mean really horrible

I am confused. If it looks horrible on normal DVD players, then doesn't that mean that you wouldn't want to encode this way? Also, it doesn't seem logical that higher bitrate encoding would look worse. Even allowing for the occassional "counter-intuitive" insight, this one just doesn't seem right to me.
john-beale wrote on 4/5/2004, 5:26 PM
For what it's worth, I just compared TMPGEnc Plus, CCE Basic and MC 1.4.2 all using 2-pass VBR at 4.8 Mbps and 4.5 Mbps (the client requires me to put 2 hours on a DVD, even if I'd rather not!). Subject matter was ballroom dancing with lots of motion, sourced from VX2k and DVX100 (3CCD MiniDV cameras). I used the same pre-filter with all three (an Avisynth script for temporal & spatial filtering).

Without filtering, this bitrate was unpleasant with any encoder. With filtering, they were all watchable but MC 1.4.2 gave me the cleanest image AND the most visible detail as well, which was interesting. It's really worthwhile to do your own tests because with different source material, bitrates, displays, and observers, the outcome might be different.

By the way- the clients are paying for my DVDs to watch on their TVs via standalone DVD player, so that seems the best way to judge quality. So far only a minority of them play back via DVD-ROM on their laptops, and most laptops I've seen have really awful video quality (they are optimized for text display- which is a very different problem).

Interesting note: my Panasonic CT-27SF36 27" TV does temporal filtering (!) and the filter cannot be turned off. It's great for VHS viewing, but deceptive for evaluating DVDs (I use a production monitor for that).
http://www.bealecorner.com/trv900/tv-image/index.html
MikeCrash wrote on 4/6/2004, 1:54 AM
The best encoder test is on this page:
http://www.tvfreak.cz/index.php?sekce=video&clanek=testmpeg2

it is in czech, but the graphs tell everything.... not the unserious rumours, but measured results
prairiedogpics wrote on 4/6/2004, 5:44 AM
In view of the suggestions above, II rendered everything again last with the following settings:
TMPGEnc: 2-pass VBR, 6000 avg. bit rate
Vegas 4.0: VBR, 6000 avg bit rate (doesn't allow 2-pass)
CCE Basic: 2-pass VBR, 6000 avg. bit rate
MC 1.4.2: 2-pass VBR, 6000 avg. bit rate

With this new round, I have to give the victory to CCE_Basic. It looks really nice. With MC 1.4.2, there is noticeable blocking in the solid green color of my son's shirt, but virtually none with CCE-Basic. I have to really be picky to see the difference between the source footage and CCE Basic's render. I compared all the renders on my set top DVD player and my Panasonic TV ("calibrated" with a Video Essentials DVD), which is the intended destination for me.

I just wish Procoder Express had a demo. (Maybe with NAB?)

Is it possible to frameserve CCE Basic in Vegas using Satish's plugin?

Dan

JJKizak wrote on 4/6/2004, 6:06 AM
I have looked at the Czeck site and concluded that I learned two
more words---forward and backward.

J. J. Kizak (former Czeck by many generations)
Hammer wrote on 4/6/2004, 1:18 PM
Did you by any chance record the time for each? I know for a of folks may not worry about render times, but I think that's an important metric to include. Also, kinda curious about the diffbetween 2 pass and 1 pass VBR. Does the difference in quality make up for the added time.
farss wrote on 4/6/2004, 3:55 PM
John,
it's quite logical unfortunately and this doesn't just apply to mpeg-2 encoders. Just limiting the issue to them though there's two parts to the process, encoding and decoding. Certainly the encoding is always the most difficult part but from what the encoder has produced something has to decode it and I keep hammering these points beacuase the decoding process doesn't get anywhere near enough attention.
You could add to that how the signal is fed from the DVD player to the TV / monitor, changing that from composite to component will also make a big difference. But that difference may be small or it may be very dramatic depending upon the content of the video.
How is that relevant to encoder tests?
If the encoder filters out HF chroma components then the results will possibly look better on poor quality player setups yet it'll be the other way around on pro gear.
As to the higher bit rates, well some decoders just cannot keep up and when that happens, yuck.. Some it seems even peg out on the audio side if the audio stream bit rate comes close to max spec.
There's a whole raft of other things that DVD players are meant to do and some don't or they do the wrong thing. Just a brief browse through vcdhelp.com will show that.
Of course if it looks bad on Joe Averages player then I doubt you'd want to encode that way but this issue is going to become more significant. The public are buying more high end gear these days, they want it to look as good as it can for them but that may mean content that looks rather bad on low end gear. This isn't an issue limited to DVDs though.
john-beale wrote on 4/6/2004, 5:37 PM
The Czech page is very interesting; there is a lot of data there. Maybe enough to support almost any subjective opinion :-). It looks like ProCoder comes off as the best in most categories (I wish it wasn't $500, otherwise I would get it!).

I did notice that in the "MAX VBR difference 4" at the bitrate closest to what I tested (4000 kbps), the MainConcept encoder had the best result.
<http://www.tvfreak.cz/index.php?sekce=video&clanek=testmpeg2&strana=21>

The things I notice most are blocky artifacts, which are probably regions of the "maximum" difference from the original video, so this may be what I consider subjectively the key criteria. Even if a large area of video is subtly different from the original, it will affect the average computed difference a lot but maybe not so much the visual perceived difference. It is the visual perceived difference (especially the biggest spatial and motion artifacts) which is most important to me, and I think that correlates most closely with "maximum error", that is if it correlates with anything. My $0.02