Intermediates Part I -- Seven Lossless Codecs

musicvid10 wrote on 7/6/2011, 12:32 PM
I took 20 intermediate codec candidates that can be rendered directly in Vegas, both RGB and YUV, 8-bit and 10-bit, and did a shootout using but one main criteria -- how well will they do on the Belle-Nuit chart, which shows primaries, resolution, and chroma subsampling comparisons. In each case I used 720p renders of the 720p version of the chart, because that's the highest either my notebook or desktop monitor will resolve.

I found seven codecs that reproduced the chart (8 bit RGB) measure-for-measure (save for the 1-bit shift):
-- AVI Uncompressed
-- MOV Uncompressed
-- MOV BMP
-- MOV PNG
-- HuffYUV (RGB mode)
-- Lagarith (RGB mode)
-- UT (RGB mode)

Some "visually lossless" codecs did not make it into this first round, because they either showed shifts in color primaries or chroma subsampling when compared to the seven above, whose results are apparently identical in every respect.

Knowing that render times and file sizes would be entirely irrelevant on static material, I then rendered a three-minute video (known as Untitled4 to some of you) from each, the relative render times and file sizes appearing in the chart below.



As can be seen, UT gives the best render time, with Lagarith or UT being the best compromise in both areas.

Here are the things I did NOT do:
-- Attempt to identify which codecs will work for a particular purpose or application in real-world conditions
-- Run multiple generations to determine where rounding errors begin to show
-- Attempt to determine how levels and gamma would behave in external applications
So even though they are lossless at various degrees of efficiency, don't assume that your favorite external encoder (x264) or NLE is going to play nice.

Judging differences in the next round of "visually lossless" intermediates gets even tougher, because some codecs fudge with primaries, show chroma subsampling errors, or both in varying ways and degrees.

These tests were all run in Vegas Pro 8 on a dual-core laptop, nothing fancy.

[EDIT] Here is the list of all the codecs run so far for this series of tests. If one of the codecs listed below does not appear in the top seven above, all it means is that it showed measurable differences in the render of the Bell-Nuit test chart as referenced to the benchmark, which is Uncompressed 8-bit RGB (same as chart).
These excellent codecs will be addressed separately, in Parts II and III of this series.

Comments

Former user wrote on 7/6/2011, 12:45 PM
Musicvid,

Thanks for your work on this. I am not surprised about HUFYUV since that has been around for so long.

Just curious, did you test the Sony YUV codecs?

Again, enjoy reading your posts and tests results.

Dave T2
musicvid10 wrote on 7/6/2011, 12:51 PM
"Just curious, did you test the Sony YUV codecs? "
Yes, and although they are very good, are technically lossy along with Cineform and some others that will show up in the next round of tests, when I am able to get to it.
Former user wrote on 7/6/2011, 12:53 PM
Will patiently wait.

Dave T2
johnmeyer wrote on 7/6/2011, 12:55 PM
If I am interpreting your results correctly, it would seem to confirm that Lagarith is the best of the bunch. I use either it, or its cousin from an earlier time, HuffYUV, when I need to create an intermediate.

Other than the speed and compression advantages of Lagarith (something confirmed by other tests I have read about), I like the fact that it will accept pretty much any resolution. I often deal with video sizes other than DV NTSC or the two main HD resolutions (and their stretched pixel equivalents), and Lagarith handles anything I throw at it.

By contrast, every time I have tried MXF (admittedly, not a lossless codec), it always seems to choke or have problems.

The only downside to Lagarith is that the files are big enough that often a garden variety disk drive seems to have trouble keeping up and giving me smooth playback.
mekelly wrote on 7/6/2011, 12:57 PM
Is the Lagarith codec supplied with Vegas Pro or do I need to get it somewhere else? If supplied, where is it buried in the 'Render As' menu?

Thanks!
musicvid10 wrote on 7/6/2011, 12:58 PM
Lagarith has a vocal fan club out there, probably for just the combination of reasons you cited, John

http://lags.leetcode.net/codec.html
It shows up in Vegas as a Custom AVI Video format (not a template).
Former user wrote on 7/6/2011, 12:58 PM
Lagarith is a codec you can download for free on the web. After installing, it should show up as a Codec option in the AVI renders.

Dave T2
musicvid10 wrote on 7/6/2011, 1:08 PM
I wanted to clarify the ambiguous term "Intermediate."
It could refer to a transitional render to be used inside Vegas, or one exported to an external app like Handbrake or AE.

Again, I have not made any judgment as to how well these will work either on the Vegas timeline, or if at all in an external app. Certainly, one would have to jump through hoops to get any of these "magnificent seven" working right in HB, if at all.

Also, the YUV mode of a given codec will give different results than RGB mode, and a 10-bit version will give different results than its 8-bit version.

All I have done is tested that they will produce identical results at the relative times and file sizes indicated when rendered in Vegas Pro 8.
In that sense, this was a clinical experiment, meaning YMWV.
NickHope wrote on 7/6/2011, 1:55 PM
Did you consider UT video codec? It seems to have growing support and I have read that it's faster than Lagarith. I have it installed but haven't used it in anger. Doom9 thread about it but the link in the first post there is an old version.
musicvid10 wrote on 7/6/2011, 1:57 PM
Nick,
Didn't know about that one. I'll d/l and test alongside the others sometime this week.
Laurence wrote on 7/6/2011, 2:02 PM
>By contrast, every time I have tried MXF (admittedly, not a lossless codec), it always seems to choke or have problems.

I use .mxf a lot. That or Cineform .avi. The important things with .mxf are the following:

1/ You can losslessly smart-render HDV into 1440x1080i .mxf, but make sure that you are using 25mbps and the interlaced preset. Change to progressive and it won't smart-render. Not only that, but successive renders won't smart-render either. What I do with progressive footage is to set the deinterlace method to "none" and render to the 1440x1080i .mxf preset. That way you can go all the way through to a .mxf master (taking for granted you aren't color correcting) with nothing rerendered but titles and little bits around the transitions.

2/ If you are dealing with DSLR or AVCHD footage that is 1920 x 1080p, I always render to either the 50Mbps or 30Mbps interlaced presets. Again I set the interlace method to "none" so that it doesn't try to deinterlace the footage. Vegas will only smart-render .mxf that it thinks is progressive and this will be important on successive renders.

3/ When working with .mxf, i just leave the deinterlace method set to "none" all the time and use interlaced and progressive footage (60i and 30p) interchangeably. A limitation of this approach are that you can't resize or zoom into any of the interlaced footage on the timeline. Zoom into or reframe the progressive footage all you want, but leave any interlaced footage at it's original size or you will get strange looking comb resized artifacts.

4/ If there is any interlaced footage on the timeline, do not render to anything other than 1080i .mxf (or 1080p with a different codec). Rendering to a different size without a deinterlace method selected will give you those horrible looking resized comb artifacts.

5/ For TV delivery just use your 1080i render (with it's mix of interlaced and progressive 1080 footage). Don't worry. The progressive parts will still look progressive and the interlaced parts will have fluid motion.

6/ For Internet delivery do your master render to Avid codec or use Nick's AVISynth script approach. Handbrake will work with .mxf but you can only select one of the stereo channels so if you have a stereo project you can't use .mxf. Handbrake will do whatever resize and deinterlace (decomb) you need for Internet delivery much better than Vegas. So will Nick's script.

I really like working with .mxf because it is small, previews really well, smart-renders really fast, and is useable on small bus powered USB drives. If I am color correcting a lot or applying movie looks I'll use Cineform, but for most of my day to day stuff, .mxf works just fine.

I never use lossless codecs unless I am working with some program that doesn't understand Cineform. To my eyes, Cineform's higher quality modes visually look exactly the same as uncompressed. I know there is a theoretical difference, but I certainly can't see it.

While there is a slight difference with .mxf, It rarely bothers me. As a general rule, .mxf is my favorite most commonly used format.
musicvid10 wrote on 7/6/2011, 2:08 PM
Those two popular codecs are included in the next round of tests, which are technically (not necessarily visually) lossy. A couple of my personal favorites are not even near the top of that group, but there is nothing wrong with them either.

A codec that smart renders, such as several AVI and MPEG-2 formats, may only be "lossless" when taken in their own context.
robwood wrote on 7/6/2011, 2:14 PM
no QuickTime PhotoJPEG or Animation codec?

PhotoJPEG i've had excellent results using as a 4:4:4 (100%) codec (i found no measurable loss tho admittedly my tests were aimed at rounding errors)... and as a bonus; change the slider to 75% and it shifts to Y'CbCr 4:2:2

and Animation codec is lossless at 100%
(edit) and supports alpha channel!

---

Lag/Huffy are/were awesome to work with. i had to bail on them when 64bit OS came out as there was no easy way to get Vista 64bit to recognize Huffy/Lag at that point... i edited a couple posts on how to do it on DVInfo, but got tired having to tweak the OS on every 64-bit computer i had to access those files on so i switched.

---

QT-PNG as an intermediate for YUV video... s--l--o--w... your render-time listing for it matches my experiences. But a great archive format for animated media being true lossless.

---

edit2: ps ~ another awesome topic. thx musicvid!
musicvid10 wrote on 7/6/2011, 2:41 PM
"no QuickTime PhotoJPEG or Animation codec?"

@rob et al,
The first post is being annotated with a list of all the codecs tested as candidates for this series of tests. If you are currently using a visually "lossless" codec in Vegas that you do not see on the list, just mention it and I will add it alongside the others.

johnmeyer wrote on 7/6/2011, 2:41 PM
Laurence,

Well, that sure is interesting ... a lossless render to MXF. I just tried it, and sure enough Vegas does say "no recompression ..." in the preview window.

But that brings me to a question: why would I want to do this? Since there is no recompression, the file size is the same as the m2t file that I could also create with no recompression. In fact, I assume that the mxf is just providing a "wrapper" for the underlying m2t.
John_Cline wrote on 7/6/2011, 2:42 PM
Huffy and DNxHD are only 32bit. Lagarith installs both 32 and 64 bit versions of the codec. I work primarily in Vegas64 and I use Lagarith exclusively for all my intermediate files. There are no color/luminance shifts when going back and forth between Vegas and any external application. The only exception is that since Handbrake relies on the ffmpeg library, it does not support Lagarith, it does support DNxHD. Regardless, I don't use Handbrake that often so it doesn't really matter much to me.
musicvid10 wrote on 7/6/2011, 2:55 PM
Note: All tests were run in 32-bit Vegas 8.

@John Cline,
Can Lagarith be exported to AE or FC, with or without colorspace issues?

"since Handbrake relies on the ffmpeg library, it does not support Lagarith, it does support DNxHD"
Yes, but unfortunately only 8 bit DNxHD, the same being true for Avisynth.
It scored quite a bit lower than 10 bit DnxHD in this test.
robwood wrote on 7/6/2011, 3:10 PM
"If you are currently using a visually "lossless" codec in Vegas that you do not see on the list, just mention it and I will add it alongside the others." - musicvid

in that case, i'd be interested in seeing PhotoJPEG & Animation codecs added. thx.
musicvid10 wrote on 7/6/2011, 3:16 PM
Rob,

In the list appended to my first post above, Quicktime Animation is abbreviated as "QT-Ani", and Photo JPEG is abbreviated "QT-phJPG. Likewise, JPEG-2000 is abbreviated "QT-JPG2K"

Sorry if this still wasn't clear, thanks.
Laurence wrote on 7/6/2011, 3:17 PM
>But that brings me to a question: why would I want to do this?

Because in HDV format the audio is data compressed and gets recompressed on each successive re-render. In MXF format, the audio is uncompressed and no damage is done on successive re-renders. Also, for some reason that I don't understand, there is less video damage in MXF in the parts that are rerendered. I have no idea why but the difference is easy to see. No it doesn't make sense, but it is definitely the case.

Also, if you have a decent graphics card, the Sony PDZ-VX10 MXF viewer software does an absolutely gorgeous job of previewing your renders at each step.

One more thing is that Vegas seems to handle a coupe of large clips with mpeg2 compression way better than it does many small clips. I think this has to do with needing to buffer the beginnings of each clip to the nearest iFrame, Even before I used. MXF i would smart-render my individual clips into larger chunks for this reason. Now I can do this without inflicting a generation of audio damage.

One last thing, I often normalize any audio tracks that have dialog in them. Sometimes I use NR and a little compression and EQ at this stage so I don't have to do it later on when I will have a lot more CPU intensive stuff going on in the project. I also often cut out a lot of junk audio at this stage.

A trailing edge computer makes you aware of a lot of this sort of stuff.
robwood wrote on 7/6/2011, 3:20 PM
Can Lagarith be exported to AE or FC, with or without colorspace issues? - musicvid

1) you can roundtrip Lagarith or Huffyuv to AE from Vegas no problem... as long as its PC to PC.
2) there is no Lagarith for Mac.
3) there is a port of Huffyuv for Mac (http://homepage.mac.com/yakc/huffyuv/) but it doesn't work for everyone.

in other words, if there was a Mac anywhere in the workflow i rendered to DNxHD rather than Lag or Huffy.
winrockpost wrote on 7/6/2011, 4:10 PM
With all the testing of #$%^ ,,do you guys have any time to edit anyhting :)
craftech wrote on 7/6/2011, 4:24 PM
With all the testing of #$%^ ,,do you guys have any time to edit anyhting
=======================
Yes,

"anyhting"..................................edit.................................anything.

John
winrockpost wrote on 7/6/2011, 4:50 PM
:) thats what I'm talking about....great edit!!!