UT or not UT

cold ones wrote on 4/18/2016, 2:51 PM
I'm updating a DV project and hoping that one of you can help me understand what's going wrong.

Thanks to people on this forum, some years ago I stopped making DV masters and began making AVI masters using the UT codec. It yields a reasonable file size and, to my eyes, looks identical to the original (fonts hold up especially well). And it works with Handbrake.

But here's my problem. Last week, I used Handbrake to export an mp4 from a UT AVI master, and I thought it looked a little washed out. Sure enough, I dragged the mp4 into Vegas, and saw that its luminance was constricted across the spectrum (peak whites are 4 or 5 IRE units lower, lowest blacks are 4 or 5 higher).

I did a test with color bars: I exported a short clip as UT, MagicYUV, DV, and Uncompressed. I brought these AVIs back into the timeline, they all metered perfectly. But when I exported each of them as an mp4 via Handbrake (and later, Adobe Media Encoder) and dragged those mp4s into a timeline, they all showed this shrunken luminance.

Except for the mp4 that was generated from the DV clip, which was perfect.

Can anyone shed some light on this? (no pun intended)

Comments

astar wrote on 4/18/2016, 3:11 PM
Why are you doing a generation loss to move to uncompressed, only to go to mp4. The DV information is as good as you are ever going to get.

The level issue likely has to do with changing color space somewhere in your conversions. Probably 411 DV, to 444 or 422 uncompressed, back to 420 h264.

I would just capture the DV information from tape, clip the stuff you want, then convert the DV straight to MP4/h264.
musicvid10 wrote on 4/18/2016, 3:18 PM
I agree with astar; native DV usually works just fine in Handbrake.

Probably what you did was render UT as YUV or YUY2, but Handbrake thought it was RGB AVI.
If you're going to use an AVI intermediate, us RGB encoding and see what turns out. That's without any levels filter on the timeline.
That said, I haven't had colorspace issues with UT, however, Lagarith behaves terribly in Handbrake.

cold ones wrote on 4/18/2016, 3:26 PM
astar, you're probably right about why this is happening.

My main reason for doing this is that any time you change the DV footage (by adding font, for instance) the program undergoes a DV generation loss in those areas. This project has tons of them: dissolves, supers, and lots of text, and the DV generation loss causes lots of little blurry pixels around the edges, while the UT copy looks perfect. (And scopes out perfect in Vegas, too.)

Why does the UT render look perfect, but mp4 renders from it don't? Is there a way to get a high quality master from a DV Vegas project?

Thanks musicvid10, I'm rendering UtVideo RGB VCM---should it be something different?
musicvid10 wrote on 4/18/2016, 6:28 PM
Should look like the original footage if all goes well.
riredale wrote on 4/19/2016, 9:20 AM
Just as an aside, what drew me to Vegas back in 2002 was the fact that the Vegas DV codec had the remarkable ability to degrade very little over multiple generations when compared to its peers. I'd suggest you take a DV source, do three or four consecutive renders, and then check to see if it looks any different from the original. I suspect you won't be able to tell the difference.
cold ones wrote on 4/19/2016, 10:31 AM
DV generation loss is OK by me, but it's really how a DV generation degrades font that is my concern (and really, anything that has to be re-rendered off the timeline, like a dissolve or super). Here's an example of how font degrades, both rendered from the same timeline:
https://www.dropbox.com/s/38h17ewns1fnbnn/UT%20vs%20DV.jpg?dl=0

Note the chunks that are missing from the DV render, and the color dropouts. And this looks worse when playing, the edges of things are a bit blocky, blurry, and squirmy. This definitely comes through in the final compressed movie (be it m2v or mp4). That's why I liked using a higher res codec like UT to master from. And now that we upload full resolution clips to YouTube, it would be great to maintain as much clarity as possible.

Try this: new Vegas project set to DV, 5 seconds of color bars. Render once to UT AVI (RGB VCM) and again to DV AVI. Now render those clips through Handbrake to mp4 and look at the difference in luma levels. That's my problem...
musicvid10 wrote on 4/19/2016, 8:33 PM
Are you allowing third party DV encoders in Preferences? If so, try turning it off.

John_Cline wrote on 4/19/2016, 11:57 PM
It's the 4:1:1 color sampling of the DV codec that gets blocky on diagonal lines of high contrast colors, which happens quite often on text. This is an inherent limitation of DV and happens on the first encoding and gets worse with every generation. The UT Video codec doesn't have this problem at all. I noticed that the last version of the UT codec had some issues, I'm using the 15.3.0 version and it works great and Handbrake can use these files, too.
cold ones wrote on 4/20/2016, 8:54 AM
John, I think your analysis is spot on. I was running UT v14.2.1, so I upgraded to 15.3.0, but same problem occurs.

Here are my test files. My original UT export from a Vegas DV timeline:
https://dl.dropboxusercontent.com/u/64768954/DV_Bars_from_Vegas_Timeline_UT.avi

and the resulting mp4 via Handbrake:
https://dl.dropboxusercontent.com/u/64768954/UT_Bars_via_Handbrake.mp4

Failing all this, is there any better way to create a high quality master for projects like these, one that Handbrake will understand better? (Again, even uncompressed AVI files don't work properly.) Thanks in advance!

For completists, here's the DV render from Vegas:
https://dl.dropboxusercontent.com/u/64768954/DV_Bars_from_Vegas_Timeline_DV.avi
Its resulting Handbraked mp4 looks fine...
musicvid10 wrote on 4/20/2016, 4:17 PM
coldones,

Your first link is broken, but I found the file.

Would it surprise you to know that your levels are exactly correct in all three examples?
The two AVI files display roughly 0-255 RGB, and the MP4 file displays 16-235 YUV,meaning Handbrake did just as it should with your UT intermediate..

I assume you are seeing the difference by comparing them on the Vegas timeline. They are supposed to look different. The timeline displays what it gets.

If, on the MP4 track, you place a Studio->Computer RGB Levels filter, do the levels now compare favorably? Vegas displays native RGB color space, and does not correct for YUV Luminance. They all look the same on a player.

And yes, the UT intermediate is cleaner than the DV.

cold ones wrote on 4/21/2016, 10:44 AM
musicvid10, thanks for taking the time to follow up, I really appreciate it!

You're right about applying the a levels filter, sRGB to cRGB corrects the signal. I think the answer is in what you said about Vegas displaying in native RGB color space without YUV Luma correction.

But I'm still confused: here's an m4v that was rendered from the DV AVI bars movie, with the same Handbrake settings:
https://dl.dropboxusercontent.com/u/64768954/DV_Bars_via_Handbrake.mp4

Its levels are correct, without filtering. And I see this whenever Handbrake ingests a DV AVI vs any flavor of "uncompressed" AVI. It's also true if I export mp4s from Vegas

Will web players (like YouTube or JWPlayer) make this correction on the fly? This all started for me because I noticed that these mp4s seemed washed out.
musicvid10 wrote on 4/21/2016, 1:38 PM
PLAYERS will play back correct levels from RGB or YUV.
The mistake is comparing them on the Vegas preview, where they will show up as apples and oranges.

The DV and the MP4 you provided play the same. If they don't on your system, you've got a stoopid dynamic contrast switch turned on someplace in your hardware. Don't do that because it raises the magnitude of possibilities for error by a power of ^2.

I've run dozens of UT files through Handbrake, and have not experienced anything unexpected.