@JN- The reason for that discrepancy in the ffmpeg results is because the render is incorrect. If you render from VP17 8-bit mode where you imported an 8-bit Limited YUV file, you're RGB black/white points will be at 16/235 in Vegas, which is not what MagicYUV or most of the VFW codecs expect when rendering. VFW codecs expect 0-255 RGB range. So both for the MagicYUV plugin render and the VFW Limited render, you have to apply a studio->pc levels conversion (in VP17), otherwise the resulting file will be incorrect. And that is what shows in the ffmpeg result. The reason it doesn't show in the HO RQ result (#1) is because you're re-importing the rendered output into Vegas 17 8-bit legacy, again incorrectly. So incorrectly importing an incorrect render makes it right (two wrongs make a right), but only inside VP17 legacy. If you import it into any other software, like ffmpeg to do SSIM comparison, or even into VP18 8-bit full, it all gets messed up.

The "best" result with the full range option is actually the worst, or at least not any better than the others, because the flags are messed up there. It is flagged as a Full range file, yet it contains Limited YUV data, ffmpeg (as I wrote a while back) doesn't care about ranges when doing SSIM, so that "looks good", and VP17 imports it and the Source.mp4 both without range expansion, so they end up the same inside VP17, so that also "looks good". But try importing it into VP18 8-bit full mode or any other NLE and you'll see that the "best" result is actually off.

I am equally curious as to why for my testing that the older VFW implementation of Magicyuv gives better results than your new Plugin. Doesn't that discrepancy bother you, or raise red flags that something is off?

Not at all, because I know precisely what is happening. The only thing missing perhaps is the full range option in the plugin, but I'm reluctant to add that, because of the misunderstandings it can cause.

The issue now is not ffmpeg or the source file, but the issue is still there as to why by using VFW with the full range box checked that I get better results than by using your new plugin.

It is not better at all. It is only better as long as you only work in VP17 legacy 8-bit, since there you incorrectly import an incorrectly exported file, which makes it right. But from all other angles, the file is messed up. Flagged as Full range internally, but in actuality containing Limited data, because of the wrong export.

@MagicYUV I think you are tying yourself in knots. Maybe you had more wine than I had tonight? Anyway, I modified the source to limited, to match the flag that VP writes, this was my attempt to normalise things.

"So both for the MagicYUV plugin render and the VFW Limited render, you have to apply a studio->pc levels conversion (in VP17), otherwise the resulting file will be incorrect."

Tried the above, it's still not as good as the best result using VFW with full range checked, which gives ...

MagicYUV ...

ffmpeg .... SSIM All 0.998408 (27.981928) ... PSNR Average 55.046681 ... VMAF 99.750428

HO RQ ... SSIM 0.9642 .................................... PSNR 46.727 ........................ MSE 1.612

By way of comparison the Sony YUV lossless below, tested with the very same source file gives ...

ffmpeg .... SSIM All 0.999984 (47.8871108) ... PSNR Average 71.989102 ... VMAF 99.750816

HO RQ ... SSIM 1.0000 .................................... PSNR 71.343 ........................ MSE 0.006

And using a new source file that's 16 to 235 all the way with flag set accordingly to limited ...

MagicYUV ...

ffmpeg .... SSIM All 0.998602 (28.546275) ... PSNR Average 56.06059 ... VMAF 99.801485.

HO RQ ... SSIM 0.9699 .................................... PSNR 47.238 ........................ MSE 1.385

By way of comparison the Sony YUV lossless below, tested with the very same new source file gives ...

ffmpeg ..... SSIM ALL 1.000000 (72.947295) ... PSNR Average 96.0914 ... VMAF 99.801485

HO RQ ... SSIM 1.0000 .................................... PSNR 72.967 ........................ MSE 0.006

For the above ... SSIM and PSNR, higher is better, for MSE lower is better.

"The reason it doesn't show in the HO RQ result (#1) is because you're re-importing the rendered output into Vegas 17 8-bit legacy, again incorrectly. "

I didn't re-import anything, I just rendered out the three files, #1, #2 and #3 as in the screengrab.

"Not at all, because I know precisely what is happening. The only thing missing perhaps is the full range option in the plugin, but I'm reluctant to add that, because of the misunderstandings it can cause."

And this is easier?

@MagicYUV For whatever reason, this discussion is going nowhere. Best of luck with your new Plugin. Over and out.

@JN- I prefer beer ;)

I didn't re-import anything, I just rendered out the three files, #1, #2 and #3 as in the screengrab.

For the HO RQ comparison, doesn't that need a re-import?

Now you say that I should apply a studio to full levels before rendering. That's not what's done, ask MusicVID.

Yes, you should definitely ensure that VFW codecs, as well as the MagicYUV 8-bit plugin, gets 0-255 RGB, where 0 is black and 255 is white. (This is also why VP18 went default on 8-bit full range, and made studio levels legacy, because almost everything expects RGB as 0-255.)

@MagicYUV For whatever reason, this discussion is going nowhere. Best of luck with your new Plugin. Over and out.

@JN- I respect your decision. Thanks.

@JN-

If you don't like the release, why are you so obsessed with it? Badgering its developer is not going to increase your understanding.

@Musicvid Perhaps this has something to do with it? JN: "Maybe you had more wine than I had tonight?"

@MagicYUV @JN-

I haven't read this thread thoroughly, and it's going to take me some time to figure out what the contention is.

However, if it's useful to either of you, I will share this information:

• MP4/M4V and MPEG-2 in the wild, comes in three flavors:

1. Conforms to REC 709 levels, designated as [16,235] inclusive, and is flagged correctly, or carries no flag (default)..
2. Does not conform to REC 709 levels [<16,>235], and carries the Apple VUI Fullrange metadata flag (yuvj420p).
3. Does not conform to REC 709 levels and carries the Apple VUI Limited flag (yuv420p}, or no flag at all, which used to be the norm.

These days, most players honor these flags correctly (it was a struggle getting there), meaning that both Fullrange and correctly conformed YUV play at expected 0-255 RGB levels in VLC, and most others.

Some Nonlinear Editors (Adobe) honor the VUI flags, and when MP4 source is encoded and/or flagged correctly, also previews and renders correctly (RGB preview, YUV render). As I think Sonic Foundry coders realized fully two decades ago, this is a hit-or-miss proposition, and chose to leave so-called "auto levels" out of its product. and subsequent brand owners chose to do the same, until a switch was introduced in VP18, with its abundant collateral of confusion, cluelessness, and divisiveness.

Vegas editing software may not natively recognize Fullrange or Limited VUI Source Flags. The default is WYSIWYG, meaning proper mp4 will preview flat in RGB space (without intervention) and will render correctly in Vegas. Notably, scenarios #2 and #3 above behave identically to each other in Vegas, meaning correct preview and incorrect blowout renders will result unless intervened. Remember, Vegas is agnostic to Apple VUI flags.

Now, as suggested by my tongue-in-cheek background article, if we introduce additional Full and Limited choices in the encoder or another upstream location, in addition to the Vegas 18 switch and graphics dynamic range switch, we have increased to odds of getting the render levels right the first time to only 1 in 128, with a number of redundant pseudo-solutions, known trivially in the wild as "frankenfiles."

This information is being offered as "what-it-is," not to choose sides, judge, or claim victory. It comes with the caveat, however, that yes, two determined parties can indeed carry on the debate for eternity ∞

Pax

Vegas editors do not natively regognize Fullrange or Limited MPEG Source Flags. The default is WYSIWYG, meaning proper mp4 previews flat (without intervention) and renders correctly in Vegas. Notably, scenarios #2 and #3 behave identically in Vegas, meaning correct preview and incorrect blowout renders (without intervention) in Vegas unless intervened. Remember, Vegas is agnostic to Apple VUI flags.

@Musicvid Do I understand it correctly, that it is exactly this paragraph that VP18 tries to address with the 8-bit full setting? Ie. to recognize and take into consideration Limited/Full when importing and range-expand accordingly to RGB 0-255, so it both previews and renders correctly?

Yes, although I do not own VP18, I do not get the impression that it reads VUI flags [edited].

https://www.vegascreativesoftware.info/us/forum/vp18-notes-on-the-8-bit-full-level-option--122749/

I'm not familiar with the terminology of an Apple VUI flag, but did spend some time testing VP 18's handling of the new default (8-bit full), and it does read and honor color range metadata (as Marco indicates in detail with his post.) If the file has data which mismatches its metadata it will display incorrectly in a predictable way. Non-video files are treated as full range.

Under render settings/project you can choose limited or full as well. VP 18 defaults to a color range of YCbCr output of limited, with corresponding metadata embedded. I just tested this to confirm- source file was limited range AVC. Working in 8-bit full mode, I rendered it as full range AVC and it has the proper corresponding full range metadata in MediaInfo.

@RogerS

Under render settings/project you can choose limited or full as well. VP 18 defaults to a color range of YCbCr output of limited, with corresponding metadata embedded. I just tested this to confirm- source file was limited range AVC. Working in 8-bit full mode, I rendered it as full range AVC and it has the proper corresponding full range metadata in MediaInfo.

One difference here is that MagicYUV 8-bit always asks RGB from Vegas during rendering, so that particular setting is not taken into account.

And this is where MagicYUV 10-bit YUV formats are different, there that setting should matter, because MagicYUV 10-bit YUV formats ask YUV from Vegas, and not RGB, so Vegas has control over the YUV range with that settings. But this has to be tested more, whether those settings do get taken into account or not. I'll need to check.

Very interesting, thanks for clarifying.

For my amusement I just took a Sony AVC 16-235 file, output Magic YUV RGB 8-bit and set as full in the render settings. The resulting file appears to have no tag in MediaInfo. Vegas reads it as undefined (so limited), and on my timeline toggling between the original and MagicYUV file they are indistinguishable in the scopes. To confirm, I put both clips into an 8-bit video levels project and both clips show as limited looking at the histogram.

I don't think I can really help with 10-bit though as I have no 10-bit footage.

Without mentioning any names, I don't think "tongue in cheek" or otherwise "cheeky" responses are warranted, along with accusations that the developers have mindlessly opened some sort of "can of worms" with VP 18, especially from persons not invested in the latest program version, which - as per this "analysis" - confers on us a "collateral of confusion and mindlessness..." (really? Mindlessness?).

I say that especially applies when talking with a third party developer who's graciously taken the time to help figure this out and make it more elegant. It just rings so disrespectful. I'm just shaking my head at these comments, honestly.

One difference here is that MagicYUV 8-bit always asks RGB from Vegas during rendering, so that particular setting is not taken into account.

And this is where MagicYUV 10-bit YUV formats are different, there that setting should matter, because MagicYUV 10-bit YUV formats ask YUV from Vegas, and not RGB, so Vegas has control over the YUV range with that settings. But this has to be tested more, whether those settings do get taken into account or not. I'll need to check.

That explains both sides of the "debate" well enough to me. It is logical based on the history of the AVI format color space, taken in light of its more recent repurposing of the format. AVI can be flagged as either, even though Y'cBcR came later.

This is reminiscent of past discussions, where the nugget of truth is obscured by a cloud of vagaries, real or imagined, relevant or not, and desirable or not, based on one party's perceptions. It has been observed from past discussions that one who hasn't gone through the process from the ground up, but only sees the outcome, doesn't really "get it," and never will.

One thing is absolutely certain:

There is no single "Clean" solution, nor is it possible to contrive one. Every time a new idiot switch is added to the processing chain, of which there are many in existence, it geometrically multiplies the chance for actual or superfluous errors by ^2. This can be and has been proven empirically by applying the simple probability formula that we all forgot from eighth grade!

“We cannot solve our problems with the same level of thinking that created them.”  -- Albert Einstein

I strongly suggest to our armchair critics that they just cool their pipes, and let an astute and successful developer continue developing viable outcomes from the entropic swamp we have been stuck with from past short-sighted mistakes. @MagicYUV can always change things after the job is done, ostensibly for the better, but never for perfection. Can we, including the entitled, all agree to accept that?

@Musicvid To complicate matters further, the AVI container in general has no flag about range whatsoever. The codecs might have. But a generic VFW reader has absolutely no means to communicate this. That is why the safest bet is to always communicate RGB with VFW (this is also what Vegas does), because there at least you can be somewhat sure that it's always computer RGB. Still, only VP18 treats RGB this way from VFW, in VP17 and earlier it was up to the user to know whether a particular AVI contained studio RGB or computer RGB.