Question about levels.

AFAIK Rec 709 uses SMPTE levels and in 8-bit digital form specifies 16-235. So a file specifying Rec 709 implies a certain range in data.

AVC/H.264 data streams define a VUI flag/option that specifies data contained is full range, aka 0-255. This option is in the AVC stream (AFAIK) and not in any file container header. The Mainconcept (Vegas) and open source (ffmpeg) decoders seem to honor this option.

So AVC has the ability to reliably support both level ranges if a decoders read the options.

The question is what about other data streams, like MPEG-2. If all they have is a color spec like Rec 709, then it would seem to me that a decoder has no option other than go with studio swing.

Where all this comes from is Rec. ITU-R BT.1886. You really need to read this carefully. It is a recommendation for non CRT based studio monitors.

It defines a electro-optical transfer function for flat panel displays used in HDTV studio production. Such a definition did not exist for CRT displays. It does say these displays should display brighter than white for values of greater than 1 but with a gamma for 4.5, the gamma below 1 remains at 2.4. A Rec 709 signal fed into such a display will display correctly.

Now this Rec makes perfect sense as many cameras output a digital signal that more or less does this.

The problem here is that I doubt there's a single consumer HDTV on the market that conforms to this specification. I can only imagine me asking the salesman at my local electrical retailer which HDTVs conform to ITU-R BT.1886.

Poynton's argument does make a lot of sense however until every manufacturer adopts ITU-R BT.1886 AND all the old standard HDTVs are replaced what should we do?

Stick to the existing standards. If we want to preserve our brighter than whites fit them into what we know will survive.

Bob.

"AFAIK Rec 709 uses SMPTE levels and in 8-bit digital form specifies 16-235."

Rec. 709 specifies that video levels are in the range of 1-254, reference black at 16 and reference white at 235.

If the distinction between the usuable video levels and the reference black and white is dropped this is no more rec. 709.

.

Now take a look into the tech. 3321 which is the EBU guideline for consumer flat panal displays …

Again, Lou said it's the Vegas AVC render which clips ranges outside 16-235. To proof in a first step what the encoder does – reimport such Vegas AVC rendered media into Vegas and compare orignal levels with rendered levels using the waveform monitor. They will be same except of minor compression losses. And I only mean 8 bit video in 8 bit Vegas projects!!!

And if such a signal will be clipped elsewhere (which certainly sometimes will be the case e. g. in most Flashplayer based web usage) then it's not the Vegas render which introduces it but anything else which does not work according to rec. 709 which is the de facto video standard for the second decade now.

Yes, unfortunately we have many systems which do not work according to rec. 709 and we also have to take care for such cases in our Vegas projects. But we should not ignore the given video standards because most of the given (digital) hd video equipment carefully adapts to rec. 709. It's up to us to realize which kind of video levels a certain task needs but with no further details mentioned about a certain task, it should be quite sure we imply given standard (which is or is based on rec. 709). Anything else which ignores rec. 709 is second quality (except of the upcoming rec. 2020 of course ;)).

.

@ Marco.

http://en.wikipedia.org/wiki/Rec._709

Digital representation[edit]

Please Note the following:


-----
Rec. 709 coding uses SMPTE reference levels (a.k.a. "studio-swing", legal-range, narrow-range) levels where reference black is defined as 8-bit interface code 16 and reference white is defined as 8-bit interface code 235. Interface codes 0 and 255 are used for synchronization, and are prohibited from video data. Eight-bit codes between 1 and 15 provide footroom, and can be used to accommodate transient signal content such as filter undershoots. Eight-bit interface codes 236 through 254 provide headroom, and can be used to accommodate transient signal content such as filter overshoots. In some camera systems, headroom in the signal is used to contain specular highlights, however, these "extended-range" signals are not allowed in the broadcast system and are clamped during final mastering. Bit-depths deeper than 8 bits are obtained by appending least-significant bits. Ten-bit systems are commonplace in studios. (Desktop computer graphic systems ordinarily use full bit-depth encoding that places reference black at code 0 and reference white at code 255, and provide no footroom or headroom.) The 16..235 limits (for luma; 16..240 for chroma) originated with ITU Rec. 601.[1]
-----


My goal is to make use of the extra video above 235 that my camera shoots. Why did Sony design it so that it shoots out of the legal range? I do not know but that is what they did. Maybe to give the impression to make it 'seem' brighter? Whatever...

I could simply use the broadcast filter and trap-out the area above 235 and that would leave the reference level at 235? But wait, it would not with my camera because the reference level is at 255 instead of 235 where it belongs. So to cut it off means to loose details in the brightest areas of the video.

An elderly guy with white-thin hair would look like he had a white blob on his head because it was no longer possible to see the skin tones mixed with the white hair. And yes that has happened on a video I was editing some time ago.

So my solution to keep the video in the "legal' range was to 'squeeze' the video down to the legal range so that the reference white is now down where it belongs!

Right now I am playing with the full 32bit and full levels from 0 to 255. I wish the industry would change from the current limitations to this so that we can finally move on.

I can probably get away with this for my own enjoyment but if my videos were to be broadcasted then I would have a problem.

The video looks absolutely fantastic with this way of working with it! Best compression I EVER saw from the Sony mp4!!! I might say the heck with the legal range and keep on doing it this way!

Please consider that there are cameras that do shoot outside of the legal range and that we DO need some way to either bring them back into the legal range or play around with it like I am doing at the moment.

It is not acceptable to cut off the illegal parts of the video and loose important details in the process. I do not want an elderly guy to look like he has a white blob on top of his head!

The original question of this thread, " Source levels 16-256... how to normalize" depends on how the camera shoots in the first place. Does it shoot 16 to 235, 16 to 255 (as in my case) or whatever range it shoots at. One must first determine what range their camera shoots at and THEN determine how to correct that to put the levels in the legal range and NOT loose the important video details.

There is no one size fits all in what the correct procedure is.

Please read the original rec. 709 document (www.itu.int/rec/R-REC-BT.709/en) and you will see it does not say anything about levels above 235 will be clamped. This is nonsense for any system adapting rec. 709 and this line was added to Wikipedia from someone who misunderstood or mistakenly meant there must be a legalizer used somewhere in the end of the digital line.

Almost same mistake is done when said a desktop computer grafic sets reference white to 255. It sets reference black to 0, yes, e. g. for print usage. But it sets peak level to 255.

.

@Marco.

Please explain to me why a man's head looked like it had a white blob on top unless I did bring the video level down?

Maybe because something in your video chain ignores what digital reference white is meant to be set to.

And that something is the fact that the camera I was using then and the ones I am using now shoot such that the brightest parts of the videos are above 235 in levels. So I have to bring those levels down to where they belong somehow.

[I]"Now take a look into the tech. 3321 which is the EBU guideline for consumer flat panal displays "[/I]

Page 6:

Section 3e: Consumer displays with an 8-bit digital interface such as DVI [10] or HDMI [11] shall correctly operate in the 8-bit coding range of digital 16 to 235 for YCBCR digital components.

Section 3f: RGB Quantisation Range: limited range (16-235)

This is sounding more and more like your own personal vendetta which has no relevance whatsoever to the topic at hand. You've even admitted that it's quite likely that signals outside 16 to 235 will be clipped, Poynton himself says the same thing.
The only certain way to ensure that the full range of what we've encoded when viewed on any display device over any possible medium is to keep it legal.

The technical rights and wrongs of this are irrelevant, that's an esoteric discussion of no relevance to the vast majority of people here who simply want to get their video as easily distributed and viewed as possible with having it mangled or rejected.

On a more relevant note another local Vegas user that I bumped into at a trade show recently informed me, much to my surprise, that significant number of the copies of his movie were sold on VHS.

Bob.

This is absolutely correct. And this is the point where we carefully should distinguish between reference white and peak levels.

If we don't have the chance while shooting to ensure reference white will be placed at luma 235, our cameras might place the brightest luminance levels available at luma 255 and nobody cares for what reference white could be. Maybe these brightest levels are any kind of light sources like a bulb, maybe some reflections. But maybe even something you later consider to be reference white. In the first case: Don't worry, this is fine. In the second case: You need a correction to set your reference white back to luma 235.

In other words:
If your camera record 16-255 (what most cameras do) you might have to bring those levels down where they belong and you might have to let those levels be where they are if it is where they belong (like light reflections which are most likely meant to be brighter than reference white belong to levels above 235).
And if your camera shoot 16-235 exactly same is the case. If the camera then places elements brighter then reference white to 235 but reference white significant below 235 you will have to put the levels up.

Again – this is for systems which adapts to current digital video standards, not for colleagues like Flashplayer.

The mentioned digital range from 16-235 covers the range of reference black to reference white, not the peaks. Remember: video data is allowed from 1-254. Now you also have to see what a LCD display luminance is meant to be for reference white coming from luma 235. It is meant to be somewhere in the range of 100 up to 140 nits for dim surround. And well calibrated the peak levels should be able to reach at least to 200 nits without doing harm to the display.
Page 5 of same document: "On displays of up to 50-inch diagonal, small-area peak white should be adjustable to least 200cd/m2 without excessive flare."
So your range of displaying levels brighter than reference white is at least 60 nits on LCDs

"You've even admitted that it's quite likely that signals outside 16 to 235 will be clipped, Poynton himself says the same thing."

This is both for me and for Poynton the same misinterpretation as which happens to Rec.709 (which indeed is not very clear at some points). We don't say that it is quite likely signals outside 16-235 will be clipped. I say it is likely those signals will be clipped when used on systems which ignore current standards. If the system adapt to the standards there will be no clipping.

Again and again: In digital video clearly distinguish between reference levels and peak levels!

"The only certain way to ensure that the full range of what we've encoded when viewed on any display device over any possible medium is to keep it legal."

I even agree to this to a certain point.
And the only correct way to fully support given color manangement at its best quality is to carefully take respect of the characters of each kind of distribution even if this means you have to export four different kind of video signals. This does not at all disagree to what you said, it just adds another piece of puzzle.

.

@ Marco.

Speaking in terms of being careful, the more careful I am with exposure settings the better my videos look especially when it comes to peoples faces.

So I now keep the zebra stripes at 70% instead of 100% to help make sure of this. As I previously said, there are those unexpected events where the video can get too bright. And that is when I have to lower it a bit if possible. This especially happens at the theater. Theater lighting is not video lighting! Church lighting takes a miracle to work with!!!

In editing and I should have said this earlier the lowering may be just one or a few events on the time-line.

I think indirectly we are in agreement but have different understandings of this. My background is having been a bench technician for approx. 30 years so I am familiar with real equipment as well as the software stuff.

Finally, after having played with the 32 bit stuff for a while I will move back to 8 bit. In 32 bit, all of the processing before the actual render, is done and then of course the actual render which will be 8 bit again because that is what Sony mp4 does. Make sense? Anyway, it takes way too long so I will leave it alone. The quality sure was a whole lot better though because the pre-processing was a whole lot better! Still, I want a faster way to get the job done.

"In 32 bit, all of the processing before the actual render, is done and then of course the actual render which will be 8 bit again because that is what Sony mp4 does. Make sense?"

Though this turns to OT: Yes. The only output out of Vegas Pro I am aware of which maintains the full advantage of floatpoint projects is EXR. You maybe would forward EXR sequences to a compositing department or to certain color correction services but it's very unlikely one would use it for final delivery.

That said: With ACES workflows this may change to a certain degree.

.

Hi Lou,

I don't understand. What does color correction have to do with levels correction?
These things are closely related to each other. In fact, you have full control over gamma/levels in color correction stage. To be honest, I don't work with camera footage too often, so instead of playing chineese whispers, I better put the link to tutorial I got the inspiration from:
http://www.glennchan.info/articles/vegas/color-correction/tutorial.htm
Note, that in color correction you can set a lot of parameters manually, including gain, shift, gamma etc.