Sony EX detail setting vs Vegas sharpening

megabit wrote on 4/15/2009, 2:32 AM

Recently I played with the two Vegas FX's: Sharpen and Unsharp Mask. The difference between the results they yield is obvious; while the former only crispens edges, the latter can - when used in conjunction with "Radius" - radically change not only the edges' sharpness, but also the overall picture contrast and latitude. For instance, on my typical mxf file from the EX1, applying Amount and Radius of some 0.3, stretches a 0-100 IRE picture well into superblacks and superwhites (much like "Levels" FX does). As such, the Unsharp Mask FX can be used as the "all in one" tool to increase the "punchiness" of the picture.

My doubts are related to the correlation between this behaviour in post, and the EX camera "Detail" setting. Many EX owners claim that detail should be turned off (or cranked down if on), in order to avoid artificial edge enhancement, and - should need be - the video be only sharpened in post.

Now, my question to those more savvy is: which of the two abovementioned Vegas tools is the more direct equivalent of the EX in-camera detail setting? In other words: when I turn detail off in my EX1, which of the two should I be using in Vegas to counteract?

This is a valid question, because if detail off in the EX camera acts more like the opposite of Unsharpen Mask, then we're effectively changing the latitude and contrast - not just edge enhancement.

I've tried to compare the scopes of some material shot with detail on and off, but before I report my findings, would like to hear your opinion...

Comments

Serena wrote on 4/15/2009, 2:57 AM

I'll have to do tests myself before attempting an opinion on the EX detail algorithm. It will be no more than an opinion, for I have no inside knowledge. I haven't observed any change in levels or contrast for detail on and off. I always use the Unsharp Masking FX in Vegas (a more sophisticated technique) and agree that it affects levels, so it isn't a good idea to apply Unsharp Masking after bringing everything within 16 to 235 levels. Well, you have to be aware of the effect it will have.

farss wrote on 4/15/2009, 3:17 AM

From my understanding there's more to the EX's capabilities than a simple Detail adjustment. Advice that I've read elsewhere that these should be adjusted on a shot by shot basis whilst watching a good vision and waveform monitor make my head spin.

Taking on board the degree of "sharpness" adjustment within the EX I don't believe there's a single FX in Vegas that comes close to the level of fine adjustment available incamera.

Bob.

megabit wrote on 4/15/2009, 3:35 AM

Advice that I've read elsewhere that these should be adjusted on a shot by shot basis whilst watching a good vision and waveform monitor make my head spin

Agree 100%, Bob.

If only the camera had a good waveform display, instead of the histogram (poorly implemented BTW, with its "left' side unresponsive to changes in the black end settings)... This could facilitate making use not only of the basic gamma selection, but also the black and knee detailed & precise tweaking - on the shot by shot basis.

farss wrote on 4/15/2009, 7:42 AM

I just tried Vegas's Unsharpen Mask on a checkerboard test pattern while watching on the waveform monitor and I'm not happy at all with what I'm seeing.
I set the test pattern's white and black to legal luma values. The FX is sending the edge enhancement to superblack and superwhite. This is then impossible to correct as far as I can see. Adding a legalizer FX would in effect extend the enhancement size or something like that.

Bob.

megabit wrote on 4/15/2009, 7:57 AM

The FX is sending the edge enhancement to superblack and superwhite

This is exactly what I mean, Bob.

Sometimes however I find it useful, actually - try it on an mxf clip recorded with the most conservative of the EX gamma curves, i.e. Cine2, and with detail off (this is the gamma recommended in the manual "for NLE processing").

Using just the Unsharpen Mask FX alone, it's possible to obtain a picture exactly as sharp as you need, and with the a vastly improved (and controlled) contrast!

When I say "contrast", I mean that you may choose the (expanded) levels range to fall into the safe 0-100 range, or - should you wish so - into the super whites and blacks a little...

After all, please tell me: what's wrong with a little super-whites/blacks if your delivery is HDTV (either from the PC or BD)? Of course monitoring to prevent loss of data in the "super" ranges?

My 50" plasma displays them just fine, if I use it to monitor the levels while adjusting Unsharp Mask...

Hence the question: with HDTV/BD delivery, is the 0-100 range still the only allowed, or "legal" one? I don't mean broadcast!

PS Remember the long discussion about the "abrupt highlights clipping" on the dvinfo forum? When playing with various setting of the Unsharp Mask FX, I'm able to obtain the exactly same, ugly effect out of perfectly "safe" recording... This confirms me in the conviction that it's a flaw in EX design - ugly artifacts might arise from choosing software parameters wrongly, but should never be possible in a camera.

GlennChan wrote on 4/15/2009, 10:30 AM

Sharpening can cause illegal values. This is normal.

2- You should clip these illegal values... one way of doing it:
http://www.glennchan.info/articles/vegas/color-correction/tutorial.htm

One big reason is so you have consistency in what is displayed... not all systems will display superwhites, and no system will display them well.

3- Sharpen (or convolution kernel) may be the closest to what the EX1 does.

Unsharp mask is more powerful.

Advice that I've read elsewhere that these should be adjusted on a shot by shot basis whilst watching a good vision and waveform monitor make my head spin.
That's just crazy and silly in my opinion.

farss wrote on 4/15/2009, 3:45 PM

I'm not so certain this is all that simple.

1) The EX provides 8 different adjustments under the heading of Detail. Some of them permit you to avoid adding detail to noise. I don't know how easily you could achieve this in Vegas.

2) The discussion so far has avoided the issue of 'why' it's recommended to reduce/turn off detail in camera. There's conflicting requirements involved here to further muddy the waters. Do you need to hand pretty pictures to a client at the end of the day or are you the one whose going to do the post work.

3) The EX would seem to record images between black and superwhite. Depending on knee and slope settings there would appear to be plenty of usefull detail between white and superwhite.

4) Vegas doesn't seem to know where 'black' is. I've always assumed this doesn't matter, so long as it's consistant then it all 'comes out in the wash' and the offset can be removed at the end of the processing chain. I'm starting to realise this might be too simplistic.
I just tried a very simple test. Add black to black. This should give black. Vegas gives grey. Vegas assumes black is superblack in it's calculations. This may explain some of the funky results I get when trying to composite in Vegas, especially in how it handles images with an alpha channel.

6) I agree with Glenn. Having the results of any edge enhancement outside legal values could be a bad thing. Perhaps not too serious in a digital signal chain but with analogue component connection I'd start to get nervous.

7) Can someone please explain what this from page 73 of the EX1 manual means:
"Select the source signal to generate vertical details from among NAM (G or R whichever is higher). Y, G, and G+R."

Bob.

Serena wrote on 4/15/2009, 8:49 PM

I've seen advice that it is preferable to set the required "look" when shooting with the EX1/3 cameras, rather than leaving that to post. I understand this is because of the limitations of 8 bit long GOP, compared to, say, the RAW output of RED. However to do that shot by shot would seem likely to lead to inconsistencies and trouble in post. In terms of detail on or off, that is also a question of "look". Detail "off" is more akin to film, but if one is then going to sharpen in post then it seems to me to be a reason for setting that up in the camera (saves time in post, at least). Mostly the argument has been that the resolution of the EX sensors is adequate without detail "on", hence allowing such decisions to be delayed until post. This becomes a circular argument; take note but make own decisions!
In terms of unsharp masking, it is a powerful tool in astrophotography where detail is usually hidden in fuzzy and somewhat noisy images; quite amazing what can be extracted and impressive are the artefacts that can be introduced. Must be used with great care.

EDIT: Darn! a thread in DVInfo has reminded me: a good reason for leaving detail "off" is to avoid things popping into focus when the camera (pan) or subject ceases moving. Case of finding a good reason for adopting a procedure and then forgetting the reason.