Always Need the Bottom of "S" Curve?

MH_Stevens wrote on 5/23/2006, 4:42 PM
I've been pushing exposure a bit as recommended by Chan but find I'm ALWAYS missing the bottom end blacks which means for nearly EVERY Vegas clip I need employ the bottom part of the famous "S" curve in Color Curves. IE I need drag the lower handle to the right. Top end always good with no need for adjustment and color correct.

Seems I should not be doing this so radically on nearly EVERY clip. Anyone got any comments? Shooting with the SONY FX1.

Michael

Comments

GlennChan wrote on 5/23/2006, 5:04 PM
Unfortunately, I don't think I quite understand your question!

Maybe if you could put some screenshots up, that would help? (i.e. imageshack will host images for free)
farss wrote on 5/23/2006, 5:21 PM
Also how are you judging this?
LCDs are pretty notorious for crushing the blacks.
MH_Stevens wrote on 5/23/2006, 5:23 PM
Hi Glenn:

In the typical shot the histogram extends up to 245 but at the bottom end is gone by 25 (IRE 7.5 not checked) and the shot looks washed out. I need use Color Curves to extend the bottom down to 0 (by pulling the Color Curves lower handle down and to the right) to give contrast, darken the image and correct color. Very similar to what the Color Curves preset "Add Contrast" does but at the bottom only and not at the top.

Michael

GlennChan wrote on 5/23/2006, 5:44 PM
Does your camera have some sort of "7.5% setup" setting? Those type of settings add fake digital setup. It should be turned off, unless you have some odd workflow.

If you're using the color curves preset I created (6 points on it)... you can drag-select two of the points and use your keyboard arrow keys to move them left/right. This will change the black level to where you want it to be (usually 16).

You might want to push some of it below 16, since compression will create some erroneous values that are slightly under digital black level. i.e. if you shoot DV, you'll notice that the very bottom of the histogram might hit around 14.
MH_Stevens wrote on 5/23/2006, 6:14 PM
No 7,5 set-up. I will look for your six point curves - I have not seen that. However I CAN correct in Vegas but why am I missing the bottom in the camera? Is it just a camera fault or a consequence of maybe over-exposing?

Is anyone else with the FX1/Z1 loosing the bottom of the histogram?
Serena wrote on 5/23/2006, 7:53 PM
Looking a number of my clips of various subject brightness ranges, the histogram always bottoms at 24. FX1E.
MH_Stevens wrote on 5/23/2006, 9:10 PM
Serena: Do you find this no problem? Do you like me use Curves to pull that black point down else the shot is washed out? If not, I'm interested in what you do do to get contrast and brightness right.

Michael

PS I'm working on your WB post but it's too late tonite - that's for the morning.
farss wrote on 5/23/2006, 9:58 PM
Just a thought from another direction.
Are the scopes in Vegas reading HDV correctly, given that HDV and DV are somewhat different. I've been working with a lot of A1 footage recently and I'll admit the HDV does look a bit washed out and the highlights blown out as well and yet rendered out to SD mpeg-2, plonked on a DVD and watched on a TV I don't think it looks so bad.

None of this is very scientific at all of course but it might be worth looking at this from a different angle.

Bob.
Serena wrote on 5/23/2006, 11:25 PM
Generally I prefer not let the highlights wash out, which can result in some under-exposure in mid-tones and crushing in blacks. So, presuming "washed out" to mean desaturated, that doesn't seem to be a problem for me. Hence applying corrections to everything is pretty much normal and, apart from frequently thinking the Z1's black stretch would be nice to have (and other features), I hadn't actually been specifically conscious of the black clip. I should have picked that it is consistent, however.

This sort of sets me off on a favourite peeve about the paucity of information made available by manufacturues. We've talked about this in relation to various Vegas behaviours and I get peeved that quantitive data on camera/sensor characteristics isn't readily available to us. Functions such as Cinetone: what on earth does "film like" actually mean? I know what I think it means, but what has Sony given us? If you shoot film you know the characteristics of your emulsion and controls available through processing etc. Mystery seems to be the catchword for video cameras. Many built-in programs are described in the vaguest qualitative terms. I had been hoping Spot would provide some of this in the 2nd addition of HDV.

A week or two ago we had a thread mentioning Cinetone (or the rival Cinegamma) and there was a diffence of view between Glenn and myself about what it is. He'd deduced that it was a change in slope (comparing with and without and using FX to get a match) whereas I said it was a "film like" half-S-curve (a roll off the top) relying on one publication that didn't describe their measurement technique. Who was right (if either) we don't know. Every now and again I think this must be determinable without pulling the camera apart and using the resources of an electronics lab, but I haven't had time. Bob probably knows how to do it! I'm thinking "fix exposure on 18% card and look at waveform with various known density greys up to white". Why not use the stepped grey wedge? Sounds too easy because then the characteristics would be readily available and I wouldn't be talking about it! First thing is to know the "density" of each step -- anyone know for the standard calibration wedge?
And colour characteristics? Knowing its 4:2:0 is equivalent information to knowing that film uses silver halides -- nothing about fidelity of colour reproduction. Seems to my eye that HDV (Sony style) is rather poor on green discrimination.

Should each of us have to independently calibrate our cameras? Might be a good investment. But if our calibration methodologies are ill designed then our results may be useless. The pragmatic "fix it in post" defeats such good intentions but I think it is unarguable that it's better to get it right in camera.
Jay Gladwell wrote on 5/24/2006, 4:39 AM

I'm interested in what you do do to get contrast and brightness right.

Expose for the highlights. You can always get some additional detail out of the shadows, but you will never get detail out of washed out, burned out, blown out highlights. "Don't be afraid of dark areas in your shots!"

I found these videos online several months ago. The man (a DP from Oz) does/suggests what I have been doing for years and years.

Please, watch these videos (and let me know who the person is speaking).


farss wrote on 5/24/2006, 4:49 AM
I think it's Pieter de Vries:

http://www.miller.com.au/news.cfm?sectid=40&subsec=4011&ID=77

You'd think I'd extract my digit and go to some of his classes!

Bob.
MH_Stevens wrote on 5/24/2006, 9:49 AM
Jay: I think I agree with you, at least for my FX1 because using Auto exposure or a more moderate level of Zebras gives me a lot of the bottom I was missing pushing the exposure.

Glenn: What do you say - when you recommended pushing exposure were you loosing some bottom end? Maybe its camera specific?

I am needing to radically color correct or exposure correct 90% of all my clips. Is this in line with others experiences? Seems too high to me.

Michael
GlennChan wrote on 5/24/2006, 12:34 PM
color curves here If you look really carefully, there's actually six points in it.

When I say pushing exposure, you could just change things so that the black peg whatever level you want. You generally don't want to lose any bottom end.

2- On cloudy days, or when there is lens flare your blacks are going to be higher than normal.
Your camera may also have a pedestal setting which can change black level. It shouldn't be the problem though, because chances are you never touched it.

3- I haven't worked much with HDV myself so I don't know how the levels are interpreted for it. What form of HDV are you working with? (i.e. intermediate)

Maybe it's how the intermediate is interpreting the levels is causing the problem??? (Speculation here, just a really wild guess.)


---
Setting exposure:
My own opinion is to exposure close to where you want the final exposure to be. You don't have all that room in color correction, as it will increase the appearance of noise and colors won't look quite as natural.
My own test:
http://www.glennchan.info/Proofs/dvinfo/color-curves.veg

Obviously auto-exposure is not going to give you the greatest exposure.

What happens to the image can be camera-specific, so I'd recommend you do a quick test with your camera. The knee function on some cameras do affect the highlights... knee adds a little more detail, and the colors get de-saturated and shift in hue a little.

GlennChan wrote on 5/24/2006, 12:43 PM
Cinematone, cinegamma:

Cinematone:
As far as I know... cinematone changes the camera's color matrix to try to mimic film's response to color (i.e. chromaticity).

What I mean by that:
Suppose you were shooting an orange with a film camera and a video camera. The film will eventually be transferred to video. Assume that color ranges from 0-100%.

The video camera might see 55%R 40%G 0%B.
The film camera might see 50%R 42%G 1%B. Totally totally arbitrary numbers here.

By changing the color matrix around, you can get the video camera to produce the same numbers as film.

However, not all film stocks + telecines and cameras have the same response to color. So you have variation there that you're not going to quite account for.

In any case, in a practical sense, use whatever setting looks better to your eye (if aesthetics are your goal).
If you want technical accuracy, then don't use cinematone. The default should provide slightly better color accuracy (except under fluorescent, in which case I think there's a setting for that).

---
Cinegamma affects the 'exposure' response / transfer function. It adds extra stops of dynamic range.

However, I don't think the transfer function is "s-shaped" like film's transfer function is. A s-shaped transfer function would affect color accuracy, increase contrast, increase saturation.
MH_Stevens wrote on 5/24/2006, 4:12 PM
Another way to make the corrections I nearly always need would be to use the Color Corrector to drop the gamma. This holds the top end and drops the blacks down and gives nice results with my footage.

As Serena rightly complains we get little technical specs from Sony so a lot of what you ask I don't know. I'm going to try cinema gamma tomorrow to see what it does.

This morning when I did my white balance tests I shot a close shot of homogeneous flowers and a white card in early morning soft light and I got a great dynamic range - like 4% to 100%, but in real life with a bright shy and strong shadows the range gets much reduced to say 14% to 100%.

I know it does not effect exposure directly but while I'm here let me ask the other FX1/Z1 owners where they have set there Zebra levels for shooting outside in full light. I think I'm over-exposing. I do wish I had a histogram on the camera.

To answer your work-flow question now I am use Cineform to intermediary but I looked back at some old straight m2t projects and the narrow dynamic range was more pronounced then than with Cineform. I have just compared a bunch of shots and I clearly see that the MORE contrast in the subject the NARROWER the dynamic range. It's the dull flat shots gives a full histogram. This is quite contrary to what I would had expected. Seems the camera is doing some processing. Serena, other FX1 owners, do you see this?

ALSO: Jays Videos would not download for me. Any advice? AND Glenns 6-point Curves would not link?????

Michael
winrockpost wrote on 5/25/2006, 2:38 PM
bump ,
i'm seeing it and on some sample clips I have downloaded it is obvious,,,, hmmmm
FuTz wrote on 5/25/2006, 5:51 PM
From the bottom up: anything to set differently inside your camera menus?
Serena wrote on 5/25/2006, 5:59 PM
Michael, I'll try to get back to this. One immediate answer is that I set zebra to 100% and wind back for min or no zebra bars showing. And use cinematone; does help with top end. Perhaps an issue with video is that the technical specs are not necessarily a reliable guide to results on the screen. There is a lot of this among people moving from film into HD for feature cinematography with a lot of "looks better than it should from the specs". This particularly applies to HDV which initially (still?) got badly mauled by the "True HD" guys. Yesterday I spent a few minutes Googling "Sony Cinematone" and pulled up a couple of articles publishing curves, but with no information about how they were measured. See p10 of:
cinematone

EDIT: that link fails for some reason [fixed now]
GlennChan wrote on 5/25/2006, 8:48 PM
Sony's Cinematone seems to be the same idea as the gamma settings on the DVX100. So by cinegamma, I actually meant cinematone.
Serena wrote on 5/26/2006, 12:25 AM
Glenn,

In general understanding CinemaTone (or Cinegamma) adds a shoulder to the CCD response curve which has the advantage of compressing the highlight end of the response and of course this has an impact on overall characteristics. The Sony article I flagged supports this understanding.

Although people talk about getting a "film-like look" this is rather a waste of time if they are talking about colour and contrast (as you rightly pointed out).

An Aside: Unfortunately, at least in my view, people completely misunderstand the advantages of film and talk rubbish about cadence, grain and other irrelevancies (such as near zero DOF -- OK some filmic use sometimes) that are mostly disadvantages of film projection. And of course included in this are people who write extensively on the video production and who have my greatest respect. Film has wide exposure latitude, so you get detail in deep shadows at the same time as detail in clouds, its has analogue resolution of gradients, it is sharp without edge effects and progressive. These are not subtle and anyone who can't differentiate a filmed image from a video is either looking at very high end HD or haven't used film.

Back to the point: The advantage of putting a shoulder on the response is to compress those highlights that otherwise blow-out in video and so keep detail over a wider range of subject illumination. It does appear to do this but we (or least I) need to get some quantitative figures. Only then do we know whether one approach or another has real (cf. perceived) advantages.
I have enormous respect for people that can eyeball stuff accurately, but like perfect pitch not many can actually do it and the Hi-Fi world, for one, is full of magical stuff that cannot be measured but some say they can hear.

We don't have to understand stuff to do very good work with it, but I find it saves me time in the long run. We need the facts, man, just the facts.

Serena
farss wrote on 5/26/2006, 12:49 AM
Serena,
whilst I couldn't agree with you more, all I can say is good luck!

It seems to me the more you spend on gear in this game the more likely you are to find it accompanied by hard facts. People in the broadcast game have the gear and the time to quantitatively check facts, so manufacturers know they have to be pretty accurate with the truth.
The rest of us are sadly left to flounder in a sea of amazingly technical terms like 'film like', 'mojo' etc. If we're lucky we might get to see something in a viewfinder that's got numbers on it, although more often than not we may have to guess as to just what those numbers mean, if anything.
Even Vegas doesn't escape this, how many FXs etc actually tell you what those numbers mean, on the audio side things aren't too bad but on the video side there's a lot of stuff that has no quantitative values assigned to it.

As for prosummer cameras like the FX1/Z1, I think the best way to put some rough numers on things is to actually test them yourself.

Re your comments about film.
Bravo, I wonder how many actually watch film in a cinema. Many comments seem to be based on film transferred to video and I think that's where a lot of the confusion comes from on this topic. I read so many people talking about wanting that 'film look' and I have to wonder just what sort of film they use as a benchmark. But what they're really trying to emulate is the look of telecined film, which is a very different beast to putting an image ona big screen in a 400 seat cinema.

I did note that Canon on the XL H1 have two cinegamma settings, one for making video that looks like film and one to use if you seriously mean to transfer to film. Sadly no curves or numbers given.

Bob.
SimonW wrote on 5/26/2006, 2:28 AM
The cine gamma settings in the FX1/Z1 etc are designed to give more of a finished look. The cinegamma curves in the Panasonics are designed to get the full 600% dynamic range of the sensors down to the recording medium. So the picture will look quite flat.

The HDW-750, and XDCAM HD cameras also have cinecurves that are designed to get as much contrast range as possible to the recording medium. Using these curves (assuming the DCC is turned off) the white level will not reach 100ire as the highlights are rolled off gently before this point is reach. Blacks are lifted also. The Sony XDCAM HD brochure shows the gamma curves behaviour, as does some of the Panasonic literature.

The FX1 and Z1 are a different matter though as the cine style gamma crushes detail.

http://www.bbc.co.uk/rd/pubs/whp/whp034.shtml
Jay Gladwell wrote on 5/26/2006, 7:28 AM

Mike, did you try to right-click on the links and try to download them? They're working for everyone else as far as I can see from my site stats.


MH_Stevens wrote on 5/26/2006, 1:34 PM
Jay-
Saw then this morning - nice little vids. Like you I'm not afraid of blacks; this post of mine is all about getting blacks from the FX1. But I can't get it with the camera exposure, I heed drag the bottom of the "S" curve in Vegas. That's what this post is about.

In the process of testing the FX1 with histograms and will post summary tonight.

Michael