Q: How to Save/Improve Grainy Low-Light Footage?

Soniclight wrote on 1/25/2011, 5:35 PM
First, I know, I know what most will say...
"Start with good lighting and footage."

Agreed, but sometimes even the best efforts with a non-pro camcorder don't pan out (I have a Canon HV30).

And indeed: the provided stills of this project in its current state will show that said footage is pretty crappy to most of you (as it does to me) - lol.
But that's the point of a forum - to ask for help.

Project Info

Vegas version: 8.c
Project properties: 960x540, progressive, 1.0 aspect ratio, 8-bit.
Viewing platform: Online either at full rez when available or proportionally smaller FLV (usually 576 or 512 width resolution). FLV rendered with On2 Standard Flix 8.

Those of you who have known me over the years here are aware that I have a particular visual style that is much different from what most of you pros work in (realistic, documentary, weddings, etc.). My style leans into the dreamy and artist-fied to reflect the spirit of my website's perspectives, CompassionSensuality.Net.

And last but not least, I'm a serious amateur not a pro and am not looking to make money on this stuff currently; I also live on a very limited fixed income so need to find in-Vegas and/or freeware/free plugin solutions. Whatever the solution, my brain is not equipped to use command line/script solutions -- I need a GUI way :o)

OK, on to the subject at hand...

The Core Problem

The granularity is of the RGB kind so the only way to wipe it out is to go for a kind of monochrome look, and that is what you will see, though it still has the grain, just not as off-putting as the RGB kind that gives it a kind of webcam look. But in that monochrome-ing, I lose range.

So ideally, I'd like to be able to still use the full color range but greatly reduce said granularity.

As Johnmeyer here once suggested, this kind of subject lends itself well to the softening or "Vaseline on lens" look, so I can kind of cheat by using that as a way to reduce granularity. But it's also not the best solution either -- for one loses crispness/detail too.

I created a full-size (1280 width x 3200 length px) composite .jpg of in-Vegas screenshots so as to show all this as accurately as possible. I also chose the most zoomed-in and hence grainiest part of the clip since that is were the flaws show up the most.

Here is what you will see in this big .jpg in descending order:

1. FX chain used for this clip.

3. Pan/zoom window with right side of frame blurred for privacy concerns.

3. No FX/raw split preview of the one core FX that has kind of "saved" this footage to a small degree by softening things: Magic Bullet Movie Looks "Mescali".

4. Split screen of all FX applied vs. No FX/raw footage (left and right)

Thanks for you input and suggestions. Hopefully one or two of my favorite "teachers" here over the years will frisbee in their wisdom ... :o)

~ Philip


Comments

musicvid10 wrote on 1/25/2011, 6:22 PM
Everything I've read says you'll save this footage better with the filters available in VirtualDub. I hope johnmeyer checks in on this discussion. He probably has as good a handle on it as anyone.
Robert Johnston wrote on 1/25/2011, 6:47 PM
The best noise removal plug-in I have ever used in any application is Neat Video. There's a plug-in specifically for Vegas. About 100 US Dollars.

http://www.neatvideo.com/


Intel Core i7 10700K CPU @ 3.80GHz (to 4.65GHz), NVIDIA GeForce RTX 2060 SUPER 8GBytes. Memory 32 GBytes DDR4. Also Intel UHD Graphics 630. Mainboard: Dell Inc. PCI-Express 3.0 (8.0 GT/s) Comet Lake. Bench CPU Multi Thread: 5500.5 per CPU-Z.

Vegas Pro 21.0 (Build 108) with Mocha Vegas

Windows 11 not pro

johnmeyer wrote on 1/25/2011, 7:06 PM
I've spent almost ten years perfecting tools to deal with exactly this situation and could definitely fix your video. However, I understand that you want to do this yourself, for various reasons.

One approach was already suggested by Robert. Neat Video does a pretty good job with video grain, although I find that the temporal portion of their filter is not particularly good and therefore most of the filtering gets done on one frame at a time. This results in rather soft video but, given your style as you've already described (and as I've seen from some of the video you've shared with me), this may not be a problem. The plug-in is slower than snails s*ot, but for a one-off project, that's probably not a problem.

The second filter most people will recommend is Mike Crash's denoising filter. I've not used this within Vegas, and therefore am not quite sure what workflow is needed when dealing with interlaced video. The original filter that Mike ported to Vegas was created for VirtualDub and was designed for progressive video. I'm pretty certain that several people have published workflows that let you use it within Vegas with interlaced video.

Prior to using any denoising filter, you'll have to correct the levels. While you can use the Levels or Color Corrector fX to do this, the most powerful -- and best -- way to deal with levels is to create color curves. This lets you keep an appropriate dark level while "gaining" the lower-midtones. You generally want to keep the video looking somewhat dark because any attempt to make it look "normal" will result in something that either looks washed out or posterized. This usually isn't a problem because the viewer will sense that the video was taken in dim lighting and therefore won't expect everything to look well-lit. However, they do expect to be able to see faces, and that's where you should concentrate when doing the levels corrections.

I do all of my restorations using AVISynth scripts. I am sure this of no interest to you, but I also know that various people in this forum use this tool. For them, let me share the script I am using as I write this. This script not only removes noise, but also can remove vertical and horizontal chroma shifts (primarily useful for VHS dubs and anything else that's been subjected to the horrible NTSC analog jungle).

#Denoiser script for interlaced video using MDegrain2
#This is the recommended script for VHS as of January 25, 2011

Loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")
LoadPlugin("c:\Program Files\AviSynth 2.5\plugins\CNR\Cnr2.dll")

SetMTMode(5) #Only use this line if you have the multi-threaded version of AVISynth

#Modify this line to point to your video file
#Change AssumeBFF() to AssumeTFF() for HD video
source=AVISource("E:\frameserver.avi").killaudio().AssumeBFF()

SetMTMode(2,0) #Only use this line if you have the multi-threaded version of AVISynth

#Only use chroma restoration for analog source material
#chroma=source.Cnr2("oxx",8,16,191,100,255,32,255,false) #VHS
chroma=source #Use this line for digital video

#Set overlap (second parameter in MDegrain function) to 0, 2, 4, 8. Higher number=better, but slower
#For VHS, 4,0 seems to work better than 8,2. Most of difference is in shadows.
#However, 8,0 is good enough and MUCH faster. 8,2 doesn't seem to make much difference on VHS.
#Set the last number to 1 if the video contains flicker -- this makes script run at about 10% of the normal speed.

output=MDegrain2i2(chroma,8,0,0)

#stackhorizontal(source,output) #Use this line to do before/after comparison
return output

#-------------------------------

function MDegrain2i2(clip source, int "blksize", int "overlap", int "dct")
{

Vshift=2 # 2 lines per bobbed-field per tape generation (PAL); original=2; copy=4 etc
Hshift=0 # determine experimentally -- I've used values up to 12 (can be + or -)
overlap=default(overlap,0) # overlap value (0 to 4 for blksize=8)
dct=default(dct,0) # use dct=1 for clip with light flicker

fields=source.SeparateFields() # separate by fields because this function is for interlaced video

#This line gets rid of vertical chroma halo -- eliminate if this isn't a problem (i.e., for digital video)
fixed_fields=MergeChroma(fields,crop(fields,Hshift,Vshift,0,0).addborders(0,0,Hshift,Vshift))
#fixed_fields=fields #Use this line instead of the one above when doing digital video

super = fixed_fields.MSuper(pel=2, sharp=1)
backward_vec2 = super.MAnalyse(isb = true, delta = 2, blksize=blksize, overlap=overlap, dct=dct)
forward_vec2 = super.MAnalyse(isb = false, delta = 2, blksize=blksize, overlap=overlap, dct=dct)
backward_vec4 = super.MAnalyse(isb = true, delta = 4, blksize=blksize, overlap=overlap, dct=dct)
forward_vec4 = super.MAnalyse(isb = false, delta = 4, blksize=blksize, overlap=overlap, dct=dct)

#Change thSAD to increase/decrease de-noising
MDegrain2(fixed_fields,super, backward_vec2,forward_vec2,backward_vec4,forward_vec4,thSAD=400) #400=original setting
unsharpmask(60,3,0) #This provides "clever" sharpening that brings out detail without adding obnoxious sharpening artifacts
Weave() #Convert fields back into interlaced video

}




Soniclight wrote on 1/25/2011, 7:28 PM
OK, thanks for all responses and I welcome more by anyone in-the-inow. My brain is too fuzzy to take in all the info right now--especially from John, so I'll be back later or tomorrow when I'm in a better cognitive state :o)
farss wrote on 1/25/2011, 8:47 PM
Seems to me that the order of the FXs is maybe mixed up. Curves should go before colour correction.
Then there's a sharpen FX which is sure not going to help the noise and then even more FXs and Magic Bullet in the middle of the chain.
I'm thinking whatever "look" is required should be doable with less FXs and less bits being shuffled up and down which surely is not helping the noise problem.

One trick from our old mate Spot. Sometimes you can do colour correction by simply adding a colour. This avoids bringing up the noise. Just use Generated Media on an upper track and set compositing mode to Add. You cannot push this too far of course.

Regarding noise reduction in general. I believe it works best when done to the original footage, before scaling and before any FXs are applied. I've used Mike's DNR to great effect, mostly on EX1 footage. The EX1 can be quite noisy even at 0dB gain but it is very fine grained noise and Mike's filter works great on that. The HV30 being single sensor would have "chunkier" noise so I cannot vouch for how well it will work but the price is right.


Bob.
bsuratt wrote on 1/25/2011, 8:51 PM
Another vote for Neat Video.... slow but works wonders most of the time. Try first applying to the raw video before any other correction..
Lou van Wijhe wrote on 1/26/2011, 6:50 AM
Philip,

For a next shoot you might try the HV30's spotlight mode. This completely switches off the gain and the gain is what causes grain or noise in the picture. If the picture is too dark you can lighten it up in Vegas (without adding noise), f.i. by increasing the gain in the second Color Corrector. Sometimes you also need to increase the saturation as in low level situations you also start loosing color.

Lou
Randy Brown wrote on 1/26/2011, 7:17 AM
I would agree definitely at least try the trial version of Neat Video to see what it will do...I had some footage that was awful and it worked "magic" on it.
Soniclight wrote on 1/26/2011, 8:17 AM
Thanks again for further responses - but got some life priorities to tend to today (one being non-painful dentist visit) and so still can't respond yet in any substantive way.

That said, since Neat Video has come up more than once: it is out of my price range as per financial restraints stated in opening post, so I'd rather not even download a trial version (why test drive a cool car you can't afford? :o)

I'll be back when things are more scribble-friendly on my end.
Thanks.
Andy_L wrote on 1/26/2011, 10:15 AM
Have you considered trying to live with the grain?

Correct color and levels using sony fx to give things a nice look, and render out some footage. If it doesn't have to be digital-perfect-sharp, you may find the grain isn't too bothersome or even adds to the ambiance of the look.
Laurence wrote on 1/26/2011, 10:43 AM
Neat video does wonders, but boy does it make for some painfully long renders!
Soniclight wrote on 1/26/2011, 8:29 PM
OK, somewhat mentally functional (still a bit loopy after my rare chug of some White Zinfandel wine - I'm such an alcohol weenie/lightweight) so I'll try to respond in more depth here. Neat Video already covered.

johnmeyer

I have a couple of the mentioned Mike Crash plugins and/but they don't always improve a clip when applied, so it's kind of touch-and-go. As far as the AVISyth stuff, as you already know, my brain can't inhale that kind of command line stuff, so I'm chicken and don't even go there.

farss/Bob

Good point on FX sequences and something of which I still need to get more aware. Since I tend to use more or less the same batch of FX, allow me to ask you to re-arrange these in an order that makes more sense. I do intersperse still .jpg images with my 60i stuff from my HV30, so that it also something to consider.

Below is a text simile of most of the ones in the screen-shot -- just copy/paste them in a more sensible order. I don't use Magic Bullet Movie Looks that often, but it would still be useful for me to know where it should be in the chain when/if I do use it.

[Color Corrector] - [Color Balance] - [Color Curves] - [Sharpen] - [Magic-Bullet Movie Looks HD] - [Smart Denoiser] - [Color Correction (Secondary)]

Lou van Wijhe

Spotlight Mode that switches off the gain and grain in the HV30. Thanks, definitely something to consider and try out the next time I shoot :o)

Andy_L

Living with the grain. Yes, I've considered that and am kind of working around with this in this particular shot by even considering adding the "scratches" of Film Looks (giving the footage the early movie looks of the silent films thing). Making champagne out of lemons, so to speak or turning a mistake into a stylistic asset.. But as the subject of this thread states, I do also want to be able to save or improve grainy footage.
Laurence wrote on 1/26/2011, 8:38 PM
Boris Continuum also has some noise reduction that looks pretty good in their advertising. I have BCC7 but haven't tried the noise reduction yet. Has anyone else? If so, how does it compare to Neat video? Is it as good? Is it any faster?
farss wrote on 1/26/2011, 11:08 PM
"[Color Corrector] - [Color Balance] - [Color Curves] - [Sharpen] - [Magic-Bullet Movie Looks HD] - [Smart Denoiser] - [Color Correction (Secondary)]"

[Smart Denoiser] - [Color Curves] - [Color Corrector] - [Magic Bullet Movie Looks]


I'm leaving Sharpen out because if you've got a noise problem then any Sharpening is very likely to make it worse.
I cannot understand why you have so many FXs adjusting color either. Less is more, try to use as few color correction tools as possible and get as much out of each one as you can in your situation.

It's been suggested that you could just ingore the noise and keep it as part of the "look". My only issue with that is when you encode your masterpiece for the web that noise / grain will eat up a lot of bandwidth and can turn into nasty looking artifacts. I'd also add noise is NOT grain.

Bob.

Soniclight wrote on 1/27/2011, 1:18 AM
Bob,

Thanks for simplified chain -- and... gee, thanks a lot for kicking me in the shins with the "I cannot understand why you have so many FXs adjusting color either. Less is more..." :o)

Truth is, I don't usually -- and I end up taking stuff out in the end or just disable them. I was just getting frustrated with this clip and got a bit carried away. My two usual color FX are one Color Corrector and one Color Balance (if needed). Color Curves is more contrast/brightness thing anyway.

Now, in my defense: as you know, I do have an artsy visual style, so having fun and experimenting with color is important. I'm not doing daylight TV-broadcast color adjusted documentaries :o)
Soniclight wrote on 2/4/2011, 10:53 PM
For those also with an HV30, this is what worked by far the best overall for me - grain be gone:

Provided above by Lou van Wijhe

For a next shoot you might try the HV30's spotlight mode. This completely switches off the gain and the gain is what causes grain or noise in the picture. If the picture is too dark you can lighten it up in Vegas (without adding noise), f.i. by increasing the gain in the second Color Corrector. Sometimes you also need to increase the saturation as in low level situations you also start loosing color.

So a BIG thanks for that, Lou :o)
farss wrote on 2/5/2011, 3:43 AM
That will work to some extent however you may well replace one problem (grain from the gain) with other problems such as colour banding and other digital artifacts.
At the end of the day and I know you don't WANT to hear this but you really should take it on board. You have to ask WHY is the camera using gain and the answer is dead simple, you are not giving it enough light to work with.
I can understand that if you have no control over the lighting but you do. Given that you're shooting in a small set then adding more light is incredibly cheap, one of the few things you can do pretty much for $0.00 in this business. Shooting properly exposed video with no gain i.e. with ENOUGH LIGHT gives you the maximum creative latitude in post.

Bob.
robwood wrote on 2/5/2011, 8:11 AM
Pretty much what Farss said.

I'd move the [Smart Denoiser] plug-in to the front of the chain; video cleaning should be first thing done. I usually use CC1, AAV6CC, and maybe Curves (esp for shot-matching) in some varying order; and maybe 2CC if there's a bunch'o'adjusts needed. Then MB Looks at end.
Grazie wrote on 2/5/2011, 8:51 AM
Ditto Rob. Nobody utilise Sony Levels anymore? I use it to give me a ball park on BnW.

Grazie

Lou van Wijhe wrote on 2/8/2011, 1:27 PM
I uploaded some shots from a holiday video to Vimeo (click here) to show the HVxx's low light capabilities in spotlight mode. As I understand it, in spotlight mode the gain (amplification) in the camera is switched off and the image is transferred as-is from the sensor to tape. Boosting the gain in post doesn't add noise, contrary to what amplification in the camera does.

The light inside the restaurant was so low that I could hardly see anything on the camera's LCD screen, yet the end result is totally acceptable.

Lou
farss wrote on 2/8/2011, 3:43 PM
"As I understand it, in spotlight mode the gain (amplification) in the camera is switched off and the image is transferred as-is from the sensor to tape."

Generally what happens when you switch a camera into Spotlight mode is it exposes for the highlights. In the example footage that'd be the lights and the well lit areas. As there's plenty of light there the camera simply doesn't need to add gain to get the highlights sitting at a reasonable level.

In the shot where the gent is looking through the menu ideally the camera should have been in Backlight mode i.e. expose for the shadows. That would get his face upto an acceptable level. Of course given the low light in the restaurant the camera is going to apply some huge amount of gain and you'll get massive amounts of noise.
Running in Spotlight as you have the face is seriously underexpsoed and I'd bet if you try to pull his face upto 70% or anything remotely correctly exposed in post you'll get some serious uglies happening.
The shots outside using Spotlight is fine although maybe shooting around 30 minutes early in the day would have been just a little better.
What you could do inside the restaurant if using an on camera light is not an option is to just get a candle or two. Anything in that situation throwing some light on the faces can make the world of difference. Faces matter, forget the rest of the frame.

Bob.
Lou van Wijhe wrote on 2/9/2011, 2:14 AM
Bob,

You are right in that the camera in spotlight mode exposes for the highlights. However, it also switches off gain. If in this mode you film in total darkness, this is what you get. As strong gain is what causes picture noise, using spotlight mode prevents it.

In the interior shots the couple (my wife and I, actually) were sitting in the darkest spot. Some light on the table (or changing to another table altogether) would indeed have been better. I never tried backlight mode in combination with spotlight mode, wonder if it works.

By the way, did you ever try the fireworks mode? Wonder what that does.

Lou

P.S.
I have another spotlight mode example here.
farss wrote on 2/9/2011, 4:09 AM
" I never tried backlight mode in combination with spotlight mode, wonder if it works."

As far as I know you get the choice of one or the other. The difference is in how the metering works, one works on highlights, the other on shadows. In theory if you had a flat white wall full frame standard, spotlight and normal exposure modes would give the same result.
We use spotlight with our HC5 for stage shows, the manual exposure on that camera is a bear to use, too much to expect the better half to cope with. It's good so long as there's no actual lights in the frame. Then the subject is under exposed and bringing that up is not good. I see the same thing happening in both your samples. There's loss of detail most noticeably in the faces. I suspect there is actually quite a bit of noise in there but the camera's noise reduction circuit is smoothing it out at the expense of detail. Unfortunately there is no perfect answer to not having enough light. Situation is not helped by cameras that don't give the user full manual control of everything.

All said and done though under the circumstance I think your choice of Spotlight in those locations was the best you could have made. The results are quite acceptable for the nature of what the video is.


[i]"By the way, did you ever try the fireworks mode?"[i]

Sure, on our HC5, again, no desire to lug the EX1 into the drunken NYE crowds. Works quite well. I think it uses a set exposure with a slow shutter to capture the fireworks.

Bob.