My Latest Denoising Tests (thanks Nick!)

johnmeyer wrote on 7/2/2011, 5:40 PM
Over in the doom9.org forum Nick Hope asked for some help denoising a few seconds of underwater test video. Here a link to that thread:

Removing low-light noise/grain from DV

Here is a link to Nick's original video (I hope you don't mind my posting this link, Nick):

Original Video

This is a 19 MB file.

I tried to help, but ended up stumbling across all sorts of problems. Believe me, you don't want to spend any time reading the entire post I linked to above. What you do want to do, if you are interested in what denoised footage can look like, is to download this final result, also in PAL DV format:

De-noised Fish Footage

This is also a 19 MB file.

Put the before and after videos on the Vegas timeline, underneath each other, match the project settings to PAL DV, and then use either the track mute or track solo buttons to A/B between them while they are playing

There is no such thing as perfect de-noising, especially with video that started out with this much noise, but having done a lot of this, I think the results here are pretty good. Many thanks to everyone at doom9 for helping me with this and, as always, thanks to Nick for coming up with a great technical and artistic challenge.

P.S. The denoising was done with a free denoiser that is part of the AVISynth plugin called MVTools2. After denoising, I did a little sharpening in Vegas, using the Unsharp Mask plugin.


[edit on July 6, 2011]
I eventually created a better denoised result, and posted a link later in this thread. Here is that link, in case you miss it below:

Fish Denoised with MDegrain3 and then sharpened with LimitedSharpenFaster script + Color Curves

[edit]January 11, 2015 - Here is a link to my attempt to denoise this footage with Neat:

Denoised with Neat Video


Comments

Jøran Toresen wrote on 7/2/2011, 6:32 PM
I did a quick test using NeatVideo (default settings and no tweaking) and I must say that NeatVideo does a (much) better job than your AviSynth filer.

Jøran
johnmeyer wrote on 7/2/2011, 7:44 PM
I would sure appreciate it if you could post the results of your Neat video test. Everyone says great things about it, but I just tried it with the default settings, and the results were really awful. I must either be doing something terribly wrong, or else we have different expectations of what denoised video should look like.

So, please post a DV PAL AVI version of the output. It should be a 19 MB file, just like the two I linked to.

Thanks!
johnmeyer wrote on 7/2/2011, 8:45 PM
P.S. When I did my Neat Video denoising test, I took my noise sample from the upper left corner of the video.
NickHope wrote on 7/3/2011, 1:59 AM
No problem with posting the file John but since the recent Dropbox rights grab it maybe better to start uploading copyrighted files elsewhere. All these sharing sites have similar TOS but Mediafire's are arguably less cheeky than Dropbox's.

I must admit I am also a little underwhelmed by your MDegrain3 output. Too much loss of detail for me.

Here is an effort using the following AviSynth script and encoded in VirtualDub with the free Cedocida DV codec. I think it's quite a good compromise between retention of detail and suppression of noise. Tweak EZDenoise for more or less denoising.

DirectShowSource("E:\batfish\Platax orbicularis-orbicular-batfish\LIBbatfish-orbicular1a.avi")
AssumeBFF()
QTGMC( Preset="Slower", EZDenoise=20.0, NoisePreset="Slow")
SeparateFields()
SelectEvery(4,0,3)
weave()
Jøran Toresen wrote on 7/3/2011, 5:51 AM
John, I'm a bit busy today, but I just did at new quick test (I did not save my previous test veg-file). I do not know how to upload files, but you can reproduce my (new) result doing this:

PROJECT SETTINGS
PAL DV
Rendering quality: Best
Motion blur type: Gaussian
Deinterlace method: Blend

Preview quality: Best (Full)

TEST A
Go to Frame 1
Open NeatVideo
Select Int[/b]erlaced
This is not the best you can achieve with NeatVideo, just a quick test. If you want to know more about how you can improve your results, take a look at the tutorials on the NeatVideo home page.
I view all versions of the file in Windows Media Player (full screen) on a 26 inch monitor.
Maybe we do not agree how the "best" denoised version should look like. For example, I think John Mayer’s version is far better than Nick Hope’s version.

Jøran Toresen
NickHope wrote on 7/3/2011, 6:36 AM
>> Maybe we do not agree how the "best" denoised version should look like. For example, I think John Mayer’s version is far better than Nick Hope’s version. <<

I am sure this is highly subjective, and also dependent on the use of the denoised clip. For example, what might look right on a computer monitor may look too soft on a CRT.

Mine is deliberately mild in order to help the clip match some others in my project that are a little noisy and that I will let pass. The strength of denoising can be increased with the EZDenoise parameter in my script. e.g. To 50.0.

I just felt John's was a bit blurry and lacking in detail. But I haven't burnt them to DVD to see how they look then.
johnmeyer wrote on 7/3/2011, 8:47 AM
Now let's see if I have this right:

I like Jøran's clip better than Nick's clip,
Jøran likes my clip better than Nick's clip;
Nick doesn't like my clip, but hasn't tried Jøran's clip

We all like our own clip the best ...

Well, if nothing else, it shows that de-noising -- and probably video itself -- is a very subjective subject.

The only thing that puzzles me is Nick's comment that my clip lacks detail. My definition of detail is the ability to resolve a particular structure or object. Some of these "objects" are very subtle, like the lines on the dorsal fins of each fish, especially the one in the foreground. Others are sharper, like the barnacles on the ship in the background (I didn't realize this was a ship wreck until I'd looked at it about 100 times). I didn't perceive that any of these details were lost in my denoised clip, and believe me, I did a lot of A/B comparisons, both in freeze frame, and while looping in Vegas.

But then there is sharpening, where contrast or edge enhancement (or both) is added in order to make detail more apparent. This doesn't actually add detail but instead simply makes it easier to discern details already in the video. Perhaps if I added even more sharpening, my clip would appear to have more detail.

If I have time today, I'll make a PAL DVD (I can now play them on my son's XBox into my big screen TV) containing all three clips and do a comparison. I'll try Jøran's settings and see if I can come up with something that looks good. I definitely am doing something wrong when using Neat because when I used that plugin, the detail was totally destroyed: it looked like a gaussian blur with a very large setting had been applied. I couldn't even see the lines on the dorsal fins.

Thanks for the critiques; it has been very useful.
craftech wrote on 7/3/2011, 9:56 AM
I like Jøran's clip better than Nick's clip,
==========
I don't see a link for Joran's clip John.

John

EDIT: Never mind. You rendered it to compare.
Jøran Toresen wrote on 7/4/2011, 6:00 AM
John, do you use the trial version of NeatVideo? If so, NeatVideo will only filter 640x480 pixels -- not 720xx576.

Jøran Toresen
johnmeyer wrote on 7/4/2011, 10:18 AM
Yes, the Neat trial only does 640x480, and it also adds a watermark. However, within the 640x480 box that it does process, it is very easy to evaluate the denoising. I created a noise profile using the upper left corner of the first of the two interlaced samples presented in the Neat video dialog. In the dialog that precedes the one for taking the noise sample, I accepted the defaults. After I created the noise profile, I did not change any of the noise settings. Here is the resulting video:

Fish Denoised With Neat Video (defaults)

To compare noise reduction, you really have to download and play each clip, along with the original, and then A/B/C/D them in order to see not only the details retained and how much noise is reduced, but also the "quality" of the resulting noise. What you often find is that the residual noise has been changed in character to something that is more distracting than the original noise.

Having said that, static comparisons are still useful. Here is a screen grab of one of the last frames on the clip, magnified, showing the original along with the three different methods of noise reduction presented so far:



Everyone should form their own conclusions. Here are mine:

1. The Neat video (lower right) gets rid of the most noise, but does so at a very high price. All the subtle details on the fish have been removed, and the details in the background are quite blurry.

2. The noise reduction I did with MDegrain3/Unsharp Mask does the second best job of noise reduction. The unsharp mask tends to add too much contrast, and you'll see that some detail is lost in the highlights in the background as a result. I used more aggressive settings for Unsharp Mask because people in the doom9.org forum seemed to prefer more contrast and apparent sharpness. If I had been doing it for myself, I would have used about half this amount of sharpening. While you cannot see the noise characteristics in the static shot, when played back, the noise in my clip looks (to my eye) more "natural" than the other clips.

3. The noise reduction done by Nick with QTMG does, by far, the best job of retaining detail. You can see this by comparing the upper right corner of the static shot. If you play back the shots and toggle between them, the difference is even more apparent.

However, much of the difference in detail between Nick's and my sharpened video is the sharpening that I applied. It actually removed some detail, even as it made the clip look "sharper." As I said, I did this excessive sharpening in response to various suggestions I received in the doom9.org forums and on the denoising examples that other people submitted. I was trying to match their tastes. However, if I were to stick with my original submission which represents what I would have done for myself, then the following is the result:

Denoised with MDegrain3, no sharpening

Download this, and then A/B with Nick's result. These two are much closer. Nick's clip still has a slight "edge" in detail, but mine does a significantly better job at noise reduction. He said that he needs to match other clips, so that will trump any advantage my approach may have in noise reduction. My other issue with Nick's result -- and you have to play the video to see this -- is that the "nature" of the noise has changed, and it has developed a certain coherence to the point that at times the noise almost seems to align into vertical columns. My goal when doing noise reduction is to have the residual noise still look like noise, and still look totally random. I think most viewers will find this less distracting.

Finally, I decided to try to see if I could combine Nick's better resolution with the better noise reduction done by MDegrain3. To do this, I used a much more complex post-noise reduction sharpening tool called limitedSharpenFaster. This is a complex AVISynth script, like QTGMC, but devoted exclusively to sharpening.

Here is one result, starting with my denoised, but unsharpened result, and then using that script with some very aggressive settings. The script must be used inside a "separatefields()/weave" combination in AVISynth, since the source is interlaced. I then realized that when you average out the noise, but have a dark background, the result is going to be too light, so I applied color curves to slightly darken the shadows. The same argument would lead to brightening the highlights, but the sharpening step seems to do that.

Here is a link to the result. I will be especially interested in Nick's reaction to this.

Fish Denoised with MDegrain3 and then sharpened with LimitedSharpenFaster script + Color Curves
musicvid10 wrote on 7/4/2011, 1:52 PM
Superb work John, Nick, et al!
NickHope wrote on 7/5/2011, 8:43 AM
Thanks John. Of all the ones you have posted, that last one is certainly the one I like best. I've been trying to replicate it here and been finding it difficult. Here's how my MDegrain3 script is looking, including the LimitedSharpenFaster line (which does work):

source=AviSource("E:\batfish\Platax orbicularis-orbicular-batfish\LIBbatfish-orbicular1a.avi")

denoised=MDegrain2i2(source,8,4,0)

return denoised

#-------------------------------

function MDegrain2i2(clip source, int "blksize", int "overlap", int "dct")
{
overlap=default(overlap,0)
dct=default(dct,0) # use dct=1 for clip with light flicker

fields=source.SeparateFields()

super = fields.MSuper(pel=2, sharp=1)

backward_vec2 = super.MAnalyse(isb = true, delta = 2, blksize=blksize, overlap=overlap, dct=dct)
forward_vec2 = super.MAnalyse(isb = false, delta = 2, blksize=blksize, overlap=overlap, dct=dct)
backward_vec4 = super.MAnalyse(isb = true, delta = 4, blksize=blksize, overlap=overlap, dct=dct)
forward_vec4 = super.MAnalyse(isb = false, delta = 4, blksize=blksize, overlap=overlap, dct=dct)
backward_vec6 = super.MAnalyse(isb = true, delta = 6, blksize=blksize, overlap=overlap, dct=dct)
forward_vec6 = super.MAnalyse(isb = false, delta = 6, blksize=blksize, overlap=overlap, dct=dct)
MDegrain3(fields,super, backward_vec2,forward_vec2,backward_vec4,forward_vec4,backward_vec6,forward_vec6,thSCD1=500,thSAD=600)

LimitedSharpenFaster(strength=500)

Weave()
}


I'm wondering what curve you used on it afterwards in Vegas. My output with MDegrain3 looks significantly different to yours.

I think that method is great for simply denoising DV for DV output. However personally I am now going to put all my DV through QTGMC on the way to publication, either as part of a PAL>NTSC conversion for DVD, or as part of an upscale to 960x720p for YouTube, so I think I'm going to stick with the denoising options inside QTGMC and refine how I'm using them. If you look in the QTGMC.html help file that came with the download, you can see that there are quite a lot of settings for both denoising and sharpness that can be tweaked.
johnmeyer wrote on 7/5/2011, 11:11 AM
Nick,

Here's a link to the preset (use Sony's preset manager to import) for the Color Curves:

Color Curves Preset to Darken Shadows

If you prefer, here's a visual snapshot of the dialog:



I copied/pasted your script, and got essentially the same results as when I first did it (the noise pixels are in slightly different positions, because I did the denoising first, then rendered, then did the sharpening second). You did it all at once, which is much more sensible. I did it as two steps only because I was trying to test out each phase of the restoration separately.


NickHope wrote on 7/6/2011, 12:45 AM
Thanks John but that's actually a link to the "denoised with neat" .avi.
johnmeyer wrote on 7/6/2011, 7:44 AM
Thanks John but that's actually a link to the "denoised with neat" .avi.Oops! Grabbed the wrong one. The link is now fixed.
mvpvideos2007 wrote on 7/6/2011, 8:33 AM
I have neat video noise remover and it works great!
johnmeyer wrote on 7/6/2011, 9:46 AM
I have neat video noise remover and it works great!If you have a moment, grab the original video from the link in the first post, and then de-noise with Neat and post the result. I followed the instructions given earlier, and posted my result, and I didn't think the video looked very good. I've posted my thoughts above. I did try to use settings other than the default, but was still not happy.

I'd love to see the results from someone who knows more about using the Neat Video noise reduction than I do.
NickHope wrote on 7/6/2011, 10:36 AM
Hmm... something wrong here. Even without sharpening, output from my script has loads more noise than yours. Sample. This was denoised with the script I posted above but without the sharpening and without your color curve. Compressed with the free Cedocida DV codec, which shouldn't be much different from the MainConcept one you're using. Somehow you've achieved a lot more denoising. Easy to see if you look in that triangular dark shadow.
plasmavideo wrote on 7/6/2011, 10:42 AM
I've used thr MSU denoiser in Virtualdub with good success in some footage, but this clip is tough. I can reduce some noise, but as John is seeing, the details sure disappear quickly when the filter is used aggressively.

If you haven't tried that one, give it a shot. I'm going to play with some more settings when I get a chance to see if I can do abetter job than using the presets.
johnmeyer wrote on 7/6/2011, 10:54 AM
Even without sharpening, output from my script has loads more noise than yours.Hmmm...

So, what could be different? I just checked your script, and it not only looks the same, but yesterday I actually copied/pasted it and used it, and as I indicated, except for very slight differences in the placement of noise dots (which I think was due to the fact that I did my denoising in two steps, with a re-coding in the middle), the two were virtually identical.

However, perhaps we are using different versions of MVTools2. Here is the version number from the MVTools2 DLL: 2.5.11.2. It is dated 3/20/2011. The documentation calls it "2.5.11.2 beta."
Robert Johnston wrote on 7/6/2011, 11:05 AM
Neat Video shouldn't be used in these tests. There isn't a sample area without detail large enough to give reliable results. I don't know if you noticed, but Neat Video tells you the reliability of the sample, which in the case of the fish video I downloaded, Neat Video couldn't get past 30 to 40 percent reliability. What is needed is a reliable noise sample from the same camera with the same ISO, zoom, and lighting conditions, but against a background without detail and with no variations in chrominance and luminance, as per the tutorial. You save that noise sample in Neat Video, then use it on the fish scene. You can use that sample to clean any other video where the same camera, zoom, iso, etc. conditions exist.

Intel Core i7 10700K CPU @ 3.80GHz (to 4.65GHz), NVIDIA GeForce RTX 2060 SUPER 8GBytes. Memory 32 GBytes DDR4. Also Intel UHD Graphics 630. Mainboard: Dell Inc. PCI-Express 3.0 (8.0 GT/s) Comet Lake. Bench CPU Multi Thread: 5500.5 per CPU-Z.

Vegas Pro 21.0 (Build 108) with Mocha Vegas

Windows 11 not pro

NickHope wrote on 7/6/2011, 12:20 PM
John, mine is the same MVTools2 DLL: 2.5.11.2 dated 3/20/2011 (although I don't see any reference to beta). I'll go through our scripts again and copy and paste yours in case I've made a mistake. Which codec did you render to in the middle and at the end? MainConcept DV? Sony Vegas DV?

The other thing is I'm on AviSynth 2.5.8 and I think you're on SET's multi-threaded build of 2.6?
johnmeyer wrote on 7/6/2011, 12:46 PM
Yes, I used the 2.6 MT AVISynth.