Real World experience of "Neat Video" denoiser?

Grazie wrote on 12/29/2007, 10:44 PM
Title says it all, but . ..

Q1: What has this over and above the freebie Smart Smoother plugin we have?

Q2: Have you noticed a reduction in file size? And has this helped DVD creation and playing?

Q3: Does it leave a "pancake-makeup" look/feel to the talent's face?

Q4: Anybody used this AFTER DeShaker?

TIA,

Grazie

Comments

Grazie wrote on 12/30/2007, 2:26 AM
http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=536179OK - I did my own research and found this.[/link]
Marco. wrote on 12/30/2007, 6:40 AM
I use Neat Video for a few month now. When I first used it I could not believe how well it works. Smart Smoother is a fine solution if you are looking for a quick working and cost-free tool. But you can't compare the results to those of Neat Video.

I never compared file sizes when it is about DVD creation and I did not yet use Neat Video after deshaking. But I know it does something like magic to old VHS stuff or to low light video or any other kind of noisy video. It's faaaaaaar beyound simply blurring the noise.

Marco
Grazie wrote on 12/30/2007, 8:05 AM
Thanks Marco - that's what I wanted to hear: "It's faaaaaaar beyound simply blurring the noise."

Excellent! I'm sold. Yah gotta a hot one?!?

Grazie
John_Cline wrote on 12/30/2007, 9:15 AM
NeatVideo seems to build a noise profile much like the Sony audio Noise Reduction plugin, then it removes just that noise from the video. It operates much differently and is much more effective than temporal noise reduction. NeatVideo does have temporal noise reduction as well, if you want to use it in conjunction with its main noise reduction capability.

Temporal Noise reduction, like Mike Crash's filter, works on the principle of looking for slight changes to the pixels from frame to frame, if it is under some threshold, it assumes these changes are noise and then blends those pixels across time.
Grazie wrote on 12/30/2007, 10:24 AM
Ah! Thanks John! So that's "temporal" = TIME!! I didn't know that - excellent. So is this the pancaking or flattening of pixels I mentioned above? This is the Mike Cash quick and easy approach? I like the analogy to the Sony AUDIO noise redux and its "footprint" of noise.

So how, in all that is Einstein, does this stuff do its thing?

I wonder if IZOtope is considering using their immense power and knowledge to dabble in this murky field of Video? Now THAT would be interesting?

Grazie

Harold Brown wrote on 12/30/2007, 10:49 AM
I used Neat Video on all of my Super8 film conversions. The main problem is that when you are using it, it cause Vegas to stutter on play back include shift-B. The renders are fine, the problem is on the time line only.

Q3) Yes it can leave a pancake look if it is over done

Here is a still of a before and after
http://www.stage6.com/user/ragtop86/image/549688/
http://www.stage6.com/user/ragtop86/image/551214/
Soniclight wrote on 12/30/2007, 11:14 AM
Grazie,

New to this topic (I'm learning just reading it) and was curious about the temporal thing. Found this at short but informative exchange at a DivX forum about using smart vs. temporal.

May or may not be useful, but my guess is that it seems to be applicable to most formats

johnmeyer wrote on 12/30/2007, 11:34 AM
I have used virtually every noise reduction product and technology. Most are either spatial or temporal. There is a third beast, however, and I have had FAR better experience with it than with anything else.

I am referring to an AVISynth plugin based on FFT (Fast Fourier Transform). This technology is used all the time for sound (including Sound Forge and Izotope), but is not normally used for video. However, it does a wonderful job.

With spatial denoisers, small objects in each frame are blurred, or at least combined in some way with their surroundings, without much care for what happened in the previous or next frame.

With temporal denoisers, each frame is compared to the adjacent frames, and if something is small and exists in only one frame, but not either the frame before or after, it is combined either with its surroundings, or is replaced with content from the adjacent frames. With the really exotic versions of this, the frame to be repaired is first "motion compensated" so that it aligns as closely as possible with either the preceding or suceeeding frame. This is done just for the purpose of getting a better comparison of what has actually changed, and also for picking what portion of the adjacent frame (or frames) to use for repairing the pixel. The frame is then returned to its original X-Y location after repair.

With FFT denoising each portion of the video is first changed into "frequencies," with the quick transitions from light to dark (if the FFT is done in the luma space) representing high frequencies. The high frequencies are then modified, and the results recombined into normal video. This is the same basic math and technology that is at the heart of JPEG compression, and JPEG compression is what is at the heart of MPEG (MPEG-2 MPEG-4, etc) compression.

Spatial noise reduction makes things look soft.

Temporal noise reduction will often yield the "pancake" face (sometimes called "clay face") effect you described. It also makes the video look like it was shot through a screen door.

FFT noise reduction will yield "mosquito noise" around objects which move against a sharply contrasting background.

Of the three, the FFT artifacts are far less (to my eye) distracting. What's more, the strength of the noise reduction can be turned up absurdly high, and the video will still be quite watchable, although over-doing any noise reduction is usually a sin worse than the original noise. I find the two other noise reductions become distracting, often before they really begin to do much to reduce the noise. Often, when people first use them, they are so happy to be rid of the noise that they don't look at the damage they have done to other aspects of each frame. As a result, over time, as they become more astute, they realize that the results are quite distracting. I'm sure all of you have had these same experiences in other aspects of video editing, where something you originally thought looked great now look so bad as to be offensive.

Not unlike wine, women, new cars, ... well, you get the idea.

If someone wants to post some problem footage (of a reasonable size -- i.e., small), I would be happy to post several noise-reduced examples. Someone else can use Neat or Mike Crash's excellent implementation of the well-known VirtualDub filter (I think Donald Graft wrote the original), or whatever noise reduction tools others may use.

While no single noise reduction technique is going to be absolutely the best under all circumstances (I use other things to remove dirt from film, for instance), I can almost guarantee better results with this technology.

For more information, use the search facility on this forum (if it works, which it often doesn't) and use "FFT" for the search term, restrict to the Vegas forum, and use my user name. I've posted the actual settings in the past.

I don't post much here anymore, but Grazie is a good guy and deserves the best.

Grazie wrote on 12/30/2007, 12:34 PM
Well thank you MR M! - You laid out the "story" so well even I understood it!!!

Whatever video noise redux I will be doing, will be light and experimental - "Just a dab'll do it" (anybody here remember Brylcreem?) - or was it: "A little dab will do ya!"

John you ARE a class act,

Cheers

Grazie
John_Cline wrote on 12/30/2007, 1:49 PM
JM,

Are you saying that NeatVideo is not FFT-based?

John
johnmeyer wrote on 12/30/2007, 5:39 PM
"Just a dab'll do it" (anybody here remember Brylcreem?) - or was it: "A little dab will do ya!"

From memory, here's the jingle:



But my memory is rotten, so that probably isn't it. I actually tried it once, back in 1965, about the same time I tried my grandfather's straight razor and chopped off part of my chin.

Are you saying that NeatVideo is not FFT-based?

Gosh, I thought it was temporal, but now that you ask, I'm not sure. I tried the demo and had a tough time figuring out the UI, so I probably never got the best results that this product has to offer. However, most other folks seem to "get" it right away, so I think it was just me being stupid.

I found this in the Neat forum:

Neat Video Forum post

and while far from conclusive, it seems to hint at Neat not being FFT.
Soniclight wrote on 12/30/2007, 5:57 PM
JM,

As usual, sparkling and useful clarity in your responses.

While a bit OT, may I ask why you stated..."I don't post much here anymore (...)"

I as many others always appreciate your input, and yes, have noticed that your posts have been sporadic. Perhaps other life priorities (such as actually getting work done). And/or if you do still participate in some forum somewhere else, I'd like to know where.

Whatever the case, you're like a favorite campus professor here and your tweed jacket and elbow patches are/will be missed.

John_Cline wrote on 12/30/2007, 8:32 PM
NeatVideo does have a temporal component, but the main de-noising algorithm is not temporal. It appears that it isn't FFT either. All I know for sure is that it works exceptionally well.
johnmeyer wrote on 12/31/2007, 12:00 AM
All I know for sure is that it works exceptionally well.

Then I should try it again. You always have the straight scoop on things.

While a bit OT, may I ask why you stated..."I don't post much here anymore (...)"

Partly because I spend too much time on it. Also, all I want to do is help, but at various times, some people seem rather disconnected from reality and, as a result, I asked myself why I should waste time making long distance calls to the Twilight Zone. Another contributing factor is the semi-broken nature of the forum software (takes too long to read stuff because the forum stalls out so often). There are other reasons, but they get into issues I have with the product marketing and strategy, and I have found it counterproductive to everyone to post those concerns here.

I have sent many dozens of emails to people since I quit posting regularly, and it seems to avoid the above problems, while still helping out. I will continue to do that, as time permits. I did post a week ago to ask help on a problem I had (and still have), and again today, because I've helped Bernie in the past, and he is a good guy (and my wife is a fellow Brit ...).

Grazie wrote on 12/31/2007, 12:33 AM
The Forum looks well attended and responded to. Maybe if JM is needing to revisit the NV abilities and re-try it, maybe the GUI needs a bit more clarity? Ah, a touch of Grazie-Proofing needed here!!!

Now, where DID I put my myopic spectacles?
Marco. wrote on 12/31/2007, 3:04 AM
"All I know for sure is that it works exceptionally well."

I can state this. When I first saw the demos on the Neat Video web site I thought these must be fakes. But then, after I learned to use it, it blew me away to see such good results. Whatever methods they use in the software - they work damned good!

I'm sure there are some other apps which does a similar good job but I can't believe there is much more improvement possible.

Marco
CClub wrote on 12/31/2007, 1:53 PM
Above, people have mentioned using the Neat Video software on VHS or Super 8 footage. I saw the example on their website regarding using with HDV... has anyone used with HDV and had as much success? I have several situations where I shot with HDV cameras with some low light moments and the gain needed to kick in, which created some noise.
johnmeyer wrote on 12/31/2007, 6:29 PM
I spent a little time with the Neat Video denoiser. It is primarily a spatial denoiser, with a very simple temporal denoiser added on. I only tested on one clip and was eventually able to get it to provide very good results on a clip with very little movement. I didn't have time to try it on dark DV camcorder footage with lots of movement. I suspect that it will not do as well (compared to the FFT3D filter), simply because of the technology being used. If I get more time, I'll try another test and see.

It is certainly, by far, the best spatial denoiser I have seen, so for clips with little movement, it seems quite good. (Edit) Although as JohnnyRoy noted, it is amazingly slow, even by denoiser standards (they are all very slow because of the processing required).
apit34356 wrote on 12/31/2007, 7:45 PM
"why I should waste time making long distance calls to the Twilight Zone", Well JM, I miss those moments when you tried to helped those on the edge of reality, but those who pass the the threshold of the BlackHole of the political insanity never do come back. But don't let spoilers rule! ;-) This forum needs your input and you help many other good members struggle thru the insanity.
Soniclight wrote on 12/31/2007, 11:38 PM
CClub said, "...has anyone used with

Once everyone has slept off their hangovers :) -- I'd welcome some follow-up on this in regards to de-noise in general since I don't deal with VHS or older formats, and will be getting an HD camcorder in '08.

I.e. how does denoise act differently or the same in HDV?

Thanks.
JohnnyRoy wrote on 1/1/2008, 7:35 AM
> ...has anyone used with HDV and had as much success?"

I used it on some low light HDV footage and it did a great job. In fact, the advantage of using it on HDV is that it requires a 128x128 pixel sample of the noise which is quite a large portion of the image for SD but a very small part of the image for HDV. So you can use a smaller noise sample relative to the entire frame with HDV.

It should be noted that Neat Video will significantly slow down your renders.I raised the levels on some dark footage on a 35 second clip and without Neat Video it rendered at 100% CPU utilization taking 04:48 to render on my Quad Core. With Neat Video I only got 50% CPU utilization and the same 35 second clip took 38 minutes to render. That's 60x slower! (but it does look great)

~jr

MarkFoley wrote on 1/2/2008, 7:46 AM
Here is a real world process...do this:

Take a short clip..drop velocity to 50%...render

Now do the same except add the neat video denoiser...render...compare the two clips

:-)
johnmeyer wrote on 1/2/2008, 10:20 AM
Now do the same except add the neat video denoiser...render...compare the two clips

I tried it. What am I supposed to see?
riredale wrote on 1/2/2008, 11:06 AM
JohnMeyer, I can sympathize with your frustration when on occasion a post here will go off on a tangent about politics and/or religion. I believe such rants don't belong here and serve no purpose other than to inflame strongly-held points of view on both sides. It has seemed to me that this often happens over the weekend, when forum moderators appear to go offline. Other that appealing to all forum posters to please ignore such topics, I guess the only other recourse is to hit the "Ignore" button for specific people, or to just go elsewhere.

I've always enjoyed your insightful technical comments, and hope that you can still find this board a fun place to hang out. At the same time, I can certainly understand that one can easily spend TOO MUCH time here, to the detriment of other more important tasks.

Signing off now to attend to those other tasks. Back later.