Anybody know of any exotic "video noise reduction" filters?

riredale wrote on 2/22/2003, 7:21 PM
Just curious: since MPEG2 encoding is clobbered by a noisy video image, is there some sort of DV Noise Reduction program? I'm not talking about just filtering the high frequencies (softening the picture), but rather some sort of program that compares frame to frame, keeping what is common and discarding the random noise.

I am convinced that one of the major reasons why Hollywood can get away with bitrates of just 4Mb/sec for beautiful DVD images versus the rest of us having to use 8Mb/sec is because the film frames are virtually noise-free when compared to typical DV.

Comments

JJKizak wrote on 2/22/2003, 8:08 PM
I agree but the software you are looking for if it exists is probably
in the six figure range. Might try NASA since all of their software is
created in house. Otherwise you are stuck with .002 gausian blur. Noisy
video is why I have not hooked up to cable. I have too many trees for
satelite. Sofo probably would have the best chance of coming up with
some superior algorithems but the main problem is you have to create
something that ain't there kind of like God. I am totally blown away
by what sofo did with the audio noise reduction. Applying techniques
as they do in data transmission (modem noise and tripple delay signals)
probably wouldn't help video but some kind of dual phasing technique
would. MTI video in radar systems was a very simple system before
they went to computerized front ends and very low noise-figure devices.
(Motion Target Indicator---anything in motion goes through but anything
not in motion is cancelled.) You need guys like Satish or the few that
work at MIT and Bell Labs. Most of the noise is "cosmic" and an algorithem
for that would net you millions and millions. There is also amount of
light and color and camera related noise to deal with and it is different
for each camera so you have to have ambidextrous algorithems working in
three dimensions. The old Tropospheric com systems had two transmitters
transmitting to four receivers with different frequencies, polarizations,
and spacing for quad diversity with the receivers (with parametric front ends)
all hooked together which provided a 3 db decrease in noise. They also had
threshold extension panels which cut the bandwidth in half and provided
a 3 db decrease in noise. Well, hope sofo comes up with something.

JJK
JJKizak wrote on 2/22/2003, 8:12 PM
PS---My knowledge of electronics stopped in 1976. I switched to maintenance.

JJK
musicvid10 wrote on 2/23/2003, 1:22 AM
riredale,
There were some plugins (3rd party?) for VirtualDub that targeted video noise. I didn't play with them too much but found one effective for a project of mine a couple of years ago. Poke around the links on their website and you should come across them.
seeker wrote on 2/23/2003, 4:30 AM
Richard,

Visual Infinity's Grain Surgery for Film & DV probably meets your requirements. It isn't available for Vegas yet. Grain Surgery is expensive. But I think it would be a good idea for SoFo to lobby Visual Infinity to make a version available for Vegas. For more information, see:

Grain Surgery for Film & DV Product Information

Grain Surgery Technology Demonstration

Grain Surgery Samples

-- Seeker --
roger_74 wrote on 2/23/2003, 6:26 AM
There are a few filters for VirtualDub that are very good if used moderately. They are free. There are even filters that deal with chroma noise only.

There are also some very good filters in the DScaler project for filtering on the fly from a TV-card (don't know if it processes files yet).
watson wrote on 2/23/2003, 9:33 AM
Grain Surgery is pretty amazing.
Good program.
Josh15 wrote on 2/23/2003, 9:58 AM
There is a superb filter called Convolution3D. It is actually an AviSynth filter (AviSynth is a utility the does simple editing tasks on video/audio files, and frameserves the result to another video application), and is included as a command in a script, which is a file with a AVS extension.

Vegas Video cannot open this script file directly, so we have to use a little utility calles "VFAPI Convert" that actually creates a fake AVI file that can be read in VV. This will allow us to use any AVISynth filter (and there are plenty wonderful filters there). Just a small note: The video editing process will be slower, because everything will pass through AVISynth and VFAPI Convert.

If you'd like to try it, you can find the utilities together with many guides and links here: www.doom9.org
Ritchie wrote on 2/23/2003, 12:08 PM
I would also recommend Convolution 3D or fluxsmooth, available through AVISynth. I have been truly amazed at what these can accomplish. These are smart noise reduction filters like you have requested. They compare nearby pixels, as well as nearby frames and use a statistical median with a threshold that you can set to try and determine if something is noise. Use them minimally, or you will get a painted look.

The Sonic Foundry Median filter is also supposed to be a similar filter, but it is very slow. For the AVISynth filters I have typically set up my avisynth file, opened it in VirtualDub and rendered it out to HuffYUV before using it in Vegas.

I haven't tried the plugin/wrapper that watson mentions with Vegas, though I have used it in other programs.

Both Convolution3D and Fluxsmooth are free, but don't expect a GUI in AviSynth, it is a text based script, but very very powerful. You will be very happy if you learn how to use this.

The Virtualdub filters also work well, but I believe there is a limitation in how many frames a filter can compare in Virtualdub, so the avisynth filters have a little more flexibility and thus better results (according to some). There are a few Avisynth filters that work only on the color channel, which is where most of the noise usually is, and where you won't notice smoothing as much as well.

Who knows, if there is enough interest, maybe SonicFoundry will add support to open .avs files directly from Vegas.
Rain Mooder wrote on 2/23/2003, 6:52 PM
Three years ago I wrote a temporal noise filter for virtualdub
and last fall I made a newer plugin for Video Vegas.

I thought about making it shareware, but this would take a
couple of weeks of polishing before I wanted to let it fly
the coup. How much interest is there in a semi-exotic noise
filter?

Also, it's a CPU killer and a memory hog...:)
JJKizak wrote on 2/23/2003, 7:36 PM
The Grain Surgery looks great and the $400.00 price is a bit much at this
time although I usually don't let that scare me.

JJK
riredale wrote on 2/25/2003, 1:14 AM
Just wanted to let everyone know that I moseyed over to VirtualDub and tried a promising filter called "Smart Smoother." I tried it at moderate levels and high levels of reduction. It is an interframe filter, and does not look ahead or back, but is quite sophisticated. The point is to reduce noise in flat areas while retaining detail.

After running a short noisy clip through this filter, I concatenated the three files and ran them through CinemaCraft Encoder. One nifty feature of this MPEG2 encoder is that one can look at a graphical representation of the bitrate and "Q" level after the first pass.

The results?--even using the "moderate" filtering level, the bitrate dropped from 8Mb/sec to 6Mb/sec for the same Q level. I was stunned, since a split-window viewing of the file showed no difference in sharpness, and the quality of the noise was only slightly different. But if this result is correct, it confirms my suspicion that MPEG2 is brought to its knees by video noise, and any efforts made to mitigate that noise level really pay off.
Ritchie wrote on 2/25/2003, 6:40 AM
I almost always use a temporal smoother with a spatial smoother. A temporal smoother works really great for MPEG2, since it can save the frame to frame compression that MPEG2 is based on. It works really well for low motion scenes. A spacial smoother can also help 'temporally' because the smoothing makes a pixel from frame to frame a little closer. Where a spacial smoother really helps is in those high motion scenes where there is very little temporal reduduncy. The smart smoother can really shine in your compression for these type of scenes. Fluxsmooth and Convolution3D are nice because they are both temporal and spacial, but again they are for AVISynth and little more difficult to get set up and play with settings. Smart Smoother has also been ported to AVISynth. There is also a 'filter' for avisynth that will try and detect if there is a lot of motion in the scene (or parts of the scene) and if there is it will apply the spacial smoother of your choice. If the scene (or section of scene) seems to have less motion it will apply the temporal smoother of your choice. I haven't used this since the above mentioned filters are both temporal and spacial. If you want, I can send you the avisynth file I use for most of my VHS filtering. For DV you would probably want to tone down the settings a bit. All you would need to do is change the path to your filter files, and the path to your source file, then you can open this file up in virtual dub and take a look.
seeker wrote on 2/25/2003, 6:59 AM
Riredale,

"It is an interframe filter, and does not look ahead or back, but is quite sophisticated."

Well, if it does not look ahead or back, wouldn't it be an intraframe filter? Perhaps the words "spatial" and "temporal" that other people are using for this are less confusing.

-- Seeker --
mcgeedo wrote on 2/25/2003, 9:41 AM
If you don't want to mess with it, I volunteer to "polish" it for you, if the sources have enough documentation for me to understand it.
riredale wrote on 2/25/2003, 10:49 AM
Seeker: you're right, I should have said "intraframe," not "interframe." Probably due to late-night Brain Fade.

Although I'm an engineer by training, I'm not a "compressionist," and I shouldn't get too deep in these dark and confusing waters. But I'll go in a little bit more (up to my knees):

MPEG2 works by doing a jpeg-like compression of a frame and then building successive frames by looking at the differences and encoding just those differences. After a certain number of frames the process repeats. I know I'm oversimplifying a lot but that's the basic idea. The primary compression method that is done is called DCT (discrete cosine transform) and the whole MPEG2 process is a very clever technique that is able to shrink a raw video stream by a factor of 20 or even more.

It's the edges of objects that make a scene a scene, and so MPEG2's task is to faithfully reproduce those edges. But random noise throws a wrench into the process, because how is the MPEG2 encoder to know just what is an edge and what is just a splotch of noise? If you or I were to look at an image, we could very quickly identify which features to preserve and which areas to smooth out. I guess this is what "Smart Smoother" and other filters are trying to do, in some limited fashion. Better to capture without any noise at all!
wcoxe1 wrote on 2/25/2003, 11:44 AM
I would be interested in your filter. Thank you for offering. Giving Satish a break here, aren't you?
Jason_Abbott wrote on 2/25/2003, 12:03 PM
potatophysics,

I'd be very interested. I have a lot of Hi-8 stuff with noise, and more and more I'm grabbing video off my brothers' inexpensive, noisy cameras. For now I'm using the median filter in Vegas.
RichMacDonald wrote on 2/25/2003, 1:15 PM
A guy named Brent Geery has a very interesting thread at rec.video.desktop. Basically, he captured his analog video 3 times and averaged it. He uses VDub.

First post:

http://groups.google.com/groups?selm=prp02v01n5o5bpcpi0q7mdhegc1b2fnv4m%404ax.com&oe=UTF-8&output=gplain

Full thread:

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&threadm=prp02v01n5o5bpcpi0q7mdhegc1b2fnv4m%404ax.com&rnum=2&prev=/groups%3Fq%3Dnoise%2Baverage%2Bgroup:rec.video.desktop%26hl%3Den%26lr%3D%26ie%3DUTF-8%26oe%3DUTF-8%26as_drrb%3Db%26as_mind%3D1%26as_minm%3D1%26as_miny%3D2003%26as_maxd%3D25%26as_maxm%3D2%26as_maxy%3D2003%26selm%3Dprp02v01n5o5bpcpi0q7mdhegc1b2fnv4m%25404ax.com%26rnum%3D2

One interesting idea was to take the 3 (or more) captures and use the median value, (i.e., discarding the "outliers" which would be expected to be noise), however, there isn't any software to do this right now.
Rain Mooder wrote on 2/25/2003, 3:06 PM
Ok, I will consider pulling up VC++ and working this filter
into a presentable form. It will be a few weeks before it
appears. Might appear as shareware for like $10 or so.
XOG wrote on 2/25/2003, 5:35 PM
PotatoP,

I'm down with your filter!

XOG
mcgeedo wrote on 2/25/2003, 9:11 PM
Josh,
If I understand what I read, you

(1) Create an AviSynth script that (a) loads your source media, and then (b) applies filters, etc. or whatever you write in the avs script.
(2) Preview the processing effect in some player (like Windows Media Player) that can open an avs file directly (calling the AviSynth dll).
(3) Run VFAPI Convert and create a VFAPI "project" that runs the AviSynth script (reads the frameserver) and outputs a pseudo-avi file that VV can open as media.

At this point, the pseudo-avi file appears as just another media file that VV can use, just more slowly than a real avi file. Can this file then be rendered out as a new, "real avi file" which can then be loaded into VV and operated on much more quickly?

Thanks, Josh,
-Don
Ritchie wrote on 2/25/2003, 9:48 PM
Don,

That is pretty close to what I do. In fact, steps 1 and 2 are identicle, but then I open the .avs file in VirtualDub and render it (save to avi) to something else. Most often I render to a .avi with the HuffYUV codec which Vegas Video will read just fine. Since the processing of avisynth is complete at this point, yes, it will run at normal speed in Vegas and not be slown down by the extra processing that avisynth would be doing out of view as the "avi" file is read.

However, this is just another extra step and the time of rendering out the intermediate .avi must be taken into consideration. The VFAPI approach lets you get right in and work with your video right away, rather than wait for the middle render. Quality wise, if you are using HuffYUV or any other lossless compression (or uncompressed), there shouldn't be a difference between the two methods.
mcgeedo wrote on 2/26/2003, 8:22 AM
Very cool, Ritchie. Thanks for your comments. After posting last night, I experimented a little more, and actually have processed a couple of VHS caps, and made significant improvements to them.

As far as the "extra" rendering step, I was doing that anyway. For the few projects I do with analog sources, I use a hardware codec to capture to mpeg. Directly editing mpegs has its own issues, as you certainly know, so I routinely render the caps to avi before starting edit work. This just adds a small amount of time to that process, so it's no big deal.

Thanks again, -Don