Getting rid of macroblocks in dark areas

moshem wrote on 2/2/2002, 11:08 AM
Hi,

When encoding source that contains small artificats or even a clean source with no artificats sometimes you get near perfect video but when dark scense are encoded you get ugly big macroblocks especially if you are encoding with Windows Media 8 or Divx which otherwise produce a very good result considering the low file size.

Is there a way in vegas to get rid of this problem or ease it a little ? Using VirtualDub you can usually use SmartSmoother or TemporalSomoother to get some reduction but never get over it , what is the Vegas equivalent or better solution ?


thanks

Comments

wvg wrote on 2/2/2002, 11:45 AM
You could try VERY SMALL amount of Gaussian blur. Like maybe .005 or less.
moshem wrote on 2/2/2002, 11:57 AM
Thanks , but even if this works won't it reduce the entire video sharpness ? the nice thing about virtual dub is the fact that it will only take care of the macroblocks and will not blur the entire video - as explained here :

http://sauron.mordor.net/dgraft/smooth.html

BTW , can VV3 use filters made for virtualdub ?
wvg wrote on 2/2/2002, 12:03 PM
Sure, it is a tradeoff, which is why I suggested you try a very small amount. You could run your files through VirtualDub for preprocessing, then continue on with your project in Vegas Video. I've done that a few times because of filters VD has that VV doesn't.
moshem wrote on 2/2/2002, 12:15 PM
I just tried it blurred the video & did not solve the problem.

I can't understand why an high-end solution like VV3 can't do what a lightware freeware can so efficently.

How about using virtual dub's filters in Vegas , is the possible ?
wvg wrote on 2/2/2002, 3:51 PM
If you can, post a small snip of the video on a web page so others can see it. If the old adage a picture is worth a 1,000 words is true, then a small portion of a video is worth 10,000 words or a dozen forum posts going back and forth and getting nowhere.
moshem wrote on 2/2/2002, 4:39 PM
Ok ,

I uploaded a 10-frame DV clip.

you can get it zipped here (1 MB):

http://www.geocities.com/moshem/dvmacroblock.zip

this is a PAL DV clip that was captured off TV using my Sony camcorder , notice the wierd animal on the left and notice the black portion on the right side of the video, there are artifacts that are hardly noticable on PC Monitor but more noticable on a TV.

Of course that compressing this video using something like MPEG4 variant will create a much worse artifact from these on the DV original. This is the reason I want to clean it first.

can you help ?
wvg wrote on 2/2/2002, 6:05 PM
I downloaded your clip. Wierd animal indeed. Anyhow, I was hard pressed to see any artifacts on my flat panel monitor. So I took a snapshot then opened it in Photoshop then enlarged it just so I could see the artifacts. I then moved my cursor around the problem area to see how far from true black the background is. I got average RGB values of 34,23,16. I then adjusted the brightness FX in VV to -.03 and cranked up the contrast to .10. I took another snapshot and this time in Photoshop the RGB values went down to 16,8,1. This made the macroblock noise disappear into the blackness.

So you may want to experiment with adjusting the brightness and contrast values. I'm afraid the encoding is still going to result in some background noise no matter what you do. Now that you got me curious I took a look at a old video clip I had from a commerical DVD of some Hubble telescope space shots. If you look close enough you can find some very slight artifacts, even on what I would suspose should be a totally dead black background.

There's another thread called Levels filtering..... by Sonic EPM that may up hide those macroblocks.
FadeToBlack wrote on 2/2/2002, 6:44 PM
moshem wrote on 2/2/2002, 7:54 PM
and it did manage to take out the artificats but it also darkened the entire video and thats not acceptable.

I am a newbie and the Levels effect was not too clear for me , When playing with the Levels I got the entire image colour & lumin. distored and could not find a combination that will take out the artifacts and leave the rest of the video untouched. I also tried searching for the thread you mentioned in the archives but could not find it.

can you help ?
moshem wrote on 2/3/2002, 1:46 PM
any help ?
wvg wrote on 2/3/2002, 3:50 PM
What I was referring to follows my comments. You last said: "...and it did manage to take out the artificats but it also darkened the entire video and thats not acceptable."

Since you say you're a newbie to video editing my best suggestion is stop trying to seek perfection, because you're not going to get it. For starters the video snip you put on the web site and I looked at and fooled around with in an effort to help you in my estimation "as is" would be fine for 99% of the people here.

Nitpicking is fine, I do it all the time. Look up nitpicker in a dictionary and you'll see my picture. Seriously, you need to learn that applying filters, fixes and tweaks is mostly a matter of trade-offs. You fix one thing, good chance you alter something else. Slightly. That's how it is.

If you want to hide some VERY MINOR background block noise one way it is cut the brightness and crank up the contrast. Slightly. As you saw it works. Complaining it also darkens the whole video and saying that's not acceptable is simply not being realastic. If applying the fix makes the video SLIGHTLY darker, and you don't like the result then lean to live with the VERY MINOR artifacts. Like I said, a trade-off. I'm not trying to lecture you. I'm just passing along my experience. If you are a perfectionist, video editing can be extremely tedious IF you don't accept the limitations of the medium, along with the present state of hardware and software. If you have an extra $10,000 or so laying around a hardware based encoder may give you slightly better results. Your call.

This may help by adding additional color saturation.


from SonicEPM 1/30
Some of you have noticed washed out color when converting DV to MPEG-2 or streaming formats. You might try adding the "Levels" filter to the entire project, settings:

input start: 0.063
input end: 0.922
output start: 0
output end: 1
gamma: 1

Turn this off when rendering to DV.

We'll be adding this as a preset in the upcoming 3.0a update, preset name "Studio RGB to Computer RGB".
moshem wrote on 2/3/2002, 4:08 PM
Hi,

thanks for the explanation, I am happy to get any tips I can.

I also think that the video I put on the website has very minor MPEG noise but that noise is greatly amplified (x20) once encoded to windows media or to divx4, if you like I can put a clip I encoded using divx4.

That is the reason I am trying to get rid of the artifacts on the original DV version first.

I also experimented by doing the brightness trick you offered and then saving it to a second DV file. Then I loaded the second DV , reversed the contrast and brightness and renderd as DIVX4 , the result was a nearly clean DIVX4 version - much much better than the the one got with direct DV->DIVX4 conversion.

The only problem with this method is if I have a 20GB DV file , I need to find disk space for another 20GB DV file, is there a way to do this process on the fly without saving to another DV version ? (using VV3)


thanks for your help
wvg wrote on 2/3/2002, 4:17 PM
What template and settings are you using when you render and how are you ultimiately going to use the finished video? Just to play off your computer, web site, burn a CD, what?

For testing I assume you know you can just render a very small area of the timeline by selecting and then selecting render loop region only from the render menu.
moshem wrote on 2/3/2002, 4:36 PM
Yes , I know that .

My goal is to get analog sources like TV shows, VHS Tapes, Old analog camcorder tapes all of which are 30-2 hours in length and transcode them to DV using my SONY camcorder , then I would like to create a copy on a CD or two , probably using those fantastic codecs like MPEG4/DIVX4. The method I described works very well and removed almost all the artifacts in the resulting DIVX4 file and keeping the original colours and brightness/contrast --- BUT I need to have twice the space for DV file storage - that is too much space. What I need is find out if I can do this process on the fly using VV3 :

1. Open the original DV
2. Reduce the brightness & increase contrast
3. Take the resulting video and add brightness & reduce contrast
4. Render as MPEG4/DIVX4

thanks very much for your help
FadeToBlack wrote on 2/3/2002, 5:57 PM
moshem wrote on 2/3/2002, 6:56 PM
Hi,

right now I am talking about converting PAL TV signal converted to DV via a SONY camcorder ,the quality is quite good except for some minor artifacts which get amplified when coverting to MPEG - otherwise the quailty is excellent both in the DV version & the final MPEG version , so I am not asking too much , just a small imporvemnt on the DV source in order to get a much bigger improvment on the MPEG version.

I kind of found a solution , but I need to find a way to do so on the fly without saving a second DV copy to disk , can you help ?
moshem wrote on 2/4/2002, 5:18 AM
Hi,

(for some reason I was not able to reply to your post)

Thanks for you help& tips , as I said before , I have found a way to greatly reduce the artifacts even in the resulting MPEG version, the problem is disk space. Can you or someone else help me with an explanation on how to create a VV3 project that will do the same on the fly without requiering me to save an additional version of the DV file.

thanks
wvg wrote on 2/4/2002, 10:27 AM
I think we're stuck on terminology. More or less everything you do in Vegas is "on the fly" or accomplished in real time. All that does little good if you don't render out and save the video, once you apply effects, filters, etc.. So if you've found a solution that works for you then the next step is to re render. There are no shortcuts. If you don't have space on your hard drive for both versions then you need to decide if or not making a new one is worth losing the older version. Doing video editing does consume lots of disk space. More so if you render multiple versions all in high bitrate formats.
Chienworks wrote on 2/4/2002, 6:25 PM
Moshem, you can put the Brightness & Contrast filter in the effects chain twice. They will operate independantly of each other. This lets you accomplish both steps at once and will save you from having to render twice. It also has the added benefit that you can see the final result in the preview window without having to do the first render.
moshem wrote on 2/4/2002, 6:34 PM
Cheinworks , thanks , I allready figured it out an hour ago , and did a test.

This actually reduces the artifacts greatly - but not all, the problem is it is also eating away at other black or near black details that are not noise , I wonder if there is a more fine tool in vegas that will allow me to be more specific of what exactly to remove...
wvg wrote on 2/4/2002, 8:37 PM
Sure, how much time you got? Take a snapshot of each frame. Export to Photoshop. Use zoom to get in real close. Adjust one pixel at a time. Inport back into Vegas and complain about something else. Have fun. Bye.

Sorry, lost my patience. You expect too much. Way too much. A wise man knows what he can change and what he can't. You need to learn that you can't take high quality DV source material, compress the crap out of it and then expect the resulting file to look as good or better than the original. Geez, someone had to tell you.
moshem wrote on 2/5/2002, 5:38 AM
wvg,

Sorry to get you upset , I am just trying to find a winning forumla for converting DV video to on of the so-called "streaming formats"

The only reason I am trying so hard is that most of the DV video got converted to DIVX4 with exceptional quality , I managed to take the 22 minutes of PAL DV , crop & resize it to 640x480 and fit about 50-70 minutes on on 700MB cdr , the quality is very very good, the only problem is dark scenes which create very ugly macroblocks and scratch like artifacts - this is weird since if it is possible to preserve complex images and details - it should also do so for much simpler dark scene but it won't. So I kept trying.

I see now that this problem has no ovious solution ,maybe it is a codec bug , maybe a flaw in all MPEG codec , I don't know , I just wish it could be addressed by a codec fix or some sort of smart filter that would do a better job than the VV3 "black restore". Until then, the awesome potential of DIVX4 will not really show - it's a shame.

anyway, thanks for your patiece and help and sorry again for upseting you.

Cheesehole wrote on 2/6/2002, 6:45 PM
moshem, remember that you are dealing with highly compressed video. one of the many tricks used by the codecs is to sacrifice the parts of the image that are less likely to be noticed by the viewer. this same concept is used for highly compressed audio.

they spend less 'bits' on the parts of the image that are harder to see, and save the bandwidth for the brighter parts of the image that are more likely to be noticed and appreciated by the viewer. on the audio side, if you compress an mp3 file with lots of dynamics and a wide frequency range, you'll notice the compression a lot more than if it's standard music/voices.

these techniques allow the codecs to deliver video/audio of exceptional quality at extremely low bitrates. even professionally encoded DVD's have visible artifacts, but the benefits of being able to deliver a movie on a small disc are so great that no one minds.

so i think you are way off to say that the potential of these highly compressed formats will not show until the quality gets better. it's a trade off, and it's a good one.

good luck on your quest...

- ben