Remove color matting a.k.a. unmultiply

Tech Diver wrote on 9/6/2016, 3:57 PM

Amongst my collection of compositing elements such as fire and smoke, there are some clips that have been filmed or rendered against a black background. To remove this background (i.e. make it transparent) such that there is no residual dark band around the element, I have very successfully used After Effects with either the built-in "Remove Color Matting" effect or the Knoll "Unmult" plugin. In Vegas however this is more troublesome, as Universe 2.0, which includes Unmult, is no longer free and the built-in "Channel Blend" effect sometimes requires lots of tweeking to get right especially for multi-colored elements (though it works really well for fire). Anyway, does anyone know of a free plugin for Vegas to unmultiply the background color?

Peter

Comments

OldJack wrote on 9/6/2016, 6:03 PM

UseVagus Pro, Video FX, Chroma Keyer, and change the color to black.

Tech Diver wrote on 9/6/2016, 7:45 PM

Thanks for the suggestion OldJack, but I think that you (and perhaps others) might not be quite familiar with image background premultiplication problem and how Chroma keying is completely different from Unmultiplication. The latter is necessary where footage is pre-multiplied with another color (i.e., part of the alpha information is contained WITHIN the RGB channels). This typically happens with fire, smoke and illumination assets (glowing balls of light) where the subject matter is partially translucent.

Here are examples of why keying does not work. Below are images of the original fire asset (first image), the effects of matt removal using Keying (second image) and Unmultiplication (third image):

Notice that keying incorrectly leaves a dark outline around the flames, while unmultiplication correctly reconstructs the original flames as they were recorded. Again, keying and unmultiplication are different mathematical operations.

Peter

NormanPCN wrote on 9/7/2016, 12:06 AM

In Universe the old plug-ins are still there as "legacy" plug-ins. The legacy ones should(?) still be free as they were before. You may have to do an extra step to install the legacy plug-ins. Can't say for sure but I see the legacy plug-ins.

edit: Some detail from the Red Giant site.

"Customers who were using the Universe plugins before June 28,2016 will have access to the legacy plugins for as long as they use Universe. New users as of June 28 will only have access to the new Universe 2.0 lineup."

"Yes, as long as you signed up for the Universe download before June 28, 2016 and the release of Universe 2.0, all the free plugins will be located in the Legacy plugin group, and you may continue to use them."

Former user wrote on 9/7/2016, 6:43 AM

In Vegas you can change the alpha channel to straight, premultilied and premultiplied dirty. This will usually clean this up for me. 

Quitter wrote on 9/7/2016, 7:04 AM

For good results you must play with the Chroma Keyer sliders

Camcorder: Sony CX 520 VE
Hardware:   Acer NG-A717-72G-71YD, Win 11 , i7-8750 H, 16GB, GTX 1060 6GB, 250GB SSD, 1TB HDD
NLE:  Sony Vegas Pro 13.0 Build 453
            Vegas Pro 14.0 Build 270
            Vegas Pro 21.0 Build 300

 

Marco. wrote on 9/7/2016, 8:18 AM

Peter, doesn't regular compositing in Vegas do the job by selecting a proper compositing mode?

You'd just have to care for the black level is zero. In the pic posted above the black level is raised a bit which would break the compositing math, so you'd need an fx like "Levels" to correct black level before.

It's correct though you won't get desired result with Vegas chroma keyer. It isn't best tool for this certain job. Right as Peter mentioned above, in this case it is not a matter of how you'd tune the chroma keyer but a matter of the math needed to work in the correct way. Chroma keyer use a different math.

Tech Diver wrote on 9/7/2016, 8:55 AM

@DonaldT: the alpha channel settings work well if the clip has BOTH an alpha channel and a premultipled image. If there is only a colored background with no alpha, it does not work.

@Quitter: Chroma Keying is NOT what I want, as there is no way that it can unmultiply the background color regardless of the settings. Mathematically it is not the same (take my word, my doctoral thesis is in computer vision).

The following DOES work, but only for fire and smoke (assets that contain information in the Red channel). Apply the Vegas "Channel Blend" effect and set the transformation to the values:

Red    = 1.000   0.000   0.000   0.000   0.000
Green = 0.000   1.000   0.000   0.000   0.000
Blue    = 0.000   0.000   1.000   0.000   0.000
Alpha  = 1.000   0.000   0.000   0.000   0.000

or as indicated in the screen shot...

If you look at the first value in the last row you will see that I assign the red channel value to the Alpha channel to make the fire visible (revealing both solid and translucent regions) but leaving the background transparent.

If you have never used this effect you really should give it a try (especially those folks who keep suggesting the use of Chroma Keying). Unfortunately the vast majority of users do not understand what this effect does.

I thank all of you who have made suggestions to me. However, my problem is that I very specifically need an unmultiply function (not from Universe) that works on both red-containing assests like fire and smoke as well as on multi-colored assets like glowing balls of light (and please do not suggest Chroma Keying).

Peter

Tech Diver wrote on 9/7/2016, 9:07 AM

@Marco: Yes, you are correct that applying the Levels effect to crush the blacks and then compositing (with either Screen or Add) comes very close to doing the job. I have used that many times and it works in a great many cases, but not for all. For the tough ones, I switch to After Effects where I unmultiply the background and render a straight alpha, which I then use in Vegas. I can continue bouncing back and forth between appications, but it would be really nice to do this all in Vegas.

Thanks,
Peter

Former user wrote on 9/7/2016, 10:30 AM

Did you try the mask generator?

 

DrLumen wrote on 9/7/2016, 10:58 AM

I'm thinking there has to be a way...

Can you post a PNG or link to an image with the particular alpha channel intact?

intel i-4790k / Asus Z97 Pro / 32GB Crucial RAM / Nvidia GTX 560Ti / 500GB Samsung SSD / 256 GB Samsung SSD / 2-WDC 4TB Black HDD's / 2-WDC 1TB HDD's / 2-HP 23" Monitors / Various MIDI gear, controllers and audio interfaces

Tech Diver wrote on 9/7/2016, 11:55 AM

@DonaldT: Thank you, but again, mask generation is mathematically not the same as Unmultiplication. Using a Vegas  luminance mask of the fire on one track multiplied with the track below of the original black-backgound fire yields incorret results as can clearly be seen below:

 

The first image is the correctly unmultiplied fire, while the second image is the one multiplied by the luminance mask. Again note the incorrect darker edges of the fire. In addition to not yielding correct results, the mask approach requires two tracks while the unmultiplication approach is merely one effect dropped on a clip with no addtional tracks or track settings.

One might ask how we know that unmultiplication is the correct approach? Just take an asset that has both an alpha channel and is premultiplied against black. One merely has to set the alpha to Premultipled black in the Media tab of the clip properties to get the correct result. Call this Image 1. Now set the same property to alpha None and you will see a black background. Next, apply the unmultiply effect to the clip and get Image 2. If you mathematically subtract the two images (I do this in a small OpenCV app that I wrote), you will find that every pixel has a value of zero. That is, images 1 and 2 are identical and therefore Unmultiplication is the correct algorithm for color matting removal.

So again I ask if anyone knows of an Unmultiply effect for Vegas that is not part of a subscription package (such as Universe). Note that you can no longer install the free Universe 1.6. Fortunately, I have one machine that does have the old version but my others do not. For those I use the Channel Blend effect that I descibed ealier in this this post.

Peter

Tech Diver wrote on 9/7/2016, 12:01 PM

@DrLumen: Yes there is a way (for fire and smoke). Use the Channel Blend effect with the parameters that I detailed earlier. If you have not yet tried it, I suggest you do to see what it does. As for a test image, are you looking for an asset with both an alpha and premltipled black background or the same but without alpha (or both versions)?

Peter

Edit: I just want to add the the Channel Blend method that I describe IS unmultiplication. However, it can only work if you use just one of the RGB channels for the Alpha information. The true unmultiplication algorithm uses the greatest value of R, G, or B at each pixel. So if the Channel Blend effect would allow the entry of equations such as "max(R,G,B)" instead of just numerical values, all this would be solved.

DrLumen wrote on 9/7/2016, 4:03 PM

Nevermind. I was thinking it may take a few tracks but apparently that is not acceptable.

intel i-4790k / Asus Z97 Pro / 32GB Crucial RAM / Nvidia GTX 560Ti / 500GB Samsung SSD / 256 GB Samsung SSD / 2-WDC 4TB Black HDD's / 2-WDC 1TB HDD's / 2-HP 23" Monitors / Various MIDI gear, controllers and audio interfaces

Marco. wrote on 9/7/2016, 6:53 PM

The one I now found which should work exacty the way supposed is HitFilm Demult which can be set to MaxRGB. But this is part of the HitFilm Ignite fx collection, not a freeware plug-in.

Tech Diver wrote on 9/7/2016, 7:56 PM

Thanks Marco. I was not aware that a Demult function is available in Ignite -- I will check it out. If there is no free version out there, I rather pay a one-time fee for the HitFilm package than a repeating subscription for Universe.

Peter

karma17 wrote on 7/11/2018, 5:21 AM

Sorry for re-visiting this old of a thread, but I was experimenting with adding an explosion from Detonate films to an HD clip I had. I was trying to keep everything in Vegas and didn't want to use AE or another program if I don't have to. I just wanted to make sure I understood what the best way to do this is. The problem with the stock explosion effect I have is that the stock effect has no alpha channel and only a black background.

Can you please tell me if this is correct?

1. Add the explosion effect on track 1 and background on track 2.

2. Set the compositing mode on track 1 to either Screen or Add. For me, screen seemed to have more detail.

3. On the explosion effect on Track 1, add the Channel Blend and Levels filters on the clip itself.

4. Set the channel blend as shown above and adjust levels to bring the blacks down as much as possible.

And that's about as good as it can get with the tools that currently exist inside Vegas, is that correct?

Any other suggestions or advice is really appreciated. I am still new to all this and trying to understand the best way to add stock effects.

Since Add or Screen blend modes are impacted by lighter tones, I notice the explosion washes out when it goes against the sky and becomes almost translucent. I've been darkening the sky trying to lessen that. I also found pumping up the Alpha on the red channel beyond 1.0 seemed to help too.

Thanks!!

St3v3 wrote on 7/11/2018, 8:57 AM

HitFilm Ignite Express (free) contains the demult effect usable in Vegas and lots of other things. That might be easier to use.

karma17 wrote on 7/13/2018, 1:04 AM

Thank you very much for the response. I finally figured it out all inside Vegas. The answer was adjusting the alpha channel on Channel Blend but not the red one, the "alpha alpha" channel, 4th column.