Smart Mask (AI assisted automatic masking) Introduction


Howard-Vigorita wrote on 2/5/2023, 9:15 PM

you were mentioning shooting infront of a uniform color to help key, did you do that here, or is the AI actually seeing a black background? In my experience black backgrounds don't work well if your subject has areas of low contrast on the boundaries (which makes sense) but I also see why someone would try with a black background, flaws in tracking not so obvious. I've given up on black/dark backgrounds so many times when using AI masks due to strobing issues. Try a well lit background and foreground. Good to see your work flow and mini tutorial

@Todd-A0 In most of their stage show there's usually lighting effects in the background but they chose to do this little brass diversion against an unlit backdrop. You're right though. Knocking out blacks often doesn't work with shadows and black mic stands all around. But it might have worked here to isolate all the performers since I knocked out to the original scene on the track underneath. But to knock out everything but one performer could only be done with a custom mask. Could also turn the group into a musical analog of Fantastic Four. By isolating each performer separately on their own track and applying a different special effect to each one.

Mindmatter wrote on 2/6/2023, 4:42 AM

FWIW, I've had pretty good results using a huge white background lighted to 100%, which then served as a luma key. It was on a project with many kids on a 6m stage where green screen was not an option. It worked so well that I started to wonder if it wasn't a better allround solution than green screen. It has its own set of issues though.

The only critical part was when the light on the kids in some scenes was not super optimal or when their clothing was too bright, in which case ther would be some transparency issues. The shadows they cast on the floor were also a problem in certain scenes.
This was the production, all keying was done with a luma key in the white background.

Last changed by Mindmatter on 2/6/2023, 4:50 AM, changed a total of 1 times.

AMD Ryzen 9 5900X, 12x 3.7 GHz
32 GB DDR4-3200 MHz (2x16GB), Dual-Channel
NVIDIA GeForce RTX 3070, 8GB GDDR6, HDMI, DP, studio drivers
7.1 (8-chanel) Surround-Sound, Digital Audio, onboard
Samsung 970 EVO Plus 250GB, NVMe M.2 PCIe x4 SSD
be quiet! System Power 9 700W CM, 80+ Bronze, modular
2x WD red 6TB
2x Samsung 2TB SSD

paul-marshall wrote on 2/7/2023, 4:59 AM

You're better off using VEGAS Motion Tracking to do that, it's better designed for motion tracking non-huma/animal objects. Tutorial here:

Thanks for the tutorial (all the tutorials) - thats how I do it at present, but you asked for thoughts on the smart mask and it has the potential to be easier with less clicks and an easier workflow than via Motion Tracking, which still needs to make a mask. I like the way the AI creates a mask that matches the shape pretty well. So then select Bezier Mask/General options and motion track it. Select Mask FX. Add Blur effect and move blur to before the Mask and it's done!
In my brief play with it, it found many different inanimate objects so I dont see why signage could not be one of them!

Roger Bansemer wrote on 2/8/2023, 8:55 AM

I've watched the posts and tutorials here on "smart mask" but they all really seem to be about other masking procedures and not the smart mask. When I use it I see a rectangular outlines around objects and I can then create a very rough mask on that object. Where do you go from here? I'm hoping someone will post a short tutorial strictly about how to use the "smart mask". Thanks

Howard-Vigorita wrote on 2/8/2023, 11:27 AM

The update notes say Smart Mask is unfinished. Making the outline and transferring to Bezier is all there is right now. From there you use Bezier as before.

fr0sty wrote on 2/24/2023, 10:14 PM

Mindmatter wrote on 2/2/2023, 9:09 PM

If I understand this feature correctly, it should be able to replace green screen keying at some point, right? I suppose it should be pretty easy when the subject is clearly distinct from a flat background.

Unfortunately based on the Resolve experience, the answer is No. it is not a replacement for a green screen. The reason is you can perfect a single frame, but when the tracking occurs you can get a different result on every frame, assuming a moving subject. It just depends if you would not be embarrassed by a Zoom meeting type background replacement in the video you were producing. Repeatedly with Resolve I've had to ditch their AI masking because if it's not perfect it looks amateurish.

Color grading is a better task, say you wanted to alter exposure on your subject but not affect your background, or to separate your subject from a background maybe you reduce exposure or blur the background. Badly tracked frames aren't' as obvious when it's not a background/foreground replacement but a color grade instead. @Mindmatter

This is especially true if you slightly feather the edges of your mask, further hiding any imperfections.



AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10


ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)