Smart Mask (AI assisted automatic masking) Introduction

fr0sty wrote on 1/23/2023, 12:03 PM

This is an incomplete feature that is being released in Beta form. We need your feedback to continue its development.

Here's his introduction written by VEGASPascal for the new smart mask feature. Please check it out, let us know what works and what doesn't for you! New updates will continue to be added to improve this functionality as it comes out of beta in the future.

Hi Community,

This is a very early drop of the "Smart Mask" feature. The status of the plugin is "work in progress" but we want you to give a first impression of the feature.

Here some quick notes:

this is only an overlay plugin... nothing happens in the render process (yet)

the goal is to create a mask the user can send to the Bezièr plugin

AiA plugin (artificial intelligence assistant): user is still able to improve bad Ai results in the edit mode of Bezièr

it is a CPU only plugin (GPU precision was too bad)

new overlay function of VEGAS (OFX draw suite)

known bug: After the first time you press the "Analyse" button nothing happens in the preview. You have to restart VEGAS and start again.

Info for the plugin

Analyse: Starts the analyse process (object detection) for the current frame

ShowLabels: Shows the confidence score and the class label for each object. (only visible after analyse process)

Confidence: Filter for objects with a given confidence score (only visible after analyse process)

Shape: Improve the contour of the selected shape (only visible after selecting an object)

Transfer: Send selected shape to Bezièr plugin (only visible after selecting an object)

Boolean parameter in timeline shows the analysed frame (only visible after analyse process)

We hope you enjoy the plugin.

 

Last changed by fr0sty

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

Comments

Cmin7669 wrote on 1/23/2023, 5:58 PM

I wonder if the Vegas team is headed down the right track here, I have recently looked at the Da Vinchi Resolve's new masking tool and ask is this the better way to go. Just wondering, maybe Vegas will get this right. I have worked with the "Trace Mask" with the Pen Tool in After Effects and that is very good also.

.

I will keep my fingers crossed for this plugin, as the possibilities are endless.

fr0sty wrote on 1/23/2023, 7:20 PM

It's literally the same thing, only that VEGAS' plugin isn't complete yet, it's still in beta form. Many planned features are not implemented yet.

Last changed by fr0sty on 1/23/2023, 7:20 PM, changed a total of 1 times.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

Cmin7669 wrote on 1/23/2023, 10:22 PM

Well I had a fiddle with it and maybe it's me but found it hard to get it to do anything useful at all. But as I said I'll keep my fingers crossed!

The DaV Resolve method is very intuitive and simple to use, I only hope Vegas takes the same approach and not over complicate it.

fr0sty wrote on 1/23/2023, 11:04 PM

There are numerous UI improvements planned, this is just the raw infrastructure that makes it work being tested. It will improve, and your feedback is important to help with that.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

paul-marshall wrote on 1/24/2023, 6:21 AM

Potentially very useful. Got it to detect and motion track a teddy bear that just happened to be in the first clip I tried, but disappointed that the AI could not recognise a vehicle registration plate. Increasingly I need to blur out a car number and that always seems very tedious.
It would identify the car and I was very impressed by how it could identify a face inside the car even though the contrast (and resolution at this point) through the windscreen was very low. There was one problem - I could not select the person inside the car so a way is need to step through 'nested' objects. Very promising start though.