VEGAS 18 Artificial Intelligence Colorize Tests

fr0sty wrote on 8/3/2020, 6:17 AM

Here's a few tests I did. Some tips below on improving your results.

1. For some reason, this plugin works best if you put it BEFORE the pan/crop in the plugin chain.

2. From there, color grade to correct any imperfections as much as you can.

3. While I didn't do it here, rotoscope/mask objects out so you can isolate them, then run the colorize effect on just those objects/people, would likely produce the best results.

4. AI won't guess the right or most desirable color every time, so use hue shift to change them to your liking.

Reefer Madness:

Wizard of Oz (note, this last one was done on an earlier version of the plugin, results have improved since):

The tech behind this, since it is identifying objects in the scene and then guessing what those objects should be colored based on what it thinks they are, can also be expanded upon in the future to enable the automatic generation of masks, even moving masks! The AI already identifies "this is a person", "this is a tree", etc... so it just needs to mask around those objects afterwards to work (I'm sure that's WAY oversimplifying it, but the basic concept is there). So, I wouldn't be surprised at all if an auto-rotoscoping feature ends up coming to VEGAS in the near future.

Post your results and techniques here!

Comments

adis-a3097 wrote on 8/3/2020, 8:43 AM

Ha!

Now I can go colorize Citizen Kane and beat Ted Turner to it! 😆

VEGASPascal wrote on 8/3/2020, 8:56 AM

@fr0sty I believe you can tweak my result (I am not so professional). Try it for your own.

fr0sty wrote on 8/3/2020, 9:05 AM

I'd say that's a pretty solid result, I don't see much need to tweak that. It doesn't have that red bias that the video clips I've seen do. I usually have to try to correct that out after the fact. Still, the plugin has come a long way since I first saw it.

animagic wrote on 8/3/2020, 12:56 PM

Although this is certainly an interesting feature, I think it is a mistake to try and "improve" classic black-and-white movies by colorizing them...🤑

Musicvid wrote on 8/3/2020, 1:34 PM

The fascination is not so much in trying to "improve" them, but in trying to "sell" them.

Mohammed_Anis wrote on 8/3/2020, 1:41 PM

@fr0sty

As per your request, here's my attempt.

KaraUSA wrote on 8/3/2020, 2:53 PM

I try on this scene from Elephant Man

Steven-Sharkey wrote on 8/3/2020, 3:46 PM

I interviewed a graduate whose disertation was to produce software that colourized B/W footage - he ran it on football (Soccer) footage - quite interesting to know what it made of the different kits! He said he "trained" his algorithm then ran it on actual footage - it was doing well until suddenly the footage zoomed in and the algorithm decided all that skin tone must be the grass and gave people green faces!

fr0sty wrote on 8/3/2020, 4:02 PM

It appears to be a tricky effect to pull off, which makes sense given the usually poor condition of black and white footage that is being worked with.

There's actually a big market for this, though, I'm getting a decent response advertising the ability to repair and colorize people's old home movies. So, this isn't just a useless "filler" feature. There are legitimate uses for it that can make us some $$$.

As far as recommendations to the team for improvements.... as of now it favors a red bias way too much, so much that some scenes, it seems like all it does is give them a slightly more red than sepia tone, which I end up having to color correct out of it as much as I can. I'd like to see it choose more greens and blues for certain objects, especially clothing.

Bertrand-H wrote on 8/5/2020, 3:46 AM

@fr0sty I have the same problem with red tints and yellow tints depending on the sequence. You say you managed to correct it. What 'effect' do you use to correct it please?
I have tried with color corrector but no luck.
Choosing the opposite color to undo the red changed the color to the opposite instead of undoing the red effect for example.
Any tip?

fr0sty wrote on 8/5/2020, 5:55 AM

First off, you want to make sure that colorize is set before "video event pan/crop" in the FX chain. For some reason, this makes it work more accurately. From there, I used the color grading panel, secondary color corrector, etc. to manipulate the colors as needed.

Bertrand-H wrote on 8/5/2020, 11:02 AM

@fr0sty Thank you for the tips.

fluffy-c wrote on 12/1/2020, 1:49 PM

Wizard of Oz (note, this last one was done on an earlier version of the plugin, results have improved since):

And would you have the Preset you used for this colorization? If yes, thank you