4-point motion tracking and object follow

DustVoice wrote on 8/26/2019, 5:03 PM

Long time lurker, first time poster, so please go easy on me 😅. Also I'm not a native speaker but I'll do my best to explain the purpose of these scripts.

I have also had the same problem as you guys, which means that the planar motion tracking features currently available in VEGAS Pro are not perfect.
You can of course use the "VEGAS Bézier Masking" plugin to create a curve mask and track that and then use the script provided with VEGAS to apply this data on to a free form Picture in Picture effect.

But let's be honest. For this to work all the stars have to be aligned and often enough it is a pain in the butt to get right. I often get jitter and distortion and it just doesn't feel right.

Because I'm a do it yourself type of guy, I wrote a script that enables you to do planar motion tracking perfectly. No special plugin, nothing. ("Copy Mask PiP Corner.cs")

It works like Hitfilms 4-point corner motion tracking. So you would apply the Bézier Masking plugin on to the video track and then create a rectangular mask for every corner you want to have tracked. Pay close attention that each center of your four masks is exactly at the location where you want the corresponding corner of your "target video"/PiP to be. After that use my script to "paste" the motion tracking data onto the PiP corner and you're done. Pretty easy and reliable. Also my plugin includes multiple dialogues, ensuring the correct use of my plugins and dialogues that let you choose which mask to use and more. This is very helpful in order for the script to stay as generic as possible.

I have also developed another script ("Copy Mask to Crop Center.cs") which enables you to follow an object, for example a head with your frame. This is currently not easily achievable in VEGAS. What I mean with "follow an object with your frame" is shown in this video: Until now one would have to do everything by hand. With my script this is now peanuts.

More explanation is given in my YouTube video (skip to 16:10 & 20:32 for what to expect) I made about my scripts, so anyone can learn to use it. I would be pleased to see it getting used and other people improving it. Hope it helps you as much as it helps me.

Please credit me, if you modify and redistribute this script.

Let me know what you think!

Download Link (GitHub page): h ttps://vegas.dustvoice.de

Greetings,
DustVoice

Comments

Marco. wrote on 8/27/2019, 3:46 AM

I get strange results trying to use your script (see screenshot below). Why could I only select one corner to get copied instead of it always uses 4 corners?

Edit:
I think I got it. You would need to create 4 masks and what get copied then isn't the corners of the mask(s) but it's the center point of each of the 4 masks? And it means you have to track 4 times and also use the script 4 times?

DustVoice wrote on 8/27/2019, 4:56 AM

EDIT:

=====

The main reason why it won't select all 4 masks at the same time and copy them to the 4 corners, is that the script doesn't want to assume which mask is which corner and also imagine having mask 1 populated with motion tracking for witness protection, and wanting to use mask 2-5 for the corners. With this method you're able to do so. Also imagine having two video events with the top corners in mask 4 and 5 of event 1 and the bottom corners in mask 4 and 5 of event 2. With the current script version this is no problem. I know it's a very unlikely case but as a software developer I'm thinking about these edge cases all the time.

=====

Yes that's exactly the case. See the YouTube video for a detailed explanation. Like in for example Hitfilm you track each of the 4 corners with an own mask. After that you have to run the script 4 times to copy each mask to the corresponding corner. This was meant to be this way in order for the script to be as generic as possible.

The main purpose of this script was to demystify and derandomize the current process (as shown in the beginning of the YouTube video). In the current process there is often times a lot of jitter and you aren't able to be very precise. That's why I used the 4 corner motion tracking method, as it is used in many other pieces of software.

In order for the motion tracking to be the most accurate, I decided I wanted to just track a single corner with a mask at a time, in order for the algorithm of the Bézier Mask plug in to "concentrate" on one corner at a time. This produces the most accurate and non jittery results.

Although this is the perfect solution for me, I plan on extending the script in order to able to assign all 4 corners at once, because having to run the script 4 times which is fairly annoying to be completely honest.

Hope this all makes sense.

And sorry about the shameless plug to the YouTube video but I just think it explains it pretty good, even if you just skip through it. I'm also not making any profit out of it or anything, it's just for explanatory purposes because I want the scripts to be as useful to others as it currently is for me. I used this method in multiple productions already and it looks gorgeous and saved me from a lot of headaches.

If you look at 16:10 of the video you'll also see the four masks with the corners of the PiP assigned to their centerpoints.

Hope this helps.

Greetings,

David Holland

DustVoice wrote on 8/27/2019, 5:00 AM

Sorry for the long answer but I want the whole thing to make any sense 😅.

And yes, again you would have to track and copy 4 times. But I think this is the best way to do it because then the algorithm can concentrate on one corner at a time and produce the most accurate and deterministic results. Also it allows you to just redo a single corner if it looks particularly off and not redo the whole thing and produce other errors in the process. This happened to me multiple times, so for example corner top-left is off, I redo the tracking, now corner bottom-right looks horrible and so on. With this method, you would retrack the top left mask and rerun the script to overwrite the top left PiP corner and you're done.

DustVoice wrote on 8/27/2019, 5:11 AM

The only problem that persists at the time of writing is, as mentioned multiple times throughout the years, that you can't zoom into the god damn preview window!

Im currently working on another script (my main focus at the moment) which enables you to do this with a trick:

The script saves the current pan and crop settings for the video clip. After that you use the pan and crop settings to "zoom in" onto the perfect location in order to precisely position your mask. After that you continue the script. The script will now calculate the exact location you "selected" and "zoom out" to the previous pan and crop settings and voila you have successfully selected the exact pixel/point you wanted to. I think this would help tremendously, that's why I'm currently focusing all of my attention on this feature.

DustVoice wrote on 8/27/2019, 7:09 AM

I have updated my script. It now removes the masks/corners already selected once (per script execution -> run script another time, all masks/corners are there again), or masks that aren't enabled from the drop down list and preselects the first element. Also after selection the mask-corner pair, you get a prompt, asking if you want to do another corner.

So spamming return will now yield the following result:
Mask 1 => Top Left Corner
Mask 2 => Top Right Corner
Mask 3 => Bottom Left Corner
Mask 4 => Bottom Right Corner
Copy from cursor onwards.

I think this is quite the improvement

I also moved the files to a GitHub repository to allow other people to contribute.

I would be very happy if you like the improvement and give me some feedback on it. I'm eager to improve.

Greeting,
DustVoice

EDIT:
I realized my comment is not very clear to the average reader. I just did some foolproofing and while maintaining the genericity, it allows you to copy all 4 masks to all 4 corners with one script execution.

DustVoice wrote on 8/27/2019, 9:35 AM

HOOORAY!
I have done it!

I have successfully written a script ("Precise adjust mask.cs") which enables you to precisely position your mask's center point by zooming into the preview window! As far as I know this has never been done before. More versions (for other things than mask center points) are to follow.

As always I have made a quick YouTube tutorial showcasing how to properly use the plugin.

Please consider trying it out and giving valuable feedback.

Greetings,
DustVoice

DustVoice wrote on 8/27/2019, 9:46 AM

Look at the first post. There is a GitHub link. YouTube video is still uploading unfortunately. But do this:

Choose a event and apply the Bézier Masking plugin
Ideally set the Blend to 0.5
Go to the pan & crop option and zoom into the part where you want your mask centerpoint to be
Position the mask precisely
If the event isn't already selected, select it while holding down CTRL in order to NOT move the cursor
Execute the script
Select the mask number you just positioned
Select yes/no (yes = mask size will also shrink, no = mask size will not be touched, only position)
Watch the magic happen

GerY wrote on 8/27/2019, 9:47 AM

Thx DustVoice i am going to test it. It looks very good to me THX

DustVoice wrote on 8/27/2019, 9:48 AM

No problem. I got a little bit excited. because its something I wanted for a long time

DustVoice wrote on 8/27/2019, 10:09 AM

Video will be avaliable under

EDIT:
YouTube apparently has to come to its senses. The video is currently marked unavailable. I don't know why. Maybe time will tell

musko wrote on 8/27/2019, 10:40 AM

@DustVoice maybe I doing something wrong.

I apply Bézier Masking FX to event.
Going to Pan/Crop and zoom.
Making mask.
Select event and select your script.
I allways have this error:

Yes I try to rename FX in script due to I have "Bézier Masking" instead of "VEGAS Bézier Masking" FX name.

Please explain what I doing wrong.

Your YT tutorial is still unavailable 😢

DustVoice wrote on 8/27/2019, 10:46 AM

Yes I try to rename FX in script due to I have "Bézier Masking" instead of "VEGAS Bézier Masking" FX name.

Please don't rename the name in the script. This is an internal name. The only reason why it's a variable is, if MAGIX decides to rename the plugin. Which script are you using? If you want to use the new zoom in feature, you have to use the "Precise Adjust Mask" script.

The "Copy Mask to PiP Corner" script is for "Bézier Masking Mask" -> "Picture in Picture Corner",
the "Copy Mask to Crop Center" script is for "Bézier Masking Mask" -> Pan & Crop Center
and the "Precise Adjust Mask" script is for the zoom in stuff.

If you have any further questions, don't hesitate to ask.

IMPORTANT EDIT:
I know what your problem is. Please apply the Bézier Masking plugin as an event fx not as a media fx. Due to the way the Scripting API is set up, all my scripts deal with Event FX, not Track FX, or Media FX.

So just click on the rightmost fx symbol apply the Bézier Masking plugin, make sure you reverted the changes you made to the script and you should be fine.

Also note, that the script currently only does this for Mask Centers! and not for individual corner points of a bézier curve!

Let me know if it works

DustVoice wrote on 8/27/2019, 10:59 AM

Video is online!
Hopefully this explains everything better...

musko wrote on 8/27/2019, 11:00 AM

IMPORTANT EDIT:
I know what your problem is. Please apply the Bézier Masking plugin as an event fx not as a media fx. Due to the way the Scripting API is set up, all my scripts deal with Event FX, not Track FX, or Media FX.

Yes this is my problem

GerY wrote on 8/27/2019, 12:26 PM

DustVoice The 4 point tracking is very good and works for me with the "Copy Mask to PiP Corner "script. This is what Planar Tracking Motion has to do. THX for the Tutorial and share it with us.👍

DustVoice wrote on 8/27/2019, 12:30 PM

No problem. I was also very frustrated at first, coming from an AfterEffects and Hitfilm background back to VEGAS. I wanted to have this SOO bad, so I even got the 17 version. After discovering that it doesn't quite work out like advertised, who would have guessed, I was really disappointed. But now I'm really into the scripting thingy and am planning to just customize the hell out of VEGAS.

Hope they eventually natively support this feature. Hell, even if they wanted to incorporate my script into an update I would be so down for it! (Like they did with motion track -> pip in version 16)

If you discover anything, or want some custom features to be baked in, just tell me.

Greeting,
DustVoice

Marco. wrote on 8/27/2019, 1:16 PM

Your updated script "Copy Mask to PiP Corner" doesn't work no more here. It throws a similar error message as the "Precise Adjust Mask" script.

DustVoice wrote on 8/27/2019, 1:18 PM

Could you tell me the error message?
It certainly is some minor incompatibility

DustVoice wrote on 8/27/2019, 1:19 PM

@GerY Also reported that it works fine for him, so this is likely some minor fix

Marco. wrote on 8/27/2019, 1:20 PM

I just sent you the error message via PM.

ScrapyardFilms wrote on 9/12/2019, 6:16 PM

Hey @DustVoice. I followed your instructions exactly on my laptop and it worked perfectly. I tried to do the same on my desktop and I'm getting the same error as @musko.

I'm making sure to apply the vegas bezier masking effect to the same clip that i used before as an event fx but when I select your precise adjust mask script, it gives the "No selected event with VEGAS bezier masking plugin found' error. I'm doing everything the exact same as I did on my other machine but it's not working the same. Reinstalled VEGAS as well with no luck.

Machine specs it worked on:

Intel i5 8300, 16GB ram, Intel graphics

Machine it's not working on:

Ryzen 1800x, 32GB ram, GTX 1060 6GB

DustVoice wrote on 9/12/2019, 6:35 PM

Are you sure the event with the Bezier masking plug in is selected properly? Just try and reselect it with Ctrl + Left mouse to eliminate any variables. Are you using the same version of Vegas? I doubt it has anything to do with the pc specs though. It most certainly is some minor error!

ScrapyardFilms wrote on 9/12/2019, 8:43 PM

Yep. Both are identical copies of VEGAS Pro 17. Using the same clip. Only difference is the computers which is weird. I'll keep messing around with it but I'm truly stumped

DustVoice wrote on 9/12/2019, 8:53 PM

Alright. Which country are you in? Maybe we can arrange a small meeting of some sort so I can overlook your problem more thoroughly. The only hurdle could be time zones.