Long time lurker, first time poster, so please go easy on me 😅. Also I'm not a native speaker but I'll do my best to explain the purpose of these scripts.
I have also had the same problem as you guys, which means that the planar motion tracking features currently available in VEGAS Pro are not perfect.
You can of course use the "VEGAS Bézier Masking" plugin to create a curve mask and track that and then use the script provided with VEGAS to apply this data on to a free form Picture in Picture effect.
But let's be honest. For this to work all the stars have to be aligned and often enough it is a pain in the butt to get right. I often get jitter and distortion and it just doesn't feel right.
Because I'm a do it yourself type of guy, I wrote a script that enables you to do planar motion tracking perfectly. No special plugin, nothing. ("Copy Mask PiP Corner.cs")
It works like Hitfilms 4-point corner motion tracking. So you would apply the Bézier Masking plugin on to the video track and then create a rectangular mask for every corner you want to have tracked. Pay close attention that each center of your four masks is exactly at the location where you want the corresponding corner of your "target video"/PiP to be. After that use my script to "paste" the motion tracking data onto the PiP corner and you're done. Pretty easy and reliable. Also my plugin includes multiple dialogues, ensuring the correct use of my plugins and dialogues that let you choose which mask to use and more. This is very helpful in order for the script to stay as generic as possible.
I have also developed another script ("Copy Mask to Crop Center.cs") which enables you to follow an object, for example a head with your frame. This is currently not easily achievable in VEGAS. What I mean with "follow an object with your frame" is shown in this video: Until now one would have to do everything by hand. With my script this is now peanuts.
More explanation is given in my YouTube video (skip to 16:10 & 20:32 for what to expect) I made about my scripts, so anyone can learn to use it. I would be pleased to see it getting used and other people improving it. Hope it helps you as much as it helps me.
Please credit me, if you modify and redistribute this script.
Let me know what you think!