Bézier masking to track and follow moving person on stage

RonB_USA wrote on 7/14/2020, 11:55 AM

Is anyone shooting a 4K stationary full stage (with no camera operator!) and then, in post-production, cropping the smaller/closer image by moving, tracking and following left and right using Pan/Crop? In other words, the cropped "PIPs" are the final full HD screen. Or even render in 4K losing a little resolution.

I have done this manually for a very long time but thought that there might be a much easier way to automate the process using a Bézier mask on the speaker's face instead of the tedious manual keying using pan/crop. The results are ultra-smooth and hindsight editing ("moving" the tripod with software) is always perfect. With practice, you can make it look like two or three cameras, all from the one full-stage wide shot. Manually I can quickly use the Insert key on the Windows 10 keyboard and pan-crop presets for the left-center-right points, or move the pan-crop image with the mouse. But if you have a script routine for this, you would save me and others a lot of precious time. Please share. Thank you. (VP17, 452)

Comments

RonB_USA wrote on 8/6/2020, 2:15 PM

Follow-up... The following is similar to what I was looking for. This script by DustVoice in Germany keeps my speaker/presenter centered automatically near the center of the frame (using Bézier mask tracking, which I have NOT been using yet for this application; I've been doing it manually). As they move on stage, the smaller Pan/Crop frame moves with them. Using Smooth keypoints (usually only one keypoint is needed for left, one for center and one for right) keeps the speaker near the center of the moving left-and-right frame (a smaller moving fixed frame of the stationary 4K stage shot). I've been using a Pan/Crop Preset for Center and the Insert key on the Windows keyboard for left and right as I adjust. This Pan/Crop is a smaller piece of the larger 4K wide stage shot. The Pan/Crop is the final production of what you see. The 4K shot never moves. No camera operator needed.

The result can be rendered in 1080p to keep the "PIP" resolution high -- no smaller than a quarter of the "4K stage" (or rendered in 4K with reduced resolution). So it is just taking a 16x9 "PIP" of the stage and making it the final smooth-moving pan/crop of the 4K stage. I usually use the "cowboy shot" of the speaker (above the head to just below the fingertips. But the Pan/Crop can easily be changed to any size within the 4K original shot.

The Script that I found is: Copy Mask to Crop Center.cs, by DustVoice in Germany. (Thank you!) I'm still experimenting with it, but that is the very same concept that I've been doing manually for years via Pan/Crop! 4K stationary shot with the final result being a smooth, smaller area following the speaker left and right of the podium. I'd love to see the developers perfect this into VEGAS Pro. It doesn't need many keypoints since speakers/presenters don't normally move fast. They normally just casually move to the left of the podium and talk, then move to the center and talk, then move to the right and talk, etc. But a ballet that I shot this way does move much quicker, so it needs to be adjustable. The Bézier mask would do the tracking, providing artificial intelligence for the "post-production, ultra-smooth electronic tripod". So VEGAS Pro becomes the camera operator and hindsight is much more accurate!

This script author DustVoice did a YouTube on the basics of this starting at 16m:30s into this video which he calls Head Tracking (and also references his script in GitHub which can be copied to anyone's VEGAS Pro script folder!):

So this is just the last part of this YouTube video. He has the Script link below this video.