Eye Contact Augmentation

Limit22 wrote on 10/19/2023, 3:30 PM

I tried a topic search here and at BorisFX and NewBlueFX to see if there is any plugin or how-to videos and no luck; however, I did an AI search, it said otherwise. I am interested in something that will adjust where my eyes are focusing and increase the whiteness. I know of a few lessor programs that market to this and offer it, so I assume VegasPro and affiliates like Boris or NewBlue would have something better and more powerful, yes?

Comments

Former user wrote on 10/19/2023, 8:12 PM

If you have an Nvidia RTX GPU look at Github etc for apps or plugins built from the Nvidia Maxine SDK

https://blogs.nvidia.com/blog/2023/08/08/maxine-3d-video-communications/

Alternatively you could use a less elegant approach of a virtual camera, OBS and Nvidia Broadcast.

The eye contact stuff looks silly to me and people mostly make fun it but I could see how it could work with short cutaways to someone who was obviously reading from a display to the side of the camera

Limit22 wrote on 10/19/2023, 8:32 PM

OMGosh, I failed to state I am looking for post production work because I did read up on the Nvidia option but from what I gather, that is real time. I have long content that i would like to spice up and center my roaming eyes and enhance the whiteness before the main edits. thank you for the response.

Former user wrote on 10/19/2023, 8:43 PM

There are apps made for what you require from Maxine SDK, I've seen one demonstrated but can't recall the details. Also with the Nvidia broadcast approach you can use a virtual camera that plays video files, Nvidia Broadcast will pick that up as a source, and OBS will pickup Nvidia Broadcast as a source. Like I say a very messy approach. There's likely a YT tutorial on how to do it I tried it near a year ago so don't recall details

Dexcon wrote on 10/19/2023, 9:15 PM

@Limit22 ... dealing with eyes has been discussed a few times over the years on BorisFX's forum:

https://forum.borisfx.com/search?q=eyes

You may or may not find some help from those discussions.

Limit22 wrote on 10/20/2023, 8:34 AM

appreciate the follow-through from both of you guys, thanks. I am interested in the source-of-a-source setup by Nvidia and OBS and will give this more of my time. I hope Vegas will include something like in their future updates. it would be a great way to usher in the AI element and hype.

I took another glace at the Borisfx list from Dexcon, and happen to click on the perfect link and how to get what I want done! It is a howto for DaVinci but still can be done with VegasPro. Thanks!

https://borisfx.com/videos/performance-replacement-with-bcc-10-and-avid-media-composer/

Former user wrote on 10/20/2023, 5:53 PM

I tried a very simple windows build based on Maxine. Not great results, messes up eyes, and eyes are very low resolution.

You wouldn't use this in anything but a social media post, but personally I don't think this example is usable at all. Try for yourself, maybe it does better with other scenes/resolutions

https://drive.google.com/u/0/uc?id=1wSYviRSL4pktIIvF6Xer5s3AOwHKbN1a&export=download

Former user wrote on 10/20/2023, 6:04 PM

Can't add to last post.

Also tried automated eye whitening in Resolve. It also failed. I left the masking on the screen to better show what goes wrong. AI tracking isn't quite there. Mocha sounds ideal for this sort of stuff, but more work