SCRIPT increase and decrease the opacity of an image when speaking

andy-0 wrote on 9/15/2023, 5:43 AM

I tried to find a plugin that would do an effect like a livestream effect where there is a png of the character and when he speaks the opacity of the character increases to indicate who is speaking at the moment.I asked this elsewhere and one of the comments said "If you could write a Vegas script to take the sidechain output of a compressor and use the presence of audio above a threshold to trigger an effect change (opacity), you'd be all set." When I heard this I tried to create a script for Vegas pro 17 and speed up my work but I was at a dead end where the script didn't work as it should already. It's the second day that I'm trying to make this script but I don't have much skill with programming codes in general so I came up with the idea of coming here to ask about it. Can anyone help me or give me a tip on how to do this script? Anything would help.

Comments

EricLNZ wrote on 9/15/2023, 5:57 AM

@andy-0 It might help if you posted, or made available, what you have done so far. One of our forum scripting experts could then have a look.

Gid wrote on 9/15/2023, 6:31 AM

@andy-0 BCC Beat reactor reacts to sound, there's a mocha mask in each fx that will limit the fx to where you want, if added to one event it would react to all audio, so you'd have to maybe duplicate the events & combine again using masks, then separate the audio of each person in the different events,.. Not sure if you could just add it twice to one event & have each fx in the fx chain only react to either person individually because like i say it would react to all audio. Edit - if each fx in the chain only uses the individual audio AIFF of each person & that fx is masked to just that person it does work,

These are the BCC fxs that have Beat Reactor in them.

Last changed by Gid on 9/15/2023, 10:44 AM, changed a total of 2 times.

Vegas Pro 21 
Magix Movie Studio Platinum 2024 - from 2004-ish - Latest version,
Magix VPX14
Boris Continuum & Sapphire, 
Silhouette Standalone + Plugin, 
Mocha Pro Standalone + Plugin, 
Boris Optics,
Desktop PC Microsoft Windows 10 Pro - 64-Bit
ASUS PRO WS WRX80E-SAGE SE WIFI AMD Motherboard
AMD Ryzen Threadripper PRO 3975WX 3.5GHz 32 Core
Corsair iCUE H150i RGB PRO XT 360mm All-in-One Liquid CPU Cooler
RAM 256GB ( 8x Micron 32GB (1x 32GB) 2666MHz DDR4 RAM )
2x Western Digital Black SN850 2TB M.2-2280 SSD, 7000MB/s Read, 5100MB/s Write
(programs on one, project files on the other)
Graphics MSI GeForce RTX 3090 SUPRIM X 24GB OC GPU
ASUS ROG Thor 1200W Semi-Modular 80+ Platinum PSU 
Fractal Design Define 7 XL Dark TG Case with 3 Fans
Dell SE3223Q 31.5 Inch 4K UHD (3840x2160) Monitor, 60Hz, 

I'm 55, I've been a UK joiner for 39yrs, apprentice trained & time served, I make videos about my work & post them on Youtube so that I can share what I've learnt over the yrs,
https://www.youtube.com/c/Gidjoiner

At the moment my filming is done with a Samsung Galaxy S23 Ultra 

andy-0 wrote on 9/15/2023, 7:50 AM

@andy-0 BCC Beat reactor reacts to sound, there's a mocha mask in each fx that will limit the fx to where you want, if added to one event it would react to all audio, so you'd have to maybe duplicate the events & combine again using masks, then separate the audio of each person in the different events,.. Not sure if you could just add it twice to one event & have each fx in the fx chain only react to either person individually because like i say it would react to all audio.

These are the BCC fxs that have Beat Reactor in them.

If I had 1 audio track for each participant and an image for each of them and used this separately for each image and audio for a participant, would it work correctly? If so, can you give me a video tutorial or a guide on how to do this for each image and audio of the 3 participants?

Gid wrote on 9/15/2023, 7:59 AM

@andy-0 Hi, I'll see if I can find a clip suitable, 👍

There's some videos on Boris's page about Beat Reactor, the ones i showed above are in the Continuum package & all work in Vegas, in these Boris videos there'll be some that are of a separate Beat Reactor fx that only shows up in Adobe AE & maybe other programs but not Vegas, the principle of it's workings are similar if not the same tho. https://borisfx.com/videos/?tags=&search=beat%20reactor

PS forgot to mention you need to create an AIFF that Beat Reactor uses but that can be done with Vegas

When you say 'opacity' do you literally mean you want the others to be 'ghostly'/faded?

Last changed by Gid on 9/15/2023, 8:23 AM, changed a total of 2 times.

Vegas Pro 21 
Magix Movie Studio Platinum 2024 - from 2004-ish - Latest version,
Magix VPX14
Boris Continuum & Sapphire, 
Silhouette Standalone + Plugin, 
Mocha Pro Standalone + Plugin, 
Boris Optics,
Desktop PC Microsoft Windows 10 Pro - 64-Bit
ASUS PRO WS WRX80E-SAGE SE WIFI AMD Motherboard
AMD Ryzen Threadripper PRO 3975WX 3.5GHz 32 Core
Corsair iCUE H150i RGB PRO XT 360mm All-in-One Liquid CPU Cooler
RAM 256GB ( 8x Micron 32GB (1x 32GB) 2666MHz DDR4 RAM )
2x Western Digital Black SN850 2TB M.2-2280 SSD, 7000MB/s Read, 5100MB/s Write
(programs on one, project files on the other)
Graphics MSI GeForce RTX 3090 SUPRIM X 24GB OC GPU
ASUS ROG Thor 1200W Semi-Modular 80+ Platinum PSU 
Fractal Design Define 7 XL Dark TG Case with 3 Fans
Dell SE3223Q 31.5 Inch 4K UHD (3840x2160) Monitor, 60Hz, 

I'm 55, I've been a UK joiner for 39yrs, apprentice trained & time served, I make videos about my work & post them on Youtube so that I can share what I've learnt over the yrs,
https://www.youtube.com/c/Gidjoiner

At the moment my filming is done with a Samsung Galaxy S23 Ultra 

andy-0 wrote on 9/15/2023, 8:00 AM

@andy-0 It might help if you posted, or made available, what you have done so far. One of our forum scripting experts could then have a look.

I even made a script that tried to do what was requested but I had no luck, I don't know much about programming and something like that and what I had done was done by an AI that had very limited functions regarding scripting for programs and that they didn't work as they should so I don't have the script that I was trying to make work and as was said in the post I came to ask here about someone who knows how to do this automatically or has the script for the same or can write a script that performs this work. But if you want, I can post this script that doesn't work and was made by an AI (certainly the script that the AI ​​made is very basic and simple so don't put too many expectations on it)

jetdv wrote on 9/15/2023, 8:37 AM

@andy-0, are the speaking parts separate events? Or will the script need to "detect" silence?

If they're separate events, this would be a fairly trivial project. If it has to "detect" the silence, it's more complicated but I've already got a script that does something similar to that in the tutorials. Take a look at this tutorial series:

The second video basically does what you're asking. The composite level of the video track is modified based on the volume level of the audio track:

 

andy-0 wrote on 9/15/2023, 10:53 AM

@andy-0, are the speaking parts separate events? Or will the script need to "detect" silence?

If they're separate events, this would be a fairly trivial project. If it has to "detect" the silence, it's more complicated but I've already got a script that does something similar to that in the tutorials. Take a look at this tutorial series:

The second video basically does what you're asking. The composite level of the video track is modified based on the volume level of the audio track:

 

First of all, thank you very much for your help, I will watch the video. The idea I have is that they are different events where you have an image that has a participant's face and an audio track of the participant's speech and when he spoke this script would detect the audio and adjust the opacity of the image that is associated with the audio to 100% and when he is not saying anything its opacity is at 50%

wwaag wrote on 9/15/2023, 1:28 PM

There's a tool in HOS, Loudness to Opacity which may work for you. Here's a link to a demo.

https://tools4vegas.com/loudtoopacity-demo/

AKA the HappyOtter at https://tools4vegas.com/. System 1: Intel i7-8700k with HD 630 graphics plus an Nvidia 1050ti graphics card. System 2: Intel i7-3770k with HD 4000 graphics plus an AMD RX550 graphics card. System 3: Laptop. Dell Inspiron Plus 16. Intel i7-11800H, Intel Graphics. Current cameras include Panasonic FZ2500, GoPro Hero11 and Hero8 Black plus a myriad of smartPhone, pocket cameras, video cameras and film cameras going back to the original Nikon S.

andy-0 wrote on 9/15/2023, 5:07 PM

@andy-0 Hi, I'll see if I can find a clip suitable, 👍

There's some videos on Boris's page about Beat Reactor, the ones i showed above are in the Continuum package & all work in Vegas, in these Boris videos there'll be some that are of a separate Beat Reactor fx that only shows up in Adobe AE & maybe other programs but not Vegas, the principle of it's workings are similar if not the same tho. https://borisfx.com/videos/?tags=&search=beat%20reactor

PS forgot to mention you need to create an AIFF that Beat Reactor uses but that can be done with Vegas

When you say 'opacity' do you literally mean you want the others to be 'ghostly'/faded?

something like this is an image with 50% opacity that increases its level to 100% when you speak. I may not have understood very well what you said about 'ghostly'/faded but if you are referring to the photo of the participant which is transparent because of its 50% opacity in its video track and which is solid in the 100% case of its opacity when the participant's audio is detected, then yes, this is the result I am looking for

An example that I can give is in the link below this message where the photo on the left would be the person when the audio is not detected and the one on the right when it is detected that they are speaking

example

andy-0 wrote on 9/15/2023, 5:10 PM

There's a tool in HOS, Loudness to Opacity which may work for you. Here's a link to a demo.

https://tools4vegas.com/loudtoopacity-demo/

Thanks for the help, I'll take a look at this tool

wwaag wrote on 9/16/2023, 8:20 AM

@andy-0

There's another tool in HOS called EventDucker that is designed to duck either an audio volume envelope or a video composite envelope. Here's a demo of that tool.https://tools4vegas.com/wp-content/uploads/EventDuckerDemo.mp4

And a screenshot of the dialog.

At the moment, you use an audio control track (e.g. voice overlay) to control the volume envelope of another audio track (e.g. music). For video you use a video control track (e.g. title) to control the opacity of another video track. Both of these are illustrated in the demo.

However, given your problem, it should be fairly easy to add a third mode where the audio control track (voiceover) controls the opacity of the video track. Since I'm always looking for new features for HOS tools, this would seem a worthwhile addition. I had already planned an option to create a new audio file after all silences had been removed from an audio track, another feature that other Vegas users had expressed a need for.

AKA the HappyOtter at https://tools4vegas.com/. System 1: Intel i7-8700k with HD 630 graphics plus an Nvidia 1050ti graphics card. System 2: Intel i7-3770k with HD 4000 graphics plus an AMD RX550 graphics card. System 3: Laptop. Dell Inspiron Plus 16. Intel i7-11800H, Intel Graphics. Current cameras include Panasonic FZ2500, GoPro Hero11 and Hero8 Black plus a myriad of smartPhone, pocket cameras, video cameras and film cameras going back to the original Nikon S.

wwaag wrote on 9/17/2023, 1:32 PM

Turned out it was pretty easy to add the option to control video opacity from an audio track. Here's the new screengrab.

And a short demo, first showing just audio ducking and then the selected audio track controlling the opacity of the video track. Note the effect of inverting envelope points.

Here's a screengrab with the inner thumbnails removed to better show the resulting envelopes.

Last changed by wwaag on 9/17/2023, 1:53 PM, changed a total of 1 times.

AKA the HappyOtter at https://tools4vegas.com/. System 1: Intel i7-8700k with HD 630 graphics plus an Nvidia 1050ti graphics card. System 2: Intel i7-3770k with HD 4000 graphics plus an AMD RX550 graphics card. System 3: Laptop. Dell Inspiron Plus 16. Intel i7-11800H, Intel Graphics. Current cameras include Panasonic FZ2500, GoPro Hero11 and Hero8 Black plus a myriad of smartPhone, pocket cameras, video cameras and film cameras going back to the original Nikon S.

Joelson wrote on 9/17/2023, 4:31 PM

@andy-0

If you organize the transcription of speakers into separate tracks, it will make your work much easier as you will know where each speaker's speech begins and ends and you can manually adjust the opacity at the corresponding points using opacity envelopes.

If you prefer an automated option to speed up your work, you can use the HOS Event Ducker tool which will do a great job, as mentioned by wwaag.

@wwaag

Is it possible to add an option to set volume or opacity on silent parts as well? This option can be useful as in the case of OP which needs the opacity to be at 50% when the speaker is silent. See my screen recording to understand better.