Automatic Event Detection (Vegas Pro 20)

JosephP wrote on 3/14/2023, 7:01 PM

Hello,

I'm looking for advice on how to perform a task.

I would like to be able to scan through the video source (or audio) and mark on the timeline when certain events occur.

I'm mostly editing video game footage, which will have repetitive and predictable events (such as a kill or point notification), whereby an image (such as some text, or an icon) will appear on screen to indicate an event of interest. I would like to be able to provide Vegas Pro with an image (or audio clip) and have it scan through my source media and mark on the timeline when the image (or audio) events are detected in the source media.

Is this possible? If not, does anyone know where I could start looking for an alternative? Being able to perform this function would save me hours upon hours of tedious manual marking work, and would be an insanely useful feature for me to have. I was considering trying to write my own script that uses ffmpeg and an image detection algorithm to perform this function, but it's quite a bit of work and I figured that Vegas might have a way to do what I'm looking for.

Comments

jetdv wrote on 3/15/2023, 8:27 AM

How is it supposed to "identify" this event? And it sounds like your "event" is in the middle of a larger "Event" on the timeline. It seems you would need to go through the timeline and identify when your "events" occur perhaps with a Marker and label that marker for the type of "event" it is.

JosephP wrote on 3/15/2023, 9:49 AM

So the answer is no? As for the "how", optical character recognition or something similar? Perhaps I could try and repurpose an image recognition model? I'm looking to avoid the tedious manual work of scrubbing through hours and hours of footage.

I'm looking to be able to mark the point in time when a specific icon or image appears within the confines of the screen. In my current case, this means I'm looking to mark the timeline whenever the word "ELIMINATED" appears within a certain region of the screen.

From there I'd want to trim X seconds before and after these points in time, that's all.

These points in time are also identifiable with a specific audio cue, but this is not the case for each and every game that I'll be editing.

If this isn't possible I'd appreciate being pointed in the right direction to start developing my own script for Vegas, I'm sure it'll be useful for others as well.

jetdv wrote on 3/15/2023, 9:58 AM

Vegas does not have anything like that built-in. You can look at my scripting tutorials but none of them will cover this particular topic. They will teach you how to write a script and go through the events on the timeline. Then you'd have to figure out how to scan the files to find the information you're wanting.

JosephP wrote on 3/15/2023, 10:05 AM

I'll definitely take a look. Just to confirm (I am new to Vegas) - would it be possible for the script to feed a separate process the video or audio information, and be able to recieve timestamp information back in order to automatically mark the timeline after processing is complete?

Thanks for the help :)

jetdv wrote on 3/15/2023, 10:16 AM

It can certainly call a separate process. It depends on how that process wants to return the information. It might be best if that other process just created a text file with the time-codes and what it found. Then you could just read through that text file and add things as needed. If the file isn't at the beginning of the timeline, you'd have to properly adjust the timecode.

JosephP wrote on 3/15/2023, 10:21 AM

I'll be heading to your informative scripting posts as soon as I have time later today.

It's a big undertaking but I'll try my best :)

jetdv wrote on 3/15/2023, 1:34 PM

Over 2 years of tutorials just waiting for your perusal!

jetdv wrote on 3/15/2023, 1:40 PM

You might take a look at this one too!

https://www.vegascreativesoftware.info/us/forum/speech-to-text-via-whisper-openai--137928/