Post Capture / Import Optical Scene Detection

Gary James wrote on 5/17/2014, 10:31 PM

I've seen this topic mentioned in this forum on numerous occasions. For what it's worth, I've just incorporated this functionality into PRE-RELEASE version v1.0.59 of Timeline Tools.

This new feature scans your media file and keeps track of scene changes in the video frames. When scenes are detected they fill a listbox where you can step through the list and view the scene boundaries in the Vegas Timeline and Preview window. There are Edit keys provided that let you either delete a scene, or merge the selected scene with either the scene that precedes or follows the selected one. When you're satisfied with your detected scenes, simply press the Accept button and Regions are created that bound each scene.

Download the PRE-RELEASE version of Timeline Tools.

Run the Setup program (which has grown considerably to incorporate this feature) then launch Sony Vegas Pro. You access the new feature from a new Context Menu option in the Main Display Grid. Highlight the Event(s) that you want to process, then right-click on the Grid and select the "Optical Scene Detect on Highlighted Events" menu item. This will bring up a dialog where you adjust a slider to set the detection sensitivity level (typical values are from 7 to 25), where lower setting are the most sensitive. A setting of Zero will return one scene for every frame. Press the Start Process button, and wait for the process to finish.

If you decide to download this PRE-RELEASE version of Timeline Tools, I'd appreciate to hear any comments or suggestions you may have.

Lastly. This new feature was only made possible through the use of FFMpeg, a program covered by the GNU General Public License, version 2. FFMpeg supports a wide range of video file formats, but the latest video formats may not have made their way into the code. However new releases are made available as soon as new formats are added. Timeline Tools installs the 32 and 64 bit versions of this DOS program, then spawns it in a process to collect it's output information. From http://ffmpeg.org/legal.html and http://ffmpeg.zeranoe.com/

Comments

PeterDuke wrote on 5/18/2014, 12:00 AM

Nice to know of in case needed.

When do you need it these days?

videoITguy wrote on 5/18/2014, 12:02 AM

While I applaud your efforts to get the FFmpeg underneath your excellent program, I would like to exercise a little caution in this area.
I have experienced some real conflicts and obnoxious interactions between some apps (including VegasPro) with various flavors of FFmpeg riding in on shirttails of other installs. This is experience with many variations in builds so I can truly say it is a problematic situation at best.
Sometimes the FFmpeg is absolutely necessary to make something work and in other instances causes problem interactions with other resident apps. It seems the problem really stems from so many flavors of the FFMpeg being distributed with a variety of installers and helper apps, that the consistency and stability of a given installation is always subject to some attack.

Gary James wrote on 5/18/2014, 12:36 AM

This version of FFMpeg was specifically compiled as a stand-alone executable for the Windows OS, and does not require a formal installation. My Setup program simply extracts and copies the 32 and 64 bit versions to a directory branch under \Documents\Vegas Application Extensions. So no interaction with any other version.

videoITguy wrote on 5/18/2014, 8:42 AM

Once I assemble a new test bed for VegasPro13 on Win7 64bit - which is probably two months out as yet, I shall want to test this.
My prior experience with FFMpeg goes back to earlier systems over the last 4 years.

There are several justifications that I know of for installing FFMpeg on an otherwise virgin NLE install on Windows platforms. Among the most weighty is being able to support writing ProRes files that will pass the test of getting into a MAC platform workflow. That is not easy to do but can be done. Another is the support of a Win OS media manager ( one much more advanced than what SCS developed many years ago). And then are dozens of other helper apps supporting timecode extraction which VegasPro needs a lot of help with.

I don' t use the tools like Handbrake or Tmpeg creation apps, but maybe others can comment on the interaction with FFMpeg in those cases.

wwaag wrote on 5/18/2014, 1:03 PM

When do you need it these days?

For anyone whose source material is analog converted to digital, it can be useful. I still have a lot of 8 and hi8 that's been converted to DV and the first step of any project is to cut these long source files into clips based on scene change. Scenalyzer used to do this. I've also used one of John Cline's scripts with some success but getting the detection sensitivity right can be a problem to the point that I now just Fast Forward, Stop, go back 15 frames, and I'm usually within a frame or two of the transition point. Plus, I then know exactly what's in the source file. Having said that, I'm eager to give this new feature of TT a go.

wwaag

Grazie wrote on 5/18/2014, 2:13 PM

Ah, Scenalyzer.....

G

wwaag wrote on 5/19/2014, 4:17 PM

Gary,
Just tried the new TT with optical scene detection. It seems to work OK once you have the sensitivity set appropriately. I have a lot of DV files from analog sources that have media markers showing where the scene changes, so it was pretty easy to test against known scene changes. A few observations:
1. Default sensitivity too low. I found 25 to 30 to be about right.
2. Detected a number of single frame clips which was obviously incorrect.
3. The start times for the last 2 suggested scenes are always the same. Observed this on all the files I tested.
4. Here's the biggie. TT slowed down V13 to a crawl. E.g. a simple split (S) is always immediate. However, with TT installed, the "busy" icon would appear and the split would actually take place a few seconds later. Removed TT, and V13 works as before.
Hope these observations will be of use to you. Regardless, I do think this will be a very useful addition once the bugs are sorted out.

wwaag

Gary James wrote on 5/19/2014, 7:02 PM

Default sensitivity too low. I found 25 to 30 to be about right.
As you play it more you'll find that each video is unique. Some require much lower settings, and some much higher.

Detected a number of single frame clips which was obviously incorrect.
This indicates a sensitivity setting too low. Bump it up a bit and it will be less prone to see small frame deltas as a new scene.

The start times for the last 2 suggested scenes are always the same. Observed this on all the files I tested.
You should download the latest update at the same link. This has been fixed.

Here's the biggie. TT slowed down V13 to a crawl. E.g. a simple split (S) is always immediate. However, with TT installed, the "busy" icon would appear and the split would actually take place a few seconds later. Removed TT, and V13 works as before.
This sounds like a memory issue. TLT is not a lightweight app. but I'm running SVP-11 64 bit on a 6GB I7 processor and I hardly notice a change when I have it running.

wwaag wrote on 5/20/2014, 11:31 AM

You should download the latest update at the same link. This has been fixed.

Still not working for me. Here are a couple of screenshots. Note the same start times for last 2 scenes. It did correctly detect the six scenes.

This sounds like a memory issue.

Could you elaborate on possible causes? I have noticed the same "slow down" using Vegasaur. Everything works fine in V12, but not V13. I have removed and reinstalled to no avail.

wwaag

Gary James wrote on 5/20/2014, 3:29 PM

By a Memory Issue I meant your available memory when you are running SVP-13. To see how much memory your system is currently using, and likewise how much is available, right-click on the taskbar, and select Start Task Manager. Click on the Performance tab and look at the lower left bar graph. This shows how much of your PC's memory is currently in use. If you don't have much available, Windows has to keep moving things around in memory to free up enough memory to do what you want to do. If you are showing very little free memory, you need to close some running programs and processes, or purchase additional memory for your PC.

I'll have to look under the hood a bit more about the last scene being off. This was a problem that appeared to be fixed on my PC. But you must be experiencing some different results that I'm not; causing the problem on your PC and not mine.

One thing you can do that will help. Before you push the Process button, press and hold the SHIFT key while you push the button. A small debugging window will appear that traps information from the spawned ffmpeg process. Once the window opens you don't need to keep pressing the Shift key. When the process finishes, copy and email me the results from that window. You can use the email address shown in the About tab.

Gary James wrote on 5/22/2014, 4:03 PM

wwaag, I've placed a TLT update on Dropbox at the same link givien in the original post.

This new version fixes the last scene's Start / End time bug. And also now generates timeline Regions immediately following the completion of Scene Detection. If you like the scene as they are just Press the Accept button. Or you can use the new Increment / Decrement buttons to add an oddset to the Start / End of a scene from the dialog.,

When you are all done press Accept, or Cancel. Cancel deletes all the Regions created by scene detection.

Gary James wrote on 6/2/2014, 10:04 AM

This feature has been added to the latest v1.0.59 release of Timeline Tools, and can be downloaded from the web site at: Timeline Tools.