GoPro timecode/timestamp access?

3d87c4 wrote on 4/25/2024, 6:09 PM

I picked up a couple of Gopro H12 blacks to try 3D with and would like to use the timestamp/timecode information for lining up the left and right clips. This information is included as a 3rd metadata track in the video file and, after reading a few threads it appears Vegas can't access it.

I've traditionally used transients on the audio tracks to do this but wonder if there's another way, using FFMPEG, scripting, or some other external program (I notice Pluraleyes is no longer available.)

Any suggestions?

I found this input on the internet, but all I get is a box with the text "hms}" in it:

ffmpeg -i inpiut -vf "drawtext=text='timecode: %{pts:hms}': x=500: y=500: fontsize=100:fontcolor=yellow@0.9: box=1: boxcolor=black@0.6" -c:a copy output.mp4

Here's the mediainfo for one of my files:

General
Complete name : XXXXXXXXXX\GX010006.MP4
Format : MPEG-4
Format profile : Base Media / Version 1
Codec ID : mp41 (mp41)
File size : 80.8 MiB
Duration : 11 s 228 ms
Overall bit rate : 60.4 Mb/s
Frame rate : 59.940 FPS
Encoded date : 2024-04-23 22:02:10 UTC
Tagged date : 2024-04-23 22:02:10 UTC

Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L6@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 11 s 228 ms
Bit rate : 60.1 Mb/s
Width : 5 312 pixels
Height : 2 988 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 59.940 (60000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.063
Stream size : 80.5 MiB (100%)
Title : GoPro H.265
Language : English
Encoded date : 2024-04-23 22:02:10 UTC
Tagged date : 2024-04-23 22:02:10 UTC
Color range : Full
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
Codec configuration box : hvcC

Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 11 s 200 ms
Bit rate mode : Constant
Bit rate : 192 kb/s
Nominal bit rate : 48.0 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 259 KiB (0%)
Title : GoPro AAC
Default : Yes
Alternate group : 1
Encoded date : 2024-04-23 22:02:10 UTC
Tagged date : 2024-04-23 22:02:10 UTC

Other #1
ID : 3
Type : Time code
Format : QuickTime TC
Duration : 11 s 228 ms
Bit rate mode : Constant
Frame rate : 59.940 (60000/1001) FPS
Time code of first frame : 16:01:12:53
Time code of last frame : 16:01:24:05
Time code, stripped : Yes
Title : GoPro TCD
Language : English
Encoded date : 2024-04-23 22:02:10 UTC
Tagged date : 2024-04-23 22:02:10 UTC

Other #2
ID : 4
Type : meta
Format : gpmd
Codec ID : gpmd
Duration : 11 s 227 ms
Bit rate mode : Variable
Title : GoPro MET
Language : English
Encoded date : 2024-04-23 22:02:10 UTC
Tagged date : 2024-04-23 22:02:10 UTC
Duration_LastFrame : -785

Back to post

Comments

3d87c4 wrote on 5/1/2024, 10:14 AM

Thank you, @NoKi!

I will try that out with my first video files.

I appreciate your making it bonehead obvious where to customize the script...I think I would have found it anyway, but...

3d87c4 wrote on 5/1/2024, 5:19 PM

@NoKi:

I'm sorry but the latest script is throwing an error. Perhaps the offset definition needs to be moved?

NoKi wrote on 5/2/2024, 12:12 AM

@3d87c4

Looks like it cannot find ffprobe. Maybe you need to adapt the path to it?

Nils

3d87c4 wrote on 5/2/2024, 11:02 AM

@NoKi: D'oh! Of course...I forgot about that. Thank you.

I tried out the gopro remote yesterday. I shot 4 60fps clips and got a consistent 5 frame difference between the left and right start times (compared to the audio spikes from my coin taps).

So, clearly I need to experiment more with both the remote and voice controls to see if either of them yield consistent results.

3d87c4 wrote on 5/2/2024, 5:12 PM

@NoKi:

I updated the ffmpeg path but changing offsetInFrames doesn't seem to have any affect.
For example...string offsetInFrames ="20"; yielded the same shift as "00". Should I remove the quotes?

Is there any reason you commented out //MessageBox.Show(panaTimecode); ? I uncommented it and it seemed to work OK.

Working with the clips I shot yesterday using the remote the original script is only 1 frame off, shifting the appropriate clip 6 frames instead of the 5 frames suggested by the audio spikes.

NoKi wrote on 5/3/2024, 9:00 AM

@3d87c4

You mean, the "Timecode In" value doesn't change when you alter the "offsetInFrames" value (with quotes!)?

Or is it the resulting event positioning that doesn't change? In case of the relative positioning script (first event always at 00:00:00:00) you wouldn't see a difference indeed, when all events have the same offset.

The MessageBox should't make a differerence, no idea why that helped...

Nils

3d87c4 wrote on 5/3/2024, 11:01 AM

@NoKi:

I mentioned the MessageBox because, with it commented out. it took me a while to see that the modified script was doing anything, LOL. I just wanted to confirm it was OK to un-comment it.

So...the offset is being applied to all events? I was hoping the modified script would shift the second event to correct for a left/right synchronization error that remains. My friend's work follows the same approach you are using and they found a consistent 1 frame error in their 30fps clips. The clips I shot using the gopro remote also had a 1 frame error. (I suspect this indicates a "rounding up" error of some sort.) The clips I shot using voice commands didn't have any error, which is a head-scratcher. Assuming further tests will confirm a consistent error, can you modify the script to shift the second event? Could this be extended to multiple camera arrays?

Thank you.

3d87c4 wrote on 5/3/2024, 11:19 AM

@NoKi

Thinking a bit more about the "rounding up error"...

Presumably the cameras aren't perfectly synchronized, so start at different times. I suspect that any difference is rounded up to a full frame if the time base is quantized to time and frames. If this is the case then an alternative might be to work in absolute time and do a proper rounding to the later clips---i.e. divide the time difference by the time for a single frame to get a real number. If the result isn't an integer, round down if the remainder is less than half a frame and up if it is more. Then go back to time&frames.

NoKi wrote on 5/3/2024, 11:31 AM

So...the offset is being applied to all events? I was hoping the modified script would shift the second event to correct for a left/right synchronization error that remains. My friend's work follows the same approach you are using and they found a consistent 1 frame error in their 30fps clips.

@3d87c4, yes, my idea was that you store two file versions of the script. One script file version with 0 offset, to get the timecode of your "master" GoPro's files and one script file with an offset of say 5 frames for the files of the other GoPro.

I thought it would be a better way to compensate the timecode offset on media file level, than later at the events, where the offset possibly gets lost.

Nils

NoKi wrote on 5/3/2024, 11:50 AM

BTW, have you read this:
https://community.gopro.com/s/article/HERO12-Black-Timecode-Sync?language=en_US

The sync precision should be < 50 ms which is about 3 frames @60 fps,

For higher precision you could also feed an external timecode into all cameras via an analog audio to USB adapter. I think the Hero 12 has this option?

This is what I do with all my cameras (no GoPros) and usually get perfect sync even with wireless timecode transmission (4 ms latency).

NIls

3d87c4 wrote on 5/3/2024, 3:11 PM

Thank you for the article, I hadn't seen the <50ms tolerance. I am following the QR code sync procedure outlined there.

I think the video I shared in an earlier comment discusses the external timecode method you mention...I'll have to watch it again.

Otherwise, the <50ms tolerance may be too wide to fix consistently... A +/- half frame tolerance can be spotted at 30fps. With my coin taps I can sync clips with a +/- half clip at 60fps and doubt that will be noticeable. The H12's can shoot at 120 fps too, so it might be possible to use the coin tap to sync that footage to a +/- half frame...I haven't tinkered with that yet.

3d87c4 wrote on 5/3/2024, 3:32 PM

@NoKi in response to your 9:30 and 9:50 comments (which I just noticed, sorry):

I have been using the script to align the left and right clips of a 3D pair prior to pairing them as a 3D subclip. I haven't thought through the practicality of doing this with multiple 3D pairs in the same Vegas project, pending getting the basic mechanics working and being sure the results are consistent.

If I align two clips in separate files, do I have to render the results to intermediate files for editing in a bigger project?

NoKi wrote on 5/4/2024, 9:38 AM

Sorry I don't know. I never looked into making 3D videos. I thought that having all left eye videos on one track and all right eye videos on another track and getting them in sync would be the start, isn't it?

So, to me seems having the timing offset already in the media timecode would be the best way, as all events derived from this file are already corrected in timing.

Nils

3d87c4 wrote on 5/4/2024, 2:28 PM

@NoKi:

Yes, initially the left clips are added to a track and the right clips are loaded onto a track below the left ones. They are then aligned and combined as 3D subclips.

Here's what I tried today:

Import the 3 left and 3 right clips to project media
Select the right clips (which I know start first) and run the synchronization script with 0 offset.
Select the left clips and run the synchronization script with a non-zero shift. (I tried several shift values).
Drag the left clips to the first timeline track.
Drag the right clips to the second timeline track.
Select all clips and run the relative position shift script (i.e. the on that places the first clip at 0).

I added a comment to show what shift was applied and enabled the comment showing each timecode.

I noticed three things:

I'm not getting the shift I expect from my input,
The second and 3rd clips are offset by what I'm guessing is the time between shooting them,
The later clips were no longer quantized to frames.

The way the later clips are shifted is an issue because they may be shot on different days or may be placed on the timeline out of order. Also, the editing itself may occur over a period of time while the videos are being shot. Would it be possible to abut them to the end of the previous clips, maybe with a small gap? I suppose I could place all clips on separate tracks for this stage of the process.

Is it possible the comma in this line of code is the reason the later clips are no longer quantized to frames?

Timecode minPosition = Timecode.FromString("23:59:59,00");

Oh, are the source files altered at all or only the version stored locally in the project media?

NoKi wrote on 5/4/2024, 4:01 PM

@3d87c4

Can you provide me these 6 files to test with, please?

BTW, the files are not altered in any way. The "Timecode In" is just a value assigned to media files in the Vegas project.

Nils

3d87c4 wrote on 5/4/2024, 4:55 PM

@NoKi

Left:

Right:

I hope the forum doesn't alter these...I can send drive links if needed.

set wrote on 5/5/2024, 4:45 AM

Unfortunately yes, the forum will change the file, Use google drive or similar links.

3d87c4 wrote on 5/5/2024, 4:18 PM

Thank you, @set. I had a notion I'd seen a comment to that effect in another thread. Here are the links:

left
https://drive.google.com/file/d/1QDVNGd5gh54n3yrCcDdjykkrx-xbrV6W/view?usp=drive_link

https://drive.google.com/file/d/1lSV_XDVcKN_EOpoQ767PQsx3_zj_eoc9/view?usp=drive_link

https://drive.google.com/file/d/17NV5KGQfzWMfoWieRKwb59J6-4Lqjcbc/view?usp=drive_link

right

https://drive.google.com/file/d/1SatEbZiq3_mWagsCir9c68FKJ8Qp7K2K/view?usp=drive_link

https://drive.google.com/file/d/1uWwrW2JXpkpnXz9KP7WVXJoXr37ZEQYW/view?usp=sharing

https://drive.google.com/file/d/1T8q6w-Db0lhkgMAhovrFsX1_WmcVAQ_f/view?usp=sharing

I just double checked the permissions.

NoKi wrote on 5/6/2024, 1:21 PM

@3d87c4

I don't know why, but all 'left' links download the same file here. So I did a test with the pair "...031.MP4" and "...016.MP4".

After running the script with "00" frame offset I get the file "Timecode In" values matching the values from MediaInfo:

Then I run the script with "05" frames offset on ...016.MP4:

Now, the "Timecode In" of ...016.MP4 has an offset of 5 frames (from 18:34:27,36 to 18:34:27,41), like it should. The "Timecode In" difference is now 2 frames in total between the two files.

I drag the media to different tracks and run the relative positioning script:

...031.MP4 is now at position 00:00:00,00 and ..016.MP4 is at position 00:00:00,02. So, still 2 frames difference.

This is exactly what I would expect to happen, so for me the scrips seem to work properly. Have you set the project properties to the correct framerate and the timeline to "Time and Frames"? This might make a difference in how Vegas does the timecode math.

Nils

3d87c4 wrote on 5/6/2024, 1:35 PM

@NoKi:

Weird about the drive links.

Here's another attempt at the left clips:

https://drive.google.com/file/d/1QDVNGd5gh54n3yrCcDdjykkrx-xbrV6W/view?usp=sharing

https://drive.google.com/file/d/1lSV_XDVcKN_EOpoQ767PQsx3_zj_eoc9/view?usp=sharing

https://drive.google.com/file/d/17NV5KGQfzWMfoWieRKwb59J6-4Lqjcbc/view?usp=sharing

I let Vegas set the project parameters based on the first clip added so those should be OK. I believe I checked the timecode settings but will try again to be sure.

Is there a way, in Vegas, to check the media properties & verify the time code changes?

NoKi wrote on 5/6/2024, 2:15 PM

Is there a way, in Vegas, to check the media properties & verify the time code changes?

To my knowledge, only in the "Project Media" and "Edit Details" windows. The media properties will also show you the timecode data and how the timecode is interpreted (e.g. "Time & Frames").

I tested now all 6 files and the results look ok. All files no offset:

Five frames offset to the "right" files:

And after positioning:

Though, I get your point that the global positioning of all events at once is not very convenient for files from takes with longer time between them.

So, please put all events on different tracks and only select them 3D-pair by pair and run this script:

using System;
using ScriptPortal.Vegas;

public class EntryPoint
{
    public void FromVegas(Vegas vegas)
    {
        Timecode minPosition = Timecode.FromString("23:59:59,00");
        foreach (Track theTrack in vegas.Project.Tracks)
        {
            foreach (TrackEvent ev in theTrack.Events)
            {
                if (!ev.Selected) continue;
                
                if (minPosition > ev.ActiveTake.Media.TimecodeIn)
                {
                    minPosition = ev.ActiveTake.Media.TimecodeIn;
                }
                
            }
        }
        // vegas.Project.Ruler.StartTime = minPosition;        
        foreach (Track theTrack in vegas.Project.Tracks)
        {
            foreach (TrackEvent ev in theTrack.Events)
            {
                if (!ev.Selected) continue;

                Timecode position = ev.ActiveTake.Media.TimecodeIn + ev.ActiveTake.Offset - minPosition;
                ev.Start = position;

                if (ev.Group != null)
                {
                    foreach (TrackEvent groupevent in ev.Group)
                    {
                    groupevent.Start = position;
                    }
                }
            }
        }
    }
}

It will only put the selected events to their relative timeline position. So only the time offset between left/right will be kept, but all pairs will be located at the timeline start.

Nils

3d87c4 wrote on 5/6/2024, 4:01 PM

@NoKi:

The fog is clearing...

I figured out how to get project media details to display.

My comment above is the clue to my basic error---I wasn't explicitly setting my project properties until dragging the media to the timeline. So, during my previous trials I was doing the initial sync step with the default 1080p 29.95 fps project settings.

Fortunately, I have a project preset for my H12 3D editing. Setting that first has the shift functions working. But that is revealing two issues:

The GoPro audio track is slightly longer than the video and that's why the later clips aren't quantized to frames properly.

Second, the clips may not have exactly the same frame shift so I am noticing 1 frame differences in their location on the timelines. (Typing this I'm wondering if this is related to the audio/video length problem. I'll check whether this happens using the other shift script.)

Ooops: I didn't notice your last post before composing this one. I'll try the workflow you suggest.

3d87c4 wrote on 5/6/2024, 5:44 PM

@NoKi:

The difference in shifts is real, the first pair needs a 2 frame shift but the other two needed 3 frames. I tried out the new shift script and have a couple of workflow (in addition to what you suggest above).

What I did this afternoon was to do the initial sync to the left and right clips in the project media area.

Then I would load a left and right clip to consecutive timelines, use the new script to shift them, pair them as a 3D subclip, and delete the audio track left by the merged right clip. I then load the next pair of 3D clips on the tracks below the first and repeat. This slightly reduces the number of tracks, but would still get cumbersome.

A different approach---after the initial sync of the left and right clips---would be to load a pair of clips to the the timelines, shift them, pair them as 3D subclips, then delete them from the timeline and repeat with the next pair until done. This would leave the paired subclips in the project media folder ready to drag to the timeline for the actual video editing.

3d87c4 wrote on 5/6/2024, 6:11 PM

So...here are the two scripts from @NoKi I'd like to flag as the solution for this thread:

This script is used to get the timecode from clips selected in the project media area. It can be edited to shift the time code, if needed, to correct synchronization errors. For this you need to create copies and edit them to reflect the number of frames, plus or minus, they need to move. For QR code synced gopros I suspect only a few copies will be needed. The line to edit is this

// set your offset in frames (project fps used) below, negative values possible
string offsetInFrames ="00";

using System;
using System.IO;
using System.Drawing;
using System.Collections.Generic;
using ScriptPortal.Vegas;
using System.Diagnostics;
using System.Windows.Forms;



public class EntryPoint
{
    public static Vegas myVegas;
    
    public void FromVegas(Vegas vegas)
    {
        myVegas = vegas;
                
        // set your offset in frames (project fps used) below, negative values possible
        string offsetInFrames ="00";
    MessageBox.Show(offsetInFrames);
        
        Timecode offset = Timecode.FromString(offsetInFrames, RulerFormat.TimeAndFrames);
        Media[] selectedMedia = myVegas.Project.MediaPool.GetSelectedMedia();
            
        foreach (Media media in selectedMedia)
        {
            if (media.HasVideo())
            {        
                FileInfo file = new FileInfo(media.FilePath);                                        
                string mediaFilePath = Convert.ToString(file);
                mediaFilePath = mediaFilePath.ToLowerInvariant();
                
                //if (mediaFilePath.EndsWith(".mov"))
                {
                    string panaTimecode = GetTimeReference(mediaFilePath);                
                    MessageBox.Show(panaTimecode);
                    
                    if (panaTimecode != "")
                    {
                        // Timecode newTimeIn = Timecode.FromString(panaTimecode, RulerFormat.Time);    
                        Timecode newTimeIn = Timecode.FromString(panaTimecode, RulerFormat.TimeAndFrames);
                        // Timecode newTimeIn = Timecode.FromString(panaTimecode, RulerFormat.SmpteEBU);                        
                        media.UseCustomTimecode = true;
                        media.RulerFormat = RulerFormat.TimeAndFrames;
                        //media.RulerFormat = RulerFormat.TimeAndFrames;
                        //media.RulerFormat = RulerFormat.SmpteEBU;
                        media.TimecodeIn = newTimeIn + offset;    
                        string commentOffset = "File TC & " + offsetInFrames + " frames offset";
                        media.Comment = commentOffset;
                    }
                }                
            }    
        }        
        myVegas.Project.MediaPool.OpenAllMedia();
    }    
    static string GetTimeReference(string mediaFilePathTmp)
        {
            string ffprobeParameter = " -v error -select_streams v:0 -show_entries stream_tags=timecode -of default=nk=1:nw=1 -i " + "\"" + @mediaFilePathTmp + "\"";
            //MessageBox.Show(ffprobeParameter);
            Process processffprobe = new Process();
            processffprobe.StartInfo.FileName = @"C:\ffmpeg\bin\ffprobe.exe";
            processffprobe.StartInfo.Arguments = ffprobeParameter;
            processffprobe.StartInfo.CreateNoWindow = true;
            processffprobe.StartInfo.UseShellExecute = false;
            processffprobe.StartInfo.RedirectStandardOutput = true;
            processffprobe.Start();

            string timeRefSampleRateTmp = processffprobe.StandardOutput.ReadToEnd();          

            return timeRefSampleRateTmp;
        }

}

This script is used to apply the synchronization adjustment to a pair of clips one, above the other, in consecutive timelines.

using System;
using ScriptPortal.Vegas;

public class EntryPoint
{
    public void FromVegas(Vegas vegas)
    {
        Timecode minPosition = Timecode.FromString("23:59:59,00");
        foreach (Track theTrack in vegas.Project.Tracks)
        {
            foreach (TrackEvent ev in theTrack.Events)
            {
                if (!ev.Selected) continue;
                
                if (minPosition > ev.ActiveTake.Media.TimecodeIn)
                {
                    minPosition = ev.ActiveTake.Media.TimecodeIn;
                }
                
            }
        }
        // vegas.Project.Ruler.StartTime = minPosition;        
        foreach (Track theTrack in vegas.Project.Tracks)
        {
            foreach (TrackEvent ev in theTrack.Events)
            {
                if (!ev.Selected) continue;

                Timecode position = ev.ActiveTake.Media.TimecodeIn + ev.ActiveTake.Offset - minPosition;
                ev.Start = position;

                if (ev.Group != null)
                {
                    foreach (TrackEvent groupevent in ev.Group)
                    {
                    groupevent.Start = position;
                    }
                }
            }
        }
    }
}

I will continue to tap the tray my Gopro's are mounted on with a coin---otherwise it's hard to know how much they really need to shift.

Thank you for your help @NoKi!