Date Created To Timecode In Or Date/Time Stamp?

doubleJ wrote on 8/28/2018, 8:15 PM

Lining up clips from various sources has always been a bit of a nightmare.
Cameras, phones, etc... always have different dates and times.
Some record audio, some don't, and some are so quiet that it isn't worth recording.
On top of that, when you copy the files to a new drive, it changes Date Created.
I've seen PluralEyes, but it seems to just use audio to align clips.

I use a program called Advanced Renamer to set the Date Created metadata using Date Taken or the filename (if it has a timestamp).
I make a copy of all the files (or just rename them) using the newly-set Date Created as the filename YYYYMMDD-HHMMSSxx-CameraName (xx is whatever is smaller than seconds).
That way I have a reasonable reference of how they should all line up.

I just noticed that Vegas Pro has two options that I haven't seen before - Lay Out Tracks Using Media Timecode and Lay Out Tracks Using Media Date/Time Stamp.
My first thought was that this would solve all of my renaming steps, since it would use the Date Created information (I would still have to set that, given that it would likely be wrong).
Unfortunately, none of my files seem to have any Timecode (I assumed that one) or Date/Time Stamp information.

Is there a way to set one (or both) of those fields by Date Created?
I didn't even see a way to manually edit them in Project Media.
JJ

Comments

doubleJ wrote on 9/1/2018, 10:59 AM

Scouring the interwebs, I found a program called DVMP Pro.
The interesting thing is that it reads some kind of timecode in every one of my video files.
Some of the sources are spot on, some are a few seconds or minutes off, and others are in GMT (+5:00 from my time).

The problem is that it's apparently different timecode than NLEs use.
Neither Vegas Pro nor Premiere Pro show any timecode.

Continuing in my never-ending quest to make my life easier, there must be some way to get this information into an NLE.
JJ

doubleJ wrote on 9/1/2018, 11:37 AM

From DVMP's website...

Embedded Metadata

 

The metadata items are stored as continuously changing values while the camera is recording. So for example if the camera's aperture changes while a recording is being made then you will be able to see this happen in "real-time" when you play the file in DVMP Pro. If you are using the Burn-in tool then you will also see the the values change in the burned-in file.

 

For files which have Embedded Metadata this is the most reliable Metadata Source because the metadata values are stored within the video file and so can not become separated from it. Embedded Metadata also tends to contain more metadata items than other Metadata Sources. Always prefer Embedded Metadata where available.

 

File Header

 

The metadata items are stored as single values in the video file's basic header or advanced header. The values usually apply to the instant that the camera created the file. Like Embedded Metadata the values are stored within the video file and so can not become separated from it - in this sense it is reliable. However the specifications for storage of metadata in basic file headers is much looser so it is less reliable than Embedded Metadata.

 

For example, MOV and MP4 files are created by many types of video recording devices, some of which may not store the date and time reliably in the basic file header, even if the device's own clock had been set correctly when the recording was made. Some recording devices may store the time without any daylight-time adjustment. Some may store the time as Greenwich Mean Time (GMT) rather than the local time, so depending on where on earth the footage was shot the time may appear to be several hours different to the local time when the recording was made. Worse still, some budget devices even store a single fixed date and time for every file that it records, so for example all files may seem to have been recorded on 1st January 2004 (or any other arbitrary date and time). And of course some devices may not store any date and time at all.

 

Fortunately, some devices also store an Advanced header that contains date, time and other information that is much more reliable than the basic header. If the video file contains an advanced header then DVMP Pro will use that instead of the basic header. When playing the video file, the presence of an advanced header is indicated by the HDA label - if there is only a basic header the label will be HDR.

 

If you have video files that only have a basic header, you need to take extra care. If you have access to the camera that recorded the files, then you can simply make a test recording at a known time of day, and then play the recorded file in DVMP Pro and compare the displayed date/time with your known time to see if they are different, and by how much. If the difference was consistent, you could then use the Adjust Date and Time by setting in Tools > Options > Metadata Sources to effectively "remove" the difference. But if you do not have access to the camera that recorded the files, then there is no way of knowing whether the date and time stored by the camcorder in the basic header was in local time, daylight saving time, GMT, etc. In that case you should treat the displayed date and time as being unreliable, and find some other independent proof of the true date and time when the files were recorded.

That's interesting information.

I think the problem is that my video files have the information in the header and not embedded.
Or, it may be a difference between Timecode and Date Recorded/Time Recorded.

•Timecode - a unique frame identifier used in professional video production (not to be confused with the date and time of recording)

•User Bits - a unique frame or clip identifier used in professional video production

•Time of Recording - the time of day when the recording was made

•Date of Recording - the date when the recording was made

JJ

doubleJ wrote on 9/1/2018, 11:48 AM

From the developer...

Some file formats do not have a method/place for
storing timecode metadata (hh:mm:ss:ff), and some/most cameras do not store timecode
metadata. For example, AVCHD format can store timecode as metadata in the video
file, but most consumer (and some pro) camera models do not actually store it.

I suspect that for MP4 and MOV files the timecode displayed by NLEs is just
"manufactured" internally by the NLE that always starts at
00:00:00:00 at the beginning of the file. Most MP4/MOV files do not have a
method of storing metadata timecode.

The values displayed by DVMP Pro when playing files are described on this page
- scroll down to the "Metadata Display" section:

https://www.dvmp.co.uk/help/v7/playing_video_files.html

For MP4 and MOV there is no timecode from metadata so it is displayed as
--:--:--:--

The grey value in the bottom left is a manufactured timecode that always starts
from 00:00:00:00 - its only purpose is to give feedback of the position when
the slider is dragged or when frame-stepping.

The time-of-day (hh:mm:ss) is displayed at the top of the second column - this
is the wall-clock time that was stored inside the video file by the camera at
the moment of recording so it will be accurate to the camera's internal clock
(though some frustratingly record wall clock time in GMT/UTC).

Time-of-day and timecode are completely separate entities.

It sounds like you want to manufacture timecode hh:mm:ss:ff from the filesystem
properties time-of-day hh:mm:ss.sss. Apart from the dangers of using filesystem
values already mentioned, most file types will not have anywhere for this to be
stored as timecode metadata in the video file. So I don't think it is possible
to do this.

So, it sounds like the limitation is the file format.
Maybe transcoding to some more professional format would give the ability to insert that information as timecode.
Of course, since the data may not be completely accurate, some massaging would need to occur before the insertion.
At that point, I question how much of a benefit it would be versus just lining up the clips manually.
:(
JJ

OldSmoke wrote on 9/1/2018, 12:04 PM

I personally don't even use PluralEyes; it takes too long and it's not accurate. I use Vegasaur and create a "Videowall", 2x2 or 3x3 at most and I use visual queues to lineup the clips which is rather easy for my kind of event recording.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)

doubleJ wrote on 9/1/2018, 3:55 PM

So, on an even wilder front...
I downloaded a bunch of sample footage (R3D, XAVC, XDCAM, DNxHD, DV, ProRes, maybe others).
I imported 17 video files into Vegas Pro 16 and Premiere Pro CC 2018.
Of all the files, only 2 showed timecode in both Vegas and Premiere (and that was an old DV AVI).
There were 8 files with timecode in Vegas and 5 files with timecode in Premiere.
It's likely that some of the other files simply didn't have timecode, but that's a pretty narrow margin.
How is a person supposed to make "standards-compliant" files?
Hehehe...

Going back to my initial topic, though...
If I was going to transcode video into one of these formats that does work, would I be able to insert timecode during that process?
Maybe if I would use FFMPEG, TMPGEnc, or VirtualDubMod?
JJ

doubleJ wrote on 9/2/2018, 6:43 PM

Ok, I'm making some head-way...

This just adds some metadata and it shows the timecode in both NLEs.

ffmpeg.exe -i originalfile.mp4 -metadata timecode="14:36:08:29" -metadata creation_time="2018-08-05 14:36:08" -c copy newfile.mp4

input file
timecode key and value in metadata
creation date/time metadata (this only seems to show in File Explorer properties details)
don’t do any encoding (just copy the video/audio)
output file

This does the same thing, but transcodes to the Avid DNxHD intermediate format (mxf didn’t open in Vegas, so I used mov).

ffmpeg.exe -i originalfile.mp4 -metadata timecode="14:36:08:29" -metadata creation_time="2018-08-05 14:36:08" -c:v dnxhd -b:v 145M -s 1920x1080 -r 30 -c:a pcm_s16le newfile.mov

I also tried ProRes, but Vegas Pro didn’t see the timecode.

I haven't been able to find ffmpeg information on encoding to CineForm.

So, now the work is to create a batch file that will read the Date Created file attribute and use it for timecode and creation_time.
I downloaded a batch script that reads a directory and echoes Date Created and Time Created, so I'm hopeful.
JJ

doubleJ wrote on 9/3/2018, 6:02 PM

They said it couldn't be done, but I did it!

https://1drv.ms/u/s!AlRf6b1u0N2PubRTXNXrhgj1RKwmbA

This reads the directories that are siblings of the batch file.
It goes into each directory and looks for *.mp4 or *.mov (you could add any formats you want, but I'm just dealing with normal DSLR/phone/etc... files).
WMIC (comes with Windows 10, but not sure about other versions) gets Date Created (you must have good Date Created for it all to work correctly).
FFMPEG (path is hard-linked - change to your preference) inserts date/time information into a new file (just a copy, not re-encoded - unless you use one of the commented-out options).
FFMPEG also uses the date/time and directory in the new filename.
NirCmd (path is hard-linked - change to your preference) uses the date/time information and changed Date Created for the copied file (since it would have fresh date/time information).

Some background information on how I organize things (if you're different, then you may need to adjust the batch).
I have a directory for the event and subdirectories for the sources.
For instance (this is all random and not representative of what I have or do)…

JukeJointGig-20180212
    BlackmagicUrsaMini
        021218121553.mp4
    CanonXl1
        File0001.mp4
    Htc10
        IMG_0142.mp4
    SonyNx80
        20180212-121426.mov

This might produce a bunch of files like this...

JukeJointGig-20180212-Transcoded
    20180212-120432-CanonXl1.mp4
    20180212-121426-SonyNx80.mp4
    20180212-121553-BlackmagicUrsaMini.mp4
    20180212-122102-Htc10.mp4

Each of those files would have Date Created, creation_time metadata, and timecode metadata matching the information in the filename.

There is one thing that may be worth noting...
You really need to pay attention to time zones and how the original creation_time metadata is stored.
I have a bunch of footage from a trip to Colorado (I live in Missouri).
Some of the devices have internet access and updated with the time zone change.
Others did not, so footage shot at the same time may have times that are an hour different (or days that are a day different, if around midnight).
Some devices stored an offset from GMT and others didn't.
That is something that should be worked out before doing this process, but it's worth noting.

I doubt anyone will have any use for this, but I'm very happy with myself.
Hehehe...
JJ

doubleJ wrote on 9/3/2018, 6:13 PM

Another thing worth pointing out...
Timecode, apparently, doesn't have date information.
So, if your footage spans multiple days, automating multicam from all the files will be problematic.
Of course, you can just select one day's worth of files (since the filenames start with the date).
JJ

doubleJ wrote on 9/3/2018, 6:15 PM

Now, this doesn't include pictures.
But, I can just have Advanced Renamer take care of that.
JJ

doubleJ wrote on 9/3/2018, 6:19 PM

Oh, and it doesn't use all subdirectory names in the output filename.
If the batch is in JukeJointGig-20180212 and you have CanonXl1\BRoll, CanonXl1\Registration, and CanonXl1\Stage, all the files will simply end with -CanonXl1.
Again, that's how I want my files to be (makes binning them easy).
JJ

doubleJ wrote on 9/7/2018, 8:47 AM

Just in case someone wants to use 29.97fps drop-frame, you need to change the framerate (-r) to 30000/1001 and timecode from 00:00:00:00 (colon) to 00:00:00;00 (semicolon).
JJ