prores 422 completely out of synch with mfx files

Mindmatter wrote on 12/7/2020, 10:19 AM

Hi all

I'm working on a multicam project with 3 mxf clips from Somny FS7 and 1 prores 422 recorded on an atomos, recorded from a Sony FS7 over HDMI. While synching ( visually with the help of the audio waves, like I always do ) all 3 cams do just fine, but the prores in completely out, video as well as audio. The audio starts off ok, then drifts massively behind pretty soon.Even within the file, the audio is dephased from from the video - it's a total mess, as I have very little visual clues to at least synch the video and take the audio from the other cams. All is 25p, all audio 48khz 24bit. so even when I resynch tho audio of the prores track to itself, I can't rely on the wav image to synch with the other tracks as the audio drifts.

I should mention that the audio came from a digital desk used for a multitrack.recording

Is there any way out of this, like transcoding the prores file first? Or is the only way out to try and find clues in the image?

Thanks!

Comments

RogerS wrote on 12/7/2020, 9:46 PM

Hi there, this sounds like a tough issue to deal with. So the 3 cameras are synced well, but the remaining ProRes is both desynced for audio and visuals?

I'd break apart the audio from the video for the defective ProRes (ungroup or u) and then use your best effort to sync it visually to the nearest other camera. Try to match them towards the start of the clip and then look towards the end and see if it drifts. If it does you could try to also match end points and compress the clip (control and drag) to match. Then do the same with the audio based on something sharp in the waveform (something drops, a clap, whatever you have that is distinct). Either make multiple cuts and sync the audio until it noticeably drifts and then make another cut and sync again, or try the start and end points to retime it.

PluralEyes may be able to automatically analyze the waveform and correct for drifted clips- I use it like that. I have one recorder that's so variable the rate of drift also drifts and PluralEyes can't help with it.

Mindmatter wrote on 12/8/2020, 4:28 AM

Thanks Roger
I actually spent a whole day looking for visual clues..and found them!

It was 7 different takes from 7 songs during a singing rehearsal...but..with masks! ( but yes, you still see when the body or eye movements are off)

So I looked for tne rare moments where that cam caught the conductor speaking or playing a note on the piano to tune the singers in, and synched by eye. It worked, but made for many hours lost.

I wonder what made the audio drift so far off though.Was it the HDMI, or the codec?

RogerS wrote on 12/8/2020, 6:37 AM

The codec isn't an issue. It's a difference in the internal clock in the recording device vs the camera.

Glad you found a way to sync them. I do recommend clap sync at start and finish to at least get some kind of reference. PluralEyes does syncing by matching the waveforms, which usually works well.

If you do a lot of this and clapping or PluralEyes isn't enough there are timecode gadgets like TentacleSync that might be worth it for you.

Dexcon wrote on 12/8/2020, 6:49 AM

I do recommend clap sync at start and finish to at least get some kind of reference.

+1 @RogerS ... And if there isn't a clapper board handy (placed upside down for the finish clap), just use 'stretched out' hands as a clapper - a long used standby when a clapper board isn't handy.

Mindmatter wrote on 12/8/2020, 1:58 PM

a clap would not help, as the issue is drifting audio.

It starts off in synch and ends up completely off / late after 10 minutes, the longer the worse. I ended up synching to the audio of another track.

RogerS wrote on 12/8/2020, 8:39 PM

A clap is a reference which helps match clips when stretching to offset drifting manually, or automatically with sync software. Try it at the start and end. I do this for long interviews with multiple cameras and recorders.

(Though I would find a better way to record so the video/audio from each camera remains in sync with itself.)

RogerS wrote on 12/8/2020, 9:33 PM

I'm still puzzling over that one ProRes camera that's out of sync within the same file. Was it recorded with the video coming in from the camera and audio coming in from an external deck? If so, I think that's your problem.

I'd run the audio from the deck into the camera through mic/line in (may need an attenuator cable depending on the levels it outputs). If the quality is acceptable like this, just run both audio and video from the camera to the Atomos and use that. If the audio quality is not acceptable, also record directly from the deck to a different recorder and use that as the master audio to match the tracks to. At least you'll have reference audio for that camera with a waveform that will perfectly match the recorder, making syncing easier.

Mindmatter wrote on 12/9/2020, 7:11 AM

Hi Roger

that's actually how it was done - the desk gave me a line signal direct to the FS7's XLRs. Both audio and video then ran to the Atomos via HDMI. Visually at least, the clap would definitely have helped, I know what you mean now.

RogerS wrote on 12/9/2020, 7:33 AM

Very interesting.

I'm still confused then of how the audio and visuals are drifting if they are run through the camera to the Atomos. Hmm. You might file a support request with Atomos and see if they have tips or advice for avoiding this? HDMI is a digital signal so it really shouldn't be the cause of a problem.

Howard-Vigorita wrote on 12/11/2020, 10:26 AM

@Mindmatter if the atomos and camera split clips into separate files as they record, you might want to try aligning the individual file clips separately to see if the atomos or the camera are dropping frames between file chunks they save. Many low end cameras do that but I wouldn't expect that of the fs7 or atomos... if that's what's happening maybe firmware updates to both the camera and the atomos might fix it.