Marrying external audio to Video clips so they are seen as one file

tim-neighbors wrote on 4/23/2021, 3:16 PM

I'm doing a multi-cam edit where they shot multiple takes and pickups with audio externally recorded. It's a mess. Is there a way to Marry 24 channels of audio to 4 channels of video where they are all stored as takes in one file that is openable in the trimmer. Like rendering a multi-cam track with 24 channels of audio where 4 video streams are embedded in the file as takes or something? I know it's probably a no ...but what is the best workflow for editing around in a multi-cam project where you're splicing together different takes that were all shot with 4 cameras and 24 channels of audio? After the edit, I'll need to render out audio stems for the mixing engineer and I want to be able to choose between cameras after pasting in the pickups, so using nested projects won't do.

Comments

jetdv wrote on 4/23/2021, 4:11 PM

As they are all separate files, you can simply group them and they will all move together but you won't be able to "open them all in the trimmer". If you want a single file you can open in the trimmer then you'll need to render out a file containing what you want. For example, you could create a 2x2 PIP and then render out that file. Then you can open that in the trimmer and see all four angles.

Musicvid wrote on 4/23/2021, 5:58 PM

Was the external audio genlocked?

tim-neighbors wrote on 4/23/2021, 6:11 PM

Was the external audio genlocked?

They did a manual jam sync, so the timecode is all within about a second of each other. ...cameras and audio.

joelsonforte.br wrote on 4/23/2021, 8:04 PM

Is there a way to Marry 24 channels of audio to 4 channels of video where they are all stored as takes in one file that is openable in the trimmer.

There are several ways to do this. In your case I would do the following:  

After synchronizing all events on the timeline using the timecode, create two multicamera tracks.  

-The multicam track 01 would be for the takes of the video clips with their respective audios. 
-The multicam track 02 would be for the takes of the 24 audio channels.  

-To group the two multicam tracks, use the group feature. 
-To switch between the various Takes on the two tracks use the Take feature or use Vegasaur. With this, it will not be necessary to use the Trimmer

Like rendering a multi-cam track with 24 channels of audio where 4 video streams are embedded in the file as takes or something? .

Vegasaur offers several ways to do this. One way is to create multiple projects with the Takes you want to render and use the Vegasaur Transcoder to import all projects and render everything in batch.

Good Luck!

tim-neighbors wrote on 4/23/2021, 9:14 PM

Is there a way to Marry 24 channels of audio to 4 channels of video where they are all stored as takes in one file that is openable in the trimmer.

There are several ways to do this. In your case I would do the following:  

After synchronizing all events on the timeline using the timecode, create two multicamera tracks.  

-The multicam track 01 would be for the takes of the video clips with their respective audios. 
-The multicam track 02 would be for the takes of the 24 audio channels.  

-To group the two multicam tracks, use the group feature. 
-To switch between the various Takes on the two tracks use the Take feature or use Vegasaur. With this, it will not be necessary to use the Trimmer

Like rendering a multi-cam track with 24 channels of audio where 4 video streams are embedded in the file as takes or something? .

Vegasaur offers several ways to do this. One way is to create multiple projects with the Takes you want to render and use the Vegasaur Transcoder to import all projects and render everything in batch.

Good Luck!

This is amazing! Thanks so much for taking the time to craft this response @joelsonforte.br . I'm touched. This approach certainly is cleaner and you've given me some ideas. If only a multi-cam track could be opened in the trimmer, I'd be set.

Musicvid wrote on 4/23/2021, 9:34 PM

They did a manual jam sync, so the timecode is all within about a second of each other. ...cameras and audio.

Meaning neither the cameras nor the audio recorders were slaved to timecode, but left to use their internal clocks?

Welcome to my world, Tim. ca. 2007.

Howard-Vigorita wrote on 4/24/2021, 2:09 PM

Call me old fashioned but when I shoot multicam concerts with a stage full of mics and electronic instruments, I just throw it all into Vegas and do both the audio mix and cuts between cameras right there. I generally start with my A-cam and align its audio track to all the external audio sources, one audio track per source. I usually do it by eye using the A-cam audio track waveform aligned to one of the the external audio track waveforms. I don't give audio stems to to the audio engineer. I get his. Preferably one continuous 24-bit wav file per channel, post head amp, but pre-fader. If done properly, all the audio tracks off the mixer are captured simultaneously and in perfect alignment relative to one another. It helps if all your cameras and the audio recorder stay on continuously from beginning to end. That minimizes the alignment challenge. I try to only ever power down and up again for intermission.

If the audio guy wants the stems back after I eq, level, and knock out dead spots, I suppose I could solo each Vegas audio track from beginning to end and render each as wav64 mono... but I have yet to meet the audio guy looking for more work. In fact all the audio mastering engineers I've ever worked with say: no stems. They just want the uncompressed mix.

I throw in the other cameras after the audio mix is done and ultimately remove all the audio except the mixdown tracks before doing the multicam edit. Saving the project with a unique name every step of the way and never changing the start point makes it possible to back up and redo a step if necessary for adjustments... I often hear things in the mix while doing multicam cuts and go back and re-tweak the audio mix.

tim-neighbors wrote on 4/24/2021, 3:06 PM

Call me old fashioned but when I shoot multicam concerts with a stage full of mics and electronic instruments, I just throw it all into Vegas and do both the audio mix and cuts between cameras right there. I generally start with my A-cam and align its audio track to all the external audio sources, one audio track per source. I usually do it by eye using the A-cam audio track waveform aligned to one of the the external audio track waveforms. I don't give audio stems to to the audio engineer. I get his. Preferably one continuous 24-bit wav file per channel, post head amp, but pre-fader. If done properly, all the audio tracks off the mixer are captured simultaneously and in perfect alignment relative to one another. It helps if all your cameras and the audio recorder stay on continuously from beginning to end. That minimizes the alignment challenge. I try to only ever power down and up again for intermission.

If the audio guy wants the stems back after I eq, level, and knock out dead spots, I suppose I could solo each Vegas audio track from beginning to end and render each as wav64 mono... but I have yet to meet the audio guy looking for more work. In fact all the audio mastering engineers I've ever worked with say: no stems. They just want the uncompressed mix.

I throw in the other cameras after the audio mix is done and ultimately remove all the audio except the mixdown tracks before doing the multicam edit. Saving the project with a unique name every step of the way and never changing the start point makes it possible to back up and redo a step if necessary for adjustments... I often hear things in the mix while doing multicam cuts and go back and re-tweak the audio mix.

That's how I'll typically work as well ...for a concert or something where it's shot as a traditional multi-cam coverage shoot. This shoot however is of a play, without audience, so they backed up and re-ran parts to fix mistakes, etc. ...but where it really gets challenging for me is that they shot some scenes where they are depending on the pickups to show certain characters when they say their lines. For example, with 4 people on stage, they covered the right two during the first run, reset and ran it again, this time covering the left 2, etc. So I have to figure out how best to edit between takes, bouncing back and forth between takes ...with 4 cameras and 28 channels of audio. I'd mix the audio myself, but I don't think I'd have time, so I'm going to try to get to picture lock, then send stems (would love to send an OMF but Vegas can't) to a sound mixer to handle that part of the job as I do color correction, titles, and do another shoot. That's the dilemma. My current plan remains having the raw takes open in a separate project and copy/pasting them in as needed, maybe using multicam tracks to keep it cleaner.

Musicvid wrote on 4/24/2021, 4:26 PM

Different performances and scene takes? Having been there, Tim, I advise you to stock up on burritos. You are in for a long spring.

Howard-Vigorita wrote on 4/26/2021, 2:23 AM

Trying to visualize how I'd approach it if I were stuck with your challenge. When I do live sound for musicals, I tend to work off the script and map out a mixer snapshot of a new sub-scene whenever an actor enters or exits the stage. A similar approach might simplify your video issue... create a separate multicam mix, perhaps in a separate project, for every such instance. Your multicam tracks would consist of not only the 4 cameras that are going simultaneously, but also treat the retakes like more cameras. If they ran all 4 for each of 2 additional takes, you would have 12 camera angles to choose from in multicam edit. You probably want to include the audio in the multicam track so when you select a camera angle, the audio selects with it... I don't ever do that myself with my own workflow but it should work... otherwise, select only the video for the multicam track and activate a take's audio manually either by muting or with velocity envelopes. Just make sure you get all the takes into alignment before creating the multicam track... kind of a mess changing the alignment while in multicam mode. Btw, you definitely want to group each video clip with its associated audio tracks before alignment so they slide around together. If you go with the multiple nested project approach, after you've finalized your take selections in each sub-scene's project, you can then nest all the sub-scenes together in a master project with transitions of your choice between sub-scenes.

tim-neighbors wrote on 4/27/2021, 9:17 PM

Trying to visualize how I'd approach it if I were stuck with your challenge. When I do live sound for musicals, I tend to work off the script and map out a mixer snapshot of a new sub-scene whenever an actor enters or exits the stage. A similar approach might simplify your video issue... create a separate multicam mix, perhaps in a separate project, for every such instance. Your multicam tracks would consist of not only the 4 cameras that are going simultaneously, but also treat the retakes like more cameras. If they ran all 4 for each of 2 additional takes, you would have 12 camera angles to choose from in multicam edit. You probably want to include the audio in the multicam track so when you select a camera angle, the audio selects with it... I don't ever do that myself with my own workflow but it should work... otherwise, select only the video for the multicam track and activate a take's audio manually either by muting or with velocity envelopes. Just make sure you get all the takes into alignment before creating the multicam track... kind of a mess changing the alignment while in multicam mode. Btw, you definitely want to group each video clip with its associated audio tracks before alignment so they slide around together. If you go with the multiple nested project approach, after you've finalized your take selections in each sub-scene's project, you can then nest all the sub-scenes together in a master project with transitions of your choice between sub-scenes.

Thanks for your suggestions. The idea of placing multiple takes under each other, creating a (for example) 12 cam split -doesn't work because the play is not being performed to a click track or anything, so they quickly fall way out of sync with each other. The nested events idea doesn't allow you to stitch together individual audio tracks at your take splice points and/or output one master EDL project or stems to go out to the audio mixer.

For anyone who ends up in this situation, what I'm doing is, I created a multicam track that has all takes for the whole show in it, re-grouped each take with it's sync'ed external audio tracks, then started cutting everything as a multicam split. When I encounter a good pickup that should be used, I splice it in along with it's audio layers. Each completed scene that will make the cut, gets moved up one video layer, then when I'm done I'll cut out all of the unused material that didn't get moved up to the top video layer and ripple edit to collapse each gap, stitching them together. I saved a separate project with all of the takes uncut to refer back to if the client or I need to look for something later. I can open that project at the same time and copy/paste portions back into the master project. It's slow going and I'm being very careful not to screw anything up with these 30 tracks. This may be one of the most challenging edit jobs I've done in my 20 yrs of editing. If the cameras and audio had been able to sync timecode properly, the entire process would be much much easier, I would've saved 4 days of syncing events, and I'd be able to quickly fix any out of sync stuff I come across. I hope this helps someone. Oh, and pluraleyes and the like would not work for this project because there are rotating actors in the greenroom with their mic on talking throughout and the cameras are not hearing audio that is close enough to each other. What a mess!

tim-neighbors wrote on 4/27/2021, 9:24 PM

They did a manual jam sync, so the timecode is all within about a second of each other. ...cameras and audio.

Meaning neither the cameras nor the audio recorders were slaved to timecode, but left to use their internal clocks?

Welcome to my world, Tim. ca. 2007.

Ooh, and you did a surround sound mix too! Nice. maybe I'll post a pic of this bear of a project when I'm done. I'll also be going through and splicing in words from different takes on individual mic channels to fix lapel rubs that happen throughout the play and color correcting to match between lighting continuity issues between take splices. There might not be enough burritos. And I want to have a rough cut done in a week.

Musicvid wrote on 4/27/2021, 9:25 PM

Our amateur production of "West Side Story" actually never reached the VHS distribution stage for many of the same reasons...

JN- wrote on 4/28/2021, 6:44 AM

@tim-neighbors “Is there a way to Marry 24 channels of audio to 4 channels of video where they are all stored as takes in one file that is openable in the trimmer.”

Audio is certainly far from my area of expertise, if any🤪. The following may not be of use, but FWIW.

 

Would creating 4 video files each with 6 audio channels be of use?

A while back I had a need to make up an ffmpeg util that extracted just a single audio and a single video stream from a video clip that contained multiple audio streams.

To test the above util I had to first generate a source video file with a single video stream and multiple audio streams, using ffmpeg. In your case, I assume that the audio streams would ideally need to be already synced before creating each file.

Multiple video streams can also get added with the audio streams. However, when I tested this in VP18 only the 1st. video stream displayed, yet playing the multiple streams output clip in MPC, Media Player Classic, revealed all video streams correctly, (and all audio streams) as it allows selection of discrete audio or video streams for playback. Also, note that adding in .mp3 audio streams is now a mess in VP18, was previously ok. This MP3 issue has been reported elsewhere in the forum.

Plus, the output audio streams, even if PCM (input as WAV) are shown in VP as AAC, and now nothing plays correctly. This was previously aok, its either a Windows or VP issue or both.

So currently my "fix" is in-operable in VP even though mediainfo and MPC/VLC display and play the different video and audio tracks aok. I tested on Nero 2018, video app, and it also plays the discrete audio tracks aok.

Last changed by JN- on 4/28/2021, 2:31 PM, changed a total of 5 times.

---------------------------------------------

VFR2CFR, Variable frame rate to Constant frame rate link to zip here.

Copies Video Converts Audio to AAC, link to zip here.

Convert 2 Lossless, link to ZIP here.

Convert Odd 2 Even (frame size), link to ZIP here

Benchmarking Continued thread + link to zip here

Codec Render Quality tables zip

---------------------------------------------

PC ... Corsair case, own build ...

CPU .. i9 9900K, iGpu UHD 630

Memory .. 32GB DDR4

Graphics card .. MSI RTX 2080 ti

Graphics driver .. latest studio

PSU .. Corsair 850i

Mboard .. Asus Z390 Code

 

Laptop… XMG

i9-11900k, iGpu n/a

Memory 64GB DDR4

Graphics card … Laptop RTX 3080

JN- wrote on 4/28/2021, 6:30 PM

I know Tim that the following is not addressing your core issue here, but since I have brought this up I might as well finish it here. "This was previously aok, its either a Windows or VP issue or both."

Just in case anyone from Magix is listening in. I know that the multiple audio stream clip was ok when I previously loaded the finished file into VP because I used VP to select, play each discrete audio stream to verify that all was in order. It was the only tool I knew of for doing this, I didn't know at that time that VLC, MPC and Nero could also be used to check out/play the individual audio streams within the multi stream clip.

So I just now installed VP18 b373 which I would have used then, it's two builds back ~ 31st-October-2020. The multi stream clip still now doesn't work properly, it was created 26th-Dec-2020, i.e. b373 era.

On that basis it may be more some Windows change that has caused all of this?

Last changed by JN- on 4/28/2021, 6:34 PM, changed a total of 1 times.

---------------------------------------------

VFR2CFR, Variable frame rate to Constant frame rate link to zip here.

Copies Video Converts Audio to AAC, link to zip here.

Convert 2 Lossless, link to ZIP here.

Convert Odd 2 Even (frame size), link to ZIP here

Benchmarking Continued thread + link to zip here

Codec Render Quality tables zip

---------------------------------------------

PC ... Corsair case, own build ...

CPU .. i9 9900K, iGpu UHD 630

Memory .. 32GB DDR4

Graphics card .. MSI RTX 2080 ti

Graphics driver .. latest studio

PSU .. Corsair 850i

Mboard .. Asus Z390 Code

 

Laptop… XMG

i9-11900k, iGpu n/a

Memory 64GB DDR4

Graphics card … Laptop RTX 3080