Lip synching: Velocity curve on video+audio?

RichMacDonald wrote on 1/8/2006, 6:01 PM
I have several clips of a band playing the same song over and over (I had a limited supply of cameras :-) Now I want to put together a video using all these clips.

All these clips need to be synced up to the final audio. Naturally, I need to speed up and slow down each clip at different parts to get the "lip syncing" lined up. I have my master audio track on a track so I can watch its waveform. I lay down each video clip on another track and must adjust its speed to match the master audio. I do this for each clip.

One simple approach is go from left to right. Match up the first point of the master audio and clip audio, then insert a split downstream and ctrl-stretch each resultant event (with ripple) to line up that event's audio. Repeat till the end of the clip. This is easy and fast because you can really eyeball two audio tracks and line them up. Unfortunately, the stretched video can look poor, because basically I've used a "step-change" velocity pattern -- one event/section at 95%, then next at 97%, then next at 106%, etc. Even if my event/sections are sliced very small, the slowdown/speedup on the video can be unnatural. The problem is that there can be very small sections (a second or less) where I have to make large adjustments of 20%-40% and these are clearly visible.

A better approach would be a velocity envelope on the audio+video, as I could provide a smoother speed transition. The problem is that I want the audio to stretch simultaneously with the video, since the audio is the best eyeball to determine the final speed.

I can't figure out how to make that happen. When I change the video velocity, the audio stays the same. So I am forced to sync by eyeballing the musicians and the corresponding audio is useless. Actually its my attempts that are useless because its really hard to get things lined up when you only have the audio from one source the video from another.

Have I missed something obvious? Is this possible? Essentially I want to add a velocity envelope to an audio-video pair simultaneously. TIA.

Comments

Chienworks wrote on 1/8/2006, 6:33 PM
Vegas has no velocity envelope for audio, such a pity. About the closest you can come to it is the Pitch Bend function in Sound Forge. You can't auto-sync it to the video though. About the best you can do is "eyeball it", and see how close you come, and if you're off then adjust the envelope and try again.
ducnbyu wrote on 1/9/2006, 2:46 PM
I have tried what you are doing... it is not easy at all. I didn't see anyway to go by the audio waveform. In my inexperience I thought I was going to be able to stretch and compress various clips to sync the timing by ear. When I discovered how tedious that was I tried to look at the waveforms. But, each waveform varied too much... maybe your eye is keener than mine. And/or your waveforms are closer due to being in the same location with basically the same room noise. I was using VMS instead of Vegas, but I don't think that really matters here.

In hindsight, if I ever get the chance/need to do this again in the future I would have the artist play each take while the pre-recorded master audio was playing at a level that he could hear from an off camera ghettoblaster. Since the audio from the video would be thrown away, it wouldn't matter that the master would be heard in the background. It would be up to him to play/sing in sync with his original good recording. We talked about it, and we figured that is they way they do it in the music videos and probably still need multiple takes.

In your case with a drummer (I assume) I think it would be even harder the way you (and I) did it because not only do you have to lip sync you also have to drumstick sync... guitar out of sync is a bit more subtle as hand movement in lead guitar is very slight and rhythm guitar tends to get lost in the total sound. Unless of course there were extreme guitar close-ups which would be cool too.

Since my friend and I live in different parts of the country it was not practical to get together again and start from scratch so I painstakingly did what you are doing. It is not fun. While he was pleased with the end result I ended up feeling dirty and obsessed with showering and washing my hands. Not really. 99% of the song you couldn't tell that is the way it was done, but the other 1% shows that it was. So for that and thinking it could have been better and a lot easier, I was dissatisfied. BTW, I taped him playing each take at different locations and cut from place to place throught the song... that part was pretty cool... I was going to surprise him and film a little story that relates to the lyrics and interleave that music video like... but I got burned out on the project just getting the lip sync as good as I could.