Audio lip sync

24Peter wrote on 7/10/2003, 9:34 PM
I recently finished editing a music video and had a lot of trouble matching the artist's lip syncing with the audio track I took off her CD and imported into Vegas. I used the camera audio (recorded as she lip synced to her song) as my reference track. Is it possible that the CD track (which I'm pretty sure was sampled at 44.1Khz) would "playback" at a different rate than the "live" boombox lip sync sound I recorded with my camera at 48Khz?

Comments

Chienworks wrote on 7/10/2003, 9:46 PM
Boombox, you say? It's entirely possible that the boombox isn't up to spec and it was playing the CD at the wrong speed.

Vegas shouldn't have any difficulty combining a 48KHz track from DV and a 44.1KHz track from CD together. It should be able to play both together at the correct speeds.
filmy wrote on 7/10/2003, 10:25 PM
Playing from CD should be the same speed on any CD player, unless it has a speed control on it. There could be several issues at play however - you say you had problems with the lip sync so did the artists keep the sync? Also if you ran either the camera(s) or the playback off battery their could be a slight speed change. Another thing is that camera to camera, tape to tape, there could be slight speed issues.

I can say that before CD's were around, for lowbudget music videoa I would simply have the artist run a cassette off their 'master' and use that to playback during the shoot. Rarely were the sync issues massive as long as everything was up to speed.

One other thing that kist crossed my mind - if you shot at one frame rate and are editing at another (30 > 24P or Pal > NTSC) there would for sure be a sync issue from the production track to the CD audio. But again - it shouldn't be massive and only noticable over time.
MUTTLEY wrote on 7/10/2003, 10:36 PM
Mentioned something similar in another thread, essentially had the same thing happen to me. I noticed that as the song progressed it became increasingly more out of sync. Never got a definitive answer, just one of those thing I guess. In my case was shot and edited at the same frame rate, cd player was not a cheapie, no variable speeds, was plugged in, and the camera was a Canon XL1 and a Sony DV-G900. The same issue happend over multiple takes so not just an isolated thing.

Weird, a pain in the ass, but easy enough to tweak.

- Ray

ray@undergroundplanet.com
www.undergroundplanet.com
videoman69 wrote on 7/10/2003, 11:27 PM
I just shot a video on BetaSP, ran the CD output to the Beta during taping. Then ripped the track into Vegas. The biggest problem I had was with the 15 year old Rap artist. He could not lip-sync at all. So it was hard over the 3 minute song to have enough material to cover. It was in a studio so no artistic cut-aways.
TorS wrote on 7/11/2003, 1:23 AM
If the distance from sound source to artist is very different from that to the recording camera you might have sync problems, but they should be consistent. If there is a slight difference in speed somewhere, the out of sync should gradually increase down the timeline (hear that, Muttley?).
If there is a significant speed difference between the playback and the CD, you should be able to hear that the one is slightly lower in pitch than the other.
If the artist is no good at lip-syncing you should still have consistent sync for some stretches and then one or a few words will be very wrong.
Tor
craftech wrote on 7/11/2003, 7:31 AM
Read this article called "Sunk" by Jay Rose

http://www.dv.com/columns/columns_item.jhtml?LookupId=/xml/feature/2003/rose0703&category=Audio%20Solutions

You may have to sign up if you don't have an account. It's free.

John
mikkie wrote on 7/11/2003, 10:28 AM
Jay Rose is great, as craftech wrote, & as TorS mentioned, might have some delay recording to camera to deal with. Otherwise, & in case it helps, audio and video use different clocks. All sorts of methods are used to compensate, but some compromise has to happen. The CD std requires 44.1 kHz, likely downsampled from 48 or 96 or better, but that in most cases shouldn't alter the timing.
MUTTLEY wrote on 7/11/2003, 12:44 PM
I will read that article when I have a few xtra moments to sign up over there but just to make the point: The room we did the video in was very small and had a great sound system, on top of that we had the cd playing purtty loud ( its actually a band rehearsal room ). I understand the concept of sound traveling faster than light but having been there I don't see that as the issue. I also did not notice any discernable difference in pitch between the two audio tracks, just an increasingly out of whack sync problem. This happened on both tapes from both cameras.

Thanks for the feedback, I mainly just posted to let 24Peter's know he's not the only one who's experience this issue. Not sure we'll be able to get to the bottom of this as there are so many variables but thought perchance someone else had dealt with this and had already found a solution.

Well see if it happens on the next one.

- Ray

ray@undergroundplanet.com
www.undergroundplanet.com
24Peter wrote on 7/11/2003, 5:34 PM
Thanks for everyone's replies. There seem to be several different opinions on the 44.1Khz v. 48 Khz issue. I'm going to try to resample the song at 48Khz and see if I can get better sync (though I'm still not sure what sample rate has to do with playback tempo.) (BTW - I shot and edited in 24p normal on my DVX 100 but the audio is still 48Khz/16 bit).

As a director I see I need to stay more on top of the talent to lip sync properly. I can see that was an issue. Sometimes she was singing more harmony than lead. Big problem. But it was my first day with the camera and her first day doing a video so we both agreed we'd learned from the experience. Still, there are parts where her singing and the reference track seem right on, but I can't get them to match the CD track. Wierd. I still feel like I'm missing something.
PeterWright wrote on 7/11/2003, 11:50 PM
I assume you have more than one "take" with different shot sizes and angles. If so you can use changes in shot to readjust her back into sync - but it does depend on how well she performed sync - wise within a continuous shot.

To show how flexible Vegas is, I recently synched a single shot live performance with a previously recorded track which in fact was 13 seconds longer - the live performance was more up tempo!

Using the video as a guide, used Ctrl-drag to shorten the pre-recorded track, and make the first and last notes synch up.

Then it was a matter of "visiting" each significant moment throughout the song, and if synch was a bit off, which it often was, splitting the audio and either dragging or stretching to put it back. Carefully placed overlaps helped disguise some of the "joins".

Good luck - at least Vegas makes it fun to meet the challenge.
mikkie wrote on 7/12/2003, 11:41 AM
"There seem to be several different opinions on the 44.1Khz v. 48 Khz issue. I'm going to try to resample the song at 48Khz and see if I can get better sync "

There is a lot of info on the web, including some quite technical stuff. In a nutshell however, boils down to the math: 48/44.1 = 1.0884353741496598639455782312925... There's no neat and tidy way to convert what is essentially the number of audio samples recorded during a set amount of time. It's not a huge deal really, but a whole lot of ways to actually do it, different methods incorporated in the software or hardware, and some are rather poor. In any case, I've never seen the actual length of the audio change, let alone by the amount needed to throw off lip sync.

Assuming the artist synced her performance with the playback off the CD, should be able to place the video on one track, the ripped audio from the CD on another, and the only discrepancies would be where the artist was off, as the source audio hasn't changed, and the camera should record time accurately. Where you can have problems is with the audio recorded onto your DV tape, and with the processing it undergoes.

If you use a camera to record audio unlocked, there are a number of things, including the camera's audio clocking that can throw things off. If you visit adamwilt.com, he has a fairly good sized explanation of what can happen, how to avoid it and such. One of his suggested methods is to record your audio from the camera to pc using an analog connection, which will in effect lock audio during playback - if that suggestion helps. If you haven't altered the fps of the original footage, should be a direct matchup - find a spot where it's relatively easy to move the audio track into sync if you don't have some sort of recorded marker, and it should be set or close throughout.
riredale wrote on 7/13/2003, 1:57 AM
As Peter Wright mentioned, Vegas is an amazing tool for sync issues. I often videotape a local semiprofessional youth choir, and there have been numerous times where I would shoot multiple performances on the same day, and then sync up the same songs so I could give the impression of a multi-camera shoot, even though in reality it was just one camera shooting the same song sung multiple times. I have gotten pretty good at eliminating all sync differences by just splitting where needed, and then CTRL-dragging to speed up or slow down portions of the songs. If done carefully, you really can't tell which performance is which, though singers will know, since they sometimes change places from performance to performance. One girl thought it was pretty amusing to see herself on both ends of the choir during one song! Nobody else noticed, however.