Smoothing out dropped frames? (when recapture of video is not an option)

kungfujoe wrote on 8/5/2003, 9:30 PM
I'm working on a project with video footage from three cameras, none of which were under control (from a stage performance, some spotlight, and some stagelight)

1 - A good quality MiniDV recording from a tripod. The trouble is that most of this is zoomed out much further than I'd like, and sometimes, the operator wasn't paying attention and the action was off-camera. This is the primary source I'm using.

2 - A pretty-good quality MicroMV recording taken without a tripod. Much of this footage is too shaky to use, and much of it is zoomed in _too_ close to use. But some of it is really good.

3 - A very noisy Hi-8 recording taken without a tripod. Some of it is too shaky, and all of it is very grainy. I've used a _little_ bit of this footage, but more or less only as a last resort, and generally not for the spotlit scenes, where the noise is worse.


The problem is that I just realized (after basically finishing the project) that the MicroMV footage has every 15th frame dropped. Every 15th frame is contains every (15+1)th frame as a placeholder. This results in the picture visibly jumping every half-second. It's very distracting, and I was hoping for some way to smooth it out. Does anyone have any suggestions?

I tried two things, based on an erroneous assumption. I figured the "jump" was caused mostly by the fields being repeated (going in the order of ...28, 29, 32, 33, 32, 33, 34, 35...), and that a slight pause every 1/30 second wouldn't be as noticeable as the back-and-forth field effect.

1 - I re-encoded part of the MicroMV footage as progressive scan, dropping every other field. This didn't help.

2 - I took about 8 seconds of footage, exporting every duplicated frame, deinterlacing twice (once with odd field, once with even field), and re-importing the deinterlaced pics in order, in place of the duplicated frames. This helped minimally, but the pause every half-second was still very noticeable.


Is there any way to smooth out this pause? I know darn well that a good deal of the quality is lost and cannot be retrieved (getting the original MicroMV tape, finding a MicroMV device to use, and re-exporting the footage isn't really an option, unfortunately, just in case the original footage on the tape doesn't have this flaw). But is there any way that I can smooth this out so that it's not as distracting? It seems that all I would need is a simple filter to create an interpolated frame between frame 14 and frame 16, but considering that my last two ideas didn't pan out, I may be off base on this, too, and an interpolated frame might not successfully smooth it out either.

Comments

XPUser2003 wrote on 8/6/2003, 1:56 AM
joe,

there's another thread going on here about software-based steadycams. you might want to look at it. they're talking about Steadyhand and Icarus- both meant to make video look smoother. I have no need for them (yet) but you may want to. Dynapel's website (just type Dynapel at Google) explains what their program can do to improve your shaky video. wouldn't hurt to look.
jeffy82 wrote on 8/6/2003, 2:27 AM
Ya, it's called MotionPerfect, from Dynapel, and it is designed for that specific purpose -- finding dropped frames and doing an intelligent interpolation to create the missing frame(s).
kungfujoe wrote on 8/6/2003, 2:01 PM
Unfortunately, Icarus and Steadyhand are for smoothing video _spatially_. I want to smooth the video _temporally_. This isn't a case of camcorder shaking, but of dropped frames causing a time-based stutter. I can't imagine that a software steadycam would help with this (though it could help smooth out the shaky parts of the two non-tripod video sources).
kungfujoe wrote on 8/6/2003, 2:06 PM
Thanks Jeffy. I'll check out MotionPerfect. It sounds like exactly what I need. Kinda stinks to have to buy additional software for this, though (unless there's a fully functional demo), when what I need is a frame predictor. I essentially already have a bunch of them - every MPEG-1 and MPEG-2 encoder does frame prediction as part of its compression algorithm. Unfortunately, they're not modular enough that I can use that part of their algorithm for this purpose.

Off to Dynapel's website I go... :)
BrianStanding wrote on 8/6/2003, 4:42 PM
VirtualDub has some plug-ins to control decimation of frames. Maybe they have some that do interpolation as well.

What about playing with the velocity of the clip in Vegas with supersampling on? Doesn't Vegas interpolate frames whenever you slow down a clip (i.e. change the frame rate)? What happens if you slow the clip down, then speed it up again (two successive filters, maybe)?

It also seems like something motion blur should take care of. If worst came to worst, maybe you could do a quick dissolve between the 14 & 15th frames. Maybe that would help smooth it out a bit.
MDVid wrote on 8/6/2003, 7:46 PM
You need to look at "motionperfect" from Dynapel, not steadyhand. They are two separate products. (I think you are getting them mixed up). Check out this link:
http://www.dynapel.com/com/private/mp_overview.htm I have used it to do eactly what you are describing.

JTH
kungfujoe wrote on 8/6/2003, 8:02 PM
The poster I replied to specifically mentioned SteadyHand.

I downloaded MotionPerfect and I'm trying it out. I don't understand what I'm doing wrong, but I can't get it to do anything. I can go into expert mode and see from the frame legend that it's properly identifying the duplicate frames (however, it doesn't do that unless I copy frame 14 onto 15. It doesn't seem to like that the duplicate frame is _before_ the actual frame on the original video). However, once I process the video that is properly analyzed, the output is identical to the input. The duplicate (dropped) frames are still there! I'm not sure what I'm doing wrong, but the program is doing nothing but making another copy of my video, with the dropped frames that were originally dropped.