Rescuing horribly shot video

Laurence wrote on 2/17/2005, 9:29 AM
My wife runs a non-profit foundation which delivers backpacks filled with school supplies and goodies to needy kids in South America. The last time she went, I stayed home with our children and she took the VX-2000 with her. Her uncle ended up shooting the footage of the distribution. I thought that with the VX-2000 it would be easy: just set everything in auto and point and shoot. Boy was I wrong!

Here I sit now, with about two hours of captured footage of beautiful subject matter, but horribly shot video. I absolutely HAVE to do this because it's my wife and it's really important to her and her work.

Here's what I'm dealing with:

1/ Handheld stuff shot at a distance with a lot of camera shake.
2/ Lots of shots where the autofocus was accidentally turned off.
3/ Lots of dirt on the lense.
4/ Lots of dark forground shots against a bright background.
5/ A couple of dozen stills saved on a CD at 640 by 480 resolution.

Here's what I'm finding so far:

The VirtualDub "Deshaker" plugin for Vegas is absolutely amazing, but incredibly time consuming. I can get some steady looking shots if I just take the time.

Spot's idea of two layering the footage with itself, sharpening the top layer, and then adjusting the opacity of the top layer gives me results that are making me hopeful that I can salvage at least some of the out of focus shots.

Does anyone have a good idea about how to handle video that is too backlit? Actually any "how to rescue badly shot video" tips and techniques would be appreciated.

Comments

Laurence wrote on 2/17/2005, 9:33 AM
The stills look pretty good resized with photozoom pro. That program really is amazing. I just resized them to twice the original size so I could animate them a little.
JJKizak wrote on 2/17/2005, 9:39 AM
About all you can do with the backlit stuff is to bump up the gamma as far as you can then increase the contrast. The background will get super bright and washed out and the faces will be ok except for a ton of pixelation. Check out SPOT's tutorial on using the cookie cutter on the faces only.

JJK
Laurence wrote on 2/17/2005, 9:43 AM
On a still photo, you can separate the frame into layers and just bump up the gamma of the stuff in the forground. Is there any practical way to do that with video?
johnmeyer wrote on 2/17/2005, 10:10 AM
Use gamma in color corrector, or use the color curves. I find color curves gives me more control. Place a control point on the upper third of the transfer function line and the move it up. You can slide the point up or down the line, and also use the two control handles to change the shape of the curve until you get the transfer function that gives you the effect you desire. Make sure to use an external monitor while you are doing this or you will probably end up with something that is washed out.
Laurence wrote on 2/17/2005, 10:39 AM
One really interesting thing is the combination of Deshaker and the dirt on the lens. Before Deshaker you have steady dirt, and a picture thats jumping around all over the place. After Deshaker you have a steady picture, and dirt jumping around all over the place. It really lets you see how well the Deshaker algorythm works.
Laurence wrote on 2/17/2005, 10:43 AM
I'll try that. My external monitor is an LCD TV/Monitor combo that doesn't have the best color resolution. I'll get as close as I can, then run some tests with the TV in the family room.
GaryKleiner wrote on 2/17/2005, 10:53 AM
You don't say what you want the final product to look like or what it is to be used for.

Trying to make footage like this look good in the usual way is not going to be possible, so I suggest letting necessity be your mother of creativity and take an "MTV" approach or somthing along those lines.

Use plenty of slow/fast motion, B&W, letterboxing, etc. Set it all to music and voila... we MEANT to shoot it this way!


Gary
RichMacDonald wrote on 2/17/2005, 11:23 AM
>The VirtualDub "Deshaker" plugin for Vegas is absolutely amazing, but incredibly time consuming. I can get some steady looking shots if I just take the time.

One trick with deshaker is to set Scale to something other than "full". I usually "half" and have perfectly acceptable results. Its much faster than "full". I keep "Use pixels" and "Color mode" at their best settings.

>Spot's idea of two layering the footage with itself, sharpening the top layer, and then adjusting the opacity of the top layer gives me results that are making me hopeful that I can salvage at least some of the out of focus shots.

I can't remember how Spot suggests you sharpen, but the Convolution filter is better than the sharpen filter and it gives you better control. Very slow, however.

Dirt sucks. You can make a second layer, offset it in X and Y, add some blur, then mask out the offending dirt sport in the main layer.

>4/ Lots of dark forground shots against a bright background.

Color curves first. If "too backlit" means your whites are blown out, then there isn't much you can do there. Try and rescue detail in the dark foreground by making it brighter. You'll find you add a great deal of noise to the dark areas, so add Mike Crash's "dynamic noise reduction" and "smart smoother" filters in that order. Bump up the former as high as you can; you'll get ghosting behind moving subjects when you've gone too far. Use the latter as little as possible since it sacrifices a lot of detail and makes subjects look "fake" and featureless. But IME these two filters can do great things with far less damage than the traditional gaussian blur filter, and they can actually look good in their own right :-)

I haven't tried this myself but there is a fascinating tutorial for photoshop that attempts to rescue dark details without sacrificing the lights. Look at contrast enhancement and restore clipped. And if these shots are stable (yeah, right; is the boundary between light and dark fixed over time?), then also look at digital neutral density filter or just use masking to tackle each region separately.
RichMacDonald wrote on 2/17/2005, 11:35 AM
Forgot another one at contrast masking. And BTW, the luminous-landscape is a phenomenal website. I steal lots of ideas from there -- seems like most of my work is trying to rescue horribly shot video :-)
Laurence wrote on 2/17/2005, 12:38 PM
The end result is going to be a dvd with standard and anamorphic widescreen playback options (courtesy of Ultimate-S).

I just discovered a new tool: DynaPel Steadyhand DV. Someone mentioned it in an earlier thread as being an alternative to VirtualDub/Deshaker. What I like about it is it's fast and you can do all your shakey files at once as a batchfile. What I don't like is that if you don't use a little zoom, your frame edges move in and out. Fortunately most of that seems to be in the "safe area" that you don't see on a regular TV. Can anybody tell me how many pixels the "safe area" actually is? I'd like to make a little masking overlay so that you don't see the fluctuating edges when you view it on a computer.

I like the idea of getting rid of the dirt spots by overlaying the video and masking off everything but the dirt spot replacement area. It reminds me of the "healing tool" in Photoshop. Unfortunately after Steadyhand or Deshaker, the dirt spots are moving around. I'll check into doing it in two passes: removing the dirt spot first then deshaking the result of that. Actually I'll probably end up letting the dirt spots go because of how much time it would take to do that.
RichMacDonald wrote on 2/17/2005, 1:40 PM
>I just discovered a new tool: DynaPel Steadyhand DV. Someone mentioned it in an earlier thread as being an alternative to VirtualDub/Deshaker. What I like about it is it's fast and you can do all your shakey files at once as a batchfile.

Seriously, deshaker is free and you can make it as fast as you want it to be if you're willing to tradeoff accuracy. I can easily get 8 fps with good accuracy, higher rates with some dropoff.

Deshaker also lets you control whether or not to zoom. I try not to zoom because the quality drop is always too severe for my eyes. Deshaker also has a setting that lets you interpolate the edges from the surrounding frames. It works brilliantly unless you have too much shake in which case it loses its mind. If that happens, just dampen the smoothing and do it again.

A batch tool would be nice for deshaker but it doesn't exist as you know. You can run multiple instances of deshaker (with different log files) but its unstable. If I run two instances overnight usually neither instance is responding by morning. I've also tried to modify the virtualdub scripts to do things in 1 script rather than 2 manual passes, do multiple files in sequence, etc, and virtualdub crashes every time.
Laurence wrote on 2/17/2005, 2:05 PM
I find the quality of Deshaker and Steadyhand to be about the same. I like that Deshaker is free and I also like the edge handling of Deshaker most of the time.

What I don't like about Deshaker is that it is slower, needs to be run in two passes manually, and puts a little extra black screen footage at the beginning.

I bought Steadyhand for the following reasons:

1/ It is much faster at high quality settings.
2/ The edge movement is in the "safe zone" anyway. Zooming sucks with either program but there seems to be enough "safe area" so as not to need to.
3/ On this project I have to do almost every clip. With Deshaker that would take several days of babysitting renders with my computer tied up and unable to do much else.
4/ It's only around $68.88. Now that I've discovered this incredibly useful tool, I'm going to use it all the time!

JJKizak wrote on 2/17/2005, 2:40 PM
Remember that sometimes you can use the velocity envelopes to reduce the shake, slow the scene down to a crawl or even freeze it, then key in some artificial movement to make it look like small camera movements. A zoom here and there helps also. This works well on scenic stuff that doesn't move so much. Doesn't your VX2000 have optical stabilization?

JJK
farss wrote on 2/17/2005, 2:57 PM
No one has mentioned this so I will, try using motion blur, with some problems it can work wonders. A few frames worth of MB doesn't make your video look too wierd and I've used it with compositing to get some pretty good results.
However I gotta say all of this is pretty much mission impossible stuff. If it wasn't a lot of cameramen would be out of work!
There are much better tools than Vegas for this stuff, both software and hardware but they all cost big time and even then they can only intelligently do what you can manually do in Vegas but for 2 hours of footage they might be worth looking into. Bear in mind you could quickly hit a point where it'd be cheaper to just fly down there for two weeks and reshoot the stuff yourself. The sort of tools I'm talking about are typically reserved for archival retrieval where it cannot be reshot or the cost of hand restoration is too high.
Bob.
Laurence wrote on 2/17/2005, 4:09 PM
The VX2000 does have optical stablization, but you wouldn't know it from looking at the footage. Actually what I've processed so far with either Steadyhand or Deshaker looks surprisingly good. I'm sure I'm even going to use this stuff on my own footage in the future. It's actually been stablized twice when I'm done, once with the optical stablization in the camera and a second time electronically with this software.

I'll try the motion blur.
Laurence wrote on 2/17/2005, 11:11 PM
OK now that I've stablized a bunch of shots I must say that I was wrong: Deshaker is way better than Steadyhand. The edges in Steadyhand are moving like crazy. I can use it if I use a safe area matt, but with Deshaker I don't need to. Deshaker is just so time consuming though when you have to correct so many clips! I need a batch fileable Deshaker big time!
apit34356 wrote on 2/18/2005, 6:44 AM
Laurence, there are a few tricks for mud on the lens. you can use the logo removeable for large spots in virtualdub or just take a snapshot of the scene, since the mud is not moving, go into photoshop and remove all the background except the mud. Now create a mask out of the remaining image, the mud shouldbe made into a trans area, Now save it as a PNG with alpha channel. Back in vegas, make a copy of video track strip over the orig. now bring the png file in and create a mask using the copy of the video track. Now move off center the copied video where the mud does not show thru, add fx's like blur, so the image on the lower track no longer shows the mud, but the results of the mask + video. One can create many masks from the orig for special needs, ie, color, less blur.....
RichMacDonald wrote on 2/18/2005, 8:12 AM
>Deshaker is just so time consuming though when you have to correct so many clips! I need a batch fileable Deshaker big time!

I agree with you completely. As I mentioned earlier, I've tried this before and failed. But I have exactly the same problem as you so I'm still thinking of a way to implement it. Maybe you'll inspire me ;-)

The workaround I use for now is to take all my clips and render them out to a single avi file, then run deshaker overnight on the single file. The downside is losing your single clip info, plus when deshaker encounters a scene change it may add a garbage correction.
PierreB wrote on 2/18/2005, 8:32 AM
I'm sure you've thought of this, but in addition to your heroic rescue efforts, supplementing the bad video with:

- freeze frames and slo-mos;
- scans of photographs taken on that trip
- scans or photos of artifacts (receipts, souvenirs) from that trip
- new video (eg, of your wife narrating the trip, of her uncle on the incredibly muddy conditions (!)
- video from previous trips
- plenty of graphics, including maps
- plenty of tricks to distract (PIP, for instance)

Good luck, I'm sure you're earning major brownie points.

Pierre
Laurence wrote on 2/18/2005, 9:09 AM
No brownie points being earned here. My wife can't understand why it's taking so long and she needs it now. I haven't resorted to the standard tricks because the Deshaker footage (once it is finally done) looks so incredible. Rendering to single AVI file is a good idea but I hate to lose the separation of the clips. For anyone who hasn't messed around with Deshaker, it really is pretty incredible. It's right up there with sampling noise reduction and autotune as far as I'm concerned. It's just one of those programs that makes you say: "wow, can you really do that!"
bStro wrote on 2/18/2005, 9:26 AM
No brownie points being earned here. My wife can't understand why it's taking so long and she needs it now.

Has she seen the original footage?

Rob

RichMacDonald wrote on 2/18/2005, 9:30 AM
>For anyone who hasn't messed around with Deshaker, it really is pretty incredible. It's right up there with sampling noise reduction and autotune as far as I'm concerned. It's just one of those programs that makes you say: "wow, can you really do that!"

And then it drives you crazy ;-) We wouldn't care if it wasn't so great, but now we need it to be better, e.g.

1) Batch mode for multiple clips.
2) Automate the vegas integration to trim the extra 30 frames and add as a take.
3) It resamples even if you're only shifting X and Y, so you cannot avoid softening.
Laurence wrote on 2/18/2005, 10:18 AM
By the way I've been using this free AVI trimmer to get rid of the 30 black intro frames:

http://www.solveigmm.com/

It works well and doesn't rerender the video.
JJKizak wrote on 2/18/2005, 11:51 AM
The problem is though if you do a clip in amongst many clips after you delete those thirty frames the clip is thirty frames smaller than it started. So you have to move everything and then worry about the sound if the sound is unlocked or narrative and it makes for a lot of screwing around.

JJK