How to rescue horribly shot video - results

Laurence wrote on 5/25/2005, 7:56 AM
A while back I started a thread entitled "How to rescue horribly shot video":

http://www.sonymediasoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=362179

This project is very close to the hearts of my wife and I, and I had to get a final result that was useful in spite of the problems with the footage.

Well I finished the project several months ago but just posted a low res version at Cheinworks site yesterday. Check it out here:

http://vegasusers.com/vidshare/textdisp?laurence-backpacks

Video was shot with a Sony VX-2000, mostly by my wife's uncle because on that trip I stayed at home with our children. Sometimes the autofocus was on, other times it wasn't and the focus was wherever it happened to be. There were lots of handheld shots zoomed way in and a lot of resulting shake. Shake was taken care of with Deshaker plug running in VirtualDub. On some of the worst out of focus shots, I tried Spot's suggestion of overlaying the video on top of itself, sharpening the bottom layer, and varying the transparency of the unsharpened layer on top of it. Because he shot so many pictures of the backs of kids instead of their smiley faces, I used a lot of stills as well so that you could see enough of their faces not to miss them.

On some of the shots, there is dirt on the lens (though it is hard to see in the low res version). On some of these I was able to get rid of this with an offset cookie cutter approach. On other clips, the dirt was hovering around a changing background such as a tree against a sky, and I had to live with it. What is interesting is that before Deshaker, the dirt was static and the shot moved about erratically. After Deshaker, it was the other way around, the image was stable but the dirt spots fluttered around. There is a shot towards the end with flares in the lens which does the same thing: the image is stable while the sun flares shake wildly. I suppose the moral is that if you think you are going to use Deshaker, watch out for flares and dirt on the lens even more than usual!

I did the music with Acid Pro 5. There are two tracks of guitar. The drums and bass were Acid Loops.

All in all it came out a lot better than I had hoped. My wife's biggest complaint was seeing herself in the video and hearing her own voice. She really wanted me to get rid of her interview shots especially. Evidently she doesn't see herself as beautifully as I do!

Comments

Laurence wrote on 5/25/2005, 8:31 AM
A couple of other details that might be of interest:

The actual DVD we are distributing to promote the nonprofit organization has both 16:9 and 4:3 versions available from a simple menu. I cropped the internet version to 16:9 letterbox because on the 4:3 version you could see the Deshaker artifacts around the edges. This isn't really a problem on a regular television because most of the Deshaker artifacts are in the safe area and not seen, but on computer playback you can see all the way to the edges. Cropping Deshaken 4:3 footage to 16:9 letterbox seems to be a good way of dealing with the Deshaker artifacts.

I recorded the Voiceover with a Rode VideoMic going directly into a tiny Sony ICD-ST25 solid state recorder at our dining room table. This is about as low budget and easy a way as you can do this.

The video as downloaded is a 24p wmv rendered with Vegas 6.

The acoustic guitar sound is actually a Line6 Acoustic Variax. I love that guitar!
RalphM wrote on 5/25/2005, 9:32 AM
Remembering your first post regarding the problems with the original video, I'd say you made a very good recovery

It's hard to tell how successful you were in correcting the out-of-focus issues given the compression used in the on-line version, but I think it delivers the message well.

Good approach with the 16:9 to eliminate the Deshaker artifacts - I'll have to try that with SteadyHand. On a casual viewing, I would not have been aware that there had been a steadiness problem (and most of your audience will be casual viewers.)

Lastly - thanks for being part of a very worthwhile endeavor in the backpack distribution. It's sometimes hard for us to imagine the gap between our everyday lives and those of the poor in developing nations. People like you and your wife make a difference.

RalphM
rs170a wrote on 5/25/2005, 11:13 AM
Laurence, let me echo Ralph's comments by offering a heartfelt congratulations on what's obviously a very rewarding project for you and your wife.

What I'd like, if possible, is for you to post is a short clip of what you consider the worst footage you had to work with. That way we'd really see just how good DeShaker is. Based on what you've said, it's definitely something I have to look into.
Thanks again.

Mike
Laurence wrote on 5/25/2005, 11:57 AM
Well one thing about Deshaker is that you need a good DV codec. I used the free Panasonic codec for the first part of the project but upgraded to the Main Concept one about half way through the project. The difference is not something you'd see on the compressed video, but on a DVD it comes accross as kind of small wormy artifacts.

Deshaken video is a little less sharp than the solid unshaky video is, but I think that that has more to do with the fact that the image moved during the sixtieth of a second that the camera shutter was open than it does with Deshaker. I tried both Steadyhand and Deshaker, but ended up using Deshaker because of how it filled in missing pieces at the edges of the frames. The end result is far superior in my opinion to shakey footage. Strangely, the end result of this project was way more stable than anything I have done thus far even though the starting footage was worse. I'm a real fan of Deshaker now and it is one of my regular tools. Now I'll zoom in to get shots I would have been afraid to get in the past. When I do I try to do so from a static position so as not to confuse Deshaker when I use it in post.

A 16:9 crop helps out Deshaken video immensely. Filled in sides are not so obvious even when you can see them, but the filled in pieces at the tops and bottoms of the frame are full of offset head tops and offset leg bottoms of the main characters in the shot! The 16:9 crop gets rid of all but the worst of these artifacts. In fact, I made another post recently saying that the advantages of being able to deshake video and crop off the artifacts at times more than made up for the lack of resolution that you got for using a 4:3 camera instead of one with native 16:9 CCDs. In this project that definately was the case.

Rather than have letterboxing around all four sides on a widescreen TV, I alway rescale to anamorphic 16:9 using Ultimate S or an aspect ratio matching script. I also do quite a lot of moving the frame up and down to follow the action. I have found that if you zoom out to 50%, all your movements happen two lines at a time and you don't mess up the field order on interlaced footage.

Yes this project is close to our hearts. We have done two trips so far and want to expand and continue the effort. Having a DVD to explain and promote this has helped immensely. Watching the video easily explains more than a half hour of discussion would.
Jøran Toresen wrote on 5/25/2005, 12:35 PM
Laurence
Thanks for your contribution. I have two questions:

1. Could you please tell us which settings you specify in Pass 1 and Pass 2 when using Deshaker?
2. Have you experimented with other Edge compensation settings than “Use previous and future frames to fill in borders”? Most of all I think of “Adaptive zoom only” or “Adaptive zoom (some borders)” in combination with the “Extra zoom factor”.

Regards,
Joran
Laurence wrote on 5/25/2005, 2:51 PM
I used the suggested settings from John Meyer's excellent Deshaker Tutorial found here:

http://www.sundancemediagroup.com/articles/deshaker_guide.htm

I also tried zooming instead of "use previous and future frames to fill in borders" but didn't like the loss of clarity I got from zooming. If anyone has better settings I'd love to know, but found the ones in the tutorial to be better than anything I could come up with on my own.

I did add a couple of things to my procedure:

First, I set the audio delay in the Audio / Interleaving window to delay the audio 1001 milliseconds. This way I didn't have to line up the audio later.

Second, many of the clips were barely long enough and I couldn't afford to lose the 30 frames at the tail end. On these clips I loaded them into Vegas, added a half second or so of black space to the end and smartrendered the clip to a temporary file which I ran Deshaker on.

Third, I made a Vegas template with a marker 30 frames in to make it easy to trim the beginning 30 frames of black off the front end of each clip. I spent about two weeks just Deshaking footage.

Once I realized how long this was going to take, that is when I bought Steadyhand so that I could batch render all the files at once painlessly. Unfortunately my two options with Steadyhand were to zoom in on the footage or have the edges moving in and out. Neither was acceptable to me so I bit the bullet and did each clip individually in Deshaker. Like I said, it took about two weeks of long days. Having to do this really sucked to be quite honest, but there was no other way to get as good results.
Jøran Toresen wrote on 5/25/2005, 5:23 PM
Laurence

Once more: I’m impressed by the video I have downloaded. Therefore I asked for the settings you used. I have read all your post on this topic. And I have read John Meyers tutorial.

You say: “I used the suggested settings from John Meyer's excellent Deshaker Tutoria”. But I can’t find any “suggested” settings. Meyer is discussing several alternatives, including “Block size”, “Scale”, “Use pixels”, “Ignore image are” and much more. This is the reason I’m interested in the settings you actually applied to achieve these great results!

Joran
apit34356 wrote on 5/25/2005, 6:18 PM
laurence, nice work on a great cause with a zero budget. Tell your wife she should be proud of her "on camera" performance and of your great editing.

If you don't a few comments, the scenes with your wife talking, appears to need some some color adjustments on her skin tone. Secondary, maybe some text on the screen about how people can donate to the program.

Again, you did a great job with the "in the field" shots.
Laurence wrote on 5/25/2005, 7:53 PM
My wife's skin tone is just like that... just kidding. I could still fix the color on her. I don't know why I haven't yet... maybe because she looks good to me even un-color-corrected!

On the DVD there is contact info both on the disc and on the cover. I figured why add contact info on the video and have them searching for a pen when it can be written down right there on the cover they already have in their hand. In the context of downloading it off of the Vegas Users site it would be nice to have it at the tail end of the video wouldn't it?

Shooting the actual interview was an excercise in patience. She didn't really want to do it and was visably angry with me as she was answering the questions. I had to really search for the little bits where she wasn't so visably impatient with the whole affair and piece them together to make the interview you see. I'd say something like "tell me how the kids reacted to getting the backpacks?" and she'd respond something like "I've told you that a million times already! You NEVER listen to me!" and I'd say "Honey, it's an interview! pretend like I know what I'm doing and answer the question." She would answer it impatiently and I'd try to get her to do it again with kindness in her voice! It didn't help much that it was one of my first interviews and I really didn't have any real experience to draw upon. I love her and she's a wonderful person, but she can be such a pain in the butt!

The Deshaker settings were (and this is quoted right from the John Meyer tutorial):



1. Click on the big “Pass 1&2” button at the top of the left column in the dialog box. Set the Source Pixel Aspect to “Standard NTSC (0.911)” or “Standard PAL (1.094).” If you are using some other type of video, then make the appropriate choice.
2. Change “Video Type” to “Interlaced, lower field first” if your video is DV. If your video is some other type, then make the appropriate choice.
3. Change the drive letter for the “Log File.” I don’t like storing things in the root directory of my C: drive. The “Log File” will be used to store the X, Y, rotational, and zoom information for each frame of video in your clip.
4. Click on the big “Pass 1” button. You can leave all the values at their defaults, but for better quality (but slower processing), change “Scale” to “Full (most precise)” and “Use pixels” to “All (most robust).” There is a significant speed penalty for doing this, and the results are often “good enough” with the defaults (which are Scale: Half, and Use Pixels: Every 4th).
5. Click on the big “Pass 2” button.
6. Change the “Destination pixel aspect” to match what you set in step 1. Set the destination video size to 720x480 for NTSC DV video or 720x576 for PAL DV video.
7. Set “Edge Compensation” to “None (large borders).”
8. Put a check mark in “Use previous and future frames to fill in borders.” Don’t change the default of 30 for the previous and future frames.
9. Set Motion Smoothness values of 3000 (NTSC). Set Zoom to zero (to turn it off). You can use larger values in order to make the motion smoother (I’ve used settings up to 18,000), but the results may look somewhat artificial, and you may begin to see unwanted artifacts that make the video look like it was placed on top of a flag rippling in the wind on top of a flag pole.
10. Set all Max Correction Limits to 99.
11. Finally — and this is important — click on the big “Pass 1” button (the one on top of the center column).

I don't have the original footage on hard drive anymore. I would have to recapture it from tape to post, and it was endless shaky out of focus shots of peoples backs for the most part... hardly the stuff anyone really wants to see.

My wife has the beginnings of a web site here:

http://givebackpacks.com/

It is a work in progress. One of her cousins in Ecuador is working on a much more involved site layout (in Spanish so far) that we will use as a basis for a bilingual site. I've been investigating compression formats to post the video in on that site. I get noticably better results using DivX or mpeg4, but the Cheinworks site does not accept those formats.
Jøran Toresen wrote on 5/26/2005, 6:26 AM
Laurence
One final question: under 4, did you choose the default fast settings or the slow settings (Scale = Full and / or Use pixels = All)?

Joran
Laurence wrote on 5/26/2005, 10:22 AM
I went with the slow setting (scale - full) and Use Pixels = ALL.

I have a small 15 inch combo TV / computer monitor that I use when I'm editing video. After I render final DVDs I always check them out on a 37" Toshiba HD CRT TV in our livingroom. All the compromises that don't seem to matter on the small monitor kind of jump out at you on the larger set. I've learned to always use the best quality settings even when you can't see the difference as you're editing.