Removing a Glitch from an old Hi-8 Capture

amendegw wrote on 2/16/2010, 6:39 AM
Experts,

I'm going thru a process of reworking a number of my old Vegas projects for web delivery. I'll be rendering at a higher bitrate because most folks viewing the videos now have higher available bandwidth.

In the process of increasing the quality of the renders, I've found a single frame glitch (should I call it a tear?) in one of my captures. Here's a three second clip - the glitch is at the two seconds mark: glitch.zip

The original video was Hi8 captured via a Canopus Acedvio card as interlaced 4:3 NTSC. The rendered output will be progressive - probably mp4 or wmv

The bottom line question is... What's the best way to remove this glitch while maintaining smooth movement?

...Jerry

PS: I recognize there's lots of chroma noise in this clip. John Meyer has posted some really good techniques to remove this via VirtualDub. I've tried his techniques on this clip and it does a great job - thanks, John!

System Model:     Alienware M18 R1
System:           Windows 11 Pro
Processor:        13th Gen Intel(R) Core(TM) i9-13980HX, 2200 Mhz, 24 Core(s), 32 Logical Processor(s)

Installed Memory: 64.0 GB
Display Adapter:  NVIDIA GeForce RTX 4090 Laptop GPU (16GB), Nvidia Studio Driver 566.14 Nov 2024
Overclock Off

Display:          1920x1200 240 hertz
Storage (8TB Total):
    OS Drive:       NVMe KIOXIA 4096GB
        Data Drive:     NVMe Samsung SSD 990 PRO 4TB
        Data Drive:     Glyph Blackbox Pro 14TB

Vegas Pro 22 Build 239

Cameras:
Canon R5 Mark II
Canon R3
Sony A9

Comments

ECB wrote on 2/16/2010, 7:54 AM
What I would do is replace the glitched video frame with the next or preceding video frame leaving the audio track in tack. Adjust the brightness/contrast and framing of the replaced frame and the surrounding frames to match the frame in the clip. The framing and exposure change was almost as noticable as the glitch. I just did a quck fix and I could not spot missing frame.

EB
DGates wrote on 2/16/2010, 8:04 AM
Ditto what EB said.
johnmeyer wrote on 2/16/2010, 9:29 AM
I just finished converting 100 reels of 8mm and Super8 film. Unfortunately, my Workprinter capture rig had a minor malfunction, and I ended up with about three hundred individual bad frames, scattered across ten hours of video. I wasn't about to re-do the whole capture, so I invented an AVISynth script that detected each bad frame. I then used an old Vegas script I wrote many years ago that lets me replace a bad frame with the previous frame (which is exactly what ECB and DGates are recommending). I've copied that script below. Just assign it to a key in Vegas. Then, position the cursor so you can see the bad frame, and then press the key. Bingo, the bad frame is gone.

Now, if your bad frame is "clean," meaning that it doesn't leave some residual artifacting that affects the next few frames, you can go one step further, and in many cases make the bad frame look identical to what should have been there. This uses the magic of motion estimation to reconstruct the bad frame from the two good frames on either side. I've been after Sony for years (back when they still had people there actually developing features) to add this. I've included that script below. You have to know AVISynth to use this one, and you have to download MVTools2 (get the "2" version). I tried it on your video, but it didn't work too well because your glitch actually causes the next 2-3 frames to be shifted slightly and also to be somewhat darker. Thus, the duplicate frame approach probably is going to be a better approach.

Here's a file with the fix done with the duplicate frame approach, and also done with the synthesized frame approach. The synthesized frame doesn't look too good in this case, but I can absolutely guarantee that in many cases it produces a perfect result -- not "near" perfect, but perfect. I am actually puzzled as to why it didn't work well here, and even spent fifteen minutes debugging my script to see if there was a flaw. Strangely, if I created duplicated elsewhere in the clip, it replaced those and the playback was near-perfect. I think the problem is that the next several frames after the glitch are also tainted.

Repaired video files

Vegas script to replace bad frame with previous frame:

/** 
* PURPOSE OF THIS SCRIPT:
*
* Fix a bad video frame by deleting bad frame and replacing with previous frame.
*
* A video track must be selected. If an audio track is selected, nothing happens.
*
* To use, select the event that contains the bad frame. Position the cursor so you can
* see the bad frame in the preview window. Then, run the script.
*
* Copyright © John Meyer 2006
* Written: June 23, 2006
*
**/

import System;
import System.IO;
import System.Windows.Forms;
import Sony.Vegas;


try {

//Global declarations
var dStart : Double;
var dLength : Double;
var dCursor : Double;
var trackEnum : Enumerator;
var evnt : VideoEvent;
var evnt2 : VideoEvent;
var one_frame = Timecode.FromFrames(1);

var track = FindSelectedTrack(); // Use this function to find the first selected track.

if (track.IsVideo()) { // Proceed only if selected track is video track.
var one_frame = Timecode.FromFrames(1);
var eventEnum = new Enumerator(track.Events);
dCursor = Vegas.Cursor.ToMilliseconds(); // Remember the cursor position.

//Go through each event on the track.
while (!eventEnum.atEnd()) {
evnt = (eventEnum.item());

// Get the event's start and length timecode, in milliseconds.
dStart = evnt.Start.ToMilliseconds();
dLength = evnt.Length.ToMilliseconds();

// If the cursor timecode is between the beginning and end of the
// event timecodes, then split the event, delete first frame from event to right of split.
// Copy last frame from event to left of split and put into the one-frame gap.

if ( (dCursor > dStart) && ( dCursor <= (dLength + dStart) ) ) {
var patch_frame : VideoEvent = VideoEvent(evnt.Copy(VideoTrack(track),Vegas.Cursor));

// Make first frame of new event equal frame to left of cursor (for all takes)
for (var i=patch_frame.Takes.Count - 1; i >= 0; i--) {
patch_frame.Takes[i].Offset = patch_frame.Takes[i].Offset + Vegas.Cursor - one_frame - evnt.Start; // Make change for ALL takes.
}

patch_frame.Start = Vegas.Cursor; // Move new event to right of cursor.
patch_frame.Length = one_frame; // Truncate event to only one frame in length.

evnt.Split (Vegas.Cursor - evnt.Start);

// Next several lines truncate start of NEXT event by one frame.
eventEnum.moveNext(); // Go to next event on this timeline.
evnt2 = VideoEvent(eventEnum.item()); // Get next event.
evnt2.AdjustStartLength(evnt2.Start+one_frame, evnt2.Length-one_frame, true);

if (evnt.IsGrouped) { // Add new events to existing grouping.
var grp : TrackEventGroup = evnt.Group;
grp.Add(evnt2);
grp.Add(patch_frame)
}

} // End if dCursor ...

eventEnum.moveNext(); // Go to next event on this timeline.

} // End While

} // End If Track

else {
MessageBox.Show("You must select a video track.");
}

} catch (e) {
MessageBox.Show(e);
}

// End of main program



// Beginning of functions

function FindSelectedTrack() : Track {
trackEnum = new Enumerator(Vegas.Project.Tracks);
while (!trackEnum.atEnd()) {
var track : Track = Track(trackEnum.item());
if (track.Selected) {
return track;
}
trackEnum.moveNext();
}
return null;
}





And here is the rather nifty AVISynth script that will find ALL duplicate frames in a video, and replace the duplicates with an interpolated frame. When this works, the result is absolute magic. For most of my film glitches, I could absolutely not tell that the new frame had been synthesized: it was perfect. For this perfection, however, the previous and next frames have to be free of any residual defect from the whatever caused the bad frame. Also, the less motion between frames, the better the result (running soccer players are about the worst-case scenario). This script would work almost perfectly at removing a frame from a still photographer's flash.

So, first run the Vegas script to replace the defect with a duplicate frame, and then if you want to go the next step, serve that video into the AVISynth script below and save the result.

[edit] The script below uses the multi-threaded version of AVISynth. If you don't have that, remove the two "setMTMode" lines.

s# Script to replace an exact duplicate frame with a synthesized frame
# Copyright 2010 John H. Meyer
#-----------------------------

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

SetMTMode(5)
AVISource("e:\frameserver.avi").KillAudio()
setMTMode(2,0)
source=ConvertToYV12(interlaced=false)
#Change the line below for DV and HDV
AssumeBFF(source)
filldropsI(last)


#---------------------------------------


function filldropsI (clip c)
{
even = c.SeparateFields().SelectEven()
super_even=MSuper(even,pel=2)
vfe=manalyse(super_even,truemotion=true,isb=false,delta=1)
vbe=manalyse(super_even,truemotion=true,isb=true,delta=1)
filldrops_e = mflowinter(even,super_even,vbe,vfe,time=50)

odd = c.SeparateFields().SelectOdd()
super_odd=MSuper(odd,pel=2)
vfo=manalyse(super_odd,truemotion=true,isb=false,delta=1)
vbo=manalyse(super_odd,truemotion=true,isb=true,delta=1)
filldrops_o = mflowinter(odd,super_odd,vbo,vfo,time=50)

# The "0.1" in the next two lines sets the threshold used to detect duplicates. For a perfect duplicate (i.e.
# one that you manufactured by duplicating a frame in Vegas) you can make this even smaller.
evenfixed = ConditionalFilter(even, filldrops_e, even, "YDifferenceFromPrevious()", "lessthan", "0.1")
oddfixed = ConditionalFilter(odd, filldrops_o, odd, "YDifferenceFromPrevious()", "lessthan", "0.1")

Interleave(evenfixed,oddfixed)
Weave()
AssumeFieldBased()
AssumeBFF()
}



ECB wrote on 2/16/2010, 10:44 AM
" I am actually puzzled as to why it didn't work well here, and even spent fifteen minutes debugging my script to see if there was a flaw."

John,

I believe the problem with the synthinized frame was caused by the shift in framing between the frames before the glitch and after the glitch. In the frame before the glitch the image is shifted to the right note the black bar on the left. After the glitch the image in the frame is centered. If you line up the images in the frames before and after the glitch I believe the synthinized frame will look great. :)

-ed
amendegw wrote on 2/16/2010, 10:54 AM
"I believe the problem is caused bythe shift in framing between the frames before the glitch and after the glitch. The frame before the glitch is shifted to the right note the black bar on the left. After the glitch the frame is centered. If you line up the frames before and after the glitch I believe synthinized wiht look great."ECB: Yeah! I think we discovered the same thing at the same time. I was cutting & pasting & uploading while you were making your post. Every frame prior to the glitch looks like this:



Every frame after the glitch looks like this:



Note the shift of the frame several pixels to the left.

The original capture of this was done several months (maybe years) ago. I think I'll try to find the original tape and recapture and see what happens.

Strange??
...Jerry

System Model:     Alienware M18 R1
System:           Windows 11 Pro
Processor:        13th Gen Intel(R) Core(TM) i9-13980HX, 2200 Mhz, 24 Core(s), 32 Logical Processor(s)

Installed Memory: 64.0 GB
Display Adapter:  NVIDIA GeForce RTX 4090 Laptop GPU (16GB), Nvidia Studio Driver 566.14 Nov 2024
Overclock Off

Display:          1920x1200 240 hertz
Storage (8TB Total):
    OS Drive:       NVMe KIOXIA 4096GB
        Data Drive:     NVMe Samsung SSD 990 PRO 4TB
        Data Drive:     Glyph Blackbox Pro 14TB

Vegas Pro 22 Build 239

Cameras:
Canon R5 Mark II
Canon R3
Sony A9

amendegw wrote on 2/16/2010, 10:59 AM
One more thing I forgot to mention in my original post - I had already tried to replace the offending frame with it's neighbor, but found the stutter as bad as the glitch. Now I know why - the whole frame shifted.

...Jerry

System Model:     Alienware M18 R1
System:           Windows 11 Pro
Processor:        13th Gen Intel(R) Core(TM) i9-13980HX, 2200 Mhz, 24 Core(s), 32 Logical Processor(s)

Installed Memory: 64.0 GB
Display Adapter:  NVIDIA GeForce RTX 4090 Laptop GPU (16GB), Nvidia Studio Driver 566.14 Nov 2024
Overclock Off

Display:          1920x1200 240 hertz
Storage (8TB Total):
    OS Drive:       NVMe KIOXIA 4096GB
        Data Drive:     NVMe Samsung SSD 990 PRO 4TB
        Data Drive:     Glyph Blackbox Pro 14TB

Vegas Pro 22 Build 239

Cameras:
Canon R5 Mark II
Canon R3
Sony A9

ECB wrote on 2/16/2010, 11:22 AM
"Unfortunately, my Workprinter capture rig had a minor malfunction, and I ended up with about three hundred individual bad frames, scattered across ten hours of video"

John,

When I first was using the Roger's Workprinter I had similar problems that required some "cut and try" adjustment of the timing cam. To fine tune the adjustment I setup the transfer camera to a capture the claw motion on the film gate in the WorkPrinter. I loaded the film and started the transfer. Now I could see exactly where the claw was when the frame was captured. I could set the timing to capture the frame when the claw was withdrawn and the film was stable.

-ed
amendegw wrote on 2/16/2010, 12:16 PM
Arrrggghh! My recapture was rock-solid - no glitch, no frame shifting.

I want to thank you guys for spending the time to look into this for me. Particular thanks to John Meyer, who went above-and-beyond. He did supply the key - when I started looking for the darker frames after the glitch, it became obvious that the frame had been shifted.

Why did this happen in the first place? I'm blaming a stray cosmic ray that collided with my silicon! Or maybe analog captures are just flaky *grin*.

...Jerry

System Model:     Alienware M18 R1
System:           Windows 11 Pro
Processor:        13th Gen Intel(R) Core(TM) i9-13980HX, 2200 Mhz, 24 Core(s), 32 Logical Processor(s)

Installed Memory: 64.0 GB
Display Adapter:  NVIDIA GeForce RTX 4090 Laptop GPU (16GB), Nvidia Studio Driver 566.14 Nov 2024
Overclock Off

Display:          1920x1200 240 hertz
Storage (8TB Total):
    OS Drive:       NVMe KIOXIA 4096GB
        Data Drive:     NVMe Samsung SSD 990 PRO 4TB
        Data Drive:     Glyph Blackbox Pro 14TB

Vegas Pro 22 Build 239

Cameras:
Canon R5 Mark II
Canon R3
Sony A9

JJKizak wrote on 2/16/2010, 2:06 PM
Instead of fixing it put in a quicky zoom or an another applicable close up shot to override the defective spot.
JJK
johnmeyer wrote on 2/16/2010, 3:21 PM
I believe the problem with the synthinized frame was caused by the shift in framing between the frames before the glitch and after the glitch.You both are correct. I don't know how I missed that, but it would explain the strange ghosting I was getting. It all makes sense now.

As for the Workprinter glitch, you are correct about the timing cam. I just had the unit refurbished (I had one of the very first ones he ever made). The adjustment is far easier on the newer unit. I think part of my problem (getting occasional bad frames) could be that I didn't get the adjustment perfect, but I also think it is a software issue. Roger has always made a point to insist that everyone use a RAID for capturing. This makes no sense, but I understand why he insists: he has a setup that works, and doesn't want to troubleshoot everyone's computer system. However, I had a 3:00 A.M. brainstorm a few days ago about what may be causing not only my own problem, but other people's as well. I need to do some testing, now that I'm finished with this huge film project. If I'm right, it is a very easy fix.

Glad the amendegw got the second capture to work better. The equipment on which you capture makes more difference than anything else when transferring old videotape. While it is nice to have a TBC and all that good stuff, you've got to have a basic mechanical setup that ensures consistent tracking, and which can handle tape at all different speeds. My old VHS VCR kicked the bucket a few months back and I found that you cannot easily get good VCRs anymore. I found a NOS still in the box JVC that has Firewire built in. The reviews on the unit were mixed, and indeed it really stinks for doing 6-hour VHS tapes. However, for 2-hour tapes it is actually better than my semi-pro Panasonic used to be.
amendegw wrote on 2/16/2010, 4:57 PM
" The equipment on which you capture makes more difference than anything else when transferring old videotape. While it is nice to have a TBC and all that good stuff, you've got to have a basic mechanical setup that ensures consistent tracking, and which can handle tape at all different speeds."My equipment is pretty good. My capture deck is a Sony EV-S7000 which has a built-in TBC. I'm still blaming the cosmic rays!

...Jerry

System Model:     Alienware M18 R1
System:           Windows 11 Pro
Processor:        13th Gen Intel(R) Core(TM) i9-13980HX, 2200 Mhz, 24 Core(s), 32 Logical Processor(s)

Installed Memory: 64.0 GB
Display Adapter:  NVIDIA GeForce RTX 4090 Laptop GPU (16GB), Nvidia Studio Driver 566.14 Nov 2024
Overclock Off

Display:          1920x1200 240 hertz
Storage (8TB Total):
    OS Drive:       NVMe KIOXIA 4096GB
        Data Drive:     NVMe Samsung SSD 990 PRO 4TB
        Data Drive:     Glyph Blackbox Pro 14TB

Vegas Pro 22 Build 239

Cameras:
Canon R5 Mark II
Canon R3
Sony A9