Twixtor for Vegas...is it necessary?

Comments

johnmeyer wrote on 12/3/2010, 10:04 AM
Supersampling does absolutely nothing with video. It is only for things generated by Vegas itself, like titles. It will not help slow motion.

Here is a link to a thread where I demonstrate how to do Twixtor-style slow motion for free, using AVISynth and a free motion estimation tool called MVTools. It is too geeky for many people, but the results are as good as the demos I've seen on Twixtor's site.

Slow Motion Using MVTools

Here's a very, very old post where I attempted to fiddle with every possible Vegas setting that would affect the quality of video slow motion generated entirely within Vegas. Supersample was one of the things I tried, and it didn't do a thing, something that was subsequently verified by others who knew a lot more about Vegas than I did.

Slow Motion Settings

Finally, here is an example I posted a long time ago showing a 1.5 second clip at normal speed followed by two Vegas-created super-slo motion versions, one using Smart Resample and the other using Disable Resample. The end of this YouTube video shows the results of using AVISynth/MVTools which, as noted above, is essentially the same thing as what you'd get with Twixtor.

As Bob (farss) noted, motion estimation software (like Twixtor) is very much like the little girl with the little curl in the old Longfellow poem who, when she was good was very, very good, but when she was bad, she was horrid.


johnmeyer wrote on 12/3/2010, 10:25 AM
P.S. to last post.

I wanted to make sure that nothing has changed, so I put a widescreen DV AVI file on the Vegas 10.0a timeline and slowed it to 1/8 speed using a combination of playback speed and velocity envelope. I set Resample to "forced" (probably not necessary because the "Smart Resample" problems were fixed about ten years ago).

I then rendered to a DV AVI Widescreen file.

I next put a Supersample envelope on the Video bus and set it to "4." I rendered another video. Finally, I set Supersample all the way to maximum ("8") and did another render.

I set the preview window to Best Full and stacked all three slow motion clips on the timeline. I played back through my camcorder to an NTSC CRT monitor. As I looped the video, I solo'd each track. There was absolutely no difference whatsoever between the slow motion done without a supersample envelope and the two clips that used supersampling. I also looked at individual frames. They were all pixel-for-pixel identical.

So, in Vegas 10.0a, supersampling still does absolutely nothing to improve video slow motion and, of course, it still dramatically increases render time.
rs170a wrote on 12/3/2010, 10:47 AM
Courtesy of Eugenia, Butter-smooth slow motion.

Mike
hazydave wrote on 12/3/2010, 12:24 PM
Well, it was worth a try.

Supersampling tweens new frames very simply, just using basic interpolation. It can help in some case, not so much in other.

Twixtor and others, like the aforementioned BCC7 plug-in I have used for just such an effect, do motion tracking interpolation. So they'll produce a more meaningful interpolation.
NickHope wrote on 12/3/2010, 11:55 PM
John Meyer, thanks very much for posting that mvtools slomo method. That looks like it will be right up my street for slowing down the odd clip such as fish predation. And can't argue with FREE :) Hopefully not too much of a learning curve if I've already done some AVISynth, stuff (right?).

Has anyone actually tried the BCC7 slomo? How were the results?
LoTN wrote on 12/4/2010, 1:42 AM
I ran some quick test and found it's more user friendly than MVTOOLS. For the result it's like with other similar tools: it depends of the scene and the desired velocity. It can be great or end with multiple and very visible artifacts.

What I really like with optical flow is that it is fully keyframable.

I have to get into this more deeply but I already think I will stop using AviSynth scripts.
RRA wrote on 12/4/2010, 1:58 AM
Hi Nick Hope,

I'm very convinced to SloMo results in BCC7. There are some obstacles :

- you have to keep pixel colour depth on 8bit level
- in some cases Vagas will crash, when redners from one format on TL to another in output file (my setup?).

If you have implemented rules described before (avoid of situation when two objects many times and very quickly will exchange position fore/back/fore/back...), then you will achieve very good quality of SloMo. I'm using SloMo from BCC7 all the time in commercial tasks from marketing - can tell you, that customers are very atracted by this content.

Best regards,

NickHope wrote on 12/5/2010, 7:24 PM
TLF, would you care to share your script (for future reference when I get around to trying this)? Even the part that allows one to load HDV might be useful. And you never know, someone might jump in and fix those problems you're getting with it.
musicvid10 wrote on 12/5/2010, 8:25 PM
TLF, Try this line in your Avisynth script:

ConvertToYV12(matrix="PC.709")


Don't credit me, I found it on the internet. But it is necessary because of a colorspace estimation error that occurs with some frequency.
As far as the pixel stretch, I have no idea, but I bet Nick Hope does.
LReavis wrote on 12/5/2010, 11:02 PM
"Deinterlacing and slowing down in Vegas produces very similar results"

Really "very similar results"? I shoot 60p, so don't need to deinterlace, and I do get pretty good results if I only cut the frame rate in half - or even tolerable results even at 1/4th the original frame rate. But not at all good for, say, 1/8th speed.

However, I remember doing some experiments with MVtools when Johnmeyer put that excellent guide on this forum in Dec., 2008. As I remember it, that procedure produced some impressive results even at 1/8th speed.

So is it really no better than what I can do in Vegas when starting with 60p? (I'd experiment - but I'm really swamped with other work just now)
NickHope wrote on 12/5/2010, 11:26 PM
>> As far as the pixel stretch, I have no idea, but I bet Nick Hope does. <<

Nope (without trawling doom9.org), but I bet John Meyer does ;)
JohnnyRoy wrote on 12/6/2010, 6:55 AM
> I'm using SloMo from BCC7 all the time in commercial tasks from marketing - can tell you, that customers are very atracted by this content.

I've used the BCC7 Optical Flow to get very smooth slo-motion as well. Far better than what you can do with Vegas alone. I'm not sure how much better Twixor would be but for the price, I'll stick with BCC7.

~jr
johnmeyer wrote on 12/6/2010, 12:08 PM
There are many subtle problems with that AVISynth script. I don't have time to list them all, but the main thing is that your HV20 produces interlaced video, and this script ends up creating progressive video. The main rule of video is that if your source is interlaced, keep it interlaced. If, in the end, you are delivering for the web, then go ahead and change it to progressive when you render and let Vegas handle the details.

I tried your script using my own 29.97 HDV clip from an FX1 camcorder, and it killed a lot of detail (due to the change to progressive) and shifted things. I didn't see any aspect shift, so that is probably due to your workflow and not to the script.

My workflow serves the original clip from the Vegas timeline using Frameserver. I serve using the YUV function in Frameserver. If you want, you can serve out using RGB24 and then do a convert in the AVISynth script. If you do this, make sure to include an "interlaced=true" statement.

Serving out of Vegas avoids the AVISynth Directshow read which, in my experience, doesn't always work well.

I test my scripts by first opening them in VirtualDub. However, once the script is debugged, I close VirtualDub and then use VFAPIConv to create an AVI signpost from the script. I then open this signpost in a second instance of Vegas (while the first instance is serving out the video). I then encode the video to the final format in Vegas. This lets me use all the professional encoders in Vegas and avoids some of the issues in going through VirtualDub and its codecs.


Here's the script I use. It uses the much better, newer, version of MVTools (MVTools2). Also, it uses the modified version of AVISynth that can utilize multiple cores. If you don't have this version of AVISynth installed, simply comment out the two SetMTMode lines.

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll") 

setMTMode(5)
AVISource("E:\frameserver.avi").Killaudio()
setMTMode(2,0)

# the following is for HDV, change to bff for DV
assumetff()

#converttoyv12(interlaced=true) #Use this if incoming video is RGB

source=Bob()
super=MSuper(source,pel=2)

#backward_vec = MAnalyse(super,blksize=16, overlap=4, isb = true, search=3 )
#forward_vec = MAnalyse(super,blksize=16, overlap=4, isb = false, search=3 )

backward_vec = MAnalyse(super,blksize=8, overlap=2, isb = true, search=3 )
forward_vec = MAnalyse(super,blksize=8, overlap=2, isb = false, search=3 )


# ml is important parameter for tuning out artifacts. Use values >100 to reduce artifacts.
# "num" equals result of 30 fps times 2 times slo-mo factor - This is for 50% slo-mo
# So, 60000 equals normal speed; 120000 equals 50% normal speed, etc.
# For PAL, use num=100, den=1 for half speed

MFlowFps(source,super,backward_vec, forward_vec, num=120000, den=1001, ml=100)

assumetff().separatefields().selectevery(4,1,2).weave()
assumefps(29.97,true) #Change this to match final framerate (25 for PAL)


johnmeyer wrote on 12/6/2010, 3:16 PM
I'm guessing that Framserver is the debugmode one? Yes.

... then use VFAPIConv to create an AVI signpost from the script ...Sorry about that. VFAPIConv is a tiny utility, written in Japanese, that can take all manner of semi-video "things," including AVISynth scripts, and turn them into a pseudo AVI file that any application -- including Vegas -- can read. Here's a link to the English version of that utility:

VFAPIConv (English version)

To use it, open your AVS script, specify a location for the AVI file, and then click on "Convert." The conversion happens very quickly. Then, put the resulting AVI file on the timeline in Vegas. If you are frameserving, as I usually do, keep the frameserving going in your first instance of Vegas, and then open a second instance of Vegas and drop the AVI file into that. Thus, the workflow is:

Original Video -> Vegas (edit, add filters) -> Debugmode Frameserver -> AVISynth script (opening the Frameserver AVI file created by Vegas/Frameserver in the first line of the script) -> VFAPIConv AVI file -> Second instance of Vegas (add titles and other things that you wouldn't want processed by the AVISynth script)

If you go to the 1:20 mark in this video:



you'll see how VFAPIConv works. In this case I was opening a signpost d2v file rather than an AVISynth AVS file. This trick lets me get MPEG-2 and VOB files onto the timeline in Vegas and play them on the timeline with the speed of an AVI file. This is how I dealt with MPEG-2 and VOB files in earlier versions of Vegas. The speed was dramatically faster than what native Vegas could provide. I haven't done benchmarks in 10.0a to see if MPEG-2 timeline performance has been improved.
johnmeyer wrote on 12/6/2010, 3:33 PM
Since I mentioned the importance of keeping interlaced video in its original interlaced form, and since most AVISynth scripts are designed to work with progressive video, you might be interested in an alternative approach to the slow motion problem, but using a technique that, once you see it, can allow you to easily use scripts designed for progressive video, even when you are working with interlaced. This script uses the common technique of separating the interlaced video into even and odd fields, and then applying the progressive script to the odd fields, and then applying it again to the even fields, finally weaving them back together again at the end. Unfortunately, it doesn't lend itself to the multi-core parallel processing, but on the other hand it always works, and you don't have to re-write an existing script.

Here is the code for this project:

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll") 

source=AVISource("E:\frameserver.avi").KillAudio().assumetff()

final=ApplyInterlacedFilter(source,"SlowMo").assumefps(29.97,false)

return final


function ApplyInterlacedFilter(clip v1, string filter)
{
v2 = separatefields(v1)
selecteven(v2)
even = Eval(filter)
selectodd(v2)
odd = Eval(filter)
interleave(even,odd)
return weave()
}

function SlowMo(clip thisclip)
{
super = MSuper(thisclip,pel=2)
backward_vectors = MAnalyse(super,blksize=8, overlap=2, isb = true, search=3)
forward_vectors = MAnalyse(super,blksize=8, overlap=2, isb = false, search=3)
MFlowFps(thisclip,super,backward_vectors, forward_vectors, num=60000, den=1001, ml=100)
}
farss wrote on 12/6/2010, 3:50 PM
Just a thought that may help with this interlaced thing.

I've found in Vegas setting project fps to Double (50 or 60fps) and de-interlace to Interpolate means I can treat 50i as 50p and then convert back to 50i.

I have zero idea how this would work out frameserving...

Bob.
johnmeyer wrote on 12/6/2010, 7:44 PM
I posted a tutorial on creating slow motion using Vegas and the AVISynth plugin called MVTools2. Here's the link:

Tutorial on creating Slo-Mo using AVISynth