Fisheye correction - any better solutions?

DDDyson wrote on 6/5/2012, 2:42 PM

I'm struggling to get proper, correct, rectilinear fisheye correction done in Vegas Movie Studio. I'm using the VIO POV.HD camera and its footage is pretty heavily curved.

The Sony Spherize plugin (suggested by some) is more or less useless for this. It MIGHT work if I was able to stretch its effect way outside the video frame. I tried shrinking the video itself just to see if it works, but... meh.

Someone tipped me off to the Sony Deform plugin (Top/Bottom/Left/Right curve). Yes, it works to an extent - but it's impossible to get proper correction from it, there is always odd curving and warping regardless of the values I use.

The NewBlue Lens Correction plugin has the same problem as the free Barrel Distotion VirtualDub plugin (by Emiliano Ferrari): there's weird curvature, regardless of what Alpha and Beta values you use. Maybe there are correct values that produce a rectilinear image, but even after hours of tweaking I just can' find them.

--> However: the Defish AviSynth plugin (by David Horman) does a PERFECT job. When I set the FOV to the right amount (101 degrees), the result is perfectly rectilinear. Excellent! Looks like there is a solution.

The problem is, I can't feed AviSynth to Vegas. Media players read the .avs files just fine, but Vegas doesn't. I actually tried this tutorial:
http://forum.doom9.org/archive/index.php/t-47194.html
... but my problem turned out to be that VFAPI doesn't support QTInput (my source videos are in .mov format), and freaks out with MP4 files (the other format POV.HD can record to). Time to give up.

Yes: I can always defish the video by just re-encoding the video with VirtualDub w/AviSynth. But that introduces an intermediate re-encoding step which is lossy (or creates a massive lossless intermediate file) and takes a long time, and is a pain to batch operate. It will take me literally days with multiple passes to create a stereoscopic, edited video. I'd prefer the whole process queue to be in one program, one pass so that I can edit and render everything in one pass without intermediate steps.

Are there any ways to do this I might have missed? (save for forking out the cash for After Effects.... a leetle beet beyond my budget, that....)

Sony guys, would it be technically feasible to make Vegas able read those AviSynth .avs files? (media players don't seem to have problems with it)

Comments

musicvid10 wrote on 6/5/2012, 3:53 PM

Vegas (a commercial product) will never support AviSynth (an open-source project) scripts or filters .

Older versions of VMS supported Debugmode Frameserver->AVISynth. It doesn't work with VMS 11 afaik, and there is no timetable from the author when this may happen.

You could download the Vegas Pro trial and frameserve to AVISynth for a month; should be long enough to get the work done. It's also easy enough to render an RGB or DNxHD intermediate in VMS, and give that to AVISynth.

If you want to dig deeper, see this article written by a friend of the forums:
http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm

DDDyson wrote on 6/5/2012, 4:29 PM

Thanks for your reply.That's a very helpful page! But doesn't DebuMode Frameserver goes the wrong way around, if I didn't understand this completely wrong?

What DMFS does:
Vegas -> AviSynth

What I want to do:
AviSynth + Defish -> Vegas (two streams, so I can create a stereoscopic video and apply rotation/convergence correction). Defishing has to be done before the other operations.

musicvid10 wrote on 6/5/2012, 9:01 PM

Vegas->Debugmode->AviSynth+Defish->Lossless Intermediate->Vegas is what I was thinking. Perhaps I missed some requirement in your workflow?

Email Nick at Bubblevision, he will give you much better advice than I can, although I don't know if he is doing stereoscopic yet.

DDDyson wrote on 6/6/2012, 12:39 AM

I can already do AviSynth + DeFish -> VirtualDub -> Two huge lossless intermediate .avi files that take hours to render (left + right camera) -> Vegas Movie Studio for pairing, adjustment and editing.

I found QTInput, which makes AviSynth able to read .mov files via the installed QuickTime codecs, so I don't need DebugMode Frameserver for that. So AviSynth->VirtualDub was enough.

But what I wanted to do, is to have the whole pipeline functional in one pass, so I wouldn't need intermediate passes and phases. If there was a plugin available for Vegas which would work like Defish ( = give perfect fisheye correction), it would be awesome. Then I could do all the corrections, edits, etc, and only then leave my computer for hours to render the final product, without having to come back every few hours to start the next phase.

A lens correction plugin that works like Defish would seem like a fairly simple plugin to do - Defish was developed for free - and since there are hordes of people who own GoPros and POV.HDs out there, there would certainly be a market for a fisheye correction plugin like this. Why doesn't someone? It would be a lucrative business. :) Is third party plugin development for Vegas a big hurdle to overcome, does it require expensive license fees or such?

Vegas MS is, interestingly enough, able to handle the distortion correctly even on paired stereoscopic clips which have been adjusted (rotated, cropped). If you add the Deform plugin to a video track that has a paired stereoscopic subclip in it, for example, the Deform is correctly added to both video tracks (left, right) before they get combined side-by-side with the rotation/convergence adjustments. This makes me believe that a native Vegas lens correction plugin would also work like this.

musicvid10 wrote on 6/6/2012, 1:39 AM

Plugin SDK for Vegas is still a free download afaik -- please let us know what you come up with, I'm sure others will be interested.

DDDyson wrote on 6/6/2012, 1:46 AM

I'm no programmer myself. But I guess I could try and contact the author of the Defish plugin (David Horman) and ask if he would be interested in porting Defish to Vegas (including Movie Studio).

He could even charge money for it, or make it donationware. I'd buy it.

DDDyson wrote on 6/6/2012, 1:57 AM

Hmm... Wax only works with Vegas Pro, but not Vegas MS, right?
http://www.debugmode.com/wax/

(since AFAIK, Vegas MS doesn't support .NET scripting)

DDDyson wrote on 6/6/2012, 2:16 AM

I registered to the Doom9 forum to contact the Defish author - but turns out I need to be registered for 5 days before I can post or message anyone. Stay tuned.........

DDDyson wrote on 6/6/2012, 2:58 PM

I've been Defishing my POH.HD videos with AviSynth + Defish -> VirtualDub, saving them to XViD videos... and after hours of conversion/rendering, turned out I ran into two problems:
- For some reason, the video is not rendered all the way to the end. The last 10% is left out
- The video's framerate changes from 29.950 to 25, even though AviSynth should not be set to change it, nor should VirtualDub. Maybe it's the video codec I'm using (XViD / DivX).

Yeah. Would be far simpler if Defish was ported to Vegas. Hope it can be done.

jetdv wrote on 6/7/2012, 8:58 AM

Take a look at the Lens Correction tool in Video Essentials 2 from NewBlueFX.com

DDDyson wrote on 6/7/2012, 4:30 PM

If you look at the original post, you may notice I already have. It gives bad, curved results. Out of the plugins I have tested, only Defish has given a perfectly rectilinear result.

DDDyson wrote on 6/9/2012, 4:36 AM

Here's what's happened in the meantime:

- The defished, intermediate DivX / XVid files turned out to be problematic to edit in Vegas. For some reason the seek times are crazy long: whenever I seek to another part of the video, it takes 4-10 seconds to render a new preview frame. (why is this? It seems to have started after I had updated my NVidia drivers)

Because of this, I started looking for a way to encode to some other format than XViD/DivX AVI via VirtualDub.

- I had an error with AviSynth with the orignal .mp4 file: it would never reach the end of the video. The conversion got interrupted, cutting off the end of the video. Googling and a few tries of various utilities revealed that the video file is missing "moov atom" metadata. ------> I purchased QuickTime Pro to re-save the files as .mov files that should contain proper metadata. (why? because none of the free utilities I tried, worked)

As a side note, KMPlayer, QuickTime Player and Vegas could handle the original .mp4 file just fine even if it was missing the metadata. Only AviSynth had this end cutting problem with it. Alas, I can only Defish the video with AviSynth...

- After re-saving to .mov, Defish started complaining that the video's dimensions are not even. And it started looking really blocky. After hours of trial and error I found out I can solve this by switching AviSynth QTSource to decode mode 3 (mode=3). For some reason the default decode mode gives a very blocky result, it looks like half size.

A new AviSynth Defish operation is currently in progress.... after a few hours I will find out if AviSynth is now able to stream the file fully all the way to the end.

P.S. Tip of the day: it's very tricky to batch operate VirtualDub or AviSynth. However, you can run multiple instances of VirtualDub or your media player to process multiple videos overnight. :D

DDDyson wrote on 6/9/2012, 12:59 PM

Because QTSource's H.264 encoding doesn't work, I have switched from that to using AVISynth -> MeGUI (with x.264) to encode the defished videos to h.264.

This should give areasonably sized intermediate file with high quality and fast seek & render times. But it's really slow to re-encode. It's going to take some 8-9 hours to create these intermediate video files, and only then can the editing and rendering start. Fortunately, though, MeGUI allows queueing of tasks, and running multiple encoding processes at once.

Though.... the encoding is probably slowed down by the fact Defish is quite slow (even the multithread version), and I've thrown a couple of other plugins in the AVISynth chain as well: MSU Deblock (before defish) and MSU Smart Sharpen (after defish).

ADB wrote on 6/11/2012, 8:41 PM

DDDyson,
I'm having trouble finding defish for virtualdaub. Do you have a download link please ?

DDDyson wrote on 6/12/2012, 2:10 AM

Here:
http://forum.doom9.org/archive/index.php/t-152860.html

NOTE: it's for AviSynth, not VirtualDub. Sorry. So it's a bit tricky to use. You'll need to install AviSynth, then write a script for it. Then this script file can be opened in VirtualDub (or a media player, like the KMPlayer) and you can see AviSynth + Defish do their magic.

Note: there is a link to a multithreaded ( = faster) version of Defish further down the forum thread. It's a pretty heavy plugin overall, definitely not realtime on my i7, but the results are excellent: rectilinear without odd warping. And it only needs the FOV parameter to be set right to work, at least with my footage. No need to worry about figuring out the polynomial parameters.

ADB wrote on 6/12/2012, 6:39 PM

Did you try Barrel Distortion Correction 1.4 with VirtualDub ? It looks a lot easier.

DDDyson wrote on 6/14/2012, 4:54 AM

The one by Emiliano Ferrari? Yes, I've tried it (as you can see in the original post), but it doesn't give a rectilinear result. There's a bit of odd warping and curvature left. Maybe you can get it right with absolutely correct Alpha and Beta values, but I just couldn't get them right even after tens of minutes of tweaking.

ADB wrote on 6/14/2012, 5:12 AM

I did see a tutorial on how to get the parameters correct but I can't seem to find it again.

DDDyson wrote on 6/14/2012, 5:20 AM

You mean this?
http://www.astrosurf.com/buil/iris/tutorial19/doc42_us.htm

Alas, I can't figure out how to apply that to the Barrel Distortion Correction plugin.

DDDyson wrote on 6/14/2012, 5:23 AM

BTW, why is this topic re-posted here by "Anne marie"? :)
http://www.designstudioschool.com/fisheye-correction-any-better-solutions-t226316.html#p2492434

vkmast wrote on 6/14/2012, 6:35 AM

DD,
these "re-post forums" seem to come and go. The one you mention used to be something like "sonytalk.com" at some time I believe. I'm afraid there will be others.

ADB wrote on 6/15/2012, 4:16 PM

No the tut I saw was a video of a wire mesh fence, being corrected directly with the virtualdub plug-in.

DDDyson wrote on 6/28/2012, 10:44 AM

Did it really make it completely straight, and not slighly oddly curved near the corners? Was there some kind of a math rule that you need to abide to?

DDDyson wrote on 1/23/2013, 7:51 AM

ProDad recently introduced a program called Defishr:
http://www.prodad.com/home/products/video_optimization/300546449,l-us.xhtml

Seems promising - but it's currently only stand alone, not a plugin. And even when they make it a plugin, I doubt it will work in Vegas Movie Studio, since their Mercalli V2 also only works in Vegas Pro.