Vegas to the Web for the Videophile - A Tutorial

NickHope wrote on 3/2/2011, 5:30 AM
Over the past few years I have been using the following workflow for rendering videos for the net, whether they be destined for self-hosting or a site like YouTube, Vimeo or Facebook.



I found this technique gives a higher quality than is possible with the tools in Vegas Pro. The Sony and MainConcept AVC codecs in Vegas Pro 10.0 have caught up a bit, but we are still limited to rudimentary blend or interpolate deinterlacing, and bicubic resizing.

In the course of this mammoth discussion instigated by Musicvid, I have refined some aspects of my workflow, in particular the deinterlacing, which I am now doing with the amazing QTGMC deinterlacing script. But much of the workflow is useful with progressive footage too.

I have written a tutorial explaining how to do it. For the time being this is written with upload to Vimeo/YouTube/Facebook in mind. I hope to add a branch for rendering for self-hosting, where lower bitrates and slightly different x264 settings will be used. I may also split the tutorial up into a number of pages (any thoughts on that?), and I also have a site redesign in the pipeline which might help readability.

This is a workflow for videophiles who are prepared to spend time and effort getting the best quality:bitrate ratio they can. With interlaced footage it is SLOW. Much slower than just using Sony AVC. But there are presets which can make it faster (or slower). If speed is your priority, just use the Sony AVC codec, but do visit the section about conforming your video's levels, which applies to all AVC/H.264 videos destined for the web.

If anyone has any comments on the workflow or the tutorial, or has success or failure with it, please let us have some feedback in this thread.

Musicvid has been using Handbrake to achieve a very similar result, and I believe he is planning to publish a tutorial for that soon.

Many thanks to Musicvid and amendegw for that engrossing discussion. Thanks to amendegw, Kimberly and Stringer for permission to use test footage, and to dxdy for hosting test renders on his Vimeo account. Thanks to Laurence, farss, A.Grandt and the rest of you who gave insight which helped with other aspects of this workflow. And thanks of course to the geniuses who write these amazing free tools that make this possible.

Comments

farss wrote on 3/2/2011, 5:54 AM
Thanks Nick, that does look interesting.
I've stayed away from Musicvid's thread if for nothing else the than the assumptions that it was based on.
To perhaps state the obvious it's long been my view that to get the best results out of YT or Vimeo you have to shoot the very best video in the first place and to do that you need to shoot progressive. That has left me with one major headache. How to get high resolution 25p onto a SD 50i DVD without massive issues with line twitter and how to avoid the overt problems of judder at such low frame rates. For once I'm jealous of those who can shoot 30p.

Recently though I've realised we face another hurdle, time to air. Part of one event I covered last year was also shot by someone with a phone camera and not even a good one at that. Uploaded to YT on the day it looked horrendous and sounded even worse. My carefully shot, edited and encoded coverage looked and sounded way, way better but it took me over a week to get it onto YT. Guess who got all views though. My one consolation was a lot of my footage did make it to air as part of a retrospective of the events of 2010, unfortunately only on a community channel with 10% coverage of Sydney.

Bob.
Laurence wrote on 3/2/2011, 6:23 AM
I am shooting interlaced again because the Handbrake decomb is that good. At this point, 60i looks the best (to my eyes) overall. 60i gives me smooth motion on HD TVs and decombed 30p that looks outstanding online. I no longer have to sacrifice one for the other.

Your approach looks like it will tweak these results even further.

I know I am on my own a little here, but I still like smartrendering my hdv into 25Mbps .mxf. This workflow involves careful shooting and use of picture profiles rather than color correction and grading in post, but I really like the look of the end result, especially as it looks sRGB to cRGB corrected and deinterlaced with the free Sony mxf player.
Laurence wrote on 3/2/2011, 6:51 AM
There are two decent Vegas deinterlacers: Mike Crash and BCC7. I've been using the 64 bit Mike Crash recompile that somebody here linked to a while back. I think it looks just as good as Handbrake's decomb. I still like Handbrake's resize better though. I imagine your Avisynth filters are even better.
altarvic wrote on 3/2/2011, 8:16 AM
Also there is another free deinterlacer for Vegas 10 - Yadif Deinterlace
I didn't test it, so it would be very interesting to hear your comments about it.
NickHope wrote on 3/2/2011, 8:28 AM
@ Bob - I bet your 25p->50i line twitter issues can be improved with the help of some AviSynth plugins. But I'm not volunteering to find out how. Maybe you could brave the doom9 forums and see if anyone has any suggestions? They live and breathe this stuff over there.

Actually my method rocks along with progressive footage. The deinterlacing part is responsible for the vast majority of the time taken. And I've shown in the tutorial exactly which parts can be left out if your source is progressive. I will do a wee test to see how the time compares to Sony AVC with a 1080p source.

@ - Laurence - You should try a 60p version using my QTGMC method sometime (by leaving out the "select even" line in the AviSynth script). Basically it's a bob deinterlace but with the bobbing fixed. I was blown away by the quality on my computer. Basically it looks like 60i on a good TV. I would be interested to know how it compares with 60i in the Sony MXF player. 50p/60p is a lot of frames but it's awesome for offline playback and potentially also for self-hosted players online as bandwidths increase.

I haven't got a BCC7 licence and I haven't done any testing recently with the Mike Crash plugin. It uses an older version of Smart Deinterlacer that didn't have an edge-directed interpolation option. I did try an x264vfw encode via VirtualDub using the latest Smart Deinterlacer with EDI. The deinterlacing result was good but the outputted .avi file was a nightmare for most software to deal with. What exact settings are you using with the Mike Crash plugin? I'd like to try it in conjunction with Sony AVC/MC AVC/x264.

@ altarvic - Wow! Nice find! Yadif is a good deinterlacer and it's relatively fast. I'll give it a try. It won't give quite the result that QTGMC does but if it works the same as the Yadif in AviSynth or Handbrake then I can imagine it becoming many Vegas users' standard deinterlacer.

Yadif: "Yet Another DeInterlacing Filter" ported from MPlayer. It checks pixels of previous, current and next frames to re-create the missed field by some local adaptive method (edge-directed interpolation) and uses spatial check to prevent most artifacts. Its operation is explained here on the Handbrake site.
musicvid10 wrote on 3/2/2011, 8:36 AM
Nick, Congratulations on a definitive work. I'm sure a few fine points will be addressed, but it is built on a solid foundation. It will take me a while to digest it all.

WRT progressive HD:
Shooting progressive is one thing.
Rendering progressive in Vegas is quite another.
I'm with laurence on this.
Very little AVCHD or HDV is being shot at the consumer level at true 1080p at this time. That is still my underlying assumption. So we look for the best way to deinterlace and resize 1080i to 720p for sane web delivery. We agree that Vegas is not the best way to do this.



NickHope wrote on 3/2/2011, 8:36 AM
I should add that Handbrake's decomb, that Musicvid and others have been using, has more to it than just Yadif. But Yadif should be significantly better than the native options in Vegas.
Jøran Toresen wrote on 3/2/2011, 8:43 AM
Is Yadif a 32 or 64 bit plugin?

Jøran
NickHope wrote on 3/2/2011, 8:52 AM
I see both in the download :)
Laurence wrote on 3/2/2011, 8:59 AM
I just wanted to add that the 60fps smart bob deinterlace is exactly what I and my clients are seeing when we look at an mxf video through the free Sony XDcam playback utility. That along with exactly the right amount of color range expansion. Not at all bad for a quick smart-render. The quality jumps out at you. The only issue I have is that when I try to put the Sony player on client's laptops, it only gives this level of playback if the have a real graphics card with separate RAM. Integrated graphics with shared RAM will work with the player, but not in the high quality accelerate mode that looks so wonderful. The non-accelerated mode looks terrible: even worse than VLC.
Jøran Toresen wrote on 3/2/2011, 9:06 AM
Nick, yes Win 32 and Win 64. But are these file for Window 32 and 64 bit, or Vegas 32 bit and 64 bit?

Jøran
NickHope wrote on 3/2/2011, 9:17 AM
@ Laurence - That would explain why I was less than impressed with it on my laptop. Well, for offline playback where you want to pack a punch but don't have a real graphics card, give QTGMC-deinterlaced 50p/60p a whirl.
NickHope wrote on 3/2/2011, 9:18 AM
@ Jøran - I have no idea, I'm afraid, and I can't test it as my 64-bit machine is out of action.
johnmeyer wrote on 3/2/2011, 10:00 AM
I've followed the lengthy original discussion, and I am in awe of the work you have put into this, and the scientific approach you used to improve the end result.

I have not read all 200+ posts in the original thread, so I am not entirely sure whether the reason for using MeGUI for the encode was entirely due to some quality improvement, or whether it was because MeGUI can read the AVS file directly. FWIW, unless the quality improvement is significant, I'd be tempted to follow your first three steps, but then use VFAPIConv to create a signpost AVI from the AVS file, and then open that in Vegas and encode there. I'd use a Levels statement at the end of the AVISynth script to adjust the levels as appropriate, something like:

Levels(16, 1, 235, 0, 255, coring=false)

I am sure that these are NOT the correct settings; I am including it just to point out that you can do these adjustments in the AVISynth script and thereby avoid having to add a bunch of steps inside of Vegas.

Also, since you mention several times that speed is an issue, I would certainly try the script with the multi-core version of AVISynth, using the SetMTMode statements to specify when to switch to multi-core. I note, in threads at doom9.org, that people ARE using SetMTMode with QTGMC. With other AVISynth filters (like mvtools2), this can provide as much as an 8x speed improvement (yes, that is eight times faster). If the script is stable using the multi-core hack (and you'll know right away) you also have to make sure that the video looks identical. I would encode a few seconds of fast moving video and compare to the non-multi-threaded version.

VFAPIConv (English Version) Download

musicvid10 wrote on 3/2/2011, 10:20 AM
Without pretending to speak for Nick (he is parsecs ahead of me on this), I think the reason for encoding in MeGUI is that x264 shows a noticeable improvement over Sony AVC or Mainconcept AVC at bitrates below 10Mbs, and the differences increase favorably as bitrates get even lower. It is my reason for using x264 in Handbrake. Even x264vfw in Vegas has some distinct disadvantages.

Our presumed "sweet spot" for uploading to Youtube and Vimeo is around 8Mbs 720p.
NickHope wrote on 3/2/2011, 11:23 AM
John, the original reason for using MeGUI was the gulf in quality between x264 and the encoders in Vegas prior to 10.0. The bundled AVC codecs have improved in 10.0, so your idea to use VFAPIConv is a good one, but x264/MeGUI still beats Sony AVC for speed and quality.

To verify this I ran a quick comparison with the deinterlacing and resizing out of the mix by rendering to 720p AVC from a 720p HuffYUV file. Sony AVC at a requested 11 Mbps (to deliver 7873 Kbps) versus x264 at the same settings as the tutorial except for crf 17.9 (delivering 7790 Kbps). x264/MeGUI just wins for speed (9.7 fps vs 9.2 fps), and comparing stills on the timeline, the x264 version is slightly more faithful to the original. So if one is going to the trouble of going in and out of 2 instances of Vegas to do deinterlacing and resizing in AviSynth, my feeling is that the encoding is better done in MeGUI.

As Musicvid says, differences increase as bitrates get lower, as amendegw's tests have shown, so x264 wins more clearly at self-hosting bitrates.

So, to backtrack a little, the actual processing time following the tutorial is only slower than Vegas if you need to deinterlace. Admittedly there is more setup to get it started.

Regarding multi-threading QTGMC to get more speed, I haven't really been able to test that as I'm only on a Core 2 Duo. I've distilled what I've learned on the doom9 discussion thread into my tutorial as best I can, but I'd welcome feedback from anyone who gets that working or otherwise.

p.s. If anyone trying John's method can't get VFAPIConv to read .avs files, read this thread where both goodrichm and I had to do some trickery with ReadAVS files to get it working.
johnmeyer wrote on 3/2/2011, 11:28 AM
I think the reason for encoding in MeGUI is that x264 shows a noticeable improvement over Sony AVC or Mainconcept AVC at bitrates below 10Mbs, and the differences increase favorably as bitrates get even lower.Thanks, that's useful insight. I wonder, however, if you fed the externally deinterlaced footage back into Vegas, would the differences still be evident? I ask this because it seems that a lot of the posts over the past few years about encoding interlaced footage for the web always seem to focus on the subject of the quality of deinterlacing. Clearly Nick has gone down this path about as far as one can go, and has found one of the more advanced deinterlacing solutions around. Since many people have found that Vegas doesn't do a very good job of deinterlacing, I'm just wondering out loud that if you completely remove that deficiency, by way of the first three steps in the tutorial overview graphic, would the encoding in either of the Vegas encoders then look pretty good?
rmack350 wrote on 3/2/2011, 11:49 AM
This is slightly off the topic but it's something I've noticed when exporting stills with recent versions of Vegas. What I'm seeing is that (in VP9 and 10) that Vegas *always* deinterlaces stills exported from the preview window in NTSC DV projects. (probably in all interlaced projects).

Example:
Start a project with an NTSC DV template (probably any interlaced template and media will work).
Load a clip onto the timeline and find a spot that exhibits a lot of combing. Mark the spot.

Test1: Save a PNG or JPG from the preview window and open it into any external viewer. The image will be aspect corrected and deinterlaced

Test2: Save the preview image to the clipboard and paste it into a new document in Photoshop or equivalent. The clipboard output is ALSO aspect corrected and deinterlaced.

Test3: Save the preview image to the clipboard and then Enable the preview Window's Split Screen view (showing clipboard content). Now drag a box in the preview window so you can see a slice of clipboard content over the preview image. You should be seeing a mushy deinterlaced slice of the clipboard.

So how do you turn off the deinterlacing? I've tried...

1. Set Deinterlace Mode to "None" in the project's properties (No effect)
2. Set the media's field order to "None" (No Effect)
3. Set the project properties field order to "None" (This stops Vegas from automatically deinterlacing still images but if you're editing an SD project you can't work like that)

Okay, so if the project template is interlaced then Vegas always deinterlaces the images saved from the preview. Obviously you wouldn't want to use a still image from your project if Vegas is just going to deintelace (and change/degrade) the image.

What lead me to look at this again today was the instructions for the Yadif deinterlacer saying to do either step 2 or 1 above when you use the deinterlacer. I wanted to test the difference on stills between Yadif and the native deinterlacing. For *stills* I get exactly the same results from the two. Layering the two together in Photoshop and setting the top one to Difference mode gives me a black image, which I'm pretty sure means they're identical.

If anyone has a solution I'm all ears but in the mean time I'm going to resubmit this as a bug report. The last time I did this I got the impression that this was by design. It's dumb stuff like this that makes me not want to recommend Vegas to people.

Rob Mack

NickHope wrote on 3/2/2011, 12:04 PM
>> ...would the encoding in either of the Vegas encoders then look pretty good? <<

The answer is yes, at around 8 Mbps it does, but not quite as faithful to the source as the encoding in x264. But you have to look REALLY closely to see the difference. For YouTube the difference doesn't matter since they are going to butcher it anyway.

I should say that I base that observation on the renderings I just did from a 720p HuffYUV file which itself was rendered from the best of my 1080-60i->720p x264 renderings (I don't have any footage that was shot at 720p). Tomorrow I will run the full 1080-60i->720p scenario exactly as John suggests and post a couple of frames.

Let me also do a little testing at 1-2 Mbps and see how big the difference is.
musicvid10 wrote on 3/2/2011, 12:08 PM
@johnmeyer,
Great question, and yes, the encoding differences held up in the tests I ran before starting the first thread, when deinterlace (and resize) were take out of the soup.
IOW, at the original source sizes, i->i looked better in x264, as did p->p, with the differences becoming more pronounced with the reduction in bitrate.

Unfortunately, I did not save the grabs from those tests, so duplicating them at some point would seem to be important to support some of the original assumptions I made, which I believe are independently supported by Nick and laurence.
NickHope wrote on 3/2/2011, 12:23 PM
@ Musicvid - But you used 8.0 for your tests didn't you? Both AVC codecs in 10.0 are better. But I need to verify that again in case I'm talking rubbish. I have both 8.0 and 10.0. I'll add a quick test to the list.
amendegw wrote on 3/2/2011, 12:30 PM
"JohnMeyer said: Thanks, that's useful insight. I wonder, however, if you fed the externally deinterlaced footage back into Vegas, would the differences still be evident?Ahh! Here's an area I think I can help - low bitrate testing. If I understand the question correctly, the answer is... The best quality is observed when rendering to a 1920x1080 interlaced intermediate followed by using HandBrake to resize & deinterlace.

Three way test all ending up at 1280x720 29.97fps 1200kbps:
1) Vegas -> DNxHD Intermediate -> HandBrake (Best)
2) Vegas -> Sony AVC (Worst)
3) Vegas->DNxHD->HandBrake (8Mbps)->Vegas->Sony AVC i.e. HandBrake did the deinterlace & resize, Sony AVC reduced the bitrate. (Middle).

View the results here: http://www.jazzythedog.com/testing/dnxhd/mediaplayers2.aspx

...Jerry

Edit: All Sony AVC renders were done in Vegas 10.0c
Edit2: Sorry, Sorry, Sorry turns out I forgot to run mp4faststart against the file rendered in case 3). It's fixed now.

System Model: Alienware Area-51m R2
System: Windows 11 Home
Processor: Intel(R) Core(TM) i7-10700K CPU @ 3.80GHz, 3792 Mhz, 8 Core(s), 16 Logical Processor(s)
Installed Memory: 64.0 GB
Display Adapter: NVIDIA GeForce RTX 2070 Super (8GB), Nvidia Studio Driver 527.56 Dec 2022)
Overclock Off

Display: 1920x1080 144 hertz
Storage (12TB Total):
OS Drive: PM981a NVMe SAMSUNG 2048GB
Data Drive1: Samsung SSD 970 EVO Plus 2TB
Data Drive2: Samsung SSD 870 QVO 8TB

USB: Thunderbolt 3 (USB Type-C) port Supports USB 3.2 Gen 2, DisplayPort 1.2, Thunderbolt 3

Cameras:
Canon R5
Canon R3
Sony A9

NickHope wrote on 3/2/2011, 12:51 PM
Well, I tried this new Yadif Vegas plugin, encoded the output with MeGUI and the result was soft and blurred. And I don't mean motion blur. Just blur. Simple Vegas interpolation looks better. I'll do a couple more to check I didn't mess something up. Edit: Turns out I did mess something up, as per the next post by Laurence. See later in the thread for YadifVegas results.

Jerry, have you jazzed up any of the x264 parameters for low bitrate/self-hosting? Could you please post a snagit of your advanced settings so I can mimic them in MeGUI? Also are you using CQ or Avg bitrate? And is your test (1) done with your latest settings?
Laurence wrote on 3/2/2011, 12:55 PM
I'm sure you know this, but you have to set Vegas up with a deinterlace method of none and make sure that the external deinterlace filter is either pre-pan-crop or inserted as a media FX. It's pretty easy to make things look soft if it isn't set right.

Having said that I haven't actually tried Yadif yet. I've downloaded and installed it, but I haven't done any tests yet.