Vegas to the Web for the Videophile - A Tutorial

NickHope wrote on 3/2/2011, 5:30 AM

Over the past few years I have been using the following workflow for rendering videos for the net, whether they be destined for self-hosting or a site like YouTube, Vimeo or Facebook.

I found this technique gives a higher quality than is possible with the tools in Vegas Pro. The Sony and MainConcept AVC codecs in Vegas Pro 10.0 have caught up a bit, but we are still limited to rudimentary blend or interpolate deinterlacing, and bicubic resizing.

In the course of this mammoth discussion instigated by Musicvid, I have refined some aspects of my workflow, in particular the deinterlacing, which I am now doing with the amazing QTGMC deinterlacing script. But much of the workflow is useful with progressive footage too.

I have written a tutorial explaining how to do it. For the time being this is written with upload to Vimeo/YouTube/Facebook in mind. I hope to add a branch for rendering for self-hosting, where lower bitrates and slightly different x264 settings will be used. I may also split the tutorial up into a number of pages (any thoughts on that?), and I also have a site redesign in the pipeline which might help readability.

This is a workflow for videophiles who are prepared to spend time and effort getting the best quality:bitrate ratio they can. With interlaced footage it is SLOW. Much slower than just using Sony AVC. But there are presets which can make it faster (or slower). If speed is your priority, just use the Sony AVC codec, but do visit the section about conforming your video's levels, which applies to all AVC/H.264 videos destined for the web.

If anyone has any comments on the workflow or the tutorial, or has success or failure with it, please let us have some feedback in this thread.

Musicvid has been using Handbrake to achieve a very similar result, and I believe he is planning to publish a tutorial for that soon.

Many thanks to Musicvid and amendegw for that engrossing discussion. Thanks to amendegw, Kimberly and Stringer for permission to use test footage, and to dxdy for hosting test renders on his Vimeo account. Thanks to Laurence, farss, A.Grandt and the rest of you who gave insight which helped with other aspects of this workflow. And thanks of course to the geniuses who write these amazing free tools that make this possible.

Back to post

Comments

amendegw wrote on 3/6/2011, 5:12 PM

So, my Spyder adjustments are stored in an ICC profile. I would assume that's the same place that any manual adjustments are stored when you "Adjust Video Color Settings" - correct?

...Jerry

johnmeyer wrote on 3/6/2011, 5:47 PM

The Spyder settings are for the monitor and not for the overlay. You have to calibrate the overlay separately.

musicvid10 wrote on 3/6/2011, 6:57 PM

Unfortunately, the overlay settings are not exposed in as many graphics cards these days, especially not in onboard cards.

NickHope wrote on 3/6/2011, 7:57 PM

The cosmic ray storm appears to be over. Today everything that should be getting blown out is now getting blown out. I checked Musicvid's YouTube test, his Vimeo Version 4, and my gradient in IE, FF and GC.

I have absolutely no idea what happened yesterday. All I can think of is possibly not enough reboots when changing Flash Player versions. Or another program I had running was upsetting things.

So we seem to be back to how we were. We should still keep everything within 16-235 for YouTube.

By the way Jerry, I suspect a difference in overlay behaviour is behind why you can snagit videos and I can't.

NickHope wrote on 3/7/2011, 12:57 AM

QTGMC was updated to version 3.20 a couple of days ago. The performance is not really different but he's separated out the help file and the noise presets. I think it's worth updating. I've updated the tutorial with those changes as well as a few minor corrections.

amendegw wrote on 3/7/2011, 3:15 AM

Just to close the loop on musicvid's request - "Wonder is "someone" might have time to run another series of tests? (Cue Jerry stage right)."

I have scoped the screen captures from JW Player, FlowPlayer, etc. and all the REC709 frames are displaying as they should using Flashplayer 10.2 (at least on my computer!!). btw, all the on-line clips were HandBrake renders from musicvid's DNxHD intermediate dated 2011-01-11

Are you guys expecting anything more from me right now?

...Jerry

NickHope wrote on 3/7/2011, 3:24 AM

No Jerry. Thanks for checking all that. I just need to keep an eye on things, maybe try a few more computers, and see if I can work out what on earth happened for 24 hours.

johnmeyer wrote on 3/7/2011, 11:02 AM

Updated to the newer version of the script. No problem.

However, I went back and re-visited the "worker" concept in MeGUI. After a few minutes of experimentation, and also reading various doom9 posts, it is now pretty clear to me that the "worker" concept is pretty much useless, and that you really don't need to assign more than one worker (two at most), regardless of how many cores you have. The reason is that MeGUI's "worker" concept is done on a per job basis, and NOT on a thread basis. What this means is that if you encode more than one video at once, you can indeed do those encodes in parallel by assigning more than one worker. However, in the workflow for this tutorial, that is not going to happen. Unlike multithreading, where a single activity can be split up into small "threads," with each thread processed individually, and the results stitched together later on, the worker concept can only deal with complete jobs.

I thought that the audio job might be done in parallel with the video encoding, but I found out that this doesn't happen. Even with four workers, MeGUI insisted on doing the entire job (audio and video and muxing) in series, and not in parallel.

I also tried the "faster" setting instead of "very fast" because Nick's deinterlacing comparison in another thread showed a small improvement with this setting. However, I ended up with MeGUI crashes when doing this. I'm not sure quite why. If I have time, I'll try without the SetMTMode statements in the AVISynth script and see if that is the source of the problem.

NickHope wrote on 3/7/2011, 12:27 PM

Thanks John, that's great feedback. I'll run a couple of tests with the default of just 1 worker and if nothing slows down I'll take that recommendation out of the tutorial. Do you think it would be worth keeping "try adding a 2nd worker" in the "speeding things up" part of the tutorial", or just leave it out?

I'm trying to pin down where this colour shift is occurring by using a belle-nuit colour chart. Will let you know what I find.

johnmeyer wrote on 3/7/2011, 1:11 PM

Do you think it would be worth keeping "try adding a 2nd worker" in the "speeding things up" part of the tutorial", or just leave it out?I think I would take it out, based on the information in this MeGUI Wiki:

MeGUI/Parallel job execution

You should still recommend that the user set up one worker, although even when I had no workers, MeGUI still worked. AFIK, extra workers don't hurt anything: in my tests, the extra workers just sat there with 0% usage reported. The reason I still recommend taking out steps for adding workers, even though it doesn't hurt anything, is that it is extra steps, and also because people may be confused (and therefore contact you) when they look and find the other workers not doing anything (which reminds me of watching PG&E workers repairing the lines in front of our house: one guy working and the other five watching!!).

NickHope wrote on 3/7/2011, 10:30 PM

I have done some testing and I agree with you John, so I have updated the tutorial to take out the recommendation to add workers. The default number of workers at MeGUI installation is 1, and to get back to that I found that I had to manually edit the C:\Program Files\MeGUI\joblists.xml file. As you say, it doesn't matter if you leave the extra workers there, but I wanted to get back to the default in order to test it.

I also corrected the following part of the script:

# Convert to YV12 so filters will work.

# Use Rec.709 coefficients, keep full range [0,255].

ConvertToYV12(matrix="PC.709")

to this:

# Convert to YV12 so filters will work.

# Use insterlaced layout for conversion.

# Change to "true" to false" for progressive source. 

# Use Rec.709 coefficients, keep full range [0,255].

ConvertToYV12(interlaced=true, matrix="PC.709")

This brings a slight improvement in the final result, and the RGB parade scope shows that the final colours match the originals more precisely.

Incidentally the QTGMC author says he is currently adding support for YUY2. Maybe that will enable us to frameserve in YUY2 and leave the Convert statement out of the script completely. But I don't know if that would improve the result.

johnmeyer wrote on 3/7/2011, 11:51 PM

The default number of workers at MeGUI installation is 1, and to get back to that I found that I had to manually edit the C:\Program Files\MeGUI\joblists.xml file.It took me awhile to figure out how to delete workers, but I did figure it out, and you don't have to edit the XML file. You show the workers which opens up a small dialog for each worker. In each dialog you'll find a setting to close down the worker. This actually removes the worker permanently.

As for the interlaced=true setting in the converttoYUV statement, I included that in the multi-thread version of your script that I posted above, but forgot to call your attention to it at the time. However, you've got it now, which I think will make a small improvement.

I'm still not sure what caused the color shift in the headlights of the car (shown in the samples you gave in your related post). I don't think I saw any color shifts in the samples I created, although I was paying more attention to the levels issues. I guess that's part of what got you started on this quest, and it sounds like you still haven't got it to the point where you know you've got it nailed. I sympathize with the problem, but you've got so many different places that the levels potentially can get changed.

NickHope wrote on 3/8/2011, 12:35 AM

Oh great! I just noticed that "interlaced=true" completely fixes the headlight issue :) ... consider that nailed!

Handbrake is getting that wrong though. One for Musicvid to report to the Handbrake devs, I reckon.

NickHope wrote on 3/8/2011, 1:13 AM

I made a quick and easy png test card to check that YouTube/Flash Player continue to do a [16,235] to [0,255] conversion.

Here it is on YouTube. I've put this in my YouTube favorites and will take a look at it randomly on different computers in the future to check that the levels behaviour doesn't change again (as it did for me a couple of days ago). If anyone at any time sees more than just pure white and pure black in this video, PLEASE let me know here or by leaving a comment on the video itself.

[url=

musicvid10 wrote on 3/8/2011, 8:08 AM

I've also created a couple of new 1920x1080 grayscale tests, available here:
http://www.sonycreativesoftware.com/forums/ShowMessage.asp?MessageID=753678&Replies=0

Here it is on Youtube, confirming (for the time being) that levels are still being mapped out:

musicvid10 wrote on 3/8/2011, 1:11 PM

Well, after burning a goat at the stake (there weren't any virgins available today), I installed the Flash 10.2 plugin for Firefox, and all I can say is the hardware acceleration does nothing for fullscreen stutter on Vimeo. Levels are the same as before. Making goat stew now . . .

NickHope wrote on 3/9/2011, 2:44 AM

Relief about the levels. Shame about the Vimeo stutter. Enjoy your stew!

I wonder if SmugMug video might overcome the problems that each of YouTube, Vimeo and Facebook video have. Worth someone trying.

I think I'm going to change my tutorial to recommend not doing the [0,255] to [16,235] conversion in Vegas but to do it in AviSynth. In my view it makes things a little simpler and more transparent for the editor, plus less chance of leaving the fx on accidentally when you want to make a full range render for another purpose.

This can be done either:

a) Changing "PC.709" to "Rec709" at the start, i.e.

ConvertToYV12(interlaced=true, matrix="Rec709")

or b) Adding a levels filter at the end:

Levels(0, 1, 255, 16, 235, coring=false)

c) which is more-or-less the same as:

ColorYUV(levels="PC->TV")

Does anyone have any opinions on which is preferable, and why this might in any way be worse than doing a C-RGB to S-RGB conversion as a master fx in Vegas as previously recommended?

I guess in theory, options (b) or (c) give the deinterlacer and resizer slightly more colour range to work with, so I'm inclined to recommend one of those.

If I do this then I will put all the stuff about levels onto a different page that is easier for people to link to when they talk about doing the levels conversion prior to rendering with Vegas codecs or by Handbrake etc..

TeetimeNC wrote on 3/9/2011, 7:23 AM

>I wonder if SmugMug video might overcome the problems that each of YouTube, Vimeo and Facebook video have. Worth someone trying.

Nick, I have a SmugMug account and a small amout of time to try something there. What would be the most useful test for me to do?

/jerry

amendegw wrote on 3/9/2011, 7:38 AM

Jerry, Any chance you could plop the .mp4 referenced in the following into SmugMug? http://www.jazzythedog.com/testing/eagles.aspx

btw: the JW Player / Flash test seems to have MUCH LESS stutter than when I originally posted this. Could it be the Hardware Acceleration in Flash 10.2??

Thanks,
...Jerry

NickHope wrote on 3/9/2011, 7:58 AM

Any Toms around to redress the balance?

Jerry1, most useful would be a render of Musicvid's latest test project which he's shortly to release. In fact either he or Jerry2 might already have a Handbrake-rendered 8Mbps mp4 from it that they could link you to???

Just to recap the biggest problems of existing video sharing sites as I see them:

YouTube: Poor quality, especially in transitions
Vimeo: Stutters
Facebook: Cannot embed HD version

amendegw wrote on 3/9/2011, 8:41 AM

Okay, I just made an 8Mbps HandBrake version from musicvid's latest-and-greatest. However, I may not have the most recent HandBrake refinements (although I doubt that the would affect the stuttering). It's here: http://www.jazzythedog.com/testing/DNxHD/DNxHD-HB-8Mbps.zip Caution: It's 172MB.

Good Luck!
...Jerry2

musicvid10 wrote on 3/9/2011, 9:47 AM

Jerry, I downloaded from that link, and it's still version 2, is that right?
The new version 4 is longer and contains the skin tone tests. Also, a watermark at beginning and end that says "Version 4."

amendegw wrote on 3/9/2011, 9:57 AM

My bad! Fat fingers! Fixed the link two posts up. Try it now.

...Jerry

musicvid10 wrote on 3/9/2011, 10:46 AM

Jerry, nice render! Only slight difference from my original settings is you used hex instead of umh, which is not going to affect anyone's viewing experience.

When you have the links on your site updated (I'll email the Youtube and Vimeo links), let me know, and we'll move this branch of the discussion back to the Handbrake thread. I seem to have effectively sabotaged Nick's higher level discussion, despite my better intentions.
;?{

I'll also be checking out the "interlaced" setting in Handbrake per Nick's observations of the effect on motion.

1 … 4 5 6 7

of 7