Incidentally, at the start of my tutorial, how should I recommend to people to conform the project to 16-235, as a general guide? Do it on each event, or run a filter on the whole project, or both depending on the project? What do you guys think?

As Bob and I discussed some time back, a stock Studio RGB filter, is a one-size-fits-all, safe, WYSIWYG solution. It does not optimize levels for a particular source, only ensures that what comes back is what was sent.

The intricacies of optimal leveling and gamma adjustment of different delivery content is an advanced art, and probably will only be mentioned in my Handbrake tutorial, not illustrated. Since your MeGUI tut will be more technical than mine, you may want to delve into it a bit deeper.

Test 18 was supposed to be my pièce de résistance but it's stuttering quite badly here in Firefox, even after fully buffering. How does it play for anyone else?

Nick, Test 18 looks very good to me. I see a small amount of stuttering in both Firefox & IE9, but it appears less than the "normal" web delivered stutter - certainly no worse and maybe better than any of the other attempts.

btw, it's one of my pet peeves that videos can play "smooth as glass" locally via WMP or VLC, but the same clips delivered via the web will show stuttering, even with no dropped frames. Maybe someday this will be fixed.

...Jerry

No differences in playback with any of the other versions including mine.
Plays smooth in the small player. I always pause and let the video load completely from the 'net before starting playback. Also, turning off Windows Sidebar and Vista screen enhancements helps.

"1 hour 8 mins on my core 2 duo T7800 @ 2600 MHz with 4GB RAM"

The combined render times (I420 + Handbrake) on my T2390 laptop (1.85 GHz) are 41:30. Even adding in a couple of minutes to set it up in Handbrake, Frameserve->MeGUI with your settings is rendering ~65% longer. As I mentioned, the sharpness using your method is better than Handbrake's.

Lacking any huge differences in our settings and bitrate, I assume it's the deinterlacer eating up the time. May explain why decomb development bogged down at Handbrake . . .

>> btw, it's one of my pet peeves that videos can play "smooth as glass" locally via WMP or VLC, but the same clips delivered via the web will show stuttering, even with no dropped frames. Maybe someday this will be fixed. <<

This is where YouTube have an advantage over Vimeo. For all their butchering of the video quality, YouTube videos never stutter (for me), whereas Vimeo ones often do. In my tests the downloaded stream from Vimeo is only 2022 Kbps, compared to 3061 Kbps from YouTube, so something fancy in the encoding that allows them to get better quality at a lower bitrate (e.g. Weighted prediction), is overly challenging the browser's Flash Player on some systems.

Mark, over here MeGUI with QTGMC with "faster" presets is taking twice as long as I420+Handbrake...

I420 + Handbrake = 20 + 14 = 34 mins
QTGMC faster = 68 mins

But there are faster QTGMC presets taking us all the way down to 37 mins for "ultra-fast". I figured that if someone is bothered enough about the quality to go through all this, then they're probably patient enough to wait a bit longer.

Here are some benchmarking notes and results. The main ones we're interested in are highlighted in yellow:

Nick, can you tell me more about QTGMC?
I found out it's an Avisynth script, but little about what's under the hood.
Does it use a combination of tritical's deinterlacers, or is it its own unique code?
Is it something that could be emulated by setting custom decomb numbers in Handbrake? Is it better than TDEint / EED12 / NNED13 / whatever, or just more convenient?

Release Candidate for Handbrake Tutorial

As Nick pointed out, it is always good to have other eyes on these tests, lest we be lulled into self-delusion:

-- Tweaked blacks and colors slightly for uniformity.
-- Added skin tone test and color chart.
-- Added end credits
-- Used Helix I420 as the intermediate
-- Original HB settings same as before

[EDIT] Helix wants to default Handbrake to Blend fields, so use this custom preset to decomb. 5:2:6:9:80:16:16:10:20:20:4:2:50:24:1:-1

I learned one thing this week -- color correcting HDV seems to be much more difficult than AVCHD; don't know what the reason for this is.

I will zip and upload the new project files to Jerry's server and begin working on the "real" Handbrake tutorial vid once everyone has checked in on this version. I'm feeling ready to put this project to bed, but still want to continue learning from Nick's branch of the project.

http://vimeo.com/20190131

>> color correcting HDV seems to be much more difficult than AVCHD; don't know what the reason for this is. <<

Possibly differing behaviour caused by the different decoders used in Vegas 8.0? You may see more uniform behaviour correcting AVCHD and HDV in 10.0, where they are both decoded by compoundplug.dll.

>> vimeo.com/20190131 <<

I'm seeing loads of motion blur in this one compared to your first one, in both the uploaded and downloaded file, when viewing stills on the timeline, almost as if it wasn't quantized to frames when you rendered it. Is it deliberate?




Been busy for a couple of days but will come back with an attempted explanation of QTGMC, later today hopefully.

No, that wasn't intended. I'll look at the renders and the new project tomorrow.

Good catch!

>> can you tell me more about QTGMC? <<

Deep breath... I'll do my best.

First, if anyone needs a deinterlacing primer, look here (somewhat outdated but still good), here (by Donald Graft, author of Smart Deinterlacer), here, and here.

From those it's worth remembering that "weave" means do no deinterlacing, and "bob" means separate the fields, restore the height by doubling it, and display one field after the other (giving double framerate of the source).

QTGMC is something I stumbled upon in the Doom9 forum. It's not an AviSynth plug-in filter as such but rather a script that uses a combination of others' filters to make a very nice job of deinterlacing. Before talking about QTGMC, I'll backtrack through some of the things I tried before it. A lot of the following is directly quoted from authors' descriptions, with a few explanations added by me.

Motion-Adaptive Deinterlacers

These analyse motion in the video and only deinterlace where necessary. They weave (= do nothing) in areas of no motion and interpolate in areas of motion.

TDeint
TDeint is a bi-directionally, motion adaptive, sharp deinterlacer by Tritical. It can adaptively choose between using per-field and per-pixel motion adaptivity, and can use cubic interpolation, kernel interpolation (with temporal direction switching), or one of two forms of modified ELA (Edge-Based Line Averaging) interpolation which help to reduce "jaggy" edges in moving areas where interpolation must be used.

Yadif
"Yet Another DeInterlacing Filter" ported from MPlayer. It checks pixels of previous, current and next frames to re-create the missed field by some local adaptive method (edge-directed interpolation) and uses spatial check to prevent most artifacts. Handbrake uses Yadif, amongst other things (Edit: ...and the author explains its operation here).

When I tried either TDeint or Yadif alone, at default settings, the result was pretty good but some jaggies were left. They were relatively fast (see table above in my previous post). Yadif was slightly faster than TDeint. TDeint alone (Test 1) was my method of choice (out of ignorance of better methods) before this project started.

Smart Deinterlacer
Donald Graft's VirtualDub filter provides a smart, motion-based deinterlacing capability. In static picture areas, interlacing artifacts do not appear, so data from both fields is used to provide full detail. In moving areas, deinterlacing is performed. Mike Crash's Vegas filter is a port of an early version of this.

Edge-Directed Interpolaters

Well, I'm not even going to try and explain what Edge-Directed Interpolation (EDI) does. I scraped an E grade in A-level maths. Happy Googling! Likewise for neural networks.

EEDI
"Enhanced (I suppose) Edge-Directed Interpolation" by Tritical. EEDI2 resizes an image by 2x in the vertical direction by copying the current image to every other line in the resized image and then interpolating the missing field. It is intended for edge-directed interpolation for deinterlacing. EEDI3 works by minimizing a cost functional involving every pixel in a scan line. It is slow.

NNEDI
"Neural Network Edge Directed Interpolation". NNEDI is also by Tritical and generally more recent than EEDI. nnedi is an intra-field only deinterlacer. It takes in a frame, throws away one field, and then interpolates the missing pixels using only information from the kept field. It has same rate and double rate modes. Tritical says he personally sees nnedi2 as completely superseded by nnedi3.

So, EEDI uses an edge mask, while NNEDI uses a neural network. EEDI smoothes edges (and the whole image) more, NNEDI is sharper and thus enhances detail (and noise) more. I tried them both. The result was pretty good. EEDI gives an extremely smooth result which I suppose would make it good for cartoons.

Smart Deinterlacer
Version 2.8 beta 1 beta adds DCDi-like edge-directed interpolation. It can significantly reduce stairstepping ("jaggies") in deinterlaced picture areas. I've read on doom9 that it's not as sophisticated as Tritical's EDI filters.

Edge-Directed Interpolater + Motion-Adaptive Deinterlacer

Tritical allows TDeint to take the weave/interpolate reasoning from EEDI or NNEDI as opposed to itself. He also modified Yadif to do the same thing, and called it Yadifmod. NNEDI3+TDeint used in this way gives a fantastic result (Test 6), and was my preferred choice before I discovered QTGMC, and may still be useful for some jobs.

QTGMC

To quote the first line of the script itself, QTGMC is a Deinterlacer using motion-compensated temporal binomial smoothing. QTGMC is written by Vit and is based on an earlier script, TempGaussMC ("TMGC") by Didée. Didée is a Doom9 stalwart and wrote MCBob which is a highly regarded deinterlacing filter. He says, regarding TGMC with most-fast and most-basic settings, versus YadifMod with NNEDI2 interpolation, "TGMC was much faster, still it let Yadifmod/NNEDI2 stand with egg in their face."

Under the hood, QTGMC uses and requires these AviSynth plugin filters:

MVTools2
MVTools is collection of functions for estimation and compensation of objects motion in video clips. Motion compensation may be used for strong temporal denoising, advanced framerate conversions, image restoration and other tasks. This is the same plugin John Meyer uses for noise reduction and slo-mo, so it must be good ;)

RemoveGrain + Repair
RemoveGrain is a purely spatial denoiser. However, in combination with Clense and the Repair plugin more powerful spatio-temporal denoisers and Deinterlacers can be built.

MaskTools v2
MaskTools is a set of AviSynth filters filters designed to create, manipulate and use masks.

NNEDI3
See above.

The core algorithm of QTGMC is this:

The "Preset" used selects sensible settings for a given encoding speed. I drew up a reference table of which settings change according to the preset, with "faster" as the benchmark:



The QTGMC3.11.avsi script itself is very well commented, so much of the explanation and options are covered in there, so I won't repeat them. I haven't really experimented beyond the presets, but I guess it might be possible to achieve a lot of what QTGMC is doing by using custom settings in Handbrake's "Video Filters" fields. But that's not something I'm personally planning to pursue.

It may well be that a major reason for any preference for the output of QTGMC over, for example, TDeint+NNEDI3 or Handbrake's default decomb is simply the sharpening. So it might be worth some experimentation in Handbrake settings to enhance that.

In my tests using single-threaded 32-bit versions of programs, my QTGMC/MeGUI workflow can only match the I420/Handbrake workflow at the "ultra fast" setting. At "faster" setting it takes twice as long. However, enabling some of the programs/plugins in multi-threaded or 64-bit mode should allow slower/higher quality presets to match the I420/Handbrake speed. When I have my quadcore desktop machine back in action I will test that.

Note that retaining all the frames (60p/50p) of the QTGMC output by commenting out the "SelectEven()" line in the AviSynth script gives a stunningly life-like result. I was amazed when I tried that on Stringer's driving clip. Great for offline playback and also possibly, as bandwidths improve, for self-hosted videos where you have control of the framerate. I'm assuming YouTube/Vimeo etc. all limit the framerate to 30p, but I would definitely consider QTGMC 50p/60p if I was using JWPlayer etc., especially at lower resolutions.

One more thing about QTGMC. The author says: "I would advise against source-match/lossless on 1080i material. It will make a difference, but a very small difference at that resolution.

I hope all that helps! Tutorial soon...

I just noticed an alternative GUI for Handbrake. VidCoder. Discussed here. It's in active development. Developer just released 0.8.2 today. Haven't tried but it seems to be well-liked. Requires .NET Framework 4. Might be worth a look.

WRT the motion blur,

Well, this is interesting. I took the original project and rendered just Stringer's clip to DNxHD and I420, both at project settings, 1080 29.97i. Put both intermediates back on the timeline, and they are indistinguishable both at the frame and field level, as I had expected.

Rendered both clips in Handbrake, using the identical original settings, and they came out the way you illustrated. Seems Handbrake is choosing "Blend" for the I420 clip and a fancier option for the DNxHD. Absolutely no idea why, but I will post the render logs on the HB forum and see if someone knows.

In the meantime, I may have to retract my accolades for I420 as an intermediate to Handbrake, not because there's anything wrong with it, but if I wanted to Blend fields, I could do that just as easily in Vegas.

https://forum.handbrake.fr/viewtopic.php?f=11&t=20249&p=93427#p93427

Actually, I had looked at the lane line in my tests, and noticed that it was smoother with I420, and immediately thought that was a good thing. It goes to show how easy it is to overlook stuff when I get tunnel vision over one particular aspect of the project.

Thanks again, Nick. Looks like our good-natured critiques are keeping us both alert.

UPDATE:
Well, using "5" as the first number in the HB decomb string fixed it, taking "Blend" out of the soup.

5:2:6:9:80:16:16:10:20:20:4:2:50:24:1:-1


Looks like I've got a lot of posts to update, here and at Handbrake. But I'll wait until I see what their devs say about the cause, before editing all of my posts that mention I420.

Latest upload is fixed. Because of the deinterlace problems, I may just leave I420 out of the tutorial and just include DNxHD.

http://vimeo.com/20190131

OK guys as usual something went right over my head!~

can you please tell me generally what the best codec/setting to use for:

Vegas SD 16:9 destined for playback on my own website, not youtube (must play on iOS and web)

@ JHendrix - I'm not familiar with what will play on iOS but generally I would say AVC (which is the same as H.264). If you have Vegas Pro 10.0 then you can do a pretty good (and fast) job using the Sony AVC codec. Go with a 360x640 or 432x768 resolution (with square pixels), or choose another resolution from here. Base your settings on one of the "internet" templates. Here's the "Internet 640x360-30p" template from 10.0.



If your footage is interlaced then set deinterlace method as either interpolate or blend in your project properties. Render a bit with moving footage and see which result you prefer.

That's the easy option. If you want to keep more quality, especially if your footage is interlaced, then you can use one of the more complicated workflows that we've been developing in this thread, involving better deinterlacing/resizing and the superior x264 codec. I'm hoping to publish my tutorial for a MeGUI-based workflow in the next couple of days. Musicvid is also working on a tutorial for a Handbrake-based workflow. The argument for using one of these fancy workflows is stronger if you have an older version of Vegas Pro such as 8.0, which had inferior Sony AVC and MainConcept AVC codecs. I do not know if they were improved at 9.0 or 10.0 as I never bought 9.0.

In any case don't forget to conform your levels to Studio RGB before you upload (which can be done by adding a Sony Levels filter to the video output and choosing the "Computer RGB to Studio RGB" preset).

JHendrix says: "Vegas SD 16:9 destined for playback on my own website,"

Nick Hope says: "If you have Vegas Pro 10.0 then you can do a pretty good (and fast) job using the Sony AVC codec."First my caveat: This thread has gone places where no man has gone before, and Nick Hope has forgotten more about this subject than I will ever know.

That said, here's my experience. If someone wants to play video from their own website, I've found that rendering to an intermediate (DNxHD or Sony MXF) and then using HandBrake to Deinterlace / Resize the video does a far superior job than rendering directly from Vegas - at bitrates needed to adequately avoid continuous buffering.

Using DNxHD->Handbrake, I've found that I can get an adequate quality 1280x720 resolution with 1.2Mbps which seem to avoid buffering fairly well on today's high bandwidth connections (us musicvid's settings except VBR 2 pass - 1200kbps). Here's the comparisons.

First the Vegas->Sony AVC 1280x720 @ 1142Kbps:


Next the Vegas->DNxHD->HandBrake @ 1200Kpbs:


You can see further testing here: http://www.jazzythedog.com/testing/DNxHD/mediaplayers.aspx

Good Luck!
...Jerry

Edit: Just noticed an abnormality. MediaInfo reports 1Mbps for the video produced using the Sony AVC encoder, even though the I set it for 1.2Mbps. Trying a render at 1.5Mbps. Strange.
Edit2: Okay, I rendered at 1.5Mbps and MediaInfo now reports 1142Kbps - close enough. Replaced the screen print above and the test clip in the above link.

I hadn't realised how big the difference is at that low bitrate. I was still thinking in terms of higher bitrates for upload to YouTube/Vimeo.

>> MediaInfo reports 1Mbps for the video produced using the Sony AVC encoder, even though the I set it for 1.2Mbps <<

This will be the same glitch with the Sony AVC encoder that delivered 8Mbps when I set it for 11Mbps.

"Vegas SD 16:9 destined for playback on my own website, not youtube"

EDIT: I just noticed that you said SD. I'll leave my reply (for HD) below intact, but you should be doing 853x480 (1.0 PAR NTSC) at around 1Mbs instead.

So as Jerry has abundantly researched, compression and file size are key to hosting your own video. 1280x720 is a nice size, agreed. DNxHD to Handbrake is a good route, agreed.

As far as bitrate for delivery, I would tend to follow the lead of Vimeo and Youtube, and encode around 2Mbs. This helps the blocking and motion detail more than one would expect.

I came up with my own advanced settings for Handbrake that maximizes compression efficiency at such low bitrates, whether you choose 1, 1.5, or 2 Mbs as your target. The other advantage to Handbrake is you can set the streaming flag in the front panel, rather than having to do it in a separate step as with the encoders in Vegas.

As mentioned in my reply to Ogul above, these settings very closely duplicate Vimeo quality at around 2Mbs. A CRF of 28 is used as a starting point, but YMMWV. You could just as easily use 2 pass VBR and choose your target bitrate or file size in Handbrake.

"I just noticed that you said SD"Whoa! I, also, have been so focused on HD, that I overlooked that as well. As such, 1280x720 surely is overkill. 853x480 would be a good choice (or maybe 640x360) and experiment with the bitrate (start with 1Mbps) to get the lowest bitrate you can with acceptable quality. High motion footage will probably require higher bitrate, whereas if you just have a talking head with no transitions, you can probably "go low". Don't forget to check the "Web Optimized" checkbox.

Good Luck!
...Jerry

Jerry, Nick,
Are you both OK with most recent version on Vimeo (untiled4.veg)?
http://vimeo.com/20190131
If so, I will zip it, upload to Jerry, and put the Handbrake tutorial together over spring break.