OT: Fifth MSU MPEG-4 AVC/H.264 Video Codec Compari

NickHope wrote on 6/26/2009, 9:16 AM
The Russian group "MSU Graphics & Media Lab (Video Group)" conduct an annual comparison of H.264 codecs. This fifth one is quite recent and is worth a quick look for anyone involved with this type of encoding.

http://compression.ru/video/codec_comparison/mpeg-4_avc_h264_2009_en.html

There is a free PDF download which tells most people more than they need to know.

Main Concept and x264 come out top, but bear in mind the MC codec tested is version 8.1, not version 2.0 as included in Vegas.

Comments

musicvid10 wrote on 6/26/2009, 9:43 AM
The report surely demonstrates the direct relationship between encoding time and quality. For instance, Elecard has impressive rendering times, but MC and x264 have the best quality across the board.

I have become a big fan of x264 and use it for a lot of my work. I think we discussed this in a previous thread. Nice to see it right up there with MC in quantitative tests.

Although I didn't understand a lot of the deep technical stuff, it is obvious these testers are very thorough. Interesting that the free version of the report was "only" eighty pages . . .
NickHope wrote on 6/26/2009, 10:09 AM
Hey musicvid, I checked out Handbrake but found I couldn't frameserve to it.

Soldiering on with MeGUI here. I really should write a tutorial for my method as it's a bit tricky. It does an auto-update of all its codecs each time at start-up and I seem to get offered a new x264 build every few days. I always just click to accept it. Probably a slightly dodgy thing to do but so far so good.

The only slight problem I notice is a slight reduction in contrast / increase in luminosity. Others have blamed that on the playback codec. It's not so bad in the Flash web player, which is what I'm delivering with, but it's definitely not quite as high in contrast as the original footage. Have you noticed this?
musicvid10 wrote on 6/26/2009, 10:30 AM
"Hey musicvid, I checked out Handbrake but found I couldn't frameserve to it."
That's a bummer. It would be a huge timesaver to be able to do this, esp. with such slow native rendering times in x264.

"The only slight problem I notice is a slight reduction in contrast / increase in luminosity. . . Have you noticed this? "
Yes I have, but had passed it off as being a byproduct of my rendering to half the original bitrate of source MPEG-2s from DVDs I have produced, which I often do to save space. Haven't really tried it much in a HD-critical workspace.

I really am going to take time to learn the ins and outs of MeGui. Possibly after my current show closes in August.
musicvid10 wrote on 6/26/2009, 10:49 AM
By the way, for those interested, there is now an easy way to export Vegas chapter markers direct to .m4v encoded in Vegas, Handbrake and probably MeGui and others.

After working with this problem, I contacted the author of Drax and he incorporated support for Vegas markers into his excellent and free chapter-import utility.

http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=655182
NickHope wrote on 6/26/2009, 11:12 AM
Off the top of your head, do you know if those chapters have any relevance at all to MPEG4 files served on the web in the Flash Player, or are they purely an iPod thing?

Send me an email when you're going to get back into MeGUI. I might even have a tutorial up by then. If not, I can certainly give you some pointers.
musicvid10 wrote on 6/26/2009, 11:24 AM
afaik, they are just for the Quicktime variety .m4v files. Since the .m4v chapters are just text, it would probably take a bit of script (not much) to convert them to cue points in Flash. Maybe somebody's already done this. (Edit: Looks like it's been done for DVD chapters, what Nick asked can't be too far down the road).
musicvid10 wrote on 6/26/2009, 7:42 PM
"I checked out Handbrake but found I couldn't frameserve to it."

John A. Stebbins, Handbrake Developer, provided this explanation:

One of handbrakes strengths is its ability to analyse the source and automatically choose many settings for the user or provide information that helps the user make better choices. Unfortunately, this analysis pass means handbrake can not take frame by frame input. It requires the entire sequence. Adding frame by frame capability would be non-trivial.
NickHope wrote on 6/26/2009, 10:42 PM
Ah, that makes sense, although, if you're processing lots of videos with similar characteristics, and you already know what settings you want, it would be nice to be able to frameserve.

I shall open up the chapters-in-MPEG4 can of worms next and let you know what I find out. Would be really useful for some long videos I'm planning on uploading.
musicvid10 wrote on 6/26/2009, 10:53 PM
Thanks.

Can you post some really basic instructions for frameserving from Vegas to x264 in either MeGui or Avisynth, whichever is easier? I assume they are similar.
NickHope wrote on 6/27/2009, 5:11 AM
AviSynth is an intergral part of the MeGUI / x264 process.

Install MeGUI with all the presets and take all the updates.

Everything you need comes with MeGUI except the Nero AAC codec (which is a separate free download from Nero site. Google it).

Using a text editor, make an Avisynth script something like this:


AviSource("d:\fs.avi")
ConvertToYV12()
TDeint(order=1)
LanczosResize(640,480)


Line 1 picks up the name of your frameserved file.
Line 2 converts to the colorspace required by x264.
Line 3 deinterlaces with a good deinterlacer plugin. Others are available. This assumes your video is interlaced of course.
Line 4 does a nice resize. Change the resolution to whatever you want.

Call your AVS script something like 640x480-TDeint.avs . I save mine in the same folder as rendered videos. I have a bunch of different AVS scripts for different resolutions in different folders.

Frameserve a AVI out of Vegas at the original size of the video. See the first part of my Vegas to YouTube tutorial for how to do that. Remember best settings etc.. Do not deinterlace in Vegas.

Open MeGUI with the input tab at the front.

Video AviSynth Script: Full path and filename to your .avs file
Video Output: Choose a path and filename (.mp4)
Encoder settings: Choose your preset (see below)
File format: MP4
Audio Input: Full path and filename to your .avs file again
Cuts: Nothing in here
Audio output: Same path and filename as Video Output
Encoder settings: I'm using Nero AAC:NDAAC-HE.64kbps. Very efficient codec. No need to go over 64K for online use.
Extension: MP4-AAC

Then click AutoEncode.

On the "MeGUI - Automatic Encoding" window select "No Target Size" then click "Queue".

Then change to the "Queue" tab in the main MeGUI window and you will see four jobs queued up (audio encode, video first pass, video second pass, and mux).

At this point you might need to create a new "worker" if you don't already have one. I can't remember. Look under the "Workers" menu and if you have one then you should be good to go. Click "Start" and the whole process begins.

When it's done I remove the "-muxed" suffix from the filename and tag the video with copyright information etc. using Mp3Tag which also supports MP4. Then I upload and embed on my site using the Longtail Flash Player, or upload to YouTube if it's for them.

Finally I clear the finished jobs out of the Queue window.

If I've missed anything, please let me know so I can put it in a full guide later.

Good luck!

PRESETS:
The preset I am using is DXVA-HD-HQ (720p output) or DXVA-SD-HQ (480p output). The only thing I change is the bitrate. 2500 Kbps for 720p videos destined for Flash (note that Facebook also encode 720p to 2500 Kbps. YouTube encode to 2000 Kbps, so 2500 Kbps is about on the mark). I keep the 1000 Kbps rate for my 640x480 vids for Flash. I double all my rates for YouTube upload to give them some overhead when they reencode.

DXVA-HD-HQ is fine for videos served in Flash. If you're looking for offline playback compatibility with a range of players and software then you'll have to choose a "lesser" preset. Compatibility is an absolute minefield. I nearly drove myself mad testing this last year and ended up concluding that you need to choose the Quicktime for Windows preset or similar. It's not something I'm doing anyway now as I want people to view my videos online where I can earn a bit of advertising cash.
NickHope wrote on 9/4/2009, 9:00 AM
I have finally, after much digging, worked out what has been causing the loss of contrast/luminosity when I encode H.264 using x264 in MeGui.

The RGB to YV12 conversion using "ConvertToYV12()" in the AviSynth script actually scales the luma to the TV range [16,235], cutting off the blacks and whites. To maintain the full range [0,255] luma you need to add matrix="pc.601". My AviSynth script typically now looks like this:

AviSource("d:\fs.avi")
ConvertToYV12(matrix="PC.601")
TDeint(order=1)
LanczosResize(640,480)

The result is much better (and way better per bit than anything Vegas can do natively).

Apparently PC.709 can also be used. I'm out of my depth with the specifics here so if anyone can shed light on the differences between Rec601, Rec604, Rec709 etc. then please do.

More at http://avisynth.org/mediawiki/ConvertToYV12.
john-beale wrote on 9/4/2009, 9:41 PM
I remember using the Debugmode frameserver with Vegas 8 (or was it 7?) and it worked fine. I haven't used it for a while. I have a new PC now and I installed Vegas 9b on it. It did not have any previous version of the frameserver software on it, but I just installed the current version 2.6 (claimed to work with Vegas 9) from Satish's webpage http://www.debugmode.com/frameserver/

Ok, the Frameserver option shows up in Vegas under uncompressed AVI and I can save, but it doesn't work the way I remember:

1) When frameserving, Vegas used to render a few frames to start, and then pause and wait for the other application to start requesting frames. Now it renders the entire video, then stops. Also I remember there was a window that gave me some frameserver options; that window did not appear.

2) I can't open the fs.avi file in VirtualDub or any other application. VD reports that the file is being used by another process. After Vegas finishes, I can open the file with VD, but all frames are black. Clearly, something is broken.
john-beale wrote on 9/4/2009, 10:14 PM
I solved my program. Turns out the Debugmode installer had a bug :-)

In the directory C:\Program Files\Sony\Vegas Pro 9.0 there is a file called
"Release.fio2007-config" which is a plain text file listing the file-i/o plugins and their locations. The debugmode line should be added to this, but instead the installer created a new file named "Frameserver.x86.fio2007-config" containing the text:

[FileIO Plug-Ins]
frameserver=C:\Program Files\DebugMode\FrameServer\dfscVegasV2Out.dll

What I did was copy the second line ("frameserver=C:\...") into the "Release.fio2007-config" file under the other items in the [FileIO Plug-Ins] section. Now Vegas shows the Frameserver as an option, and it actually works as a frameserver. Before, it had appeared only within Avi:Uncompressed as a sub-type of uncompressed.

...then to get MeGUI working (specifically the audio encoding), it turns out I needed to download neroAacEnc.exe from Nero.com

...Zounds! The mp4 video quality from MeGUI is significantly better than the Vegas built-in MainConcept mp4. As a bonus, it hasn't yet crashed due to memory leak either.
musicvid10 wrote on 9/5/2009, 7:33 AM
Thanks Nick for tracking down that little booger. I'm going to see if I can use it with x264 in Handbrake through the cli terminal (no, I still haven't "graduated" yet) . . .

Also, jbeale, I think you should post your Frameserver trick in a new thread where it won't be hidden -- I am sure there are several people who will be interested, including Satish.
NickHope wrote on 9/5/2009, 11:01 PM
OK, a little more clarification on this luminance fix...

Apparently if the width is 1280 (e.g. 720p with square pixels) or more we should use PC.709. Otherwise PC.601. Haven't tested it yet though.

http://forum.doom9.org/showthread.php?p=1321983#post1321983
Sebaz wrote on 9/8/2009, 6:20 PM
Thanks to Nick for the tutorial on MeGUI usage, I learned a lot from it.

However, even though I tried all day to get a DVD Architect compliant 1080i AVC file, I can't do it. The MeGui presets for either AVCHD and blu-ray don't give me a file that DVDA will author without recompressing, even though I managed to get one that Encore CS4 will accept after playing with the advanced settings. But that same file is not accepted by DVDA without reencoding.

Has anybody achieved this, frameserving from Vegas 8 to MeGUI and producing a file that DVDA 5.0b will take without recompressing?
john-beale wrote on 9/8/2009, 11:26 PM
I'm using Nick Hope's Vegas->Frameserver->Avisynth-> Megui process and the output looks fine. However I am curious; on my quad core I see that no processor is working above 30% while rendering, and overall I am encoding only about 7 fps. When encoding in Vegas without frameserving, it goes 2x or 3x faster.

Is the bottleneck here something about the frameserving process, I assume? Is there any way to make this faster? It's not really physically writing all those uncompressed frames to disk and reading it back, is it? I thought that was just a "virtual" file and the inter-process communication was all on some internal bus that should be very fast. (?)
NickHope wrote on 9/9/2009, 4:13 AM
@ Sebaz: When you run into AVC compatibility problems it seems to be usually down to software not supporting certain AVC levels/profiles or advanced stuff like B-pyramid. Quicktime, especially on Windows, is an absolute nightmare in this respect. So try the "x264 unrestricted 2-pass Quicktime for Windows" template from the MeGui encoder settings and see if that works. It dumbs down some of the advanced settings but the quality per bit is still great. That's the template I'm using for stock footage previews where I expect viewers might want to download clips for viewing offline (where Quicktime is highly likely to get used as the player) as opposed to online in Flash (which supports lots of AVC features).

Please let me know what happens so I can include it in a better online guide at some point.

@jbeale: Do some research into the "Workers" pulldown. It might be that "Create new worker" will help.

I'm no expert on this. I've just found a method that works for me. I'm sure there is more that can be done such as setting up one-click encoding in MeGui. For more advanced help visit the the doom9.org forums where the developers of both x264 and MeGui hang out.
musicvid10 wrote on 9/10/2009, 8:39 PM
Apparently if the width is 1280 (e.g. 720p with square pixels) or more we should use PC.709. Otherwise PC.601. Haven't tested it yet though.

I double-checked HandBrake's instructions to x264 and it's already doing just that.
Just satisfying my curiosity.