HDV to Blu-ray: why is it recompressing?

Comments

Kimberly wrote on 10/10/2011, 10:45 PM
I just remembered that both MXF and XDCAM-EX (.mp4) render an HDV timeline at better quality than MainConcept MPEG-2. By that I mean the parts of the timeline that need to recompress (i.e. the transitions, fx, and the parts at the ends of HDV events that are outside the normal half-second GOPs). They'll also smart render the untouched HDV parts. The are basically MPEG-2 streams inside an MXF or MP4 container, but they seem to use a newer, better MPEG-2 codec than the MainConcept one. In addition to that, the XDCAM-EX mp4 render is blisteringly fast.

Indeed XDCAM is fast and the quality is great.

I noticed on the XDCAM "SP" setting I got the no recompresion required message. But on the "HQ" setting I did not see this message and it tool considerably longer. The extra time makes sense in part because the bit rate is higher. But not seeing the no recompression message? Does this mean it IS recompressing on the HQ setting?
PeterDuke wrote on 10/10/2011, 10:51 PM
If I render an HDV .m2t file to an HDV template in Vegas, it produces a .m2t file without recompression. (This file has a menu stream, according to Mediainfo. I don't know if that is significant.) This file needs to be recompressed by DVDA.

If I render an HDV .m2t file to a Blu-ray 25 Mb/s template in Vegas it produces a .m2v file, but is recompressed. This file does not recompress in DVDA.

So you can take your pick whether to recompress in Vegas or in DVDA.

A raw .m2ts file from my camera needs both the video and audio to be compressed.
NickHope wrote on 10/10/2011, 10:52 PM
I've been testing simply by dragging an HDV file to the project overview window on the left, double clicking it, which brings it up in the timeline and preview windows, then clicking "Make Blu-ray Disc" then "Prepare". Then see if you get the recompression warning or not. Really fast to test.
malowz wrote on 10/10/2011, 10:59 PM
DVD Architect 5.2 124b, format: MPEG-2 1440X1080-60i, 6:9. single movie only

captured HDV file direct in DVDA:
"The video 'Clip 011.m2t' on track 2 'Video' in 'Clip 011' will be compressed."

same HDV file in vegas, HDV project, exporting in HDV (no recompress required). put in DVDA:
"The video 'Untitled.m2t' on track 2 'Video' in 'Untitled' will be compressed."

same HDV file in vegas, HDV project, exporting in Blu-ray remplate (the one with "=") = Vegas recompressing.

previous HDV exported from vegas, exporting in Blu-ray remplate (the one with "=") = Vegas recompressing

------------------------------------------------------------------------
now testing my patch:

exporting HDV file from vegas, using my HDV modified template = ("no recompression required" in vegas). open in DVDA:
"The video 'Untitled.mpg' on track 2 'Video' in 'Untitled' will be recompressed."

get same file, APPLY PATCH, test again in DVDA:
NO RECOMPRESS MESSAGE.
Optimize button, Video 1, recompress = NO

anyone got different results?
NickHope wrote on 10/10/2011, 11:27 PM
That's the same as I'm finding Malowz.

Kimberley, the SP setting with smart render because the bitrate matches your HDV bitrate. The HQ setting will recompress because it's a higher bitrate. So use SP, not HQ.

I'm opening up the old thread to discuss this, as I have more findings.
Kimberly wrote on 10/11/2011, 12:41 AM
This is probably addessed in the Malowz fix, but I noticed if you change the Profile Level on the HDV template from 1440 High to High, it recompresses in Vegas but does not recompress in DVD-A. Anyone know what the Profile Levels control?

A side by side on HDV versus BD template:



I've been messing with variations on the setting for each template but cannot figure out what the man behind the curtain is doing on some of them.
PeterDuke wrote on 10/11/2011, 3:15 AM
As I pointed out in my previous post, when you render HDV to the HDV template, Vegas smart renders and puts the video into a .m2t wrapper. When you render HDV to a Blu-ray template, Vegas re-renders it and puts the video in an elementary stream.

I think High with or without 1440 just means High, since we are talking about 1440x1080 at all times.
malowz wrote on 10/11/2011, 3:40 AM
the main difference (but not the only one) between HDV and Blu-ray templates are the Level.

you can export from vegas HDV compatible video as elementary, transport or program. all without recompress.

High Level CAN have higher resolution, but its not required. so, it means, a 1440x1080 video can be High and High 1440, but a 1920x1080 can only be High.

BD specs says video need to be High, EVEN if it is 1440x1080. as HDV is High 1440, some programs may reject. even if the video IS "technically" the same, the information on the video will make programs reject it.

so, you can export a HDV file from vegas, in High 1440 level, without recompress. THEN you patch the file to make it appears to be as High level. so authoring programs can accept it.

Levels are "limitations" to make sure compatibility is maintained, and to avoid "out of specs" video. for example, you cannot make a DVD with mpeg2 compressed with 4:2:2 color sampling. why? cause players would be unable to decode it. so a profile/level is established to make "impossible to decode" videos cannot end on a DVD.

the main problem, is that DVD architect does not "scan" the file to make sure it is compatible, it just read the information on the file itself. if the information says "incompatible", it wont accept it.

DVD Architect is not "wrong" by not accepting HDV files. but it could detect if the file is HDV, and "treat as if" it where a Main Profile/High Level. wont cause any problems, and solve a lot of them ;)

a good analogy is this:
let's say you can carry a 5KG box. and you have a box that it's written 10KG on it. you say: "i can't carry it, its too heavy". i say: "no, wait, the box SAYS 10KG, but it actually weight 5KG" - "sorry, it's written 10kg, i can't carry". then you put a sticker over the 10KG, saying 5KG. "oh, NOW I CAN CARRY IT" ;p

http://en.wikipedia.org/wiki/H.262/MPEG-2_Part_2#Video_profiles_and_levels
Kimberly wrote on 10/11/2011, 8:40 AM
I get it! I finally get it! Thanks a million, Malowz!

And I was wrong in my earlier post. I believed that a template with an ( = ) equals sign automatically meant no recompression required. Now I know that this isn't always true. Perhaps a better description of such templates is that they have been optimized for a particular format, but that doesn't automatically mean no recompression required.

Thanks again.

NickHope wrote on 10/11/2011, 9:13 AM
From the help:

"Templates that match your project settings (frame size, pixel aspect ratio, and frame rate) are displayed with an = symbol in the drop-down list"
Arthur.S wrote on 12/10/2011, 6:18 AM
Just tried this. No option to change stream type to 'program'. The option is greyed out and set to 'transport'. Went ahead anyway, but DVDA wants to recompress. Vegas 11 also wants to recompress it on the timeline. I used the HDV 50i template as a base. (I'm in PAL land) Could that be where it's going wrong? A bit of a strange decision to change the file name to video.mpg - how do you ID which file is which afterwards??
Arthur.S wrote on 12/10/2011, 6:44 AM
Realised I didn't change HDV to MPEG-2. Changed that. Resulting file smart renders in Vegas - good. DVDA still wants to recompress it though - bad. :-(
Arthur.S wrote on 12/10/2011, 7:02 AM
Tried again...and it works! :-) Smart renders in V11 and no recompression in DVDA.
tfc wrote on 12/10/2011, 2:30 PM
Arthur S:

"Tried again...and it works! :-) Smart renders in V11 and no recompression in DVDA."

So, Arthur, what exactly did you do to make sure there was no recompression in either V11 or DVDA?
NickHope wrote on 12/10/2011, 10:58 PM
Lest it get lost, Malowz' post of 10/10/2011 3:20:56 PM holds the magic.

Here is his patch. The download includes instructions in a readme file.
Arthur.S wrote on 12/11/2011, 7:32 AM
I went back over the steps a bit more carefully. :-)
The first rendered file doesn't recompress in V11. Once the patch is run, the resulting file also doesn't recompress in DVDA. However...they are 2 separate files. So, we now have 2 workarounds for DVDA recompressing - neither of them from Sony. Shame on them.
malowz wrote on 12/11/2011, 9:39 AM
2 workarounds?

what is the other one besides patching the file?

cause you said vegas > dvda with no recompress, but in our tests we always get recompress
Arthur.S wrote on 12/11/2011, 10:03 AM
The TMPGEnc Authoring Works 4 method.
malowz wrote on 12/11/2011, 10:18 AM
ahhh, tnks ;)
Arthur.S wrote on 12/11/2011, 1:51 PM
Just 'patched' a 3hr file...worked perfectly. BIG thanks malowz!!
athomasl wrote on 12/28/2011, 12:15 AM
FYI, patch.dll did not work on my computer until I renamed it patch.exe.

IN SUMMARY == THIS ONLY SEEMS TO WORK WITH SHORT VIDEOS!

Over the years I have been archiving hi-def versions of my projects on hard drives. I did this by rendering to HDV 1080-60i .m2t files, which matched what I had been starting with (Sony HDR-HC3 camera). These VIDEO.m2t files are accompanied by VIDEO.m2t.sfl files which contain the chapter markers.

The flow to go from these archived files to Blu-Ray:

1) Create a new Vegas Video project and pull the archived VIDEO.m2t file into it.

2) Render As: Main Concept MPEG-2 type.
a) Start with the HDV 1080-60i template, then click Custom.
b) In Output Type, select MPEG-2, then change bitrate to 25,000,000.
c) In Advanced Video Tab, check Write sequence end code and uncheck Write sequence display extension.
d) In Audio Tab you can leave Include Audio Stream checked.
e) In System make sure Save as Elementary Streams is unchecked and make sure Stream Type is Program.
f) For future use, save the template with a name like "Blu-Ray HDV 1080-60i Patched"
g) Render the output to a new empty directory. The base of the output file should match the name used for the original archived files. The render step will create VIDEO.mpg and VIDEO.mpg.sfl files.

3) a) Delete the VIDEO.mpg.sfl file.
b) Rename VIDEO.mpg to VIDEO.m2t (do not overwrite the original archive).
c) Copy or move the original VIDEO.m2t.sfl file into this directory.

4) a) Edit the patch.cmd file to process the right file.
b) Execute that patch.cmd file.

5) Start new DVD Architect Project, Blu-Ray, 1440x1080-60i, AC3.

6) Drag the new VIDEO.m2t file into the project. Check the markers. (I had trouble loading markers with some sets of my archived files because I renamed files after they were created. Open the .sfl file with Notepad to see what base file name it is expecting and rename files as needed.)

7) When you go to create the disc, it should only report that the audio track will be compressed/converted. I prefer to let DVDA do the AC3 audio conversion vs. creating a separate audio file from Vegas.

8) If you have the old NTSC DVD Architect file, you can start with that instead of step 5. Change the properties of the project to Blu-Ray Disc 1440x1080 29.970i, AC3. Click into your video link and Set the Video and Audio to the new m2t.

=====

One remaining question: Are we guaranteed that the patch only affects the header information and not any video data? I am guessing so because that many zeros in a row might not happen in compressed video, but I'd like to know for sure. Sony Vegas and DVD Architect should be fixed so we don't need to bother with this clumsy workaround!!! >:-(

=====

* THIS PROCEDURE WORKED FOR ME WITH A SHORT (10 MINUTE) VIDEO. IT DID NOT WORK FOR ME WITH LONG (2 HOUR) VIDEOS. The process seems to work fine, but when I burn and play back the Blu-Ray disk, the audio and video get horribly out of sync. When I try to mount and play back the .iso files with PowerDVD, I see the same problem with audio/video sync. I wish I could figure out why, but I can't, so I'm just going to re-render things to AVCHD instead.
malowz wrote on 12/28/2011, 7:25 AM
i tested the patch in win7 x64, worked fine. what error it gives?

the patcher is not guaranteed to patch only header info, but is very very very unlikely to patch something besides the header, as you mentioned, due to zeroes, and i used the "longest" possible string of bytes to help avoid wrong patching.
malowz wrote on 12/30/2011, 9:59 PM
rename the patch.dll to patch.exe.

open the .cmd in notepad and also change the patch,dll there to patch.exe

all systems i tested (i do batch scripts for long years) running a file from commandline execute it even with dll extension. to be visually better and "cleaner" i rename to dll. but its not a requirement, the file is a .exe.
athomasl wrote on 12/31/2011, 12:59 PM
Renaming patch.dll to patch.exe worked.

Although the procedure to avoid video recompression above works, I might re-render to AVC anyway to save me space in my hard-drive archive. Most of my footage is slow-moving. What average AVC bitrate is "visually equivalent" to HDV? Looks like Vegas recommends 8Mbps in the Render-As options for Blu-Ray 1440x1080i, but DVDA defaults to 18Mbps.