Hi Nick

It's really late for me and Im tired so I hope this message makes sense...lol

Alright...results from the test. I rendered out a bunch of files and here is what I came up with:

These are for 720p renders...the XDCAM EX render settting was adjusted to 18.3 Mbps


for individual clips on timeline:

"original" = clip directly from camera

MC render (default settings) of original: first frame recompressed with noticeable difference (expected).

EX render of original: first frame did not recompress and no noticeable difference to the original...so every frame smart rendered out.

MC render of original with "Length of GOPs" set to "none short": first frame recompressed and that frame looks MUCH closer to original.

The ex renders look exactly like the original clips...no difference at all. the recompressed MC rendered frames look like they lost some quality...especially in the blacks....as if a purpleish tint was taken out. it also looks a tad blockier and a tad less detail.

Now compatibility. All files played back fine with VLC media player (love that program..lol). However...the EX file in the .mp4 container did not play back in Windows 7's media player...but the .m2t file played back fine. bummer.

Since the .mp4 will not playback in Windows media player...I thought about something else: having several .m2t files in the timeline with straight cuts and smart rendering them out as one file. If I rendered with MC...the first/last frame/few frames of each cut will be recompressed and degraded like I was saying in the original message. So I thought I'll render out with EX as one file. Since the .mp4 isnt very compatible in playback, I'll reimport that same file into Vegas and then smart render out as .m2t. It'll be better to have the first few frames of that whole clip recompressed instead of the first frames of each cut. That failed with one set I did it with. It created white boxes in ccertain areas on some clips near the cuts. So I tried it with another set of .m2t clips and it worked fine. Strange. So the idea with that is edit normally and render out with xdcam ex. Then reimport that rendered .mp4 file and then render that out in MC .m2t.

Ok, my head is spinning now and getting really tired...lol. You are right about the full GOP and first and last I-frame. I have tried moving the slider in HDV setting to the right...same image...or nothing noticeable. Oh...and yes I have project settings at best render and 32-bit (video). At this point I agree that it isnt worthwhile to convert them to xdcam-ex. I am only storing these individual clips to archive. If I edit with them again (maybe for a reel or something)...I will not be using the first and last frames of those clips anyway. I didnt notice any problems with editing the mp4 in the timeline..although I didnt do anything complicated..just basic cuts and trimming. The mp4 clips playback fine from timeline. The other reason it doesnt make sense to have them stored as xdcam ex is the files wont playback in windows media player. So if I render out a project in that format for someone, they wont be able to watch the video (unless they have VLC).

Its just strange how MC's mpeg2 codec loses a hint of color when rendering while Sony's xdcam ex does not when rendering the same file...even when the file format is something else. EX ends up looking like the original. I will experiment later on with rendering out the timeline in xdcam ex and re-rendering that clip out to MC m2t again. I think there is something there.

The other thing I noticed is that if I smart render just one m2t clip...it only re-renders the first frame. If I have mulitple clips rendering out as one, then it may be more than one clip and can be in the front or end of the clip.

I just realized I basically just kept on writing..haha.

I did some testing with 1080-60i HDV and pixel format 8-bit (I've never used 32-bit for anything) and my findings are largely similar to yours. By smart rendering to XDCAM-EX, you have definitely hit on a better way of preserving the quality of the very ends of the clips for archiving, even if you then render back to HDV. Thanks for the idea! I may well use it myself for future archiving.

Likewise, my rendered XDCAM-EX clips will play in VLC but not in Windows Media Player or GOM Player.

I'm a little confused in what you did with the GOPs. "None Short" is the default for HDV. I ticked the box "Use closed GOPs" to make sure and indeed it made no difference to the render. So are you saying that the previous testing you did was with "First short" or "All short" set?

I'm also wondering why you would line a lot of clips up and render them together as you describe. As the XDCAM-EX codec is smart rendering every frame anyway right from the beginning to end, then I don't see that you are gaining anything. Have I missed something here? One thing you could do is separately render each clip to XDCAM-EX and then batch render the lot back to HDV using something like the free proxy stream script. In theory (I think) you'll then get a lossless render from start to finish.

I've never liked the MainConcept MPEG-2 codec. I never used it for standard definition DVDs because in my opinion the quality was horrible compared to others such as Cinemacraft or TMPGEnc or Procoder. Back at Vegas Pro 7.0, before HDV smart rendering was introduced, I used to archive my HDV using Womble Mpeg Video wizard which smart-rendered to MPEG program stream files. I later rendered all of them again to HDV-compliant transport stream using Vegas because Vegas was limited in the number of program stream MPEG files that could be put on the timeline.

@ Nick
The thought occurred to me that many of your original clips might not have usable sound. If you like the fidelity of the XDCAM-EX clips rendered from Vegas using the Sony codec but find the .mp4 wrapper distasteful because of player incompatibility, you can rewrap them with tsMuxer to ts “with no loss”. I’ve found that tsMuxer does not recognize the sound in the Sony MP4 wrapper but seems to handle the video easily. Having the wrong wrapper is not the end of the world if it can be changed easily. Even if you need audio you could render it separately then add it when you remux.
While you’re doing mental gymnastics, you could try that one.

Thanks for the suggestion John. I'll check out tsMuxer next time I'm archiving. I normally do try to keep the sound on underwater stuff but you're right, it's often not worth keeping.

Let's bear in mind that we're only talking about a quality difference at the very most during half a second at the start and end of clips, so in most cases it's just easier to make sure the clip has plenty of head and tail. But this technique is definitely worth remembering for those clips where the action is right at the start of the clip.

Wow! I started a thread about trying to do this a long time ago but was unsuccessful in getting the mpeg 2 video to smart-render into a .mxf container. I just tried it now and it works perfectly so long as I match the bitrate!

No Nick, it isn't just the first couple of frames. It is also the audio which is uncompressed in the .mxf format. I have never wanted to go beyond a generation or maybe two with successive m2t renders because of the multiple generations of recompression of the audio. This is a way better way to archive footage! I will use it on every single project! Boy do I love this forum sometimes! Alan, I can't thank you enough!

In case Alan or anyone else following this thread isn't aware of this, use the following Sony utility to play back your .mxf files instead of VLC or whatever else you are using:

https://www.servicesplus.sel.sony.com/sony-software-model-PDZVX10.aspx

After you install it, go into Tools/Options and check the "Use alternate renderer" tab. Otherwise it won't use your graphics card GPU and the video quality will suck.

This PDZVX10 utility will give you hands down the best looking video playback you have ever seen. It will deinterlace on the fly perfectly. It will convert the sRGB colorspace to a more computer cRGB colorspace on playback. It will barely touch your CPU and give you one heck of a good image. It is worth converting from HDV to XDCAM for this playback utility alone!

Also kind of cool is that Handbrake will work with a source .mxf file. So will nVidia's "Badaboom" if you want a lessor quality (but unbelievably quick and CPU free) h.264 render.

So just to be clear, Laurence, are you saying that we should consider archiving HDV by smart rendering to Sony MXF rather than XDCAM EX, and that the two are effectively the same thing but in different containers (.mxf versus .mp4)?

>> No Nick, it isn't just the first couple of frames. It is also the audio which is uncompressed in the .mxf format. I have never wanted to go beyond a generation or maybe two with successive m2t renders because of the multiple generations of recompression of the audio. <<

In an HDV > HDV smart render, wouldn't the audio just get copied losslessly, probably right from the start to the end of the file? I'm thinking the audio stream doesn't conform to the same long-GOP structure as the video stream.

My understanding is that the audio in m2t clips isn't smart-rendered, just the video. I'm 99% sure about this, but please, somebody correct me if I am wrong. In any case, I would much rather be dealing with uncompressed audio. With m2t, if you smart render into an m2t master, your source audio, your narration, your music, your sound effects: everything is going to be data compressed, then data compressed again if you render to AC3 or use something like Handbrake. Yes, in the future, I will be working with a smart-rendered .mxf format any time when I would have in the past smart-rendered to .m2t. For me this is a huge improvement in my workflow. I can't believe there isn't more attention to this thread.

I'm getting a bit of a format headache here...

Would anybody fancy summarising (for a newbie on this subject) the differences between .mxf and XDCAM-EX at 'like-for-like' resolution/framerate etc.? As far as I can see from the Vegas Pro templates in 10.0 they are effectively the same things in different containers (.mxf and .mp4).

Are they any other differences or advantages?

Why might .mxf be better for archiving than XDCAM-EX .mp4?

Will the Sony mxf player Laurence links to play both?

A further question... If I captured from a Sony EX1 or similar I'd get XDCAM-EX .mp4 files, right?

I have no experience with XDCAM-EX mp4, but I can answer your questions about .mxf:

>Are they any other differences or advantages?

.mxf as it is used by Vegas is mpeg2 video with uncompressed audio with several high quality format options.

>Why might .mxf be better for archiving than XDCAM-EX .mp4?

The mpeg2 video of .mxf previews wonderfully in Vegas and unlike mp4, it smart-renders. The audio is uncompressed and can be 16 or 24 bit. Since the video has never been re-encoded since the initial mpeg2 compression by the camera it is still in it's original format except that the audio is no longer data compressed. This makes it the perfect archival format IMHO.

>A further question... If I captured from a Sony EX1 or similar I'd get XDCAM-EX .mp4 files, right?

As far as I can tell, HDV smart-rendered into 25Mbps .mxf is the same format you would get from an EX 1 if you set it to shoot in the slightly lower quality 1440x1080 25Mbps mode.

I think that the title of this thread is misleading:

Sony EX .mp4 vs Mainconcept .m2t should actually read Sony EX .mxf vs Mainconcept .m25 since that is what is actually being described in the text that follows.

Laurence, I think you may have missed the XDCAM-EX .mp4 render templates (bottom of the list of formats) that are separate from the Sony MXF templates and DO create .mp4 files (which can themselves be smart-rendered). These XDCAM-EX .mp4 templates were not in Vegas 8.0. I don't know if they were in 9.0 or not.

If you think being able to smart render HDV to MXF is significant, perhaps a new thread should be started?

From the original post:

>I never tried it before because I thought that setting would render out using an mpeg4 codec and not and mpeg2 considering the wrapper and extention of the file.

Now I see my confusion. Alan was rendering mpeg2 files into a format that had a .mp4 extension even though the actual codec in use was still mpeg2. That is confusing! I'll have to try it out to see it for myself.

Did a little more testing here...

In Vegas Pro 10.0b on my Core 2 Duo T7800 laptop I smart-rendered from HDV 1080-60i to 3 different formats:

1. HDV .m2t with the quality slider all the way to 31
2. Sony MXF .mxf (template HD 1440x1080-60i with the bitrate dropped to 25 Mbps)
3. Sony XDCAM EX .mp4 (template SP 1440-x1080-60i, 25 Mbps CBR)

Video Quality
M2T: There is a very small difference in quality during the initial and final few frames of the HDV render.
MXF: No perceptible difference from original
MP4: No perceptible difference from original

Speed
The XDCAM-EX mp4 render is much faster than the others. So fast in fact that I wondered if it had actually completed the render.
M2T: 25 secs
MXF: 29 secs
MP4: 5 secs

File Size
They are all within a percent or two of the same file size. Interesting that the HDV is not smaller, despite having compressed audio.

Portability
All 3 play in VLC. Only the .m2t will play in WMP. Only the .mxf will play in the Sony XDCAM Viewer but it doesn't look any better on my system than VLC (0.9.8a). In my experience .m2t can't be opened by FCP. I have no idea if the .mxf and .mp4 versions can be, or if there is a related change in gamma by doing so. No idea how the 3 formats perform in other NLEs either. I guess .mxf may be more portable, but in reality perhaps not.

[ Edit: Some brief tests in VP14 here: https://www.vegascreativesoftware.info/us/forum/vp14-cannot-import-m2t-files--104009/?page=last#ca643573 ]

Hey guys...

Sorry I've been mia for a bit. I got busy and never came back to the forum. I'm actually going to do more testing of different formats. I actually love tsmuxer and use it often. Another program that I love to demux streams with is an encoder called Super. I always use super to demux the video/audio streams. At times Vegas doesnt like the audio waveforms when importing clips and it crashes when it doesnt So I use Super to demux the streams and make new clips with the new audio stream and the original video.

I have Vegas 9.0e...so xdcam ex format is available there. I also got excited when I finally figured out a way to smart render hdv 720/60p video...lol. Now I am finally able to trim those clips to archive.

Nick, here is what I meant about lining up the clips. Lets say I have project I am working on and all the clips are in .m2t format. Its not a fancy project...just all straight cuts. If i want to keep a master of the edit..I would have to smart render the entire timeline. However, if I do, then the first few frames of each cut will be degraded in the final file. So then the best solution I found for that so far was rendering out to xdcam ex .mp4. Even though it still renders (sometimes it doesnt) the first frames of each clip, they will match the original in quality. Then take that .mp4 file and smart render it out into .m2t. Sure the first frame will render, but the rest of the frames will look like the original. I hope that makes it clearer.

As far as the open gops thing...my default is "First short" When I switched that to "none short" I had a better quality render of the first few frames when using Mainconcept. It not like the original but much better than using the "first short" setting. Maybe the default is now "non short" in Vegas 10.

Laurence, thanks for the link. I tried to go to the site but I need a servicesPlus account. Does it cost anything? I tried logging in with this account but it didnt let me. I never tried hanbrake. I recently downloaded it but didnt have time to mess with it yet. If you guys havent tried the Super encoder I mentioned...it is worth a try. Just do a search for "super encoder". its on the erightsoft.com site. Once on the site you have to search for the link...lol. you'll eventually find it. just following the links that say "download and use" or something similar and you'll eventually get to it. You can render to a whole bunch of different formats.

I will not be able to smart render to .mxf since HDV1 encodes at 18.3mbps and the lowest bitrate for .mxf is 25mbps. I've never tried it with HDV2...but I assume it would smart render out since that has a bitrate of 25mbps.

I was confused about it too...lol. I was always hesitant to use the xdcam ex format because it had a .mp4. I didnt know much about it at the time and assumed it used an mpeg4 codec given the wrapper it has. Turns out it uses mpeg2...lol. I'll some more tests involving tsmuxer and super when I have lots of time to spare. I always have trouble muxing files in tsmuxer for some reason...it always muffles the audio. I usually just use it to convert the video to .ts or .m2ts.

Alan, The Sony XDCAM EX player is free but yes, you have to sign up for an account. The one I downloaded was the top of the list, part FPDZVX10E. It's nice but in my brief tests I didn't see much of an advantage over VLC, but I get the feeling I may be in the minority on that score.

I have checked it out and "first short" is the default for 720p HDV. "None short" is the default for 1080i HDV. This is true of both Vegas 8.0c and 10.0b.

Is it just me or is it confusing that the .mp4 extension just refers to a video format container has nothing whatsoever to do with mpeg 4 compression? Who'd have thought....

Well I did some tests as well and can confirm what you are saying about render times Nick. Wow is the XDCAM mp4 render fast! I did a project that was a mix of smart and regular rendering and it finished in half the time of a .mxf or .m2t render. The regular rendering is about the same but the smart-rendering sections just fly by!

As far as differences go, the XDCAM mp4 render won't play back for me in Windows Media Player but it works really well in VLC even with on the fly deinterlacing. It is much more solid the .mxf playback in VLC which only seems to work properly if you start playback and don't touch it. If I move around within the clip in VLC with a .mxf file, I lose audio. XDCAM mp4 VLC playback is rock solid and I can move around within the clip.

Preview of either XDCAM .mp4 or .mxf in Vegas is about the same: that is to say absolutely wonderful! I can preview at best full resolution on my lowly Intel Core2Duo laptop.

On a side note, while I can smart-render from m2t to XDCAM .mp4, I cannot smart-render from XDCAM .mxf to .mp4. I can however smart-render from .mp4 to .mxf.

The only advantage I see of .mxf at this point is that if you watch .mxf with the PDZVX10 player, you get to see your video with sRGB to cRGB color correction which looks nice and vivid and less washed out. Also, with .mxf you can do 24bit uncompressed audio (.mp4 only does 16 bits).

None the less, I think I will be working with XDCAM .mp4 from now on. I don't see any need to batch convert the original clips since it doesn't really gain you anything, but I will use it for any re-renders of my original .m2t source clips. This actually affects me quite a bit because the way I usually work is to go through my .m2t footage, pre-edit all the junk out, and smart-render what's left into more manageable large sections with markers and cleaned up and normalized audio on any dialog. These sections are what I typically archive. I've always hated the way this adds a generation of data compression to the audio but I have worked this way anyway because it is so much easier for me. I will use the XDCAM mp4 format for all these pre-renders now. Thanks guys! :-)

In my experience .m2t can't be opened by FCP. I have no idea if the .mxf and .mp4 versions can be, or if there is a related change in gamma by doing so. No idea how the 3 formats perform in other NLEs either. I guess .mxf may be more portable, but in reality perhaps not.

I have rewrapped .m2t into FCP compatible .mov format with a program called "ClipWrap" and it worked well. I have heard that .mxf can be read by FCP if one installs the right plugin into FCP. I wouldn't be surprised however if XDCAM .mp4 didn't just work with FCP right off the bat. I don't have a Mac to try this, but boy would I love to know if this is the case. It would be wonderful to have a more portable format!

That's basically my conclusion too, pending someone coming up with any other drawbacks.

Laurence, did you ever buy TMPGEnc Authoring Works, in order to get smart rendering of HDV to Blu-Ray? I'm wondering if it will read and smart-render XDCAM EX .mp4 (or MXF) like it does HDV .m2t. If so then that should be a superior intermediate format to render to for that purpose as well. I will try and check that out some time.

I'm also wondering if DVDA might smart render one of these formats, but I highly doubt it.

OK I think I've found a bit of a fly in the ointment:

My main format that I use is HDV 30p but with it captured on the tape or memory card as 60i. I do this because if I use the true 30p setting it won't smart-render.

When I did my tests that I based my earlier comments on, I was using some true 60i footage for the testing.

Now what I'm finding is that while my 30p (flagged as 60i) footage will smart-render into a .mxf container, it won't smart-render into an XDCAM .mp4 container.

I'm still better off than I was. HDV smart-rendered into .mxf is still way cool as the footage now has uncompressed audio, plays back with sRGB to cRGB color correction with the PDZVX10 XDCAM viewer, will work with FCP with a .mxf plugin and will smart-render again into a .mxf project.

What I'm missing however is that incredibly fast smart-render that I would get if I could use the .mp4 container.

The XDCAM .mp4 approach should still work well though for compatible HDV formats, just not for those of us working with 30p.

I wont be able to smart render my footage to mxf as I shoot in 720p and there isnt an 18.3mbps setting. VLC actually has an option to do the studio-computer conversion. If you go tools->preferences->video..uncheck "use hardware YUV->RGB conversions" There is a newer version of VLC out...they are on version 1.1.5. I think Nick has the older one.

Laurence, I have tried giving a friend video in multiple formats to see if it would work in FCP. I have tried .m2t, xdcam ex, and sony mxf and none worked right off the bat. I think they ended up using a conversion program or a plug in. Do they make ClipWrap for PC as well? I was only able to find it for Mac. Thats strange how you can only render from format to another but not the other way around. Im not sure why but did you check field order? I was checking default settings and xdcam defaults to "progressive" Since your video is 30p in a 60i stream...it is still flagged as 60i. When you converted from xdcam to mxf..you might have had your field order set to "upper field first" so it smart rendered out.

My sleep clock is really messed up right now. I live in Miami but I wake up and go to bed at the same time with the people in Hawaii..lol. I need to force myself to get it back to normal again.