Selective Prerendering...

megabit wrote on 9/17/2008, 6:32 AM

OK, so this one has nothing to do with the recent 8.0c/8.1 hype; in fact it's a question so basic that I'm sort of embarrassed to ask it. So far, for most part of it, I only did some very basic editing (no compositing or fancy transitions); therefore - even with full HD material from my EX1 - I really didn't need prerendering. The mxf's play from my timeline at Best/Full for the most part, and when doing Multicam I switch to Preview/Half (with 3 cameras, it still gives me the full 25psf playback). Those short transitions or other effects I'm using, do of course slow it down to some 15 fps - but lasting for a second or so, do not prevent me from enjoying the full quality, full fps playback.

With some more complicated editing, I'm trying to use Selective Prerendering now (as even with the 8.1 and 8GB of RAM, the RAM preview is still too short for me). And I really cannot settle with any specific prerender format! I mean, isn't the slightly compressed AVI (like SONY YUV) supposed to give me the highest speed playback? Well, it doesn't - with the prerendered areas playing back at a maximum of some 20 fps (regardless of the Preview quality), I'd suspect my HDD is not fast enough - but using 2xRAID0 configuration (yes - two separate RAID's for reading and writing, each capable of over 200MBps), I somehow don't believe this is too slow...

But what do I know - please advice on the following:

1. Is the low fps of my prerendered avis's due to my HDD's still not fast enough?

2. If so, which format is the best compromise for selective prerenders, when working with the EX HQ 1920x1080/25p?

Oh, and one more thing: I don't attempt to use my prerenders as "cumulative" to facilitate the final render out; I need them only to give me full speed, full quality playback of heavily edited areas - which they don't seem to be able to deliver...

EDIT: After the priceless contribution from fellow members, specifically Rob - I now actually DO intend to use my pre-renders as the final render "time-savers" - please see below :)

Comments

farss wrote on 9/17/2008, 6:53 AM

1. Yes, look to double your STR. If you might have 10bit on the horizon double it again.

2. YUV, you're already using it. Light CPU load, just need FAST disk arrays. You could also try Cineform.

Bob.

megabit wrote on 9/17/2008, 7:04 AM

Bob, while I don't know what you mean by "STR", I take it my 200 MBps should be doubled (at least). Any practical hints on how to do it, apart from building (or buying for a lot of $$$) a 4-disk striped array?

megabit wrote on 9/17/2008, 7:38 AM

Not being quite convinced that a Sony YUV 1920x1080 AVI cannot playback at full fps from an 200MBps + HDD array, I redirected my Vegas Explorer to the appropriate preneders folder, and played one of these avi's in the preview window, but NOT from the timeline... And guess what - full 25fps !

Just what is Vegas trying to do with it, when playing the prerendered contents from inside the project, and slowing it down almost 50% ?!!

I still think I'm missing something important about the very idea of prerendering... Anyone?

Piotr

Sebaz wrote on 9/17/2008, 9:18 AM

Vegas has a bug in prerenders that appears after a while. What it does is that after a certain amount of prerenders, the next time you do one it will show you the progress as if it was prerendering, but when it finishes, it doesn't update the timeline, so even if you play the loop, you will still see the raw material. You can tell when this happens because other than the footage playing slower than real time, it won't show the blue bars on top of the loop area which indicates a prerendered loop. The only solution for this is to restart Vegas.

One prerender template you might want to setup is open the Mainconcept Mpeg 2 module and start with the Blu-Ray 1920x1080 50i (I'm not at home so I can't remember if it's that name exactly) and then modify it slightly to make it a program stream instead of a transport stream, and make sure "Save as separate elementary streams" is unchecked. Also set the bitrate to 30 Mbps CBR and that will give you an excellent preview quality with a bitrate that any current hard drive should be able to handle even without a RAID.

TheHappyFriar wrote on 9/17/2008, 9:48 AM

megabit wrote on 9/17/2008, 10:20 AM

Why are you using YUV for preview renders? I use mpeg-2 HD (same as project). Vegas just copies frames, doesn't recompress/render, if nothing's changed so the same quality it kept with direct previews. You can't improve quality by rendering to YUV as a preview

Whether to use avi or the project (and final delivery) format, was the original question I asked in this thread. Bob opted for avi (which I tried, and couldn't even playback at full speed - most certainly NOT due to my disk being too slow, as Bob suggested). You're suggesting the same format as the project uses - which is more convincing, especially if I could use smart rendering at prerenders, and in turn use the prerendered areas (saved onto a new track) to speed up the final render. Well, I must admit this is more convincing, and answers my original question. There is only one "BUT":

- in order for smart rendering to work on mxf clips, I must use SONY MXF output with video settings matching those of my project. Now, I don't need mxf files in my final (delivery) output; I need BD-compatible MPEG-2's. Hence my question again, which I kept asking in other threads as well:

- Has anyone succeeded to achieve smart rendering of mxf-based project into the MainConcept MPEG2 of 1920x1080p/25, 35Mbps VBR, format? If so, how did you set up your template?

Cause I tried many various setting combinations to achieve that, to no avail... In 8.0c and 8.1, smart rendering from mxf to mxf works fine, but it's NOT what I need!

rmack350 wrote on 9/17/2008, 12:50 PM

"smart rendering from mxf to mxf works fine, but it's NOT what I need! "

But that's what the smart render feature does. Direct copies. So if your media won't play on the timeline at full fps a smartrender won't change that.

As to why a Sony YUV prerender won't play at full fps... that's an interesting mystery. First test is whether a similar SONY YUV file would play in the trimmer at full fps. That's your baseline, and maybe you tried that-I'll read again.

The Vegas timeline has overhead compared to the trimmer so the performance isn't ever as good, but you'd think that the timeline would ignore all the vision under a prerender. If that's the case then could Vegas be processing audio?

Rob

farss wrote on 9/17/2008, 3:05 PM

Vegas has a lot more overhead than a media player. It'd be a fair bet that the media player also uses all the capabilities of whatever instruction set your CPU has. Pretty certain Vegas doesn't.
If your RAID controller is using the CPU to do work then things get even worse.

STR = Sustained Transfer Rate.

Bob.

rmack350 wrote on 9/17/2008, 3:41 PM

Just rendered a second of sonyYUV at 1920x1080 and it comes out to about 120MB/second. My first thought is that you aren't really getting 200MB/s off that 2-disk RAID, but if ya say you get that then you get that. If it plays in the trimmer at full framerate then you've got the throughput for that one clip.

On the timeline, Vegas still needs to access the other clips to get their audio, and even if Vegas were properly ignoring the vision under the prerender it still needs to do some work just to figure out that it needs to ignore them. So obviously there's overhead.

I had a 2 disk striped array running off a motherboard chipset for a while and was just getting a bit over 100MB/second throughput. I'm sure you could get a lot more with a dedicated controller card on a fast bus with faster drives but 200MB/s is really good for 2 disks under any circumstances.

Rob

TheHappyFriar wrote on 9/17/2008, 6:59 PM

hey... if you are only using it for preview and NOT for a final render saver, you could use the "render to media player" feature instead. then it renders what you want & opens it up in the default media player for the filetype you choose.

don't forget that if you have project wide FX vegas still does those, even on prerendered footage. And it's most likely still using any edited audio you have. You're streaming several media fiels from different parts of the drive, that could be the issue.

megabit wrote on 9/18/2008, 12:53 AM

Thanks guys for your answers so far. Very helpful.

Q: ""smart rendering from mxf to mxf works fine, but it's NOT what I need! "
A: But that's what the smart render feature does. Direct copies

With regards to the above excerpts: mxf is just a wrapper for the Long-GOP MPEG2, so technically it should be perfectly viable to smart render from mxf to MPGEG-2 - provided Vegas allowed this by design, and the output video properties were picked correctly to match the mxf clips on the timeline. Well, just another item on the wish-list, Sony!

rmack350 wrote on 9/18/2008, 9:37 AM

Fair enough. I suspected that that's what you were thinking, that mxf and mpeg2 could be identical data by different names. I certainly can't argue that one way or another, but if they can be identical then do you really need to prerender as mpeg2 rather than mxf?

Maybe that's a dumb question but I just logged on for the day and haven't reread the thread yet, maybe the answer is in there somewhere.

Rob

megabit wrote on 9/18/2008, 10:01 AM

MXF can be loaded into DVDA, but will BD's made this way fully BR-compliant? Also, Vegas doesn't seem to allow separate, video-only MXF renders, so adding DD sound track would be difficult..

This is why what I need is MPEG-2 video stream smart rendering!

rmack350 wrote on 9/18/2008, 10:04 AM

Okay, went back over the posts. In the end you need BD compatible MPEG-2, so the workflow you're hoping for is to:

--Edit
--prerender (preferably smartrender into BD-compatible MPEG2)
--export as BD compatible mpeg2 (Another smartrender based on your prerenders)

You're saying that the MXF data is exactly the same as the MPEG2 data so you'd like to get a smart render (data copy) out of it. Because Vegas won't do it, you end up with an extra generation of mpeg compression, I guess, and a hit on the render time.

I guess the question might be whether there's an external way to use MXF output. Can you stick to the MXF format, output that, and rewrap it for burning to disc?

<Edit>Okay, the workflow isn't that linear in practice but the steps represent the rendering flow. The main reason people suggest to prerender to something like SonyYUV is to give you many options on the last render step. In theory, you are creating a good general purpose prerender from which you can strike several different types of final renders. In your case the render path is very strictly defined so you don't need a general purpose prerender.

Rob

megabit wrote on 9/18/2008, 11:29 AM

Rob, I guess I found a solution: the Snell & Wilcox MXF Desktop will unwrap the mxf into native essence files; in fact it only works for the video stream of the Vegas-produced mxf, but the m2v that I'm getting is exactly what I need in my workflow. The audio I am rendering as AC3 separately, anyway.

Interestingly, Vegas cannot read this m2v, but DVDA can - which is what I need. However, DVDA needs to recompress it - will appreciate suggestions on how to avoid it, as it ruins this workflow!

The only question is whether a BD authored from such files will be fully BR-specs-compatible - but actually, I don't see why it shouldn't.

megabit wrote on 9/18/2008, 2:43 PM

OK, I did some more testing as the mxf smart rendering available in the 8.0c/8.1 versions of Vegas looks promising from the point view of speeding up the workflow with XDCAM EX...

When I smart-render my 1920x180p/25 material to the mxf HQ 50i template, the process is not very fast at all, with only occasional "No recompression required" messages in the preview window. After demuxing such a file, it IS compatible with DVDA in that it does NOT require recompression.

However, to take the full speed advantage of smart rendering, I must render out to progressive, as my source material is 25p. Indeed, mxf smart rendering is a breeze, alas - DVDA needs to recompress the resultant file, as being 25p it's not BR-compliant! This of course denies any speed advantages of mxf smart rendering in Vegas.

Can anyone think of a workaround, or am I stuck? Of course, shooting 24p rather than 25p would resolve this issue, but being in a PAL country I'd prefer using 25 fps...Also, 50i would work - but I will never go back to shooting interlaced!

rmack350 wrote on 9/18/2008, 5:50 PM

Well, yeah, the fact that DVDa recompresses the file negates the whole point since what you're after is a fast workflow with the fewest number of recompressions. But you're trying the path I was suggesting. I'm curious if this can be made to work.

This would be worthwhile to submit as a support request to see if you can get anyone at SCS interested, and if you haven't yet posted in the DVDa forum you should do that too.

I wonder if the solution is to use some other BD authoring tool? Probably not, but worth looking into.

Rob

megabit wrote on 9/18/2008, 11:24 PM

I inspected the two output mxf files (the progressive and interlaced one), and now think that Vegas was not even supposed to "smart" render the latter, as all clips in my timeline were 25p. Why did it even display the "No recompression required" message beats me - would it be a bug?

Also, the two files differ in size; the interlaced being bigger than the progressive one like 13:9 - is this normal?

They both play back flawlessly, though...

PS. Rob, I just saw your comment:

"I wonder if the solution is to use some other BD authoring tool? Probably not, but worth looking into"

- yes, Ulead MF 6plus HD will take a progressive m2v and compile a BD without recompresssion, as long as the resolution and bitrate are the same as the project settings - but I can't be sure such an BD would play back in every standalone BD player out there!

megabit wrote on 9/19/2008, 1:12 AM

I have just submitted a ticket to SCS support, with the suggestionsbased on these tests; here it goes:

"As a Polish (PAL area) Vegas Pro user, I have an inquiry; please see my thread on the SCS Vegas forum:"

Now, I wonder what the response will be, if any :)

blink3times wrote on 9/19/2008, 3:14 AM

"- yes, Ulead MF 6plus HD will take a progressive m2v and compile a BD without recompresssion, as long as the resolution and bitrate are the same as the project settings - but I can't be sure such an BD would play back in every standalone BD player out there!"
======================================

I use it pretty religiously for burning BD and have yet to burn a coaster.

megabit wrote on 9/20/2008, 2:31 AM

Just to update you on the answer I got from SCS support. As could be expected (even if hoped for otherwise), they didn't engage into a discussion o my idea at all. In fact, they didn't even notice being PAL is an important factor in my scheme

"Hi Piotr,"

Oh, well.

Laurence wrote on 9/20/2008, 8:33 AM

Basically what I find is that everything is prerendered in HD. Where in SD DV codec projects, just the transitions, filters and text overlays are prerendered, in HD you get a prerender of the entire project. This feature is a lot like 32bit color in that it is best to just pretend that it isn't even there. It is a lot less frustrating that way.