A rendering option that would be useful

Robert W wrote on 10/23/2008, 11:14 PM
Here is a way they could speed up renders and make better use of multiple cores when you need to render to several formats.

Presently, when you render to multiple formats, you have either use the antiquated script driven batch renderer or push each one through by hand. Every time you render to a format, the renderer has to process each frame and then pass it to the codec to encode in you desired format.

So why not have a renderer that produces each frame, then passes it to four codecs? Or seven codecs? When not run each codec on it's own core? That way instead of wasting time rendering the same stuff over and over again just so it can pass the same frames to different codecs.

Actually I think the Vegas architecture is not quite as simple as that. I think the way it redners frames is determined by settings in the codec, for exampel as with codec conversions. However whichever way you look at it, there is a certain stage where there is identical work being done over and over again in multiple renders to different formats. With the huge amounts of redundant processor time some people are experiencing with multi core systems (I have a dual core here that never seems to run above 70%), using the redundant time to make the most of the reusable data for other codecs seems like a potential massive benefit.

Of course, it would not be useful for mixing multiple renders to Studio RGB and Computer RGB formats, but you can't do that with the clunky old rendering script anyway. Where it would be very useful is for say two pass renders. I've just completed a one pass render and straight away started a two pass render of the same thing. Common sense would mean that two codec tasks are called, one producing the one pass render, while the other task does the first pass of the two pass render. We check the one pass version for and if we find an error, we can stop the second pass in it's tracks. Otherwise we have saved over a day on the final render.

I have a render on a dual core at the moment going that will not finish until Sunday afternoon. Even if we had a similarly clocked quad core processor running on that machine, it would most liekly not give us much of a speed improvement as it is barely using 58% of the available processor time on the dual core. I have criticised Sony for not moving on the multi threaded code, and this is one example of where they need to think in new ways. It is quite something when computer architecture moves beyond the ability the developers to actually exploit it. Poor coding used to mean that it would create too much of a processor overhead. Now you get less overhead and more redundancy, and things just happen slower.

Comments

farss wrote on 10/23/2008, 11:56 PM
"So why not have a renderer that produces each frame, then passes it to four codecs? Or seven codecs?"

Damn good idea, After Affects already does this. Then again given how complex an AE composite can be it'd be kind of nuts if it didn't.

With Vegas you can use the Multirender in Peachrock's Veggie Toolkit. However while that is very, very good and I use it to give Vegas a good workout it does do multiple passes to render to different codecs but at least you can sleep while it works.

Bob.

TheHappyFriar wrote on 10/24/2008, 5:00 AM
big drawback is that if you rendered to multiple codec's at a time you'd take longer to render. As it is right now, you render to codec 1 relatively fast, #2 relatively fast, etc. So if it takes 5 minutes a render every 5 minutes you get a file.

If you do them all at once it would take 5x#codec's render time, this increasing the chance that if something went wrong (ie power failure) you'd loose ALL the work. Plus if there's multi-core codec's they loose a lot of advantage of being multicore.

A way to do this now is to use multiple instances of Vegas. For a big project you can have multiple instances render different sections, then you join them later.
farss wrote on 10/24/2008, 5:14 AM
That's not entirely true. If the project involves complex composites or effects then most of the work is in building the final frame, some of my projects have taken seconds per frame. Once Vegas has that frame in RAM it's then a relatively trivial task to encode / render it. In my scenario 1 codec would take x time, two would take way, way less than 2 x X time, in fact I could render to 5 codecs in under 2 x X time.
Not to say that you don't raise some valid concerns though. One fix for your concerns and to still get things done faster is to render first to a lossless / near lossless codec and then encode that to the various codecs / resolutions you need.

Bob.
Robert W wrote on 10/24/2008, 5:54 AM
I'm not sure rendering to lossless be an equivalent solution. First of all, for my current project, I would need about 4 terrabytes of storage to get a lossless HD lagarith master. It would also take a long time to encode, probably longer than most of the target formats. Not to mention that it still would not make use of the masses of redundant processor time multi-core users seem to be reporting. Also, I think frame rate conversion is handled better straight from the project.

The short of it is that the development is so skewif that they can not manage to build a integrated batch renderer, so building a batch renderer that does parallel encoding to more than one format is obviously a bit too clever for them. Even though it is a completely easy quick win.

Sony, why don't you give me a call? I can give you more of these great ideas to make your product worthwhile for a small fee only. Or you can keep on making us dance like monkeys.
kairosmatt wrote on 10/24/2008, 6:32 AM
I second, or 5th or whatever, the need for a better batch renderer. How about something where we can set the names and locations for different files?

Also, I'll bite, what is this knitting and pearling going on lately?

kairosmatt
tcbetka wrote on 10/24/2008, 7:27 AM
Excuse the ignorance here, as I am only trying to learn...

But what sorts of things are you guys doing that require you render with so many different codecs? I haven't used but two different codecs yet in my vast experience (of 2 months) with Vegas, so forgive me for asking.

Interesting topic though--a batch rendering script that assigns one CPU core to each different render. I would like to know more about the kinds of projects that require this many different codecs.

I am finding that, like with most new complex topics a person learns, the most difficult thing about learning Vegas is finding out what it is that I don't know. Once that's known, it is not all that difficult to go learn it.

Thanks guys.

TB
kairosmatt wrote on 10/24/2008, 8:00 AM
I don't have vast amounts of experience either, but when you make a DVD you need to render both MPEG2 and AC-3.
And if your done with the project, say you want make a master AVI with a lossless codec. Or an uncompressed one. And maybe you need to make a Quicktime as well to give to someone who uses Final Cut.
Plus there are dozens of formats for the web. Windows Media, MP4, Quicktime, Real etc. And within all of those you can have different frame sizes and bit rates.


So sometimes its nice just to set it all up to go one time. And it that is an interesting idea that Robert has, and I for one didn't even know that After Effects could do that or that it was even possible.

kairosmatt
tcbetka wrote on 10/24/2008, 8:06 AM
Ah...I understand perfectly. Makes complete sense...

I haven't done any of that yet, and simply use the NTSC DVD MPEG-2 template and include 224Kbps audio with my DVDs. They are only for my daughter's volleyball games right now, so it really doesn't need more complex audio than that. However I have used the AC3 codec before--and I know about the different web formats. Somehow I guess I just thought you guys were working on stuff that's much more complicated than that. I was probably just making it more complicated than it has to be, lol.

Thanks for clearing it up.

Vegas Pro DVDs, the HD Survival guide, DVD Architect tutorials...

So much to learn, so little time. I need to just quit my job and work on this stuff 80 hours per week!

TB
TheHappyFriar wrote on 10/24/2008, 8:28 AM
If vegas it told to use the final compiled frame for different codec's (wasn't thinking of that, good idea!!) it would be no different then rendering to an uncompressed format.

How long is your project to require 4TB?? I'm sure there's some lossless compression schemes out there that would greatly reduce the file size. IE PNG compression.
johnmeyer wrote on 10/24/2008, 8:30 AM
It is a very good idea.

However, the chance of Sony implementing this are somewhere between zero and none. Therefore, as a way to achieve the same objective, but without using uncompressed, I'd suggest rendering to Cineform and then use that to render to the multiple codecs. Yes, there is some loss, but unless you are planning to release to theaters and view on the big screen, I'm not sure you'd be able to tell the difference.
rmack350 wrote on 10/24/2008, 10:31 AM
Using satish's frameserver would get you uncompressed output, in 8-bit, so you'd need a compressor that could do multiple renders in lock step.

Rob Mack