Rendering Intermediary Files/Preprocessing - Best Practices

Comments

Musicvid wrote on 9/12/2021, 8:46 PM

Curious-does MagicYUV perform similarly, Howard?

Also, which ProRes are you testing here? 422?

In tests I ran a few years back, ProRes 422 wias 4.3 times larger than MagicYUV 422, at the same quality (98%). No render speed comparisons were done, but I surmise that MagicYUV was quite a bit faster. since it's intraframe.

Of course, the source "is" 4:2:0, so an XAVC-S intermediate would be just dandy....

The disadvantage to AVI is, of course, that it is unsuitable for handoff to other OS'.

RogerS wrote on 9/12/2021, 10:05 PM

Thank you, I saved a screenshot of that test result and appreciate it.

Going from X-AVC-S 8 bit 4:2:0 source to XAVC-S intermediate doesn't risk compression artifacts? Just give it enough bitrate?

I was curious if Howard did comparisons esp. given the unexpected results he found with HEVC decoding and render quality. I surmise it would probably affect MagicYUV the same way (once controlled for you'd end up with same quality).

RogerS wrote on 9/12/2021, 10:31 PM

Curious-does MagicYUV perform similarly, Howard?

Also, which ProRes are you testing here? 422?

Funny you asked. 😀 I just tried that too. I did ProRes 422 and ProRes422hq but came up with negligible quality difference between the two. MagicYUV was marginally higher quality but the air's pretty thin up there and ProRes422 is a fair amount smaller. Here's a google sheet with all the details:

https://docs.google.com/spreadsheets/d/1EY4lDiq7AxQ3aarRNurz6hqqHRubL3n-bM_CM7YqPrU/edit?usp=sharing

 

 

Thanks for the testing, Howard. I'm wondering why your render size is different than MusicVid's experience? Quality looks very comparable in your tests with legacy HEVC enabled.

Howard-Vigorita wrote on 9/12/2021, 11:21 PM

Likely the source material. If it's full of air or lacks much motion, it compresses quite easily. For instance a still photo turned into a video would compress down to a single frame with a repeat command. A photo of a black frame would compress even smaller. My source was of moving blue/green surf shot 10-bit taken directly off of a raw sensor which would be on the other end of the difficulty scale.

RogerS wrote on 9/13/2021, 2:36 AM

That makes a lot of sense. I'll do some tests myself to get a feel for how both work.

Musicvid wrote on 9/13/2021, 5:47 AM

If it was staying in Vegas, I think I would use MagicYUV 420 because it has a small footprint and will withstand generational loss. Xavc might not hold up as well after a few generations, and that is the concern, not second generation artifacts.

If I was going to hand off for editing, ProRes is still the preferred choice.

Considering that the source is pretty light weight for 4k, it all becomes a coffee table discussion, because it all doesn't make one whit of difference in any practical production sense. If it was 60 Mbps, we might have a discussion.

Musicvid wrote on 9/13/2021, 6:24 AM

A photo of a black frame would compress even smaller.

In integer space? It's a common misconception; have you run the tests? (Hint: zero is an integer.)

sakendrick wrote on 9/15/2021, 7:45 AM

Lots of discussion - thanks all for the info (not sure I'm following it all). My main goal was just to not lose quality when rendering intermediate files, through compression (no misconceptions that I would gain quality) - and was looking for best practices considering my source. When I used the ProRes native in Vegas Pro - it took a while, created giant files, but I got through it and they seem to edit and render fine when pulled back in to my main project. Not sure if it's overkill for my main goal.

I have started playing with ThrottleStop (thanks for the suggestion). I liked this description: https://beebom.com/how-use-throttlestop-control-cpu/. If I understand it... Dell puts your processor into Turbo, but then it overheats, so it throttles it down to let it cool down, impacting performance. Doesn't sound too smart. The concern with throttlestop I suppose is that you can leave your CPU unprotected while it overheats. And yes, I guess because of my current bios I was unable to set the multiplier down to try to reduce heat (it's disabled). I adjusted the clock modulation down to about 80% hoping for the same result, but I have no idea what that does.

 

RogerS wrote on 9/15/2021, 8:18 AM

ProRes 422 is fine for intermediates- I use that and the paid MagicYUV. They perform well and aren't taxing on the CPU either for playback.

For throttling your concern is misplaced. Throttlestop does not leave your CPU unprotected. If you uninstalled the Intel Dynamic Thermal Platform and unchecked BD Prochot, yes you could fry your CPU. Assuming you don't do that, when the CPU gets hot it will still throttle (or turn off the computer if it really gets too hot) so there is zero risk of damage.

What Throttlestop can do is help you use the computer for longer before throttling in a few ways. The best thing to do is to undervolt. You need to roll back your bios to a previous version for that (about 1.5 years ago) as Dell solved another problem but disabled undervolting at the same time. Undervolting doesn't reduce performance but does reduce heat. Now if you take it too far you'll get BSODs so if you see any, back it off a bit. Mine's at -135mV for CPU and cache. You should be able to do at least -100.

The second is you can set up different profiles for different use scenarios. For rendering (and most editing) I'd check disable turbo and set speedshift to a modest value like 160. Keep an eye on your CPU and GPU temps and total wattage used as you work and maybe that will be enough to keep it well above 1500MHZ.

The problems with Dells are several- designing laptops to work at peak performance for only a few seconds before overheating is one of them (turbo). The heatsink/fan assembly is inadequate and the cases too tight. On computers of our era the VRMs that supply power to the processors aren't linked to the heat sink so even when CPU and GPU temps are controlled these overheat and choke off the amount of power that can be supplied. Really dismal overall... Next up I'm trying an actively cooled laptop stand and will see if it helps.

Musicvid wrote on 9/15/2021, 8:26 AM

My main goal was just to not lose quality when rendering intermediate files, through compression (no misconceptions that I would gain quality) - and was looking for best practices considering my source.

You can do so without raising your modest source bitrate by 3,300%, and it is desirable, from a number of physical considerations, not to do so.

As for your CPU heating concerns, that is better left to a separate discussion.

 

monoparadox wrote on 9/15/2021, 10:47 AM

Anyone have MagicYUV running in Vegas 19? The 18 installer won’t work for me.

Musicvid wrote on 9/15/2021, 10:58 AM

It's working here. Don't remember doing anything special.

Grazie wrote on 9/15/2021, 11:37 AM

Anyone have MagicYUV running in Vegas 19? The 18 installer won’t work for me.


Here he be in Render Event in HOS, so very fast:

 

Grazie wrote on 9/15/2021, 11:40 AM

Anyone have MagicYUV running in Vegas 19? The 18 installer won’t work for me.


@monoparadox - Check the Changelog: https://www.magicyuv.com/change-log/

You may have an earlier one for VP18? Maybe?

sakendrick wrote on 9/15/2021, 12:51 PM

@Musicvid, I don't see anywhere in the native ProRes rendering dialog box to change the bit rate. What am I missing, or is that a reason to use Voukoder where I have more control (actually I don't see how to changes this in Voukoder either). I'm also surprised at my low bitrate from source.. assuming this is a setting I should seek to change (video is just captured using phones) in the future. Guess I'm still at a loss on my original question. Based on my source video, what is the best intermediate file render settings should I use so I'm not unnecessarily creating bloated files.

Musicvid wrote on 9/15/2021, 1:18 PM

 

  1. Is the intermediate staying in Vegas, or is it going to a Mac?
  2. Will your intermediate be subject to multiple generations of renders, or a single generation?

If it was staying in Vegas, I think I would use MagicYUV 420 because it has a small footprint and will withstand generational loss. Xavc might not hold up as well after a few generations, and that is the concern, not second generation artifacts.

If I was going to hand off for editing, ProRes is still the preferred choice.

sakendrick wrote on 9/16/2021, 8:24 AM
  1. Staying in Vegas.
  2. Just one re render... I have several source files. I'm taking each source, denoising, doing some contrast/color adjustment, rendering out. Then taking those files to do my final edit/composition, which will be rendered one more time for final

thanks!

RogerS wrote on 9/16/2021, 8:46 AM

Anyone have MagicYUV running in Vegas 19? The 18 installer won’t work for me.

It works but I copied files over manually from my VP 18 install.

Musicvid wrote on 9/16/2021, 8:48 AM

Plan on making several intermediates along the way, so you can revert when you change your mind on editing decisions, and also save your projects incrementally. Projects are cheap.

So a fast high quality intermediate is MagicYUV (or UT) 420, possibly at the cost of size. It's lossless through multiple generations, so you could make an intermediate from an intermediate.

[EDIT /] I see the UT 4:2:0 encoder is available in Voukoder, so I would start wirh that and see how it goes. And the files may be bigger than you want to work with. [/]

There are third choices, of course, there may be an MPEG-2 variant that would render fast and retain quality? I would certainly test XAVC-S for size comparison.

sakendrick wrote on 9/16/2021, 1:22 PM

I saw a number posts here about MagicYUV - will the free version suffice, or do I need one of the paid tiers?

Musicvid wrote on 9/16/2021, 3:53 PM

The free version will work just fine for your source.

sakendrick wrote on 9/16/2021, 8:30 PM

cool - will give it a shot, but otherwise will stick with ProRes. I rendered using Voukoder UT. Took about 3.5 hours (different source file), 35 GB, won't open in Vegas :(

RogerS wrote on 9/16/2021, 8:55 PM

Try just testing a few seconds with new file formats in Voukoder and ensure it works as expected in Vegas.

Howard-Vigorita wrote on 9/19/2021, 12:27 AM

Just tested a whole bunch of render formats for maximal quality and got the best results in Vegas 19 so far transcoding hevc lossless to MagicYUV or ProRes with Legacy decoding. Spot checking indicates Legacy hevc decoding would impact quality similarly in v18 which gave me identical quality and performance results to v19. Only differences I found were with Voukoder which changed a few preset options from v8 to v9. Updated charts are here. Use of Legacy decoding is indicated by "leg dec" in the Quality column.

I haven't done any testing yet using an AVC reference clip but I think it would be valid to do that using one of the near lossless 10-bit AVC transcodes from hevc lossless that I identified. I'm curious to find out if Legacy decoding also benefits quality results from AVC source clips.