Rendering Intermediary Files/Preprocessing - Best Practices

sakendrick wrote on 9/10/2021, 10:34 AM

Not sure if "intermediary" files is the right word - but I'm working on a video and one of the sources is dark and has a lot of noise. To reduce the amount of processing, I was considering trying to improve those source files separately from my video composition - using Neat for noise removal, improving brightness and contrast, sharpness etc. Is this a good practice, or by re-rendering am I only going to degrade quality further? Assuming if I do this I should try to use a very high quality format (maybe ProRes - although I'm not sure how well Vegas works with ProRes files in the editing stage as in previous issues I've had, community members have recommended converting MOV to MP4).

Any other tips, or best practices to help clean up my video are greatly appreciated. The source video had VFR but I converted to CFR using FFMPEG scripts - here is the result:

 

-----------

General
Complete name                            : C:\RWTC\20210620_144221-[CFR].mp4
Format                                   : MPEG-4
Format profile                           : Base Media
Codec ID                                 : isom (isom/iso2/avc1/mp41)
File size                                : 4.07 GiB
Duration                                 : 7 min 1 s
Overall bit rate mode                    : Variable
Overall bit rate                         : 82.8 Mb/s
Writing application                      : Lavf58.45.100

Video
ID                                       : 1
Format                                   : AVC
Format/Info                              : Advanced Video Codec
Format profile                           : Main@L5.1
Format settings                          : CABAC / 1 Ref Frames
Format settings, CABAC                   : Yes
Format settings, Reference frames        : 1 frame
Format settings, GOP                     : M=1, N=30
Codec ID                                 : avc1
Codec ID/Info                            : Advanced Video Coding
Duration                                 : 7 min 1 s
Bit rate mode                            : Variable
Bit rate                                 : 82.5 Mb/s
Maximum bit rate                         : 120 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 30.000 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.332
Stream size                              : 4.05 GiB (100%)
Title                                    : VideoHandle
Language                                 : English
Codec configuration box                  : avcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 7 min 1 s
Bit rate mode                            : Constant
Bit rate                                 : 256 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 12.9 MiB (0%)
Title                                    : SoundHandle
Language                                 : English
Default                                  : Yes
Alternate group                          : 1

Comments

sakendrick wrote on 9/10/2021, 11:11 AM

one thing I realized I could do... is skip my step of converting to CFR and do this when I do my denoising and sharpening. Same question stands on best practices, file format to use, and if this is common practice...

Musicvid wrote on 9/10/2021, 12:51 PM

The source video had VFR but I converted to CFR using FFMPEG scripts - here is the result:

That alone is a big limiting factor. It's a long-GOP lossy recode, definitely not x264, so I suspect a hardware encoder?

Either way, that would be the source of shadow noise not associated with your original VFR file. Let's start there and get you the best intermediate, not a dubious third generation recode. Post those MediaInfo properties and we'll help you.

sakendrick wrote on 9/10/2021, 2:00 PM

Here's my original source. I have it in a project and got some Neat, contrast, and sharpening settings that look a lot better (I'm no expert)... I started to render using ProRes 422 but it was going to take 10 hours... I abandoned for now and looking at Voukoder - not sure if it will help with ProRes rendering, but seems to offer other lossless options. Let me know what you think! working on my next music video :)

----------

General
Complete name                            : C:\Users\scott.kendrick\Documents\Video_local\KPLG\RWTC\20210620_144221.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/mp42)
File size                                : 1.39 GiB
Duration                                 : 7 min 1 s
Overall bit rate                         : 28.2 Mb/s
Encoded date                             : UTC 2021-06-20 18:49:24
Tagged date                              : UTC 2021-06-20 18:49:24
com.android.version                      : 11

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main@L5.1@Main
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 7 min 1 s
Source duration                          : 7 min 1 s
Bit rate                                 : 28.0 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 30.000 FPS
Minimum frame rate                       : 15.005 FPS
Maximum frame rate                       : 42.898 FPS
Standard                                 : PAL
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 0.112
Stream size                              : 1.37 GiB (99%)
Source stream size                       : 1.37 GiB (99%)
Title                                    : VideoHandle
Language                                 : English
Encoded date                             : UTC 2021-06-20 18:49:24
Tagged date                              : UTC 2021-06-20 18:49:24
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709
mdhd_Duration                            : 421651
Codec configuration box                  : hvcC

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : mp4a-40-2
Duration                                 : 7 min 1 s
Bit rate mode                            : Constant
Bit rate                                 : 256 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Stream size                              : 12.9 MiB (1%)
Title                                    : SoundHandle
Language                                 : English
Encoded date                             : UTC 2021-06-20 18:49:24
Tagged date                              : UTC 2021-06-20 18:49:24

Musicvid wrote on 9/10/2021, 3:37 PM

You can have best quality, best speed, or best file size, but not all three.

Where is your ProRes intermediate going to, is it Final Cut?

sakendrick wrote on 9/10/2021, 3:59 PM

No... Back into Vegas for editing the full composition with other videos

sakendrick wrote on 9/10/2021, 4:11 PM

will add... that I just started doing my editing with proxy files (which is fantastic for all the years I suffered with laggy playback)... so while I will pull back into vegas, I'll create proxy files to speed up the editing process, swap out for final rendering.

RogerS wrote on 9/10/2021, 8:56 PM

ProRes 422 is a fine choice for intermediates and plays back well in Vegas.

I would skip sharpening (or do it within NeatVideo) and save some time.

sakendrick wrote on 9/11/2021, 3:14 AM

I finished rendering after 10 hours and it won't open in anything including vegas... perhaps I shouldn't have used Voukoder?

resulting file

---------------

General
Unique ID                                : 268429716404066350039999714597958925697 (0xC9F1AED121B71AC9F36BD214D1193D81)
Complete name                            : C:\Users\scott.kendrick\Documents\Video_local\KPLG\RWTC\SCOTT_CFR\20210620_144221.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 45.5 GiB
Duration                                 : 7 min 2 s
Overall bit rate                         : 927 Mb/s
Writing application                      : Voukoder (VEGAS)
Writing library                          : Lavf58.76.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : ProRes
Codec ID                                 : V_PRORES
Duration                                 : 7 min 2 s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Default                                  : Yes
Forced                                   : No

Audio
ID                                       : 2
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 7 min 2 s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : 23 ms
Default                                  : Yes
Forced                                   : No

Marco. wrote on 9/11/2021, 4:14 AM

Why don't you use VP's core ProRes encoder? Why do you wrap ProRes into a MKV container instead of MOV? Why do you use AAC audio along with ProRes video instead of PCM audio? You wildly mix up media properties which are not meant to be used this way. It's no surprise the resulting file won't work.

RogerS wrote on 9/11/2021, 5:14 AM

What Marco said- stick to Apple ProRes in Vegas. If you want to use Voukoder, pay careful attention to settings and try rendering a few seconds and test before doing the whole project!

Still baffled as to why a 7 min. clip takes 10 hours to render... how slow is this CPU?

Photo_G wrote on 9/11/2021, 4:45 PM

Why don't you use VP's core ProRes encoder? Why do you wrap ProRes into a MKV container instead of MOV? Why do you use AAC audio along with ProRes video instead of PCM audio?

@sakendrick, these are the Voukoder defaults for ProRes KS, but you can change them to MOV and PCM in the codec dialog. Are you using VEGAS Pro v19?

sakendrick wrote on 9/11/2021, 9:59 PM

Yes, I didn't choose to use mkv, aac etc... Those were just the defaults for voukoder for proRes. So I went back to VP default ProRes (I was using Voukoder hoping it would be faster - it wasn't). I changed my Neat quality settings from high to normal. That cut my 10 hour rendering time to about 5 or 6 hours. I pulled the resulting mov back into my project and it looked good and rendered fine. So I've been rendering the rest of my source files to the ProRes intermediate files. I think I'm in good shape. Thanks

sakendrick wrote on 9/11/2021, 10:04 PM

@RogerS, re the render time, I have no idea why it takes so long. I'm certain it's the deNoising, and as mentioned in the above, changing quality to Normal cut it almost in half but still, why 7min takes 5 or 6 hours? Would love to know. Here's my specs attached

 

RogerS wrote on 9/11/2021, 11:42 PM

That's still too slow!

I find NeatVideo fairly quick for denoising assuming it is finding your GPU and using it okay.

This page suggests the laptop may use the NVIDIA® Quadro P2000?
https://www.dell.com/en-us/work/shop/dell-laptops-and-notebooks/precision-5530-mobile-workstation/spd/precision-15-5530-laptop

Since you're in VP 18 or 19, can you first go to help/driver update and see if any GPU updates are suggested for NVIDIA or the integrated GPU? If so, please install.

Second is Dell states its "compact design fits a 15.6-inch display into a 14-inch chassis." Well, that could be a problem. While it boasts revolutionary thermals it looks pretty standard. If the VRMs aren't under the heatsink as with other Dells of the era, it could be throttling after a short period of time and killing performance.

To see if this is the case you can watch the CPU GHz and if it starts plummeting into the render you know that's the issue. Its base clock appears to be 2.6 GHz, so if it falls under that it's throttling.

A better tool to diagnose and mitigate the issue is Throttlestop. As soon as you get power limit throttling or other throttling it will tell you. You can also use the settings to automatically disable turbo after you hit a certain CPU or GPU temp, for example to mitigate the issue for long stretches of high GPU and CPU activity. It's free and safe. https://www.ultrabookreview.com/31385-the-throttlestop-guide/

Last changed by RogerS on 9/11/2021, 11:48 PM, changed a total of 1 times.

Custom PC (2022) Intel i5-13600K with UHD 770 iGPU with latest driver, MSI z690 Tomahawk motherboard, 64GB Corsair DDR5 5200 ram, NVIDIA 2080 Super (8GB) with latest studio driver, 2TB Hynix P41 SSD and 2TB Samsung 980 Pro cache drive, Windows 11 Pro 64 bit https://pcpartpicker.com/b/rZ9NnQ

ASUS Zenbook Pro 14 Intel i9-13900H with Intel graphics iGPU with latest ASUS driver, NVIDIA 4060 (8GB) with latest studio driver, 48GB system ram, Windows 11 Home, 1TB Samsung SSD.

VEGAS Pro 21.208
VEGAS Pro 22.250

Try the
VEGAS 4K "sample project" benchmark (works with VP 16+): https://forms.gle/ypyrrbUghEiaf2aC7
VEGAS Pro 20 "Ad" benchmark (works with VP 20+): https://forms.gle/eErJTR87K2bbJc4Q7

RogerS wrote on 9/11/2021, 11:51 PM

Also try going into the NeatVideo Fx and going to preferences/performance and click "optimize settings". Accept and apply its recommendation.

sakendrick wrote on 9/12/2021, 8:11 AM

I don't have any more intermediate files to render at this time but will play with this when I get a chance. Yes the GPU drivers are up to date. I was rendering some proxy files and observed this for CPU usage. But this be indicative of throttling and the possible cause for long renders?

RogerS wrote on 9/12/2021, 8:21 AM

Maybe, but is this time period just 60 seconds? At the end where it spikes to 100% and then plummets is more likely throttling, but as I mentioned above the processor GHz frequency is more useful as a signal it's cutting performance to reduce heat.

You can also watch Throttlestop (without changing any settings) or HWINFO64 to see if any throttling is occurring. Both programs are free.

sakendrick wrote on 9/12/2021, 8:22 AM

after a while it cut down to around 50% cpu usage

RogerS wrote on 9/12/2021, 8:35 AM

Utilization could just mean the GPU is doing more work, but look at the speed- 1.50 GHz while 2.59GHz is the base speed. Yeah, that will slow things down. Better than my laptop which ends up at 800MHz, though!

I highly recommend reading the page on Throttlestop and going down the path of monitoring and controlling temperatures.

I'd start by turning off turbo clocks when doing such renders (you won't remain at turbo speeds for long anyway) and you may need to downgrade to an older bios to get undervolting to work, which is very effective at reducing heat build-up. My own Dell XPS 15 is at -135MV with turbo off for my work in Vegas. I also ended up removing and cleaning the fans, bridging the VRMs to the heat sink using thermal pads and other simple mods.

Musicvid wrote on 9/12/2021, 9:56 AM

File size                                : 45.5 GiB
Duration                                 : 7 min 2 s
Overall bit rate                         : 927 Mb/s

Of ccourse your render time will be long, especially with 927 Mbps bitrate and 45.5GB file size.

And of course it won't play.

Your source is 28 Mbps and 1.39 GB. You will not gain one iota of quality by such overkill, if that was your notion.

Keep a sane bitrate, and stick to the Vegas ProRes stock templates, and don't let your expectations rule your thinking.. It takes years of experience to be able to adjust Voukoder's parameters sensibly.

/Welcome to the forums.

https://www.vegascreativesoftware.info/us/forum/speaking-good-video-a-beginner-s-guide--104463/

@sakendrick @RogerS

Howard-Vigorita wrote on 9/12/2021, 7:01 PM

Just did some quality and performance testing to see if I could do better than ProRes as an Intermediate. I compared ProRes with Voukoder x264 lossless. ProRes rendering was the fastest of the two on my 9900k machine. But x264 lossless yielded substantially higher quality. Note that neither of these are anywhere near lossless and this was only true if none of the legacy boxes were checked off on the file I/O page. I would have loved to try hevc lossless as an Intermediate but Vegas cannot do that. Ffmpeg can do it and it is in fact truly lossless. I've dropped a note to the Voukoder developer on that so perhaps it'll appear in an update some day.

But checking off the legacy decoding boxes changes everything. x264 vmaf quality jumps from the 64's to the 68's. But ProRes quality jumps from the 51's to the 97's which is fairly close to lossless. Render times for both are somewhat slower but ProRes is still the faster of the two.

My testing was done with an Intel 9900k but I think similar results would obtain on a Ryzen. My theory is that the legacy checkbox enables an internal cpu-based decoding runtime which seems pretty smart yielding identical quality results with or without an Intel igpu. I did the quality measurements with ffmetrics using a losslessly compressed 4K 10-bit test clip but an uncompressed avi would probably tell a similar story.

RogerS wrote on 9/12/2021, 7:18 PM

Curious-does MagicYUV perform similarly, Howard?

Also, which ProRes are you testing here? 422?

RogerS wrote on 9/12/2021, 7:22 PM

Of ccourse your render time will be long, especially with 927 Mbps bitrate and 45.5GB file size.

And of course it won't play.

Your source is 28 Mbps and 1.39 GB. You will not gain one iota of quality by such overkill, if that was your notion.

Keep a sane bitrate, and stick to the Vegas ProRes stock templates, and don't let your expectations rule your thinking.. It takes years of experience to be able to adjust Voukoder's parameters sensibly.

/Welcome to the forums.

https://www.vegascreativesoftware.info/us/forum/speaking-good-video-a-beginner-s-guide--104463/

Note that 10 hours was also the estimate for stock ProRes 422 before the attempt to switch to Voukoder to save time, according to an earlier post.

Last changed by RogerS on 9/12/2021, 10:23 PM, changed a total of 1 times.

Custom PC (2022) Intel i5-13600K with UHD 770 iGPU with latest driver, MSI z690 Tomahawk motherboard, 64GB Corsair DDR5 5200 ram, NVIDIA 2080 Super (8GB) with latest studio driver, 2TB Hynix P41 SSD and 2TB Samsung 980 Pro cache drive, Windows 11 Pro 64 bit https://pcpartpicker.com/b/rZ9NnQ

ASUS Zenbook Pro 14 Intel i9-13900H with Intel graphics iGPU with latest ASUS driver, NVIDIA 4060 (8GB) with latest studio driver, 48GB system ram, Windows 11 Home, 1TB Samsung SSD.

VEGAS Pro 21.208
VEGAS Pro 22.250

Try the
VEGAS 4K "sample project" benchmark (works with VP 16+): https://forms.gle/ypyrrbUghEiaf2aC7
VEGAS Pro 20 "Ad" benchmark (works with VP 20+): https://forms.gle/eErJTR87K2bbJc4Q7

Howard-Vigorita wrote on 9/12/2021, 7:59 PM

Curious-does MagicYUV perform similarly, Howard?

Also, which ProRes are you testing here? 422?

Funny you asked. 😀 I just tried that too. I did ProRes 422 and ProRes422hq but came up with negligible quality difference between the two. MagicYUV was marginally higher quality but the air's pretty thin up there and ProRes422 is a fair amount smaller. Here's a google sheet with all the details:

https://docs.google.com/spreadsheets/d/1EY4lDiq7AxQ3aarRNurz6hqqHRubL3n-bM_CM7YqPrU/edit?usp=sharing