Unable to render HEVC using NVENC Lossless Preset (VP 18)

john-w2170 wrote on 9/26/2020, 5:56 AM

As per title, on Vegas Pro 18 (build 334), I cannot render a HEVC MP4 using the NVENC encoder, when the encoder preset is set to "Lossless" or "Lossless - High Performance".

The error message that comes up immediately upon the render starting states:

"An error occurred while creating the media file [FILENAME].

Error 0x80660008 (message missing)".

This error code appears in one other topic on this forum: https://www.vegascreativesoftware.info/us/forum/can-t-render-fullhd-50i-footage-with-nv-encoder-in-vegas-16-pro--115337/

Rendering an AVC MP4 (using the "Lossless" NVENC preset) works.

Rendering HEVC under any other preset also works.

I'm almost certain that it was an NVIDIA driver update that caused this problem to start. My current driver is 456.38 (the lastest one). I've rolled back the driver, reinstalled VP and reinstalled my OS to no avail.

My GPU is an NVIDIA GTX 1060 Mobile.

My OS is Win10 Home, ver.2004.

The footage being used is recorded gameplay from OBS.

I will try rolling back to a recent driver and a much older one to see if the problem persists.

Can anyone help?

Comments

Musicvid wrote on 9/26/2020, 8:23 AM

Are you sure to install the Nvidia Studio driver as recommended, not the Gaming one?

j-v wrote on 9/26/2020, 8:38 AM

I'm also seeing it, but what might be the sense of that setting. FHD files of max 135 MB/s bitrate with a HEVC codec?

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 24H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver 566.14 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2130
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 576.02 and Intel UHD Graphics 630 with driver 31.0.101.2130
Vegas software: VP 10 to 22 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)

 

JN- wrote on 9/26/2020, 9:57 AM

@john-w2170 This may not address your immediate issue, but the vp lossless hw encoder has had issues with corruption. If you do get it working do play the rendered video to make sure that its ok.

---------------------------------------------

VFR2CFR, Variable frame rate to Constant frame rate link to zip here.

Copies Video Converts Audio to AAC, link to zip here.

Convert 2 Lossless, link to ZIP here.

Convert Odd 2 Even (frame size), link to ZIP here

Benchmarking Continued thread + link to zip here

Codec Render Quality tables zip

---------------------------------------------

PC ... Corsair case, own build ...

CPU .. i9 9900K, iGpu UHD 630

Memory .. 32GB DDR4

Graphics card .. MSI RTX 2080 ti

Graphics driver .. latest studio

PSU .. Corsair 850i

Mboard .. Asus Z390 Code

 

Laptop… XMG

i9-11900k, iGpu n/a

Memory 64GB DDR4

Graphics card … Laptop RTX 3080

john-w6850 wrote on 9/26/2020, 10:39 AM

Are you sure to install the Nvidia Studio driver as recommended, not the Gaming one?


The lossless preset has worked on previous Gaming drivers. But it stopped working, I believe, when I updated to the latest one.

I've rolled my Nvidia driver back to 388.16. This is the (now) two year old driver that shipped with my laptop. This, for some reason, has fixed the problem. Vegas is again able to render HEVC, with NVENC's lossless - high performance preset.

But I don't want to be stuck using an old driver. I'll try updating to the latest studio driver and see what happens.

@john-w2170 This may not address your immediate issue, but the vp lossless hw encoder has had issues with corruption. If you do get it working do play the rendered video to make sure that its ok.


I've done so on my latest render. It came out OK, no issues.

I'm also seeing it, but what might be the sense of that setting. FHD files of max 135 MB/s bitrate with a HEVC codec?


Do you mean you are seeing the same issue?

j-v wrote on 9/26/2020, 11:03 AM

Do you mean you are seeing the same issue?

On my laptop yess

 

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 24H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver 566.14 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2130
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 576.02 and Intel UHD Graphics 630 with driver 31.0.101.2130
Vegas software: VP 10 to 22 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)

 

wwaag wrote on 9/26/2020, 11:21 AM

Even more importantly, why would you want to use lossless HEVC? HEVC is not designed to be used as a digital intermediate due to the increased demands for proper decoding when using Vegas. HEVC is best used as a delivery format. AVC (which apparently works OK) is a much better editing format for use in Vegas.

Last changed by wwaag on 9/26/2020, 11:21 AM, changed a total of 1 times.

AKA the HappyOtter at https://tools4vegas.com/. System 1: Intel i7-8700k with HD 630 graphics plus an Nvidia RTX4070 graphics card. System 2: Intel i7-3770k with HD 4000 graphics plus an AMD RX550 graphics card. System 3: Laptop. Dell Inspiron Plus 16. Intel i7-11800H, Intel Graphics. Current cameras include Panasonic FZ2500, GoPro Hero11 and Hero8 Black plus a myriad of smartPhone, pocket cameras, video cameras and film cameras going back to the original Nikon S.

john-w6850 wrote on 9/26/2020, 11:36 AM

Even more importantly, why would you want to use lossless HEVC? HEVC is not designed to be used as a digital intermediate due to the increased demands for proper decoding when using Vegas. HEVC is best used as a delivery format. AVC (which apparently works OK) is a much better editing format for use in Vegas.

I am using it as a delivery format. Not sure what you mean, but in any case this would probably be suited better to another thread. Have you any thoughts on the issue in the OP?

Musicvid wrote on 9/26/2020, 12:14 PM

Even more importantly, why would you want to use lossless HEVC? HEVC is not designed to be used as a digital intermediate due to the increased demands for proper decoding when using Vegas. HEVC is best used as a delivery format. AVC (which apparently works OK) is a much better editing format for use in Vegas.

+1.

I am using it as a delivery format. Not sure what you mean, but in any case this would probably be suited better to another thread. Have you any thoughts on the issue in the OP?

I wouldn't use a machine codec for delivery outside my own studio walls unless: a) the client needed it yesterday and was paying me double for the service; or b), the KGB was holding a gun to my head.

And to echo what we have known for 15 years, "Size, Quality, Speed. Pick two."

john-w6850 wrote on 9/26/2020, 2:23 PM
I wouldn't use a machine codec for delivery outside my own studio walls unless: a) the client needed it yesterday and was paying me double for the service; or b), the KGB was holding a gun to my head.

And to echo what we have known for 15 years, "Size, Quality, Speed. Pick two."

Fascinating. Again, not sure what this has to do with my specific, reproducible problem that I've detailed in the OP.

Musicvid wrote on 9/26/2020, 2:55 PM

I know, john, but those of us who place quality, rather than speed first in our work flows are understandably biased regarding machine encoders. The designation of "lossless" begs the question, "related to what?," and I haven't seen enough interest in the answer for someone to produce controlled, repeatable test data to back such an overreaching claim, especially in comparison to established software intermediate and delivery solutions. Certainly I don't.

Oh, by the way, what players and streaming delivery services have you tested that actually support "lossless" NVENC? Can you be specific and provide some examples that I can test?

Musicvid wrote on 9/26/2020, 3:40 PM

@MagicYUV

As I suspected, with version 2.3.0, I cannot encode 10 bit on my Vegas Pro 14, Are there any thoughts of extending the plugin compatibility backward to include the earlier Magix versions?

MagicYUV wrote on 9/26/2020, 5:47 PM

@Musicvid The codec installer only installs for VP17/18 currently. I don't know if the plugin binary interface changed or not between VP14 and VP17. If it didn't, it can work. If it did, it'll crash the program. For example, the plugin interface changed between VP17 and VP18, meaning the VP17 plugin doesn't work (will crash at some point) in VP18 and vice-versa.

I have a friend who still has VP14, I'll ask him to try it out for me and report back.

In the meantime, copy the following two files from the VP17 folder (by default at: "c:\Program Files\VEGAS\VEGAS Pro 17.0") to the VP14 folder (into same subfolder as it was in VP17):

magicyuvplug.x64.fio2007-config

FileIO Plug-Ins\magicyuvplug\magicyuvplug.dll

If it works, then we're happy.

(EDIT: Sure you wanted to post in this thread?)

Musicvid wrote on 9/26/2020, 6:07 PM

Ha, I was looking all over for my post that I'd thought I'd lost. Sorry, @john-w6850. If @vkmast can move these last three replies to one of the more appropriate discussions, I'm sure we'll all be much happier.

I believe @wwaag has made his extension available to all Magix Vegas Pro versions by way of a single common .dll. What relevance this might have to my question I of course do not know. Meanwhile, I will try your suggestion because I learn best from breaking stuff! Thanks again.

wwaag wrote on 9/26/2020, 6:59 PM

@MagicYUV

Works in Magix versions 14-16 plus Sony 13. Didn't try V12.

AKA the HappyOtter at https://tools4vegas.com/. System 1: Intel i7-8700k with HD 630 graphics plus an Nvidia RTX4070 graphics card. System 2: Intel i7-3770k with HD 4000 graphics plus an AMD RX550 graphics card. System 3: Laptop. Dell Inspiron Plus 16. Intel i7-11800H, Intel Graphics. Current cameras include Panasonic FZ2500, GoPro Hero11 and Hero8 Black plus a myriad of smartPhone, pocket cameras, video cameras and film cameras going back to the original Nikon S.

Musicvid wrote on 9/26/2020, 9:09 PM

@MagicYUV

I was able to obtain those two files by creating a new folder in Program Files, Vegas Pro 17.0, and a FileIOPlugins folder inside that. I then reinstalled the Magic 2.3.0 trial, and copied the two files over to my VP14 folders.

Unfortunately, it did not work with 10 bit, but still rendered with 8 bit. Oh well, it was just wishful thinking on my part, so don't belabor yourself. Thanks again for a great product, and I'll follow the development progress.

Musicvid wrote on 9/27/2020, 11:47 AM

This misplaced discussion about Magic YUV has been moved here:

https://www.vegascreativesoftware.info/us/forum/magicyuv-2-3-0-released-with-10-bit-plugin-for-vegas--124012/

john-w6850 wrote on 9/29/2020, 3:13 PM

To get back on topic, is the error code in the OP known to anyone?

wwaag wrote on 9/30/2020, 1:17 AM

I can confirm that it's a bug in V18. V17 works OK. You'll just have to wait till Magix fixes it in a future build.

AKA the HappyOtter at https://tools4vegas.com/. System 1: Intel i7-8700k with HD 630 graphics plus an Nvidia RTX4070 graphics card. System 2: Intel i7-3770k with HD 4000 graphics plus an AMD RX550 graphics card. System 3: Laptop. Dell Inspiron Plus 16. Intel i7-11800H, Intel Graphics. Current cameras include Panasonic FZ2500, GoPro Hero11 and Hero8 Black plus a myriad of smartPhone, pocket cameras, video cameras and film cameras going back to the original Nikon S.

john-w6850 wrote on 10/8/2020, 5:07 PM

Still broken with latest driver 456.71. Yes, it's probably due to V18. Are MAGIX aware? Should I report it?

VEGASHeman wrote on 10/9/2020, 6:22 PM

@john-w6850 : Thank you for bringing this to our notice, we will look into this and get back to you.