Rendering issues after GPU replacement (NVENC)

Rifane wrote on 12/6/2025, 2:19 PM

Hi everyone,
Well, I recently upgraded my GPU! I previously had a GTX 1060 and now I have an RTX 3060, but since the upgrade, whenever I try to render a video using Nvenc, I get this error: "An error occurred while creating the media file Untitled.mp4. Error 0x80660008 (message missing)"

I've already removed the old GPU drivers using DDU, changed the Nvidia drivers to the studio version, uninstalled Vegas, and still get the error. I tested it on V17, which is still on my PC, and I get the same error there. Does anyone have a solution?

Comments

RogerS wrote on 12/6/2025, 8:44 PM

Does resetting VEGAS help? https://www.vegascreativesoftware.info/us/forum/faq-how-can-i-reset-vegas-pro-to-default-settings--104646/

Which versions of VEGAS are you using? 17 and ?

Rifane wrote on 12/6/2025, 9:22 PM

Hi @RogerS! Thank you so much for your help! Sorry, I forgot to mention, but I'm using V22 and yes, I reset Vegas, I tried practically everything I found on the internet, chatgpt also helped and nothing worked. I ended up opting to update my Windows to 11 and installed V23 and it worked perfectly.

RogerS wrote on 12/6/2025, 10:38 PM

Strange, that sounds like maybe a Windows install issue. I'm glad you got it to work.

walter-i. wrote on 12/7/2025, 1:30 PM

I ended up opting to update my Windows to 11 and installed V23 and it worked perfectly.

@Rifane
I would appreciate hearing about your experience and how much the change has benefited you, as I am currently considering taking a similar step. Namely, upgrading to Win11 and changing my graphics card from Nvidia GTX 1660 Super to RTX 3060 (possibly TI?).  
Walter

vegas-edit-user wrote on 12/7/2025, 1:44 PM

Most likely, the problem is with Nvidia's 591.44 driver, which means that AVC and HEVC can no longer be encoded with GPU acceleration in Vegas 22 and earlier versions.

The workaround for now is to stick with driver 581...

Vegas 23 and editing programs from other manufacturers work with the new driver.

BruceUSA wrote on 12/7/2025, 2:34 PM

Most likely, the problem is with Nvidia's 591.44 driver, which means that AVC and HEVC can no longer be encoded with GPU acceleration in Vegas 22 and earlier versions.

The workaround for now is to stick with driver 581...

Vegas 23 and editing programs from other manufacturers work with the new driver.

Agreed, you will get erorr message with 591.44 driver. But if you use Voukoder or HOS its will work fine without issue.

Last changed by BruceUSA on 12/7/2025, 2:36 PM, changed a total of 1 times.

CPU:  i9 Core Ultra 285K OCed @5.6Ghz  
MBO: MSI Z890 MEG ACE Gaming Wifi 7 10G Super Lan, thunderbolt 4
RAM: 48GB RGB DDR5 8200mhz
GPU: NVidia RTX 5080 16GB Triple fan OCed 3100mhz, Bandwidth 1152 GB/s     
NVMe: 2TB T705 Gen5 OS, 4TB Gen4 storage
MSI PSU 1250W. OS: Windows 11 Pro. Custom built hard tube watercooling

 

                                   

                 

               

 

vegas-edit-user wrote on 12/7/2025, 2:51 PM

Okay, so this increasingly points to an NVENC implementation problem with the Magix encoders for AVC and HEVC. (Except for Vegas Pro 23)
See link point 2.6 - Discontinued Support:

https://us.download.nvidia.com/Windows/591.44/591.44-win10-win11-nsd-release-notes.pdf

andyrpsmith wrote on 12/7/2025, 3:34 PM

Yes I get same error, I agree it is likely a driver issue. I'm running the 591.44 driver. I will now try the AV1 codec with HOS to see if that is affected.

(Intel 3rd gen i5@4.1GHz, 32GB RAM, SSD, 1080Ti GPU, Windows 10) Not now used with Vegas.

13th gen i913900K - water cooled, 96GB RAM, 4TB M2 drive, 4TB games SSD, 2TB video SSD, GPU RTX 4080 Super, Windows 11 pro

andyrpsmith wrote on 12/7/2025, 3:43 PM

No issue with HOS Render+ AV1, no problem with V23 AVC NVENC. So seems to affect V22.

(Intel 3rd gen i5@4.1GHz, 32GB RAM, SSD, 1080Ti GPU, Windows 10) Not now used with Vegas.

13th gen i913900K - water cooled, 96GB RAM, 4TB M2 drive, 4TB games SSD, 2TB video SSD, GPU RTX 4080 Super, Windows 11 pro

EricLNZ wrote on 12/7/2025, 3:48 PM

@vegas-edit-user Thanks for the heads up as I'd just updated my driver to 591.44 and can confirm VP21 is affected. Fortunately I prefer rendering to avc with the Sony AVC templates.

As a user I find it annoying when something that worked ceases to work.

EricLNZ wrote on 12/7/2025, 4:13 PM

With VP21 it also affects Magix mp4 renders using NVE.

vegas-edit-user wrote on 12/7/2025, 4:32 PM

A user with Vegas 15 also posted error 0x80660008. I think there is a problem with the encoder presets for Magix AVC NVENC and HEVC NVENC.
AV1 is newer and was added most recently, and Vegas 23 is also relatively new.

I hope it's not because there's an old video codec SDK or something like that in the affected Magix programs, which according to Nvidia had to be adjusted. If that were the case, wouldn't all Vegas versions below 23 have to be updated in order to use drivers above 581.xx?

https://us.download.nvidia.com/Windows/591.44/591.44-win10-win11-nsd-release-notes.pdf
(2.6 - Discontinued Support)

https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-preset-migration-guide/index.html

Rifane wrote on 12/7/2025, 7:34 PM

I ended up opting to update my Windows to 11 and installed V23 and it worked perfectly.

@Rifane
I would appreciate hearing about your experience and how much the change has benefited you, as I am currently considering taking a similar step. Namely, upgrading to Win11 and changing my graphics card from Nvidia GTX 1660 Super to RTX 3060 (possibly TI?).  
Walter

Although I'm experiencing some freezes in V23, I'm really enjoying the experience! You feel that everything runs smoother; visually the difference is minimal, so I wasn't too surprised. What bothered me initially was the fact that the space bar didn't work as play/stop like it did in V22, but I was able to easily fix that. Otherwise, it's practically all the same. Regarding Windows 11, I'm also enjoying it, although some changes bother me a little, but it's easy to get used to. Overall, I think it was worth it.

andyrpsmith wrote on 12/8/2025, 4:05 AM

A user with Vegas 15 also posted error 0x80660008. I think there is a problem with the encoder presets for Magix AVC NVENC and HEVC NVENC.
AV1 is newer and was added most recently, and Vegas 23 is also relatively new.

I hope it's not because there's an old video codec SDK or something like that in the affected Magix programs, which according to Nvidia had to be adjusted. If that were the case, wouldn't all Vegas versions below 23 have to be updated in order to use drivers above 581.xx?

https://us.download.nvidia.com/Windows/591.44/591.44-win10-win11-nsd-release-notes.pdf
(2.6 - Discontinued Support)

https://docs.nvidia.com/video-technologies/video-codec-sdk/12.1/nvenc-preset-migration-guide/index.html

A note about AV1. AV1 is an open source replacement for HEVC and has been around for some years. The implementation in Vegas certainly in V22 has been a bit patchy with artifacts showing when compared with the now depreciated free Voukoder. Vegas claim to have improved the implementation in V23, however I have experienced issues with the template in crashing/not being able to render if choosing certain bit rates. Additionally Magix have not provided GPU encoding in their template so it renders much slower with CPU. To get round this I use Happy Otter Render+ which provides a much better experience also allowing a choice of audio codec other than AAC.

seetharamaiah-velicheti wrote on 12/8/2025, 8:46 AM

Friends, the problem got solved. Here it goes how. I have RTX-3060 Graphics Card on my PC. After reinstallation of Windows 11 Pro, I naturally found that a Graphics Driver 560.94 was already installed by the Windows. Now, after downloading NVIDIA App it was showing a latest Driver No.591.44. While installing two options were shown. 1. Keep the old driver's settings and update, 2. Clean install. I clicked clean install in NVIDIA APP and it completely uninstalled the previous driver 560.94 installed by OS and clean-installed its latest driver 591.44. This latest driver did not match with Vegas Pro render process. Then I uninstalled NVIDA App and all apps installed by NVIDIA, disk-cleaned the 'C', deleted all temporary files, restarted the PC which freshly installed the original driver 560.94. Now Vegas Pro works fine. Afterwords when I install NVIDIA app again, I will only click on the option "Keep the old settings and update". Thankyou.

Howard-Vigorita wrote on 12/8/2025, 12:55 PM

Just had the same problem with vp22 b250 after I updated to 591.44. Btw, all I had to do to recover was to run the previous v581.57 msi which left the newer nvidia app and settings in place.

RogerS wrote on 12/9/2025, 12:38 AM

One developer suggests a cause here: https://forums.developer.nvidia.com/t/drivers-591-44-broke-nvenc-getencodepresetconfig-no-longer-works/353613/2

Howard-Vigorita wrote on 12/9/2025, 11:09 AM

I'm suspecting Nvidia removed or renamed a driver function of long standing which causes all older implementations that call it to fail

Harons wrote on 12/10/2025, 4:26 AM

Hi everyone,
Well, I recently upgraded my GPU! I previously had a GTX 1060 and now I have an RTX 3060, but since the upgrade, whenever I try to render a video using Nvenc, I get this error: "An error occurred while creating the media file Untitled.mp4. Error 0x80660008 (message missing)"

I've already removed the old GPU drivers using DDU, changed the Nvidia drivers to the studio version, uninstalled Vegas, and still get the error. I tested it on V17, which is still on my PC, and I get the same error there. Does anyone have a solution?


 

Yes, the issue with the new 591.44 drivers dated December 4, 2025 is confirmed — I tested it on two graphics cards: the RTX 4070 Ti and the RTX 5080.
If you are experiencing the same problem, uninstall the drivers and install version 581.57 from October 14, 2025.

You can download the drivers here:
https://www.nvidia.com/en-us/drivers/results/

Operating system - Windows 11 Pro
AMD Ryzen 9 7900X3D Processor
Gigabyte B650 AORUS ELITE AX motherboard
RAM ADATA XPG Lancer 64 Gb DDR5 6000MHz
Video card - Palit GeForce RTX 4070 Ti GameRock Classic
Power Supply - Seasonic Focus GX-850 80+ Gold
Dropsy - NZXT Kraken X73
Housing - LIAN LI O11D Dynamic Evo
SSD: 2000 GB M.2 ADATA LEGEND 960 MAX
SSD - 500 GB M.2 Samsung 970 EVO

diamond204 wrote on 12/14/2025, 8:28 PM

Having same issue (the error with the message missing). Tried a clean install of 22 and no luck. Also deactivating GPU in the settings has zero effect. It seems the new branch of drivers break the NVencoder used in perhaps all older versions of Vegas which IMO is a major deal. I rolled back to October 2015 drivers and can render again.

But the big question is this an issue Nvidia will fix or is Magix able to patch or are we stuck in a bad place for the future?

RogerS wrote on 12/14/2025, 8:44 PM

@diamond204 It's an issue for NVIDIA to fix as they broke a few applications use of NV Encoder. VP 23 is working with the latest driver but not others. I'd complain to NVIDIA as there's no way Magix will reopen VP 15-22 to patch them all.

VEGASDerek wrote on 12/15/2025, 11:10 AM

Ideally this would be something Nvidia will address (as there are other applications out there that are experiencing similar issues). We are analyzing if we can do anything to address this, but it looks like it would be several months of work to address the problem and properly test the solution. The new Vegas video engine uses the newest Nvidia SDK which is not dependent on a specific property that the latest Nvidia drivers have removed. Any application using this property will have problems using NVENC right now. The uplift to use a new version of the Nvidia SDK in older versions of Vegas is not trivial as the old rendering engine is using the now unavailable property.

diamond204 wrote on 12/15/2025, 6:33 PM

I sent a report to Nvidia and I hope others will do the same.

 

I can also add that Driver version 581.80 is the last version that currently works. Users of 5000 series cards will want to download this particular version of the driver as it added big updates with a new version of CUDA that works on the 5000 series.

RogerS wrote on 12/15/2025, 7:14 PM

Looking at the release notes the CUDA update to 13 was done with the 580 series so studio driver 581.57 should also be fine (581.80 was a gaming driver).