Make sure you are using the latest version of Vegas 17 (build 387).

Make sure you have the latest Nvidia Studio driver installed.

I see you are using a customized render template, try to use one of the stock templates and see if the problem persists. If it does not, try to make a new custom template.

Also, what GPU are you using?

Nvidea GTX 1060 6gb

system settings in a screenshot from msinfo

problem still persists, I reinstalled my driver just before I made this post.

stock didnt work either.

Check your preferences. You have a 4 core CPU, each core can process 2 threads each. You've got Vegas set to throw 32 threads at once at that processor. Might want to dial that back. Try dialing it all the way back to 1 to see what happens, then dial up from there until it crashes again. Also, set the dynamic RAM preview to its stock 200. That too can cause issues.

Check your preferences. You have a 4 core CPU, each core can process 2 threads each. You've got Vegas set to throw 32 threads at once at that processor. Might want to dial that back. Try dialing it all the way back to 1 to see what happens, then dial up from there until it crashes again. Also, set the dynamic RAM preview to its stock 200. That too can cause issues.

tried it, had put off GPU acceleration and the GPU in pref as well, then it did render but didn't utilize the GPU.

Then when I put back on the GPU and Allow legacy rendering, it started looping again.

I also tried it in Vegas 16, same problem in both 17 and 16.

CPU threads didn't matter, it does render with 32 threads if I use AVC instead of the NV encoder.

Also tried it with 1 thread, still did happen.

Problem is with my GPU if NV encoder, I think

When rendering you don't use Dyn RAM Preview, so it can be set to 0.
On my laptop I see the same with that kind of template

while rendering the same to FHD gives no problem and also rendering to 4K NVENC HEVC no problem.
On my desktop with a bit other GPU I don't see those problems.
You can try to render a small section to FHD AVC 4K with NVENC or to HEVC 4K with NVENC.
See here my report of that issue of a few months ago: https://www.vegascreativesoftware.info/us/forum/vp-17-issue-still-only-on-laptop-again--117092/
Maybe an issue with the videocard?

When rendering you don't use Dyn RAM Preview, so it can be set to 0.
On my laptop I see the same with that kind of template

while rendering the same to FHD gives no problem and also rendering to 4K NVENC HEVC no problem.
On my desktop with a bit other GPU I don't see those problems.
You can try to render a small section to AVC 4K with NVENC or to HEVC 4K with NVENC.
See here my report of that issue of a few months ago: https://www.vegascreativesoftware.info/us/forum/vp-17-issue-still-only-on-laptop-again--117092/
Maybe an issue with the videocard?

tried it, at first it didn't work.

Then I removed the dyn ram preview(from both 16 and 17), restarted vegas 17, still not working.

Started vegas 16, edited one of my templates and it worked with the following settings:

I'm testing Vegas 17 now

** edit

vegas 17 appears not to work, this time its different.

found a simalir issue here

It appears to be something with Vegas 17 in regard to using the render template out of the Appdata folder.

Vegas 16 is working fine now.

Maybe it is better to stick to one issue, the one of your opening post and open another one for other issues.

Maybe it is better to stick to one issue, the one of your opening post and open another one for other issues.

Had the issue on both V16 and 17, 16 is solved now, but 17 still presists.

and yeah I removed the part that wasn't needed.

Did you try my proposal to render a small part to FHD with NVENC and the same to Magix 4K HEVC with NVENC ?
My solution that you could try if you have exact the same is to use the solution I give in the video of my report about it.
Maybe what I experienced is the same as you see, but you did not confirm that you saw the same things I saw and still see.

Did you try using the stock template in 17 again? I know when 17 launched, there was a bug that caused a crash if you used custom render templates that had been carried over from Vegas 16.

Did you try my proposal to render a small part to FHD with NVENC and the same to Magix 4K HEVC with NVENC ?
My solution that you could try if you have exact the same is to use the solution I give in the video of my report about it.
Maybe what I experienced is the same as you see, but you did not confirm that you saw the same things I saw and still see.

Both seem to work, both FHD and UHD Nvenc from MAGIX.

HEVC with FHD & UHD NVENC works too, but as soon as I use a custom template that I've got saved I get the following error.

Did you try using the stock template in 17 again? I know when 17 launched, there was a bug that caused a crash if you used custom render templates that had been carried over from Vegas 16.

Yeah I did, stock works, but as soon a custom one is used that I did save on Vegas 17, I get the above error.

However, when I use a custom one that I did not save, then it renders out.

At least I can now use a custom one, just cannot save the template.

Than I cannot help you further because my problems were different than yours.
I get your first error message by rendering at a Magix AVC default UDH template with NVENC. Your last error message is totally different and thus another problem.