No idea what is wrong at you, can be hardware related, Windows version and -settings related and so on. This how a render ( as yours I prosume) goes with me on laptop with VPro 18. Processor is very high due to also making a screenvideo.
I assume you are using a GPU accelerated render template? Does it also do this with a CPU only render? Also try setting the number of rendering threads to either the number of CPU cores or twice that number if your CPU has hyperthreading.
This software declares GPU support, I will not use only CPU for rendering. I think it's a problem with supporting this format from the camera. Vegas often has a problem with some mp4 files, but I want to solve this. My customer will not wait half a year for an update, which may not even solve it. My OS is fine, the hardware is standard (i7 4790k, GTX 1080, 16GB RAM). God, in this case I'm rendering 720p, not 8K video.
I rendered this resolution just fine a few years ago with 6GB of RAM and Sony Vegas 12 😐
Vegas Pro 18 is great, but this is a weird problem.
I like to help you because on my laptop has approximately the same hardware with a little weaker GPU and a a little better CPU (7th generation i7) and there the software goes without problems. So I like to try a little project of you that has the error. Can you post such a veg included sourcefiles on a cloudserver we can download it with a link you give here? You may also send me that link on PM.
wrote on 10/28/2020, 9:08 PM
Try turning "GPU acceleration of video" OFF, when you go to encode. You can leave video decoding on, and still use NVENC or QSV hardware gpu encoding. See if that fixes the high memory fault
New info: For a project with my YI 4K + camera, I must to turn on the GPU on the processor, otherwise Vegas causes artifacts(black frames, blinking frames....), or it gets stuck and crash during editing.