Upscale Fx causes freeze when changing preview quality?

andrewcg wrote on 2/17/2024, 2:25 PM

Hi, do you guys have any issue cropping with Upscale Fx in 4K project and changing the preview quality from preview to good/best? My system always freezes at that point. I use Neat Video and I believe that it automatically changes the preview to good/best so it always freezes for me. I'm using 4K H.264 29.97 fps source media and my specs are in signature.

Comments

mark-y wrote on 2/17/2024, 4:23 PM

It seems a bit counterintutive to use both at the same time; does it still freeze if you eliminate Neat from the filter chain?

Neat is called a kernel level filter, which are about the most computationally intensive out there -- except for AI modeling, such as Vegas Upscale, which has its own denoising.

Instead of Neat, you can apply a super light Unsharp Mask before the Upscale in Media Fx, and accomplish about the same thing -- at a far lower overload on your system resources.

Former user wrote on 2/17/2024, 5:39 PM

It looks to be a bug, Vegas eats up all your ram, crashes it'self and stalls or perhaps even BSOD's your computer, as Windows will expect to have ram, but there is none. This is true for you and me who only have 32GB of ram, it may not happen if you have 64GB or more, not sure. The performance will absolutely tank if Vegas doesn't crash because Vegas is allocating too much data to VRAM, it then leaks out into system memory and that's noy how GPU's are supposed to operate (especially Nvidia GPU's)

This occurred with the upscale settings sharp and denoise, it did not happen with smooth

andrewcg wrote on 2/17/2024, 7:08 PM

Neat is called a kernel level filter, which are about the most computationally intensive out there -- except for AI modeling, such as Vegas Upscale, which has its own denoising.

Thanks for the alternative suggestion. I was using the sharp model and didn't see apparent noise reduction happening so I used Neat as well. I haven't tried Upscale denoise model yet and how it compares to Neat.

This is true for you and me who only have 32GB of ram, it may not happen if you have 64GB or more, not sure. The performance will absolutely tank if Vegas doesn't crash because Vegas is allocating too much data to VRAM, it then leaks out into system memory and that's noy how GPU's are supposed to operate (especially Nvidia GPU's)

Interesting. One time I was able to get it to render with both Upscale sharp and Neat and got some blank frames. I checked my ram usage and noticed that the blank frames happened as my ram usage spiked from around 31GB to 32GB before dropping back down. I was thinking about getting more ram but didn't want to encounter the same issue even with the larger ram. I hope this issue can be fixed or maybe Vegas team can suggest ram amount to use Upscale sharp reliably? Is it true that the smooth mode doesn't use ai model?

andrewcg wrote on 2/17/2024, 7:18 PM

It seems a bit counterintutive to use both at the same time; does it still freeze if you eliminate Neat from the filter chain?

It didn't outright freeze when I removed Neat but the preview window was still unstable and gave me error code a few times.

Former user wrote on 2/17/2024, 7:40 PM

. I hope this issue can be fixed or maybe Vegas team can suggest ram amount to use Upscale sharp reliably? Is it true that the smooth mode doesn't use ai model?

 

I think it's using the AI models, but uses less Vram and processing power, and doesn't seem to do much. Some have suggested Smooth model plus sharpening such as Unsharp Mask that Mark_Y mentions may give results resembling sharp model but faster and use less memory. I haven't looked into that.

mark-y wrote on 2/17/2024, 8:34 PM

Thanks for the alternative suggestion. I was using the sharp model and didn't see apparent noise reduction happening so I used Neat as well. I haven't tried Upscale denoise model yet and how it compares to Neat.

The Sharp Model in Vegas Sharpen Fx, in and of itself, was unstable and unusable when it was first introduced. It has been changed, but is still profoundly system-intensive, and I am unable to use it on my i5 system.

That model is almost certainly the "1,000 pound elephant" in your room; again, I suggest respectfully that you start with Vegas Upscale (with Smooth, which is default model), preceded in the filter chain with a very conservative Unsharp Mask Fx, confirm that you have stable renders, examine your results, and add to them conservatively

https://www.vegascreativesoftware.info/us/forum/upscaling-sd-video--134650/#ca840174

I wouldn't mix conventional kernel filters and the Sharpen AI model under any circumstances.

Keep in mind that all Upscaling is a destructive process. This is as true with paid solutions like Topaz, as well as with Vegas.

You may be simply hoping for a little too much; despite the promises of AI. Despite everything you read and hear on the internet, AI is not Alchemy.

Big Note: The reason you aren't seeing the improved results you want with Upscale "may" be due to one of the following reasons:

  • GPU Acceleration of Video Processing must be turned ON. That is a specific setting in your Video Preferences, it is not trivial.
  • The Project Properties must be set to exactly match your new Output Dimensions, not your Source Dimensions.
  • The Upscale Effect must be applied at the Media Level, or at the Event Level, Pre-Pan/Crop only.
  • (If I've missed the mark for Vegas 21, or left something out, I welcome being corrected.)
  • If you overlook any of these steps, Vegas will default to "oldschool" bicubic scaling, which is not what you want. If you're not getting results like this from a 2.5x upscale from interlaced source, your "smart" upscale filter may be defaulting to "dumb"

We know that as time goes on, the developers will hone the "smart filter" process to make it more automatic, but right now it's kind of superimposed on the old technology that Vegas once used.

 

 

andrewcg wrote on 2/18/2024, 6:18 PM

I think it's using the AI models, but uses less Vram and processing power, and doesn't seem to do much.

Yes, I found similar result. I just tested cropping with Upscale smooth and regular Pan/Crop. Upscale only added ever so slight contrast but no new detail added at all.

You may be simply hoping for a little too much; despite the promises of AI. Despite everything you read and hear on the internet, AI is not Alchemy.

Yes I definitely did lol! Especially after seeing the stuffs that Stable Diffusion is doing with AI. I just want to be able to crop in without losing too much or at least better quality than bicubic. I tested cropping with Upscale smooth vs regular Pan/Crop and it didn't look like the sample above (with correct settings as in the big note). Was the test done as cropping in or upscaling the whole media?

The Sharp Model in Vegas Sharpen Fx, in and of itself, was unstable and unusable when it was first introduced. It has been changed, but is still profoundly system-intensive, and I am unable to use it on my i5 system.

Interesting, I didn't know that. I just noticed my ram usage was about 26 GB doing Upscale sharp crop render. I can't believe I added Neat on top of it. Thanks for the suggestions everyone. I got good info. Upscale sharp works ok when I upscale a video but I can't use it as a cropping tool at the moment. I definitely look forward to future improvement in crop/scale.

mark-y wrote on 2/18/2024, 7:10 PM

The biggest challenge of newer technologies is remembering to practice moderation, and to accept any improvement on its own merits, expectations notwithstanding.

Former user wrote on 2/18/2024, 10:40 PM

Upscale sharp works ok when I upscale a video but I can't use it as a cropping tool at the moment.

Good observation. The first run is upscaling 1080P video to 4K, 2nd is upscaling a 1080P crop of a 4K video to 4K. Although what i'm doing looks strange it does seem to accurately simulate what happens in an encode although memory use may build as the encode continues. This time the sharp crop only just remains within Vram and doesn't cause a problem.

 

 

 

andrewcg wrote on 2/18/2024, 11:46 PM

The biggest challenge of newer technologies is remembering to practice moderation, and to accept any improvement on its own merits, expectations notwithstanding.

Can't disagree with that. I was bit excited with the Upcale feature because that would mean I could get pristine 4K render out of reframed 4K source without having to invest in 8K cameras. Expectations now safely curbed lol.

Although what i'm doing looks strange it does seem to accurately simulate what happens in an encode although memory use may build as the encode continues.

Thanks for testing. Very interesting. I was only watching system ram. I thought Vram was ok cause GPU utilization was low but it apparently was also affected. Btw, when I used Upscale crop what I meant was using the "cut out" function within Upscale fx. Is that what you did as well in the 2nd video?

Former user wrote on 2/19/2024, 1:03 AM

Btw, when I used Upscale crop what I meant was using the "cut out" function within Upscale fx. Is that what you did as well in the 2nd video?

NO! and it turns out I was doing it the wrong way in Vegas, so the reason for the huge amount of extra VRAM use is because Vegas must have been manipulating 4K of data and not only the cropped 1080P

I tried your way (the correct way) and VRAM use was similar as the 1080P clip upscaled to 4K. Hopefully another user tests this. I am more use to Resolve, where you do just crop the video, and turn on the upscaler, the cropped version is using much higher processing power, and maybe Vegas gets around that by their cutout method.

Can compare Vram use between the 2, Vegas is on the high side currently.

mark-y wrote on 2/19/2024, 9:21 AM

The biggest challenge of newer technologies is remembering to practice moderation, and to accept any improvement on its own merits, expectations notwithstanding.

Can't disagree with that. I was bit excited with the Upcale feature because that would mean I could get pristine 4K render out of reframed 4K source without having to invest in 8K cameras. Expectations now safely curbed lol.

Heh, I taught in G/T/2E education for 26 years; it is an unbelievably difficult challenge to coax brilliant, unchanneled imaginations into the structures imposed by our universal physical confines. But when they get it, it's a lifechanger.

I like to start with the Laws of Conservation of Energy and Mass, and Newton's 1st and 2nd principles and expand from there.

I've even received threats from gamers and magical thinkers on the internet who claim to turn water into wine, lead into gold, and SD into HDR (and who, coincidentally, hate teachers).

Welcome to the discussions.

;?)

 

mark-y wrote on 2/19/2024, 9:47 AM

I was bit excited with the Upcale feature because that would mean I could get pristine 4K render out of reframed 4K source without having to invest in 8K cameras.

Still the best way to do that is shoot 8k and crop to 4k, or shoot 4k and crop to FHD.

The best that AI upscaling can do is fill the holes with made-up or contrived pixel blocks, so it is not a constructive process in the strictest sense, just a chance for a modest perceptual improvement over "blind" bicubic and bilinear algorithms of the past.

andrewcg wrote on 2/19/2024, 11:12 PM

Can compare Vram use between the 2, Vegas is on the high side currently.

Thanks again for testing in both Vegas and Resolve. I think a possible solution is for Vegas to not do AI processing until render time. That way the preview will not be bogged down when using cut out function in Upscale. Ram/Vram usage still needs to be addressed though cause it seems like Vegas will take all available ram to the point of crashing.

I've even received threats from gamers and magical thinkers on the internet who claim to turn water into wine, lead into gold, and SD into HDR (and who, coincidentally, hate teachers).

Welcome to the discussions.

Threats? Wow people can be very passionate on the internets. I definitely wish I learned about universal physical confines much earlier. It would help bring my gigantic illusion of grandeur down to earth a bit lol. I am a bit dissapointed I can't use Upscale fx sharp to reliably reframe 4K project but I have a workaround that still looks pretty good. I rendered the finished project as 1080p prores then used Upscale sharp to bring it back to 4K. It looked quite good and my system didn't explode.

Yes, 8K cameras would be great for work that need lots of cropping. I haven't looked at recent camera market. I think Canon dslr has 8K capability but it still very expensive and I think it has time limit due to heat. AI graphics have made astonishing improvements recently. Have you seen Sora, text to video ai creation from openai? It is insane.

mark-y wrote on 2/20/2024, 12:14 AM

Yes, I started a thread about SORA in the OT area. Can't wait until it starts mimicking political debates. Should be a hoot.