AI FX in VP20 build 326 - working great for some

Howard-Vigorita wrote on 1/31/2023, 3:49 AM

Been beating up the AI-FX in the recent VP20 update. Style Transfer not only won't crash anymore, it's outperforming both Colorize and Upscale. So much that I had to update the older ai-fx test project I mentioned in this discussion of a Techgage Review. The review griped about the Vegas AI implementation. I'm calling the updated test project the AI Torture Test. I lengthened it from a 10-second media-length to 60 seconds. My original version was just running too quickly to compare performance of different machines. Too many things were running in under 10 seconds. If anyone wants to try the new version with the VP20 update, you can download it from here:

https://drive.google.com/file/d/10wujTAGcylOCJtVhVk0Y7GPf7rTtvJwo/view?usp=share_link

Ran the updated Torture Test on my relatively new laptop and a much older Intel NUC and threw the results into google sheets here: https://docs.google.com/spreadsheets/d/1oqSnW0U5mN16BAF3wGL0MwisWz_kY1Vf4HFfZ9Ie_ec/edit?usp=share_link

Looks like the default now is for Vegas to run AI stuff on the same gpu that is selected in Video Prefs. The activity shows up on my laptop's Nvidia and on my Nuc's AMD VegaM. If the Intel igpu is selected in Video Prefs, it runs like a dying dog. In older versions of Vegas, AI activity showed up on my Intel igpu where Style Transfer crashed allot. It's looking real good now on everything but Intel. And seems to love gobbling up Vram. But somehow manages to run, though not all that well, in the VegaM which only has 4gb Vram... grabbing it all. I imagine it'll run much better on a bigger gpu with a whole lot more Vram. I won't be at home base for a while, so someone else will have to check that out.

Btw, the Torture Test will run on VP19 but if Style Transfer is not disabled in Track FX, it'll probably crash. Same for VP20 build 214. The new VP20 update is pretty solid with it, however. Though I did instigate a crash once when I killed the render while it was running.

EDIT: Here's a Response Submission Form with links to the Project and the Responses.

Comments

Former user wrote on 1/31/2023, 4:53 AM

Crashed at 65%, error related to MagixAiFx.ofx Vp20b316 , Ram and VRAM were both mid range and not the cause.

I saw some GPU use on Nvidia and high CPU use when encoding NVENC

Former user wrote on 1/31/2023, 5:17 AM

StyleTransfer and colorization, impressive GPU use, but it looks like 10gb VRAM isn't enough. That's possibly causing a slow down the same way using High Quality textures in gaming will slow down a game if it can't all fit in the Vram.

Howard-Vigorita wrote on 1/31/2023, 11:45 AM

@Former user Thanks for giving it a try. Did you get any success running any of the FX one at a time? Should indicate which one caused the crash. I'm also experimenting with an input form if want to try it... inspired by @RogerS. It's very, very, alpha at this point. Can pm you a link if you're interested in sharing some data.

jetdv wrote on 1/31/2023, 12:45 PM

I decided to give this a try. Tried rendering using NVENC. Did great until it got to from 1499 (83%) at which point it did not move forward for a few minutes so I finally killed it.

Then tried again using QSV. Render was going great until it got to 83%. Fortunately, it didn't just stop like the previous one did. But it did slow to a horrible crawl. What looked like it was going to be under 4 minutes ended up taking over 16.5 minutes...

In Options - Preferences - video tab, the nVidia GeForce 3060 card was chosen.

Yelandkeil wrote on 1/31/2023, 2:09 PM

With about 200W power consumption rendered in 3 and a half minutes, very proud and satisfied with my HW and VP20b326!
Timeline playback smooth like butter.

Howard-Vigorita wrote on 1/31/2023, 2:30 PM

@jetdv render format should not make much difference if you keep it simple. I did the render tests @ 1080p with Magix AVC Nvec or Vce defaults. The 83% point is where the Upscale crosses the 1.1x line so it's normal for it to creep along after that if Upscale FX is enabled. That FX can be a bone crusher. You should be able to get at least as good a time as my laptop which has a 3060 mobile gpu. It's using the the 522.30 studio drivers. Also check the internal dnn setting and see if it's at the default DML,0... might be some kind of OV if Windows settings interfere. In which case AI load may go to the cpu or the Intel igpu and performance will get really bad.

j-v wrote on 1/31/2023, 2:31 PM

I tried also rendering that Ai Torture test with 3 AVC templates on my laptop from signature

- Magix with NVenc..... crash at 89%

- Magix with Intel QSV..freeze at 86%

- Voukoder rendered complete after almost 40 minutes, look

jetdv wrote on 1/31/2023, 3:43 PM

Ok, I tried rendering to HD instead of 4K and all tests succeeded and were much faster:

NVENC was actually the slowest with QSV in the middle.

Former user wrote on 1/31/2023, 6:18 PM

@Former user Thanks for giving it a try. Did you get any success running any of the FX one at a time? Should indicate which one caused the crash.

i certainly know it's crashing with colorize, did not crash with styletransfer when I tried it alone. It sometimes doesn't crash with colorize.

Below, shows original/colorize(basic)/colorize(enhanced) . crashes when it gets to colorize, error related to MagixAiFx.ofx

Also the enhance model seems to want to put red everywhere at a high intensity where it doesn't belong

Real time colorization at 720P, an impressive improvement in performance . Intel OpenVino looks to have got the boot, and a coincidental but good thing too, Intel is having a terrible time trying to sell their CPU's, while AMD is doing much better. Next to fix is Nvidia/AMD GPU decoder/encoders and remove this dependence on Intel IGPU's for best performance.

Howard-Vigorita wrote on 2/1/2023, 7:38 PM

Since I seem to be in the minority having a good experience with AI-FX, which I'm sorry to learn, I changed the thread title. And put a few tips into a data collection form that have helped me along:

https://docs.google.com/forms/d/e/1FAIpQLSedu2yBPAJHYN-qAW8r2O0c_NbJf2_XF3zsRlB1lUGNFQp8rg/viewform?usp=sharing

If rendering at 720p is less crash prone than 1080p which I used, do that and report. If anyone has any other suggestions that work for them, I'll try and wire them into form options and/or add them to the tip list. The data might be useful for developers if they look at it.

Former user wrote on 2/1/2023, 11:31 PM

@Howard-Vigorita like I say it crashed at 65% so based only on those figures, your benchmark showed me low GPU use, and medium high CPU use, yet I know for styletransfer and colorize it gives me close to 100% GPU usage, and previously using AI effects on Vegas that use CPU would use close to 100% CPU.

I haven't investigated how your project works, but if you are combining them together, that may be the problem, there's no need to do that to use up all the resources, if that's what's happening, it's creating a bottleneck unrelated to GPU and CPU, which isn't very useful for a benchmark.

@VEGASDerek the colorizer using Enhanced models is a game changer, it looks maybe better then deoldify it just has the one problem of splashing red everywhere. In my testing of other colorizers, this is often caused by a low render factor, and I was able to replicate the over abundance of red. There are precision/processing/accuracy controls but they either don't work, or they don't control 'render factor' .

Whatever is the actual problem, once you fix that, there will be no NLE or upscaler that works as well as Vegas for colorizing (that I know about) It can be a Halo feature you use in marketing

1.Vegas basic 2.Vegas Enhanced 3.Deoldify

Illusion wrote on 2/2/2023, 7:49 AM

Tested on my system. Total render time: 3:18 GPU is not utilized much.

j-v wrote on 2/2/2023, 9:07 AM

If rendering at 720p is less crash prone than 1080p which I used, do that and report.

Rendered to 1280 x 720 AVC at laptop 3 times:

- Magix AVC QSV in 96 sec > 58,6 MB
- Magix AVC Nvenc in 97 sec > 59,2 MB
- Voukoder AVC in 99 sec > 45,7 MB

Yelandkeil wrote on 2/2/2023, 10:25 AM

Tested on my system. Total render time: 3:18 GPU is not utilized much.

Hmm, 8 sec less than me; whether your HW config better or Winpro11 better. 👍👍

Howard-Vigorita wrote on 2/2/2023, 10:44 AM

... it crashed at 65% so based only on those figures, your benchmark showed me low GPU use, and medium high CPU use, yet I know for styletransfer and colorize it gives me close to 100% GPU usage, and previously using AI effects on Vegas that use CPU would use close to 100% CPU.

@Former user Is it possible your gpu was set to off in video prefs? Otherwise with a gpu selected that's tagged (Optimal), dnn inspection should reveal mode=DML, dnn device=0, and AI processing load should go to the gpu. Reflected as higher 3d utilization and vram usage. Was totally different in build 214... seemed to all go to an Intel igpu if you had one or to the cpu & ram otherwise. I think this build will also fall back to ram when vram gets low. Probably good idea to run this test immediately after starting Vegas. I also restarted Vegas between runs and kept wireless off.

I did set Colorize to use the enhanced model to give stronger gpus more to chew on. All the other FX use default presets. Maybe I should set it back to basic. I could also lower the max on the Upscale from 1.2x to 1.1x. When Upscale goes above 1.1x (at the 83% mark), the load skyrockets. Each AI-FX can be run in isolation by unchecking boxes in Track-FX... I suppose I could provide 5 different projects with none, all, and 1 FX in each. Anyone else what to weigh in?

Howard-Vigorita wrote on 2/2/2023, 10:54 AM

Tested on my system. Total render time: 3:18 GPU is not utilized much.

@Illusion Thanks for the submission. Never did a google form before and hope it worked smoothly for you.

Illusion wrote on 2/2/2023, 12:20 PM

Tested on my system. Total render time: 3:18 GPU is not utilized much.

@Illusion Thanks for the submission. Never did a google form before and hope it worked smoothly for you.

No issue there.

Vincent-Brice wrote on 2/2/2023, 2:12 PM

No crashes here. At project settings

Magix AVC (AMD VCE) 2160 = 03:01.77

Magix AVC (AMD VCE) 1080 = 00:51.21

Howard-Vigorita wrote on 2/2/2023, 3:04 PM

@Vincent-Brice Those are really great run times. Best so far mentioned here. I see an AMD rx6800xt in your signature. I haven't been brave enough yet to try this thing with a 4k render on my laptop after others failed. If you have the time, could you run the fx one by one and post them to the response form?

Vincent-Brice wrote on 2/3/2023, 5:45 AM

@Howard-Vigorita Done. Not a glitch, so stable now with v20 🙂

Don't know if my Task Manager of all FX is of any use to you:

RogerS wrote on 2/3/2023, 6:51 AM

I'm 1.5 hours in on the render with my laptop and I think it hung at 66%.
|

Gave up on that one and the rest completed faster than expected. I reviewed each render and it looks like they all worked though times are better than I would expect.

Reyfox wrote on 2/3/2023, 7:05 AM

Tossing my "hat" into the ring. Total render time with Magix AVC (AMD VCE) all at 29.97fps. All the AI FX at the track level were on. Colorization, Style Transfer and Upscale.

UHD 4K 07:26

1080 HD 01:16

720 HD 0:55.

@Howard-Vigorita what should the Project Properties>Full-Resolution rendering quality be set to? Looking at mine, it was set to Good.

Video ram was at 11.1GB on my 12GB card rendering in UHD 4K. It was less with the lower resolution renders.

Howard-Vigorita wrote on 2/3/2023, 9:43 AM

I'm 1.5 hours in on the render with my laptop and I think it hung at 66%.

@RogerS Your 13600k looks pretty good... quite a bit better than my 12900h. But wow, the hung xps15 looks like it's stuck at 100% cpu usage with DML,0 which is odd... is the Nvidia listed 1st, selected, and marked (Optimal) in Video Prefs? If the igpu was loaded before before it hung with all fx on, maybe it crashed and sent the cpu off the deep end. The Style Transfer time when run by itself looks decent on that machine... developers must have put in extra effort when fixing that one.

Howard-Vigorita wrote on 2/3/2023, 10:01 AM

what should the Project Properties>Full-Resolution rendering quality be set to? Looking at mine, it was set to Good.

@Reyfox Yes. I left rendering quality set to the default of Good to better isolate AI performance. At least decoding is totally out of the picture with generated media.

The fact you can get 2060p without locking up on all fx is pretty good. I finally got my laptop to do it last night if I rendered Qsv. The moving of a little rendering load from my Nvidia gpu seemed to let it run cool enough to make it through. When I rendered 2060p Nvenc, I could render the fx individually but not all together. I'll post those details.