New project to use for benchmarks...

fr0sty wrote on 12/23/2022, 8:00 AM

John-Ivar has released the project files for his spectacular user-made VEGAS ad... I thought it might make a great benchmark for testing Vegas performance. Here is the link to download it.

https://drive.google.com/drive/mobile/folders/1m2Ok_HIFO7hxkfeK9pgCSwu4RAKtTSW0?usp=share_link

 

Comments

John-Ivar wrote on 12/23/2022, 10:06 AM

Thanks for posting @fr0sty!​

fr0sty wrote on 12/23/2022, 10:08 AM

Thanks for sharing!

Howard-Vigorita wrote on 12/23/2022, 5:34 PM

Wow. Uses allot of Vegas features I still don't have my head wrapped around. Studying it will be a real learning experience. Customized the default Magix AVC 4k render templates, changing the height to 2700, width to 2160, frame rate to 24p, and got the following on my 11900k/6900xt system ("VEGAS Pro Fanmade Ad.veg"):

vce: 2:10; qsv: 2:16, and MainConcept: 3:20

Former user wrote on 12/23/2022, 8:15 PM

@Howard-Vigorita Could you and others with modern intel cpu's render only the section between markers 13 and 14. It is where the Intels should excel over AMD's. If people bench this section it's important to have nothing else running as my CPU loads up to 100%, background activity will give you wildly different results.

Amd 5900x = 1m18s

RogerS wrote on 12/23/2022, 8:23 PM

Let me test this out and see it seems like a good second benchmark (vs Red Car, etc.) Happy to make a system to upload results for this one as well if it will sort systems in a different way.

RogerS wrote on 12/24/2022, 2:24 AM

I'm trying to test this out. I couldn't get the VEGAS Pro Fanmade Ad - Windows 11 Update project to open- it crashed VP 20 within seconds.

With the All New Vegas Pro project, it's very complex and creative.

I noticed that there is some misalignment here- looks like the y offset in ProType is pushing it out of the frame. (Not sure why the frame is so small when the text box size is project resolution). 5.96 offset keeps it from being cut off at bottom.

I wasn't able to complete a render as Vegas crashed- will try again later. This is the template I customized- let me know if that matches your settings..

John-Ivar wrote on 12/24/2022, 3:37 AM

Hi @RogerS! Thank for the positive feedback! Too bad about the crashing, I wonder what's causing it. The Windows 11 project is a pretty light project in terms of external media and graphics use, I figured the 'All New VEGAS' project would be more challenging somehow.

Anyways, I took a look at your template and I'm guessing the render crash is because of your output resolution. It looks like you've got the width/height mixed up. The videos were made to be 'social media first', meaning a square or a vertical aspect ratio. In other words, you want the higher number for height. So 2160x2700 for a 4:5 aspect ratio at 4K quality, and 1080x1350 for 1080p quality.

The 'out of frame' text effect is a deliberate choice. I think the text sliding out from nothing gives it a modern almost Apple like look. This is obviously subjective though. You can remove the effect by turning off the masking on the event. As the sliding animation is done inside the ProType Titler generated media, you're able to freely mask on top of the animation.

RogerS wrote on 12/24/2022, 5:00 AM

Hi John, I think it's likely AI-enabled Fx? I haven't used any on this machine to date and need to test it.

Thanks for the correction re: rendering- I just default to horizontal being the larger number (oops).

For out of frame I have no problem with that but is it intentional that it never come fully in frame? The bottom remains cut off even when it's fully loaded.

Last changed by RogerS on 12/24/2022, 5:06 AM, changed a total of 1 times.

Custom PC (2022) Intel i5-13600K with UHD 770 iGPU with 31.0.101.3790 driver, MSI z690 Tomahawk motherboard, 64GB Corsair DDR5 5200 ram, NVIDIA 2080 Super (8GB) with latest studio driver, 2TB Hynix P41 SSD, Windows 11 Pro 64 bit

Dell XPS 15 laptop (2017) 32GB ram, NVIDIA 1050 (4GB) with latest studio driver, Intel i7-7700HQ with Intel 630 iGPU (driver 31.0.101.2115), dual internal SSD (256GB; 1TB), Windows 10 64 bit

Vegas 19.648
Vegas 20.214

VEGAS 4K benchmark: https://docs.google.com/forms/d/1Exbi4K3hbxw6snJuisR1ble-0tCPVNcIcNnx0BAtSIM/

John-Ivar wrote on 12/24/2022, 5:23 AM

@RogerS very strange and interesting.

Maybe try removing or re adding the style transfer effect? It appears at about 40 seconds in the video. Maybe it's an incompatability issue caused by us having different builds of the VEGAS Deep Learning model?

The text should be fully in frame shortly after appearing, that's also strange. It could be because of differing fonts maybe? I should probably have switched the fonts to something widely available before uploading it. The Epilogue font family can be downloaded here.

Have a look at the finished video, the text does not spend much time not being fully visible. Maybe it could be a GPU thing? I've had many strange results with the ProType Titler with GPU acceleration is enabled.

 

RogerS wrote on 12/24/2022, 5:32 AM

Hi John, I searched through the project and did just that. Without style transfer enabled (unchecked the Fx) it plays back and renders. Let me experiment a bit more on a project I create and see if that's an issue I'll also test on my system.

I noticed color correction and the p and g for compositing are cut off which could be a font issue as you state. I looked and Epilogue is not a font I have on my system so it may be substituting for one that is larger and that's the cause of the issue?

Edit: looks like a completely different font.

I was thinking this could be a new benchmark project for the community if that's something you'd approve. I'm the maintainer of the current crowdsourced benchmark (link in my signature) but it uses a somewhat odd selection of Fx and if this project scales in different ways could be useful as a second benchmark. It just needs to be as user-proof as possible : )

Last changed by RogerS on 12/24/2022, 5:34 AM, changed a total of 1 times.

Custom PC (2022) Intel i5-13600K with UHD 770 iGPU with 31.0.101.3790 driver, MSI z690 Tomahawk motherboard, 64GB Corsair DDR5 5200 ram, NVIDIA 2080 Super (8GB) with latest studio driver, 2TB Hynix P41 SSD, Windows 11 Pro 64 bit

Dell XPS 15 laptop (2017) 32GB ram, NVIDIA 1050 (4GB) with latest studio driver, Intel i7-7700HQ with Intel 630 iGPU (driver 31.0.101.2115), dual internal SSD (256GB; 1TB), Windows 10 64 bit

Vegas 19.648
Vegas 20.214

VEGAS 4K benchmark: https://docs.google.com/forms/d/1Exbi4K3hbxw6snJuisR1ble-0tCPVNcIcNnx0BAtSIM/

John-Ivar wrote on 12/24/2022, 6:12 AM

@RogerS Cool! I'd certainly be fine with using this as a benchmark in some way. And I'm happy to make any modification necessary to make it as user-friendly as possible. I should have thought about the font issue before putting this up for download, I'll get right on that over the holidays. In the meanwhile, the font can be downloaded here.

Also, feel free to make any changes you see fit and repurpose it for any use case. I'm just happy to contribute to the community in any way. 👍

RogerS wrote on 12/24/2022, 7:23 AM

Hi John, if you wouldn't mind substituting with a Windows standard font and ensuring the size works I think that's enough for now.

If a few others can test the AI part and confirm which versions of deep learning it works with or which GPUs it works with, that will be helpful. If AI compatibility turns out to be a big issue I guess that part could be pre-rendered, though I like the idea of leaving it in as the performance should really vary by system.

John-Ivar wrote on 12/24/2022, 7:34 AM

@RogerS I'll get on that.

For anyone who'd like to test the project as it is right now, you can download the Epilogue font here.

I can leave the Style Transfer effect disabled/unchecked. This would avoid potential crashes and also enable users to test it, remove it or change it at their leisure.

RogerS wrote on 12/24/2022, 8:29 AM

Interestingly, my underpowered laptop could play back both projects albeit slowly. It froze on frame 163 while doing a NVENC render. Interestingly, style transfer does work on my desktop with my own media. More testing is in order.

The Epilogue font did fix the alignment issue so thank you.

John-Ivar wrote on 12/25/2022, 2:58 AM

Interesting. Thanks for testing and playing around with the project @Howard-Vigorita

Former user wrote on 12/25/2022, 6:00 PM

Could you and others with modern intel cpu's render only the section between markers 13 and 14. It is where the Intels should excel over AMD's.

Amd 5900x = 1m18s

@Former user I think when it comes to the ai stuff, a really high powered gpu takes the cake. Ran the 13-14 section on my 11900k/6900xt in 8 seconds with vp20. Moving the project over to vp19, that section took 13 seconds. Same section on my 12900h/rtx3060 laptop took vp20: 1:10 and vp19 1:17.

@Howard-Vigorita I don't get any GPU activity between markers 13 and 14, are you sure your 6900XT is doing anything at all?

The Intel acceleration should be done on CPU and IGPU right? (and Intel GPU if installed)

If you are seeing what looks to be large amount of GPU use on your AMD try turning off the GPU decoder, I've noticed it can either give the appearance of high GPU usage or due to the poor efficiency of the decoder it is using a lot of GPU, a premise could be that due to my encoding of this section being so slow (1m18s) I don't get any GPU activity from decoder, but for an 8second encode you do. If turning off the decoder also turns off any IGPU AI processing, then that presents a problem

If it is actually the case that the Intel AI is using your AMD GPU that would be interesting too, but would seem unlikely

Musicvid wrote on 12/25/2022, 7:12 PM

2160x2700 Project? Is that something new on social media -- I haven't encountered it?

Preview hangs at 1711 frames . . .

;?(

John-Ivar wrote on 12/26/2022, 3:29 AM

Hi @Musicvid!

The 4:5 aspect ratio is very common in social media, along with 1:1 (square) and 9:16 (vertical). When making content for social media, you want to optimize for smartphone screens held vertically, where a standard 16:9 aspect ratio will take up too little of the screen.

Attached you can see some examples of videos in 9:16, 4:5 and 1:1 aspect ratios respectively. Personally, I like 4:5 and 1:1, as they work well for smartphones, without necessarily alienating desktop users.

Hulk wrote on 12/26/2022, 8:43 AM

Was this created in VP20? Won't open in latest VP19 version for me?

RogerS wrote on 12/26/2022, 9:22 AM

It was created in 20.

Howard-Vigorita wrote on 12/26/2022, 1:44 PM
I don't get any GPU activity between markers 13 and 14, are you sure your 6900XT is doing anything at all?

The Intel acceleration should be done on CPU and IGPU right? (and Intel GPU if installed)

If you are seeing what looks to be large amount of GPU use on your AMD try turning off the GPU decoder, I've noticed it can either give the appearance of high GPU usage or due to the poor efficiency of the decoder it is using a lot of GPU, a premise could be that due to my encoding of this section being so slow (1m18s) I don't get any GPU activity from decoder, but for an 8second encode you do. If turning off the decoder also turns off any IGPU AI processing, then that presents a problem

If it is actually the case that the Intel AI is using your AMD GPU that would be interesting too, but would seem unlikely

@Former user I don't think the decoder or render format should have anything to do ai processing unless they swamp your gpu utilization or gpu memory. Things that seem to influence it are Windows/System/Display/Graphics (mine is set to High Performance (6900xt) for vp20), the gpu selected in Vegas Video prefs (mine is set to 6900xt), and internal settings for dnn (my 1st two are set to SMART, 0). Laptop is same except that it selects the 3060. You might try playing with the dnn settings and see if anything changes in your gpu 3d utilization vrs cpu utilization... settings may end up different from mine with a single gpu and no igpu. The little ai_fx test project here is what I used to suss it out. Be careful with enabling the Style Transfer fx in it. It's kind of a torture test animating between all the supplied models. I got allot of crashes doing that with the older build of vp20 back then but it's been getting better with dlm updates since then... the new bench only uses one Style Transfer w/o animating it so should be fine.

Former user wrote on 12/26/2022, 7:35 PM
I don't get any GPU activity between markers 13 and 14, are you sure your 6900XT is doing anything at all?

If you are seeing what looks to be large amount of GPU use on your AMD try turning off the GPU decoder, I've noticed it can either give the appearance of high GPU usage or due to the poor efficiency of the decoder it is using a lot of GPU

@Former user I don't think the decoder or render format should have anything to do ai processing unless they swamp your gpu utilization or gpu memory.

@Howard-Vigorita This is what I'm referring to, Vegas uses a lot of GPU processing for no real reason for certain files when GPU decode is turned on, That excess GPU use can be made to go away by turning off the GPU Decoding

So what I was suggesting is that just because you see GPU activity on your AMD GPU it doesn't mean the AI effect Style transfer is using it, but you seem to be indicating you believe it is? In this example The Vegas decoder is using 65 extra watts of GPU processing power simply playing the video (No GPU fx)

Admittedly the difference between your 8 second render and my 78 second render is huge, but couldn't this come down to the Intel AI instructions on your CPU that I don't have on AMD that also allow processing to be done on your IGU that I don't have.

Your Intel laptop with 3060 is also interesting, why is it so slow in comparison, if what you're saying is correct and i'm understanding your properly you seem to be suggesting Style transfer is using your AMD GPU either with assistance from Intel OpenVIno AI, or it's use of the AMD GPU for AI processing is unrelated to OpenVino, the way most other software is able to switch between AI engines depending on the GPU in use or if CPU only.

Can you prove your AMD GPU is doing the work, cause if it is maybe Nvidia is next, although the need for an Intel CPU is still not clear.

Btw I"ll look into all that technical info you've provided, but I"ll have to google it first

 

 

 

 

Howard-Vigorita wrote on 12/26/2022, 9:50 PM

The Woman in Field in 13-14 doesn't seem to generate much gpu activity if I uncheck the Style Transfer box in the event-fx on track#10.

I have no trouble, btw, moving the Style Transfer load around on e̵i̵t̵h̵e̵r̵ ̵v̵p̵1̵9̵ ̵b̵6̵4̵8̵ ̵o̵r̵ vp20 between Amd, Nvidia, or Intel gpus or the cpu. On my Xeon, the Nvidia 1660 handles colorization and upscale almost as quickly as my 5700xt and doesn't crash on my ai_fx Style Transfer torture test. I suspect a high-end Nvidia will beat a high-end Amd at this... they've been doing ai stuff for so long.

EDIT: after further testing I can only get vp19 Deep Learning to use Intel graphics or the cpu.

RogerS wrote on 12/26/2022, 10:38 PM

@John-Ivar Since so far 3 systems haven't been able to complete the benchmark so I think I'll wait until the next build of VP 20 is out to test again.

AI compatibility seems specific to the GPU/iGPU in use and seems like a work in progress from 19 to 20 and between builds of 20. Hopefully any issues will be addressed by the developers as I think having style transfer in it makes for a better benchmark.