Benchmarking results continued

Comments

RogerS wrote on 11/22/2021, 9:42 AM

That's a first- it opens here in VP 19.

If you can open it in audio software just resave as a .wav and replace it? It shouldn't have any impact on render times.

In case somehow it got corrupted here is a direct download.
https://www.dropbox.com/s/91kz701f8m6t243/Song.ogg?dl=0

Howard-Vigorita wrote on 11/22/2021, 10:16 AM

Hello everyone! I just created an account here specifically to try and get this benchmarking deal working and submit a result. Sadly, the .ogg song file is erroring no matter how I try to import it. Has anyone else had this issue? Using Vegas Pro 19. The OGG file plays in all media players (Windows Groove, VLC, etc..), but I can't drag and drop it in, and importing doesn't work either. All I get is this error. Any help would be appreciated!
 


@DDMan11 Only time I've ever seen that error is when the media is already open and locked by another application, like a browser for instance. Suggest you try a reboot and fire up Vegas before opening any of the media with another app. You might also have to turn off "Restart Apps" in Windows Sign-on settings if the offending app restarts automatically after a reboot.

Cameras: Z-Cam E2 w/Canon EF turbo mount, Canon XF605 & XF305, JVC GV-LS2, Canon 6D w/L-glass line.
Laptop: Dell XPS15-9570; i7-8750h 32gb (igpu Intel UHD-630 {.8935} & Nvidia GTX-1050Ti {512.96}) Win11
Road: Intel NUC8 i7-8809g 32gb (igpu AMD VegaM 4gb {21.8.2 [Mod]} & integrated Intel HD630 {.9415}) Win11
Workstation: i9-11900k 64gb (Sapphire ToxicLC 6900xt 16 gb {Adrenalin 22.5.1 } & Intel UHD750 {.1994} ) Win11
Workstation: i9-9900k 32 gb (Sapphire AMD Radeon VII 16gb {22.5.1} & integrated Intel UHD630 {.1994}) Win10

Workstation2: XeonE5-1650v4 128 gb (Sapphire Nitro+ Vega64 8gb {22.5.1} & Nvidia GTX-1660 8gb {512.96}) Win10
Workstation3: i7-980X 24gb (Sapphire Nitro+ RX580 8gb {21.q3}) Win10
currently active Vegas 17.455, Vegas 18.527, Vegas 19.643.

Red Car & Pip Benchmarks: https://drive.google.com/drive/folders/1TK2lyFZSlzU9JQ8xO1mvoj-zl9tyj2lc?usp=sharing
Benchmark Tables: RoundUp21 , RoundUp2020, and earlier
(ffMetrics by fifonik) Encoder Quality Tables by me: http://www.rtpress.com/ffmetrics.htm
LutP utility to create OpenFX xml preset files for the Vegas LUT fx in Vegas 17, 18, & 19.

 

DDMan11 wrote on 11/22/2021, 3:03 PM

I tried the suggestion and just converted it to a WAV file. No problems after that, but nothing with the OGG file worked. *shrug*

Ran things 4x through and will submit my findings, but here's my breakdown:

Ryzen 2700x (Stock clocks)
Radeon 6800 16gb
32GB 3200 DDR4
ASRock x470 Taichi
Samsung 970 1TB NVME SSD (PCI-e 3.0)

Playback, after 3 or 4 loops averaged to around 4.9 FPS. At least that's what it looked like...

4k AMD VCE: 4:01, avg 4.33 FPS

4k Mainconcepts: 5:25, avg 3.21 FPS

1080p AMD VCE: 1:26, avg 12.09 FPS

1080p Mainconcepts: 1:46, avg 9.84 FPS

My only issue is the lack of CPU and GPU usage, even with the 4k version. With an NVME, read and write speeds are not remotely an issue. Vegas never uses more than 2gb of RAM though and CPU/GPU usage doesn't ramp up until about halfway through. Between the 10% and 45% mark it runs like a SNAIL, and then suddenly after half it more than triples its speed (and I show about 40% CPU/GPU usage). It does this for all versions (VCE or CPU rendering), so I just assume it has to do with the files it's rendering and the difficulty more than anything else. Comparing my hardware to others, I think I'm running a bit slow so I need to see what's up.

RogerS wrote on 11/22/2021, 6:23 PM

The inconsistent performance as Vegas manages text and Fx are expected for this benchmark. The media itself isn't challenging for Vegas.

Scores of 4 and 5 minutes aren't expected for 4K- I get that with my laptop. Can you keep an eye on GHz speed of your CPU while rendering? If it drops a lot maybe use a tool like HWInfo64 (or Ryzen master?) to see if there are elevated temps. Beyond that I'm not sure what could cause this. I assume you have recent Pro or Enterprise drivers for your GPU and system bios, chipset, etc. are up to date and functioning normally. Under File i/o in Vegas is it using your AMD card for decoding? AMD is active under preferences/video? There's not a whole lot else to configure in Vegas assuming you haven't made any unexpected changes in internal preferences, etc.

DDMan11 wrote on 11/22/2021, 11:54 PM

@RogerS Thank you for the reply. I've got up to date drivers, and BIOS is as up to date as they will let me (They don't recommend any higher bios update with my current CPU as newer updates for for 3000/5000 series CPU's).

File I/O was set to Auto (AMD)
File/Preferences was set to AMD GPU acceleration
Also Dynamic preview set to 0

I took 3 different measures (along with screenshots) during rendering:

5% (5 seconds into rendering):

 

12% (1:05 into rendering)

 

50% (3:36 in)...



And of course it finished that last 50% in about 30 seconds.

Less than 20% CPU usage the entire time, MODERATE GPU usage...sometimes... but during the slowdown (which occurs between 9% and 30%) it's just a few percent usage. Temps never get over 55* (I can run Prime95 on MAX for 30 mins and only see CPU temps up in the upper 70's). Disk usage is super low (2.5MB/s or so when it's "working"). I'm just super torn as to what is up. I noticed this behavior before which is why I looked up this benchmark, but I'm not going to plug this data in if it isn't accurate or will skew data.

RogerS wrote on 11/23/2021, 12:07 AM

Just a note, dynamic ram preview should be at whatever you normally use for the render (for VP 19, 5% or 10% is normal), it is just 0% for the preview to see how the system actually plays back the footage.

You're doing a good job tracking your system so I'd say your data reflects how it works today. I'd encourage you to submit results and if you find something was clearly broken just DM me later with your build of Vegas and new render times and I'll update the sheet directly. Some data is better than no data!

DDMan11 wrote on 11/25/2021, 5:19 PM

Just a note, dynamic ram preview should be at whatever you normally use for the render (for VP 19, 5% or 10% is normal), it is just 0% for the preview to see how the system actually plays back the footage.

You're doing a good job tracking your system so I'd say your data reflects how it works today. I'd encourage you to submit results and if you find something was clearly broken just DM me later with your build of Vegas and new render times and I'll update the sheet directly. Some data is better than no data!

You hit the nail on the head right here! Wow....

Benchmarking 4k with VCE with 0 dynamic preview (Which is what I was told to set it to by some others, to help with compatibility since I have 32gb of RAM), gave me a 3:53 as my fastest.

Sure enough, I turned on dynamic RAM and set to 1% (then tested 5% too just to be sure), and BAM. 1:25/1:24 rendering duration with no other changes.

I'm going to re-run all my benchmarks and then give you an update. I also tried rendering with VEGAS 17, which rendered faster than 19 (1:13 vs 1:24) and used WAY more RAM (7gb on 17 vs only 2gb on 19). Not sure why that's the case, but who knows.

Definitely some interesting stuff here!

RogerS wrote on 11/25/2021, 7:13 PM

Please report VP17 times too. It's interesting for comparisons. Haven't seen a change between 18 and 19 but 19 is still being improved.

todd-b wrote on 11/25/2021, 7:47 PM
.


Benchmarking 4k with VCE with 0 dynamic preview (Which is what I was told to set it to by some others, to help with compatibility since I have 32gb of RAM), gave me a 3:53 as my fastest.

Sure enough, I turned on dynamic RAM and set to 1% (then tested 5% too just to be sure), and BAM. 1:25/1:24 rendering duration with no other changes.

I'm going to re-run all my benchmarks and then give you an update.

Yes you should try that again, I have seen up to 25% faster playback speeds on timeline with Dynamic ram set to 50mb or above, and we do know that carries accross to rendering speed, and will be faster , maybe 15-20%. Your results show a 275% increase in speed

Your dynamic ram 'on' values show expected results but dynamic ram 0 are unusually slow @DDMan11

 

DDMan11 wrote on 11/25/2021, 7:55 PM

I wish I could just USE my RAM. I didn't buy 32gb to use so little. Most of my projects are 10-30 mins in length and usually have lots of speeding up, slowing, and clips from longer videos. Maybe my projects just aren't big enough.

Ill re-run the benchmarks with dynamic ram and go from there.

Former user wrote on 11/25/2021, 11:57 PM

Hi @DDMan11 You're not alone with noticing the lack of use of CPU/RAM/GPU etc.

in your pics above it says Google (45) in Taskmanager, I think, it's a bit sml for me to read, but have you got Google open & other programs while doing this test? I notice a drop in speeds when Google is open.

can you try it with & without it open,

& my Google only reads Google (31) even when i opened 19tabs just as a test ?

DDMan11 wrote on 11/26/2021, 12:22 AM

Hi @DDMan11 You're not alone with noticing the lack of use of CPU/RAM/GPU etc.

in your pics above it says Google (45) in Taskmanager, I think, it's a bit sml for me to read, but have you got Google open & other programs while doing this test? I notice a drop in speeds when Google is open.

can you try it with & without it open,

& my Google only reads Google (31) even when i opened 19tabs just as a test ?

Yeah, I had some tabs open, but it was crystal clear that it was the dynamic preview that sped everything up for me. Even with a fresh restart, 0 apps open, it was still running dog slow until I upped the dynamic preview just a tiny bit.

wilri001 wrote on 12/12/2021, 7:49 AM

I just submitted my 5950x with 6800x benchmark results with CPU & VCE in FHD and UHD that are quite good.
But then I did VCE renders at UHD and FHD with Dynamic RAM at 0, instead of the 50 that I normally use. It took WAY longer! FHD took 4 times longer! Increasing Dynamic RAM to 2000 just took off 1 second from 50.

Well, I didn't think Dynamic RAM was used at all when rendering.

And if it makes such a huge difference, why isn't Dynamic RAM a parameter in the benchmark?

RogerS wrote on 12/12/2021, 7:57 AM

Thanks for posting your scores- you are at about the top of UHD and FHD.

The one thing I didn't understand is when you said GPU acceleration is off- that is supposed to mean in preferences/video the GPU is set to none. But did you mean by it dynamic ram preview was set to 0 (off)? If you clarify, I'll update the chart accordingly.

Dynamic ram preview is a preview buffer and setting to 0 hurts performance but can be needed to avoid corruption when using GPU encoding pre VP 19 (if you see corrupt frames or black frames in your rendered video, don't use those times). It usually doesn't have that much of an impact and this benchmark has been around since VP 16. I could add a column for dynamic ram preview, though it will only be meaningful for scores going forward.

TheRhino wrote on 12/12/2021, 10:30 PM

I submitted my stock 11700K & 6800 XT results & am getting about the same 6800 XT results as others. It looks to be one of the best GPUs for timeline performance and it allows me to stick to VCE renders so clients' files match renders from my other systems with AMD GPUs...

Vegas almost never utilizes 100% of the CPU or GPU, so to maximize system potential I often have multiple instances of Vegas batch-rendering long projects at once... With AMD GPUs, I can open 4-6 instances of Vegas, one for each client, but with NVENC, I think the limit is still just 3, and then the 4th reports an error...

Workstation C with $600 USD of upgrades in April, 2021
--$360 11700K @ 5.0ghz
--$200 ASRock W480 Creator (onboard 10G net, TB3, etc.)
Borrowed from my 9900K until prices drop:
--32GB of G.Skill DDR4 3200 ($100 on Black Friday...)
Reused from same Tower Case that housed the Xeon:
--Used VEGA 56 GPU ($200 on eBay before mining craze...)
--Noctua Cooler, 750W PSU, OS SSD, LSI RAID Controller, SATAs, etc.

Performs VERY close to my overclocked 9900K (below), but at stock settings with no tweaking...

Workstation D with $1,350 USD of upgrades in April, 2019
--$500 9900K @ 5.0ghz
--$140 Corsair H150i liquid cooling with 360mm radiator (3 fans)
--$200 open box Asus Z390 WS (PLX chip manages 4/5 PCIe slots)
--$160 32GB of G.Skill DDR4 3000 (added another 32GB later...)
--$350 refurbished, but like-new Radeon Vega 64 LQ (liquid cooled)

Renders Vegas11 "Red Car Test" (AMD VCE) in 13s when clocked at 4.9 ghz
(note: BOTH onboard Intel & Vega64 show utilization during QSV & VCE renders...)

Source Video1 = 4TB RAID0--(2) 2TB M.2 on motherboard in RAID0
Source Video2 = 4TB RAID0--(2) 2TB M.2 (1) via U.2 adapter & (1) on separate PCIe card
Target Video1 = 32TB RAID0--(4) 8TB SATA hot-swap drives on PCIe RAID card with backups elsewhere

10G Network using used $30 Mellanox2 Adapters & Qnap QSW-M408-2C 10G Switch
Copy of Work Files, Source & Output Video, OS Images on QNAP 653b NAS with (6) 14TB WD RED
Blackmagic Decklink PCie card for capturing from tape, etc.
(2) internal BR Burners connected via USB 3.0 to SATA adapters
Old Cooler Master CM Stacker ATX case with (13) 5.25" front drive-bays holds & cools everything.

Workstations A & B are the 2 remaining 6-core 4.0ghz Xeon 5660 or I7 980x on Asus P6T6 motherboards.

$999 Walmart Evoo 17 Laptop with I7-9750H 6-core CPU, RTX 2060, (2) M.2 bays & (1) SSD bay...

RogerS wrote on 12/12/2021, 10:34 PM

Thanks TheRhino- you're near the top of the results and your times look in line with what can be expected for that CPU/GPU combination. Very interesting about the multiple batch renders to more fully maximize system resource use.

RogerS wrote on 12/21/2021, 2:33 AM

Minor update: I added filters (data/filter view) to make it easier to sort by encode mode. I also added a bit of color coding to make it easier to see where the FHD results end and UHD starts and to call out the few HEVC files from the rest of the AVC.

If you have custom filters you want, feel free to message me. Or if you know how to do more robust data visualization in Sheets I'd like to improve the presentation of the results.

I also added a new column and question for dynamic ram preview value as it's important to render time.

todd-b wrote on 4/3/2022, 7:15 PM

@jetdv Is it possible to benchmark Vegas via scripting. Run a number of benchmark encodes back to back?

In the case of PugetBench, it's an executable that runs Premiere and Resolve without the user even opening the software or seeing it, it collects all results in FPS, then uploads results. It comes with the media files. In the case of Vegas maybe the user opens Vegas, runs a script and collects the benchmark info themselves?

The current benchmark is of limited use, would be good to have a transcode benchmark, different media such as HEVC, and output to different encoders. With this benchmark as example we would have no idea Vegas can't use the NVIDIA encoder efficiently via the native MagixAVC setting but if a script could encode to both MagixAVC and Voukoder Nvenc for a transcode we would. A benchmark comparing the different decoders would be good too but that would be outside the scope of an internal automated script.

This is what PugetBench runs on it's basic test for Resolve

jetdv wrote on 4/4/2022, 8:59 AM

@todd-b, yes a script could automatically run multiple renders. It could also keep track of how much time each render took. The proper projects and media files would be required along with the script.

todd-b wrote on 4/6/2022, 9:31 PM

@jetdv definitely want a transcode AND a GPU heavy benchmark. Current benchmark includes both in same benchmark which confuses issues when looking at figures. It is too GPU heavy to be representative of the average project but also the end section is pure transcode. It skews results relating to GPU heavy figures, and hides problems relating to transcode only.

The automated part, especially obtaining the results is needed because so many people made mistakes, had to redo until fixed, and some of the results are probably not correct. Making things as simple as possible also means more people will bother doing the benchmark

Current benchmark could be a part of the automated bench, and be titled legacy benchmark

jetdv wrote on 4/7/2022, 9:26 AM

@todd-b You can find all the information needed to get VEGAS to render to a desired preset in my tutorials at http://www.jetdv.com (or direct on my youtube channel). It's really just a matter of picking the right preset and telling it to render. For timing, you'd need a variable that get's the start time and end time and then just displays the difference.

todd-b wrote on 6/26/2022, 5:05 AM

@RogerS Can you fix my recent entry, it is not showing when looking at the page with default setting, it shows if I list by most recent contributor. Also in the description I confused kb with mb

They are voukoder entries I think it's fair for us Nvidia people due to Vegas's 'heart beat' style encoding where it constantly starts and stops slowing the encode. Voukoder gives a better representation of GPU processing on Nvidia. If you think the entries don't belong due to not using MagixAVC, then that's fine, maybe Voukoder options could be included in the future.

 

RogerS wrote on 6/26/2022, 5:14 AM

Hi Todd, it shows up in the right place after I sort it by render time. Not sure if there's a way to force it but I always do it manually. Just give me a bit of time after submitting as I will get to it!

Since you made a note of it there, I'll keep the Voukoder renders. You switched from a 3070 to a 3080?

Fixed the description to million bits.

todd-b wrote on 6/26/2022, 8:19 PM

@RogerS Yes 3070 - 3080, but it's difficult to make comparisons at 4K because Voukoder encodes 13% faster then MagixAVC with Nvidia. At 1080P a better comparison can be made because in my testing there is no difference between voukoder NVENC and MagixAVC encoding speeds, the difference between the 2 GPU's is 3 seconds(41s vs 44s) Not much to write home about