Benchmarking

Former user wrote on 7/16/2019, 2:27 PM

Moderator's note:

This thread has been closed, and discussion continues here:

https://www.vegascreativesoftware.info/us/forum/benchmarking-results-continued--118503/

------------------------------------------------------

Magix Sample Project 4K link ...

https://www.dropbox.com/sh/qoaym1jgpgo8l05/AABec7Jhg_c3jTDJZNxDXB1Wa?dl=0

A user data input file, spreadsheet, render templates for FHD and UHD, etc are also at this location.

Templates and User data input with examples ...

The above are Magix Avc h264 templates, for Hevc use them also, same data rates etc, just change the “Render As” to Magix Hevc.

The spreadsheet and the spreadsheet screen captures below are updated in the download link location as new results come in. If i’m away from my PC it might be a while.

Spreadsheet screen captures ...

[1] Sorted on user name and [2] Sorted on render times

[3] Sorted on render times (FHD) [4] Sorted on render times (UHD)

[5] Sorted on Average FPS

The [n] items are there simply to easily reference a particular screen capture in discussions.

There's a single region marked out on the project, region 1. Please use it to check the Min. and Max. fps playback rates, and so get the average. This was added later, so if you have previously downloaded the zip, no need to do again, I've uploaded the project file as a single separate download. The same updated .veg is also in the zip. If possible give your playback rates in the form min-max e.g. 7-14. If you have say full playback, 25fps I'll enter it as 25-25, with a view to assist in sorting on the FPS column, not possible at the moment.

The following may be of little interest but I'll include it anyway.

Back to post

Comments

Kinvermark wrote on 7/26/2019, 12:24 AM

Do you wear sunglasses at night? You're either a very early riser or never sleep. 😎

Grazie wrote on 7/26/2019, 1:14 AM

Do you wear sunglasses at night? You're either a very early riser or never sleep. 😎

@Kinvermark - Lol. It was a toss up between Kool Dood or this one 🤪.

Former user wrote on 7/26/2019, 2:55 AM

@Howard-Vigorita "Please remove my results."

Is now updated.

Former user wrote on 7/26/2019, 5:42 AM

A bit of memory lane .. “Red Car” VP 12 benchmarking.. http://www.hyperactivemusic.com/vegaspro/vegaspro.html

http://www.hyperactivemusic.com/vegaspro/Sony%20Vegas%20Pro%2012%20and%20Haswell%20GT2%20Graphics%20Performance.pdf

Kinvermark wrote on 7/26/2019, 10:25 AM

I wanted to say a BIG thanks to @Former user for all the effort he put into getting this ball rolling. There is some good news about coming developments re benchmarking.... but I will let JN_ tell more. :)

Former user wrote on 7/26/2019, 10:44 AM

@Kinvermark See my last PM?

Howard-Vigorita wrote on 7/26/2019, 6:45 PM

Yes, let's keep this all in perspective. We have a few dozen results being seen by about the same number of moderately interested forum members. No big deal if someone is slightly "mislead" by minuscule sematic differences. The benchmark methodology is in its infancy and will no doubt need tweaking over time - assuming it even survives to adolescence. :)

Actually any fudging on a study's methods and materials alters the ability to draw correct conclusions from the data. To wit, fr0sty honestly believing the gpu is not enabled and used in all 3 Magix AVC Render Mode options. After I went to the trouble of testing and timing them with the gpu in my system and enabled. And then with it sitting on my desk and a plain vanilla vga card with no gpu whatsoever in my system instead. It looks like cooking the books to prevent disproving a favored myth. The settings are what they are and should be reported accurately and honestly. And not rewritten or interpreted. The data needs to speak for itself.

Howard-Vigorita wrote on 7/26/2019, 7:02 PM

Mainconcept IS CPU ENCODING. I don't know how many times we have to say that. The timeline rendering will be GPU accelerated as long as you have it enabled in preferences, but the encoding by the Mainconcept encoder is CPU based, just like every other encoder in Vegas Pro is except for NVENC/AVC/Quicksync/Sony AVC.

@fr0sty ... I don't think you'd be saying that if you saw my data before the methods got rewritten. I ran timing tests on all 3 encoders with a gpu and with no gpu. All the with-gpu timings were in a similar range and quicker that even my 9900k could possibly run them. And without gpu, all were dramatically slower. Proving without a doubt they all use the gpu similarly. And regrettably, my 9900k may be good, but not that good. Also, btw, I figured out what you mean when you claim the timeline uses the gpu. It's not the timeline. It's the FX plugins. They in fact do use the gpu without regard to render template settings in the Sony AVC which has a cpu-only render option. The Magix AVC template doesn't have that option. Neither do the Mainconcept template groups for dvd and bluray which don't render with gpu at all but the fx plugins do anyway. This is a big deal in dvd and bluray authoring where fx-gpu usage can add sparkle in dark shadow areas. Turning off gpu in video settings brings back the creamy black.

Btw, loved your hybrid animation video. Wow!

fr0sty wrote on 7/26/2019, 7:16 PM

The only difference between the Sony AVC CPU only and Mainconcept is that MC doesn't refer to it as CPU only, even though it actually is.

"And without gpu, all were dramatically slower. Proving without a doubt they all use the gpu similarly"

This is because the GPU is being utilized during the render process (even on Sony AVC CPU only), as you said, to aid with timeline rendering, which includes effects, compositing, etc., anything done to the video from the time it is decoded until it is sent to the encoder, the GPU will help with that no matter which format, CPU only or not, you select, unless you go into preferences and disable the GPU entirely.

Even when you select CPU only in the Sony codec, it still uses the GPU if you do not go into Vegas Preferences and disable the GPU there. However, the Sony AVC codec has always been said to give higher quality results than the Mainconcept/NVENC/VCE/QS encoders do, from what I gather reading posts around here... with the highest quality option being rendering out to x264 using something like Happy Otter Scripts.

I do a lot of animation, so forgive me for not knowing exactly which video you refer to, but thanks!

Howard-Vigorita wrote on 7/26/2019, 8:01 PM

The only difference between the Sony AVC CPU only and Mainconcept is that MC doesn't refer to it as CPU only, even though it actually is.

I think that's exactly right if you are talking about the Sony AVC preset group and the Mainconcept for dvd and BlurRay preset group. But in the Magix AVC presets, selecting Mainconcept defaults to gpu. That was the whole point of my doing the timing tests. My uber-fast cpu couldn't possibly encode or render that fast. Try it yourself on any piece of video. In Magix AVC select Mainconcept, render, and write it down. Then set your gpu to off in video settings and repeat. If you suspect the timeline is still using the gpu anyway, pull your video card and replace it with a plain vanilla one and do it again. I did all that but double checking and verifying is always good.

Kinvermark wrote on 7/26/2019, 9:12 PM

As I mentioned above, we have had some good news regarding the future of benchmarking for Vegas Pro:

Magix have recognized the importance of these kinds of performance tests, and indicated an interest in developing a benchmark test in the future. I don't know of any specifics or timeline, but personally, I am happy suspend any further effort to find an alternate host for our current test, and wait for the "official" one.

In the meanwhile, I hope users will continue to post results for @Former user to add to the spreadsheet, as well as to use this thread as an opportunity to discuss what the ideal test or tests would look like.

N.B For readability, please keep comments to a reasonable length, and if you feel the need to have an elaborate discussion of technical minutia, then start another thread for that. THANKS!

bitman wrote on 7/27/2019, 4:00 AM

@Former user

Here are the results from Belgium taken during the historic all time record breaking temperatures (> 40 C) with the beats of Tomorrowland in my backyard (I can still hear the dance festival from 8 km away).

User name :bitman
Machine    :OB, screen LG 38 inch ultra-wide (21x9)
VP version :16 (B424)
CPU   :i7-8700K
Cores :6
GPU   :NVIDEA RTX 2080Ti (Founders edition) + Intel UHD 630
OC    :No
Render frame size :FHD
Encode mode :QSV
Render time :1 min 16s
FPS   :8-16
Comments :Nvidia selected as gpu, DRAM prev max 200MB, max NBR rendering threads 48

User name :bitman
Machine    :OB, screen LG 38 inch ultra-wide (21x9)
VP version :16 (B424)
CPU   :i7-8700K
Cores :6
GPU   :NVIDEA RTX 2080Ti (Founders edition) + Intel UHD 630
OC    :No
Render frame size :UHD
Encode mode :Nvenc
Render time :1 min 37s
FPS   :8-16
Comments :Nvidia selected as gpu, DRAM prev max 200MB, max NBR rendering threads 48

Grazie wrote on 7/27/2019, 4:31 AM

@bitman - That’s an interesting comparison of your 50sec i7 and my 51sec on my i9 with an almost similar GPU. You’ve done awfully well with that i7.

bitman wrote on 7/27/2019, 5:25 AM

@Grazie

Do not feel bad, I had 50 s on one run and 51 s on another run 😃. Guess which I posted!

Indeed interesting enough, although the rendering time is the same, our systems are quite different. Also the GPU is not almost similar as you state, RTX 2070 is not quite like a RTX 2080ti FE.

Grazie wrote on 7/27/2019, 6:02 AM

@bitman

Nope, not feeling bad, I was wanting to hear the IT science behind the convergence, and you've now supplied it. Thank you.

My next question set/s are/is, what is the value of these tables? Can we see/identify convergence and possibly suggest what hardware to make VP fly and consequently how much to spend or nopt WHAT to spend? Hopefully MAGIX can use this info to direct their own engineers on what to be made possible.

Kinvermark wrote on 7/27/2019, 6:55 AM

@Grazie Well you answered your own question really: Yes, the tables can help identify best configurations, and yes, Magix engineers could use the information for further optimization and/or fixing weak links.

Former user wrote on 7/27/2019, 7:44 AM

@bitman is now updated. Yes I could easily identify the variations in your playback rates, heatwaves and sound waves all converging at once, maybe Grazie can identify with his convergence🤣

Former user wrote on 7/27/2019, 7:59 AM

Not going off topic, but just to say I am curious to see if there will be a substantial difference in playback FPS when VP17 lands, maybe small difference in render as well, you never know,

fr0sty wrote on 7/27/2019, 11:44 AM

Since we're on the topic of render speed, Vegas benched the fastest render time of all Windows NLEs:

Kinvermark wrote on 7/28/2019, 8:09 AM

Vegas has improved a lot with the addition of NVENC and VCE rendering - I bought an RX580 second hand just to take advantage of this.

But... this is only one specific render type / use case. Overall I would say the fastest rendering can be found with Davinci, who seem to have put a lot of effort into in-house development (I am speculating) of highly optimized, but fewer, render types. I would like to see Vegas add a highly optimized cineform render, as that format seems to be the best intermediate option for Windows (considering widespread support in addition to technical factors.)

@Former user Yes, VP 17 features list claims timeline GPU acceleration so playback should improve. Great!

fr0sty wrote on 7/28/2019, 1:56 PM

True, but the m majority of Vegas users are outputting AVC/HEVC files of their projects, so good to know it leads the pack in that format.

OldSmoke wrote on 7/28/2019, 4:53 PM

I finally had all parts together for the new machine, was waiting for the waterblock for the AMD Vega FE.

User name :OldSmoke
Machine    :OB, screens 2x DELL Ultrasharp 27" 2560x1440
VP version :16 (B424)
CPU   :i7-9800K
Cores :8
GPU   :AMD VEGA Pro Frontier Edition
OC    :4.6GHz
Render frame size :FHD
Encode mode :VCE
Render time :47s
FPS   :9-16
Comments :AMD selected as gpu, DRAM prev max 200MB, max rendering threads 16

User name :OldSmoke
Machine    :OB, screens 2x DELL Ultrasharp 27" 2560x1440
VP version :16 (B424)
CPU   :i7-9800K
Cores :8
GPU: AMD VEGA Pro Frontier Edition
OC: 4.6GHz
Render frame size :UHD
Encode mode :VCE
Render time :91s
FPS   :9-16
Comments :AMD selected as gpu, DRAM prev max 200MB, max rendering threads 16

Former user wrote on 7/28/2019, 5:57 PM

@OldSmoke Is updated now. I took a while looking for the iGpu for the i7-9800K, penny dropped eventually and checked out your specs. Nice rig.

fr0sty wrote on 7/28/2019, 6:58 PM

Request: Create a list showing the top 5 machine configs for both render time and playback performance, so we have a post to refer people to when we get the inevitable "help me build my PC" requests. This setup is nice, but FHD and UHD results are clustered together, it would be nice to have a easy to refer to list where fhd, uhd, and timeline performance are separated into ranked categories.

1 … 5 6 7 … 10

of 10