@Kinvermark and others ...

Maybe for usability and simplicity and sorting, I think it would be better to convert the existing FPS Min-Max values from region 1 to an average single value derived from the min/max values. E.G. 7-14 becomes an average FPS of 10.5 for region 1.

This way users can then supply either their observed min-max values or their calculated average single fps value. Only the single average value gets entered into the spreadsheet. This will then facilitate sorting.

Adding an extra column to accommodate the Average FPS just makes it more complex to the viewer I think, not an issue for me adding another column, but my aim is to keep things as simple as is possible.

Best I think then is to add the original Min-Max values to the end of the comments field, so no extra columns added.

I realise that modifying the project settings to get better playback fps is possible, but unless we all do that, doubtful, as it’s one extra job for each user to do, then better to do no modifications to any project settings.

So although it might not give the best fps values, leaving the project fps to uhd as it is, is best since it gives a level playing field for comparing fps between machines.

I am assuming that up to now mostly all users have tested without modifying the project.

On a side note, when @BruceUSA used his own setup, his results probably would have been identical if he had been able to access/use the supplied zip, with what he got, but at the end of the day the testing is only really valid if the original zip is used, unmodified, with the exception of the added region 1.

This is the way I see to display the highest to lowest Avg. FPS. Note that the original Min-Max are not lost, they are added to the end of the comments.

Unless anyone has a major objection to this, I can publish this also at my 1st. post with the others.

I temporarily left out the Rhino's until I hear from him.

My first post is now updated with 5 screen captures of ss.

Looks good. 👍

Ultimately we will need a specialized test for playback smoothness etc. Need to think about what this should look like in Vegas. Best example I have seen so far is the testing done by Puget Systems for DR, PPRO, etc.

e.g.

https://www.pugetsystems.com/labs/articles/Premiere-Pro-CPU-Roundup-AMD-Ryzen-3-AMD-Threadripper-2-Intel-9th-Gen-Intel-X-series-1535/

Ok @Kinvermark, will proceed accordingly.

The thing is, as you probably well know, retrofitting after the event is not easy. In an ideal world everything would have been setup 100% from the get go. So getting users to go back and do more testing after they’ve done their main testing is maybe a bit of an ask?

Frankly, I’ll be happy enough if TheRhino and any others that feel the need to give any more feedback on the FPS do so.

What your suggesting is ideal, but I'm not sure that the numbers, users, are there to warrant it.

Although I know that the current method of getting playback benchmarks in this project isn’t very accurate, without some script say, I don’t know of any better way.

Looks good. 👍

Ultimately we will need a specialized test for playback smoothness etc. Need to think about what this should look like in Vegas. Best example I have seen so far is the testing done by Puget Systems for DR, PPRO, etc.

e.g.

https://www.pugetsystems.com/labs/articles/Premiere-Pro-CPU-Roundup-AMD-Ryzen-3-AMD-Threadripper-2-Intel-9th-Gen-Intel-X-series-1535/

I hope you're not advocating benchmarking with Neat Video as discussed in that article. Only because it makes the Ryzens look so good. Their benchmark is designed to measure the performance of their digital noise filter plugin. Which is a great plugin which might come with Vegas one day. It raises frame rate and clarity by removing video noise from the video processing stream. And would make your benchmark project run and display much faster by cancelling out most of the low visibility content added by the Vegas Video Noise media generator employed throughout. Which I think is perfectly legitimate for separating the wheat from the chaff or measuring the quality of a clip of raw video. But not so much for benchmarking Vegas as a video editor unless your concern is unwanted noise it or a gpu might insert all on its own.

@Former user No, I am not suggesting you make any changes or invent a new test / spreadsheet. My comment is meant to start a discussion to help inform Magix about what we would, ideally, like to see in performance testing.

@Howard-Vigorita Not particularly. Best to have a good look around the Puget site at the various tests. They may have some ideas you like - or not :)

@Former user I have some thoughts on your fps performance data. I think there's a better way to measure it in Vegas. My problem is that my ocular frame rate isn't good enough to capture an accurate reading from a display screen if the tiny numbers are in constant flux. Granted sometimes it just comes up at a low rate and stays there. Or varies within a narrow range. But other times it's full frame rate most of the way though till it plummets to something like .2 for the last quarter of the region... not sure a weighted average is even a useful metric there. I would suggest something simpler, more objective, and more meaningful to users. Just ask users to adjust and report their 'best, good, preview' setting to what yields a full frame rate. I know when I'm in edit mode, I tend to leave it at good(auto) till near the end of my workflow when I try for best(full). Also getting rid of region 1 would prevent an accidental region-only rendering... I just did a 1st time install of Vegas 16 on an old dog last night and didn't realize I had to start over running your benchmark because Vegas turns on the region-only render option by default on fresh installs.

@Former user No, I am not suggesting you make any changes or invent a new test / spreadsheet. My comment is meant to start a discussion to help inform Magix about what we would, ideally, like to see in performance testing.

@Kinvermark Gotcha.

My first post is now updated with 5 screen captures of ss.

Just a reminder that users can download the ss at any time and do their own sorting and filtering. It's created in Excel 2016 but saved as version 97-2003 for backward compatibility. However, when I loaded it in Excel 2000 I found that it displayed ok, filtered ok, but macro threw up an error.

 

@Howard-Vigorita Not particularly. Best to have a good look around the Puget site at the various tests. They may have some ideas you like - or not :)

@Kinvermark Know them well. In fact my suggestion about frame rate measurement is based on how they do it. Adapted to the controls in Vegas. Here's a direct quote of what they do in Adobe Premier:

How does the scoring work?

The scoring system used in our benchmark is based on the performance relative to the FPS of the test media. If the media is 29.97FPS and the system plays it back at 29.97FPS, that results in a score of "100". Likewise, if it can only play back at half the FPS, the score would be "50". Scores for Live Playback cannot go above 100 (since you cannot play at faster than the media's FPS), although scores for Exporting can go well above 100 depending on the performance of the system.

I personally do not approve of or recommend their numerical grading system, btw. I would much prefer they report the exact setting they used to reach their evaluation. So I can repeat it myself on my own systems and make my own evaluation.

The HW Acceleration column was introduced much later so users mainly would have had no reason to mention it, and supply that data. I overlooked Damian's mention, Oldsmoke's is entered with mine also.

What I’ll do is assume that all other users used their main Gpu as HW Acc. and fill in the blanks accordingly.

If any users did otherwise then simply let me know and I’ll change it.

Here are my results, updated to show FPS on Region 1:

User name: The_Rhino
Machine: OB
VP version:16 (B424)
CPU: 9900K OC to 4.9 GHz
Cores: 8
GPU: Vega 64 Liquid Cooled
OC: Yes @ 4.9 GHz
Render frame size: HD 1080 (1920x1080)
Encode mode: AMD VCE
FHD Render time: 0:48; UHD Render time: 1:39
FPS: 13.25 average (9-17.5)
Comments: CPU Useage=7%-36% & OC to 4.9Ghz; GPU=19%-28% & OC to 1750 Mhz.

Here are my UPDATED FPS (FHD & UHD)results for Region 1. 9 was lowest & 17.5 was highest displayed. I see 12, 13 & 14-something a lot. Vega 64 is Liquid-Cooled & overclocked to 1750 MHz while undervolted to 1250 Mv.

Old Xeon System, updated to show FPS on Region 1:

User name: The_Rhino
Machine: OB
VP version:16 (B424)
CPU: Xeon 5660 OC to 3.9 GHz
Cores: 6
GPU: AMD RX 570
OC: Yes @ 3.9 GHz
Render frame size: HD 1080 (1920x1080)
Encode mode: AMD VCE
FHD Render time: 1:38; UHD Render Time: 2:48
FPS: 4.25 average; (2.5 - 6)
Comments: Vegas CPU=15%-39%, GPU=0%-15% (35% - 45% for UHD).

Old Xeon preview is unbearable to watch at BEST setting... 4.25 fps average...

@TheRhino Thanks for getting back, all are updated now.

"Old Xeon preview is unbearable to watch at BEST setting... 4.25 fps average..."

Try watching my not so old at all Laptop 1.15 fps 😫

@TheRhino I slipped up earlier, I didn’t see that you had included FHD and UHD render times for both machines. Is now updated. Maybe I did it incorrectly before when you first posted, it’s ok now.

Your top result for render time isn't accurate, as they listed 91 seconds instead of 1 minute 31 seconds, and it messed up the sorting putting them ahead of render times that were actually faster.

Your top result for render time isn't accurate, as they listed 91 seconds instead of 1 minute 31 seconds, and it messed up the sorting putting them ahead of render times that were actually faster.

@fr0sty - Oo-er! Well spotted Sir! Keeping it FROSTY.

Thanks @Former user, hadn't seen that. Quite a bit slower at 2:16 vs 1:49 with AMD VCE

@petecarney Hi Peter, I may have overlooked that you might wish to have this Mainconcept AKA CPU 2:16s render time reading entered? Do let me know and I'll be glad to update it accordingly. I'll assume all of the rest of your data items were the same, thanks.

@Grazie and other contributers … If at any stage I slip up and changes need to be made do point it out, but be specific as to what, where, who's users data is incorrectly entered/displayed. To help in this I have numbered the 5 main screen grabs at my first post as [1], [2], [3], [4], and [5]. Also there's a line number, currently 1 to 31 on the left of the spreadsheet that references the data.

@Former user . You’re doing immensely valuable work for us; for Scopers of new builds AND MAGIX to garner Hardware feedback hopefully to influence their plans going forward.

@Grazie Hi Grazie, much appreciated, RE: the discrepancy, who, where exactly? I can find no 1:31s entry. Oldsmokes is :91s all right.

@Former user So, did you find it? If not have a look at this:

@Grazie No I didn’t find it? You’ve circled 0:91s, but it is in it’s correctly sorted order with the rest, what is wrong? I just don’t see the issue?