Feature request: Standardized 4k test project.

Kinvermark wrote on 7/7/2019, 1:31 PM

I think we need a standard 4k project designed and hosted by Magix. Purpose would be to allow forum members to test the performance of various hardware configurations. Doesn't have to be too complicated or long, just enough to show differences in performance.

A simple table of results could be compiled.

IMO, we would need preset render templates for various hardware accelerated renders (VCE, NVENC, QSV, Open-CL) and the timeline media should be a common UHD camera format like 100-150 Mbps 8-bit 4-2-0 h.264 footage.

Opinions?

Comments

fr0sty wrote on 7/7/2019, 1:36 PM

Vegas 16 did come with a sample project that could be used.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

Former user wrote on 7/7/2019, 5:51 PM

@Kinvermark I have already comprehensively commented in another thread, no need to repeat it all here, best of luck with it ... https://www.vegascreativesoftware.info/us/forum/red-car-test-where--115995/?page=2

Kinvermark wrote on 7/7/2019, 6:05 PM

@Former user

That's too bad. I thought that this was something you and several others had a keen interest in. Unfortunately, the other threads have become a confusing mess of inaccurate information sprinkled with disagreements.

Anyway, I repect the fact you have already contributed your time.

Former user wrote on 7/7/2019, 6:16 PM

@Kinvermark Ive spent a lot of my time today already, writing up recommendations and testing another users download. The disagreements are there in the other thread for sure, still ongoing. I gave a link above to all of my suggestions, rather than copy and paste it all over your new thread. I decided to copy it all over his new thread anyway, LOL, see further down.

I do have an interest in it, but only if its done properly, and not a mess.

Ive already contributed quite a lot in the other thread and don’t want to be seen to monopolise the topic.

Best of luck with it.

Musicvid wrote on 7/7/2019, 6:28 PM

@Kinvermark

I hope this doesn't sound insipid, but it may be one of those things that won't exist until you publish it. There is so much original camera footage on Vimeo, and ideas for synthetic benchmarks should reveal themselves. There was a synthetic FHD benchmark on SCS, but it was user-created.

fr0sty wrote on 7/7/2019, 6:40 PM

Just use the demo project that came with Vegas 16, set everything to 4K, and go. I don't see why that project doesn't fit the requirements, it is more complex than the red car test everyone seems to love so much.

Systems:

Desktop

AMD Ryzen 7 1800x 8 core 16 thread at stock speed

64GB 3000mhz DDR4

Geforce RTX 3090

Windows 10

Laptop:

ASUS Zenbook Pro Duo 32GB (9980HK CPU, RTX 2060 GPU, dual 4K touch screens, main one OLED HDR)

Former user wrote on 7/8/2019, 5:44 AM

When the updated 4K Red Car test is again available for download, just a bit of feedback, in case anyone else runs into this problem.  All this is if rendering to a 4K output file.

I can't render past say 16% mark using PC or laptop.  The culprit within the project is in the screenshot, once I muted it it fully rendered.  Maybe Marco or people more knowledgeable could explain what's happening within this small section?  Anyway, anyone else having this issue are forewarned.  It starts at 15s;23f position.

I fixed it by un-muting and then disabling Nvidia HW Acceleration, but then everything is slower.

Also worth mentioning that the introduction of a quicktime .mov piece of media into the 4K version maybe isn’t such a good idea.

Former user wrote on 7/8/2019, 7:21 AM

Whenever we finally get going, and all is agreed beforehand, with this @Kinvermark I think we should start a new thread, Benchmarking.

I've uploaded a "User data input" form .txt file here ... https://www.dropbox.com/sh/we0paonvz9fmdce/AABTA_DgOiRFQ67kaYeoBOAra?dl=0

This data would be entered into say a spreadsheet that's available to download, sort accordingly by anyone.

Goes without saying that is just my suggestion of the way forward.

I've also uploaded a copy of my post that was in @Grazie's thread. I'll try pasting into my next post here.

 

It needs to be decided also on the output render type file 2K or 4K uhd.

Former user wrote on 7/8/2019, 7:35 AM

Before launching a new thread “Benchmarking” a few points need to be agreed upon. It isn’t necessary to get everything right before the starting whistle blows, but for sure a good portion of them.
 
1. Is it legitimate to use the Red Car project in 4K modified or any form.
2. Agree on a single output render template, to 2K or to 4K or both?
3. Agree on rendering to say h264 or Hevc or both?
4. Agree on the format of data to use in say a spread sheet.
5. Agree on the format that users will post their results in, should be fixed mostly.
6. Agree that the Red Car 4K modified version is aok, i.e. that it won’t need any further changes.
7. Agree that the data, say a spread sheet would be regularly made available to a downloadable location.
Add more suggestions as needed ...
 
Item 1 above .. If there's an issue with it then an alternative might be converting to 4K the Magix 2k benchmark thats supplied with VP, Magix is unlikely to object.
 

Item 6 above .. A few users need to download the suggested Red Car 4K project and confirm that its 2k to 4k media conversions are correctly done, that no further changes will be needed. For example are the fps settings correct, should the data rate be higher or lower, is the codec used ok for the new 4K media.
 

Item 4 below ..order of items not finalised, suggestions are welcome ...
 
User name: Machine: VP version: CPU: Cores: GPU: OC: Encoding: Render time: FPS: Comments: etc
 
The Machine entry item could be say [OB] or a specific manufacturer model example [HP dv7]. OB is own build.
VP version .. example .. VP16.

The Cpu and Gpu items would have say multiple entries each, Gpu .. manufacturer and model, example .. [Nvidia]   [Gtx 1070]. Cpu .. manufacturer, model, cores, example .. [Intel] [i7 4790k] [4].

The OC overclock item would be say a .. [yes], [no] or [partially] entry. For example if the cpu wasn’t overclocked but the gpu was then maybe partially covers it? Or the cpu had a mild OC.

Encoding could be two fields .. H264 or Hevc and Cpu Nvenc Vce Qsv Cuda etc. Example [h264] [CPU].

The FPS entry could be entered as say [full] if it plays at 29.97 or an actual value. Use the comments field to clarify if necessary, I have entered what happens say 95% of the time. It would be necessary to do the following before testing preview playback … 1) Make sure that proxy files were not in use. 2) That preview was at Best/Full. 3) Adjust size and quality for optimal playback is not checked, right click the preview screen to confirm on or off.

 

The comments entry could be used to clarify that its a dual cpu, gpu etc.

j-v wrote on 7/8/2019, 7:49 AM

I modified the SampleProject that became delivered with Vegas Pro and VMS 16 to a "SampleProject 4K", much newer and more modern than the old Red Car benchmark.
If you want you may use it here as a 4K test project for others.
I like to hear if it is wanted.
 

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 23H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver
560.81 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2127
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 560.81 and Intel UHD Graphics 630 with driver 31.0.101.2127
Vegas software: VP 10 to 21 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)

 

Former user wrote on 7/8/2019, 7:58 AM

@j-v Fine by me. @Howard-Vigorita has one also, so we have to choose one of them, thanks to you both for doing the conversions and going to the trouble.

j-v wrote on 7/8/2019, 8:04 AM

Ok, I shall upload it to my dropbox from where it can be downloaded during 1 week.
Tonight I try to make it all right, now I'm busy with other things

Last changed by j-v on 7/8/2019, 8:43 AM, changed a total of 1 times.

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 23H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver
560.81 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2127
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 560.81 and Intel UHD Graphics 630 with driver 31.0.101.2127
Vegas software: VP 10 to 21 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)

 

Grazie wrote on 7/8/2019, 9:16 AM

Thanks Guys!

So, how many variations of the Stress Test were out there? And did I hit the Stress Test that is has that questionable Region 5 ? If this is the case then, ostensibly, I don’t need to swap out my GPU? Any comments?

Presently my rig is performing like a speed demon.

Former user wrote on 7/8/2019, 9:25 AM

@Grazie There was a simpler test for playback put up by @OldSmoke and another one called the running man by @bitman, another simple enough 4K clip. But when a cross fade for either is implemented it can show issues, use 2 or more pieces on the timeline. Very important to disable proxies, run at Best/Full to really stress test playback.

 

These were my own test results ... I think no cross fade used here, more a render time test ...

Running Man test... (3 pieces of it end to end, 37s 12f duration.)

Corsair i9900K PC build ... November 2018.

Graphics card Nvidia, MSI GTX 1080 z, 32gb. ram, VP16, build 307.

 

HW Acceleration with either Nvidia, Intel graphics or none.

Render with either Nvenc, Intel QSV or CPU only.  Render templates set at default for Nvenc and QSV.  The Hevc template render values are in brackets, {Hevc}.

Overclock in brackets, (Overclocked).

 

HW Acceleration = Nvidia

Render with Nvenc .......  0:23s ... (0:20s ...... {0:29s})

Render with QSV ..........  0:22s ... (0:18s ....... {0:34s})

Render with Cpu only ...  1:55s ... (1:48s ....... {1:04s} cpu 75 degrees.)

 

HW Acceleration = Intel Graphics

Render with Nvenc .......  (0:21s ... {0:30s})

Render with QSV ..........  (0:19s ... {0:33s})

Render with Cpu only ...  (1:51s .... {1:05s})

 

HW Acceleration = None

Render with Nvenc .......  0:23s ... (0:22s ......... {0:29s}

Render with QSV ..........  0:22s ... (0:19s .......... {0:34s})

Render with Cpu only ...  1:55s  ... (1:49s ......... {1:04})

=============================================================

Former user wrote on 7/8/2019, 9:32 AM

@Grazie Theres also the Magix sample project, included with VP ...

Magix Sample project... 41s duration.

Corsair i9900K PC build ... November 2018.

Graphics card Nvidia, MSI GTX 1080 z, 32gb. ram, VP16, build 307.

 

HW Acceleration with either Nvidia, Intel graphics or none.

Render with either Nvenc, Intel QSV or CPU only.  Render templates set at default for Nvenc and QSV, i.e. 1920  1080, 25fps, 24/12 data rate.  I used the  “Magix Sample project” benchmark, Overclock in brackets, (Overclocked).

 

HW Acceleration = Nvidia

Render with Nvenc .......  (0:33s) ..... [4790K 0:32s].

Render with QSV ..........  (0:32s)

Render with Cpu only ...  (0:39s)

 

HW Acceleration = Intel Graphics

Render with Nvenc .......  

Render with QSV ..........  

Render with Cpu only ...  

 

HW Acceleration = None

Render with Nvenc .......  (2:24s)

Render with QSV ..........  (2:36s)

Render with Cpu only ...  (2:39s)

Former user wrote on 7/8/2019, 10:11 AM

@Grazie this is @OldSmoke's link to his xavc-s files ... https://onedrive.live.com/?authkey=%21AIzZvr%2DdgrDDYAU&id=4FC8B86C60F99F79%2112406&cid=4FC8B86C60F99F79

John Rofano's test ...

https://f1.creativecow.net/file.php?id=7248&folder=vegas-pro-120-gpu-render-test-project

Former user wrote on 7/8/2019, 10:20 AM

@Grazie This is my link to the running man test, I couldn't find @bitman's link.

https://www.dropbox.com/sh/zfjra27jeyba80c/AAA1X0jzwsyWw5nroaFbq4-za?dl=0

j-v wrote on 7/8/2019, 11:15 AM

My "Sample Project 4K": https://www.dropbox.com/s/huk6mfdfjaxawb8/SampleProject%204K.zip?dl=0

Link is one week from now available.

met vriendelijke groet
Marten

Camera : Pan X900, GoPro Hero7 Hero Black, DJI Osmo Pocket, Samsung Galaxy A8
Desktop :MB Gigabyte Z390M, W11 home version 23H2, i7 9700 4.7Ghz,16 DDR4 GB RAM, Gef. GTX 1660 Ti with driver
560.81 Studiodriver and Intel HD graphics 630 with driver 31.0.101.2127
Laptop  :Asus ROG Str G712L, W11 home version 23H2, CPU i7-10875H, 16 GB RAM, NVIDIA GeForce RTX 2070 with Studiodriver 560.81 and Intel UHD Graphics 630 with driver 31.0.101.2127
Vegas software: VP 10 to 21 and VMS(pl) 10,12 to 17.
TV      :LG 4K 55EG960V

My slogan is: BE OR BECOME A STEM CELL DONOR!!! (because it saved my life in 2016)

 

Kinvermark wrote on 7/8/2019, 11:52 AM

@j-v

I am happy to use that project, but we will still need somewhere to host it permanently and give it a unique identifier so as to avoid future confusion about which test is being run.

Maybe one of the MODERATORS can suggest a way to have both the project test file and some kind of results table (TBD) hosted on this forum?

Grazie wrote on 7/8/2019, 12:03 PM

Ladies and Gentlemen, am I the only one recognising the large Pachyderm squatting in the corner? Am I?

Former user wrote on 7/8/2019, 12:57 PM

@j-v Way to go, for purely selfish reasons, my laptop managed to render it out fully to 4K uhd in Nvenc. @Howard-Vigorita version is just too hard on my system, but anyway, lets see what others have to say. Maybe harder is better?

Render time 2:01s, playback most of the time was 1.1fps. If the moderators can't host it maybe as I said earlier several users can, I have no problem.doing so. https://www.vegascreativesoftware.info/us/proxy/f5df4c408b36cf42b3c7/?link=https%3A//www.dropbox.com/sh/we0paonvz9fmdce/AABTA_DgOiRFQ67kaYeoBOAra%3Fdl%3D0

I used this template. We can get going with this soon enough, we need to agree on a few things, template, 2k/4k render etc see my previous post.

@Grazie "Pachyderm" what? Let me guess, will Magix give permission, maybe best not to ask? They should be delighted.

Grazie wrote on 7/8/2019, 1:13 PM

@Grazie "Pachyderm" what? Let me guess, will Magix give permission, maybe best not to ask? They should be delighted.

@Former user Lol.... No, not MAGIX, but us using variations on the Red Car Theme. Those that had had success would needed to have tested the penultimate Veg to have got “failure”, like I have with my week old MONSTA!

Im hoping to success with at least playback Preview on Best. We’ll see.

Last changed by Grazie on 7/8/2019, 1:14 PM, changed a total of 1 times.

Grazie

PC 10 64-bit 64gb * Intel Core i9 10900X s2066 * EVGA RTX 3080 XC3 Ultra 10GB - Studio Driver 551.23 * 4x16G CorsVengLPX DDR4 2666C16 * Asus TUF X299 MK 2


Cameras: Canon XF300 + PowerShot SX60HS Bridge

Former user wrote on 7/8/2019, 6:08 PM

@Kinvermark Just to know that @Howard-Vigorita has updated his Red Car 4K download ... https://www.vegascreativesoftware.info/us/proxy/a08857d93e81b208e8a5/?link=https%3A//drive.google.com/open%3Fid%3D1TK2lyFZSlzU9JQ8xO1mvoj-zl9tyj2lc

 

Comments here in this thread ...

https://www.vegascreativesoftware.info/us/forum/red-car-test-where--115995/?page=4#ca721987

Former user wrote on 7/9/2019, 2:40 AM

@Kinvermark @Howard-Vigorita @j-v I see we are at a bit of an impasse, two 4K contenders for the Benchmark testing. Our ringmaster has gone asleep. May I suggest a way to move this forward?

How about we take a week to allow any and all users that have an interest in this topic to vote on it.

If one of the moderators is willing to offer their time to collect and count the votes, @vkmast, @marco etc @nick-hope, other moderators that I don’t know of.

The voting can be done via a private message to the moderator that volunteers to do the counting.

The possible choices would be, in no particular order ...

JV

Howard

Both

None

Users would simply send in one of the above as their choice to be counted and the result announced by the moderator a week from when the moderator opened the voting. Its private voting so no one is going to be embarrassed, result only to be announced, plus total votes cast, just to give an idea of the interest or otherwise in this endeavour.

——————————————————————————————————————————

By all means, make any suggestions, modifications on my idea above.