Rendertest.veg results somewhat surprising

Jayster wrote on 6/14/2006, 9:46 PM
Thanks, Glenn Chan, for giving me the link to this test a few days ago! This rendertest is a great idea, this way you can learn what's the best configuration for your setup. I don't know if the parameters or results would be different if we had an HDV rendertest, but nonetheless it's quite useful to see what happens using the old rendertest.

My "best speed" is quite consistently 39s. I am running 64-bit Windows XP on an AMD Athlon X2 4400+ OC'd to 2420 MHz (other details are in the system spec of my profile).

Some things about the configuration for this speed surprised me a bit:
* Video Preview RAM at 0 slows it down a factor of almost 3 (to 1:10). 64MB preview RAM or higher was best
* Turning on or off the "show progress in video preview window while rendering" setting in the Vegas preferences dialog made no difference.
* Closing the preview window also was irrelevant
* Having an external preview via FW was irrelevant
* Decreasing threads to 1 slowed it x3. Same (much better) max speed with threads at 2 and 4
* Drive choice was irrelevant (one is RAID0 other is single drive)

I'm not surprised about the drives being irrelevant. This veg file uses only generated media, thus the only I/O going on is output. The 18MB output file is peanuts.

The external preview monitor being on was irrelevant. Maybe that's because the whole test is in NTSC DV. If I were rendering an HDV clip, Vegas would have to do an additional "render" to convert HDV to DV for the preview monitor. That might be significant.

The fact that having preview ram at zero made it slower goes against lots of forum posts where somebody said to set it at zero for best speed. But then I think they were not using Vegas 6.0 d for those tests. Setting RAM preview to 512MB gave exactly the same results as 64MB. But I noticed that the RAM in use by Vegas was usually at ~60MB, so I guess the excess RAM for preview was irrelevant. I don't know what differences would happen in a more RAM hogging test.

I don't know if this differs much from other people's results, but I found it to be easily repeatable. I logged my times in an Excel file (for my own benefit, but if anyone is actually interested [doubtful!] I can send it.

Comments

JJKizak wrote on 6/15/2006, 5:22 AM
My AMD 4600+ X2 comes out at 41 sec no overclocking 4 gig ram with V6d at defaults and set to best. The XP pro"gadgets" are all unchecked as well as indexing and restore. The processor runs most of the time at 100% and sometimes at 55% when doing straight NTSC 4 x 3 with no fx's, etc.

JJK
fldave wrote on 6/15/2006, 5:49 AM
Jayster,
I did some benchmark testing with a different veg to test the dynamic ram settings. For that veg on my machine I found 16MB and 64MB to be the worst settings, it was HDV, also. It probably has to do with which effects are in place.

Interestingly, not everyone saw improvements, but many who said that Vegas 6 was slower than Vegas 5 saw improvements when they changed their settings from the default 16MB dynamic ram.
Jayster wrote on 6/15/2006, 7:22 AM
fldave:
I set off another test last night before I called it a night, this time using HDV. It's the rendertest that DSE posted last November at VASST.

I was using my fastest configuration (preview RAM at 64MB) and next morning I found the test had completed in 1:27:09 (almost one and a half hours). I'll try it again tonight because I'm curious about your finding that HDV is slower at this preview RAM setting.

Interestingly, my time of almost 1.5 hours seems to be a lot faster than what others were reporting. I see that all their tests were done before Vegas came out with 6.0d, so it must be that Sony made some serious rendering improvements in 6.0d.

Also, I explicitly followed the instructions on the web page, which led to doing the HDV render with the project properties unchanged (render quality at "good", not "best"). If the times reported on the page were done at "best" quality (which wasn't called for on the web page) then this would also explain the speed differences between mine and theirs. But, for all our benefits, I'm hoping it's because Vegas got better in 6.0d. We all appreciate improvements:-)
fldave wrote on 6/15/2006, 7:38 AM
Spot's HDV render test was grueling. My 3.2 HT pentium never did finish it, with any setting. Interestingly, my trusty old dual 1Ghz Pentium 3 finished it...in 10+hours!

I'm running some tests again with the original rendertest on the 3.2 pentium, preview window closed/open at various ram settings.
Initial results: Best 0ram 1:17; Best 16ram 1:33.
continued...
fldave wrote on 6/15/2006, 6:58 PM
Testing with the official rendertest.veg, all runs with 2 threads. I rebooted between each run to remove any memory advantage.

Preview Window
Closed Open
RAM MB Best Good Best

000 1:14 :45 1:14
016 1:33 :58 1:41 ==> Vegas Default Install Setting
064 1:02 :39 1:02
128 1:02 :39 1:02
512 1:02 :38 1:02


I still stand by the fact that there are problems with some Vegas settings for dynamic ram. I'm sure it is purely a factor of FX on the timeline and format. My HDV render testing showed that the 64MB definitely had problems, not this SD testing though. My HD test included a chroma key FX, so I would expect some additional ram requirements.


Edited: sorry for the crunched numbers. Preview window closed was run with both Best, then Good; Preview window open (last column) run only with Best.
Jayster wrote on 6/16/2006, 7:23 AM
Thanks fldave for posting. I did some more tests and between us I think we may have some interesting findings.

I resumed testing using Spot's HDV Render test. His project also has ChromaKey portions, and more. I kicked it off with preview RAM set to 0. At 2 hours it had only crunched through to the 80% point. I had previously completed the whole thing in less than 1.5 hours with preview RAM at 64, so I cancelled the render.

The key observation I noted is that with preview RAM at 0, Vegas wasn't fully utilizing both CPU cores. One of the cores was running at about 90% utilization and the other was at quite a low utilization (the Task Manager reported an average of 50%). As soon as I put the preview RAM back to 64MB, both cores went back to being pegged at 100% and the render went dramatically faster.

I notice on your profile that your system apparently is a single core P4. If this is correct, you wouldn't have the same issue I observed. You noted it was slowest at 16MB on the old rendertest. I had found 0 and 16 were both equally slow on my system. For the benefit of single-core systems, you might want to check and see if CPU utilization differs, i.e. is it at less than 100% for 0 and 16?

I also wanted to test my theory that doing an HDV render while running an external, FW-connected preview monitor would slow things down. I ran Spot's grueling HDV rendertest with the external SD monitor active and found that Vegas didn't update it during the render. Instead, it updated the Vegas preview window. The external monitor just kept displaying the first frame. Interesting. And the test finished at 1 hour and 34 minutes, only 7 minutes slower than when I ran it with no preview window.
Jayster wrote on 6/16/2006, 11:12 AM
There were some other things I noticed that are worth mentioning. If you read the original thread from last November about Spot's HDV rendertest there are a lot of people saying they couldn't complete the test because they were getting out of memory errors. Spot also wrote in his thread that at one point in time only two of their machines could do it.

On my own machine, I did the test successfully twice with an average of 1.5 hours. This beats all the other scores posted on the website (they range from 2:16:24 on a dual Opteron system to 10 hours on a dual Intel P3 system). [There may be times a good bit faster out there, but they haven't been posted. Spot - any updates or new feedback?] So what's the deal with my resutls?

My system is no miracle-worker. I am not ready to put it on the "bragging forum" thread that someone else started:-) Someone would surely embarrass me quickly with a dual Opteron machine...

I can see two possible explanations for why I was able to complete this HDV render test and in record time. First is that I tested with Vegas 6.0d (everyone else who posted did this on 6.0c or earlier). Second, and I think this is the big one, I was using the 64-bit Windows XP x64, which handles memory allocation a lot differently than the 32-bit version of Windows XP.

32 bits means 4GB of address space. Most of the 32 bit Windows operating systems split this address space in half (2GB for the OS and 2GB for applications). A few months ago I read on a Microsoft technical page that the OS also takes up some of the application's memory space with shared dlls and things like this.

When the sum of RAM and virtual memory that Vegas consumes reaches 2GB (minus whatever the 32 bit OS is taking), it will get "out of memory" errors and probably crash. This is regardless of how much HD or RAM is physically in the machine. The limitation is the 32 bit addressing space and how the OS divides it, NOT the actual amount of memory on the machine.

According to Microsoft, even if you are running in a 64 bit Windows XP environment, a 32 bit application still uses 32 bits for memory addressing (because it is a 32 bit app). The big difference when running on 64 bit XP is this: the operating system doesn't share (i.e. take away) the application's memory space like a 32 bit OS would.

Running this rendertest illustrated what Microsoft is talking about. When I was doing projects in 32 bit XP, Vegas would die after RAM + VM reached about 1.4 GB, indicating that XP was "sharing" about 600 MB of Vegas's address space (probably for stuff like the .NET framework, system dlls which Vegas is dependent on, etc.). In contrast, Vegas is making use of the full 2GB address space on my Windows XP x64 system, which allowed the test to complete without any problems.

While I was watching the task manager during the render I noticed that after getting pretty far along in this grueling test project the physical RAM allocated shot up to 1.4GB and (at different times) the virtual memory went as high as 800 MB! So, to put it mildly, this particular rendertest was pushing Vegas to the extreme limit for memory usage. I could have never got this far when I had the 32 bit version of XP on my machine.

This means that for me (and this would vary for others depending on what other services, drivers, etc. are running on their machines), switching to 64 bit Windows gave Vegas about 30% more address capacity to work with (600MB out of 2GB). That was just enough to get this rendertest to complete successfully. I'm not suggesting that you must have a 64 bit OS to complete the HDV rendertest. I'm only saying that it helps and for me it was the make/break point.

I would also *guess* that with more of the RAM available to Vegas, there was less dependence on the swap file, and this could account for the higher speed that I observed compared to others' posted results.

I know that a few others on this forum are using a 64 bit Windows OS. I would be interested to hear their observations. And I would mention that before I upgraded my OS to 64 bit, I made sure all my hardware was supported by the vendors with 64 bit drivers.

Finally, there are some ways that Sony could increase the amount of memory space available for Vegas while living on a 32 bit OS. (Specifically, there is a compiler switch called \LARGEADDRESSAWARE that would enable up to 3GB of address space to be used by Vegas).

I wonder what Sony will do to improve the situation in Vegas 7? Will they start compiling the 32-bit version to use more address space? And wIll they make it (or a future update) 64-bit native for Windows Vista?
Coursedesign wrote on 6/16/2006, 11:37 AM
Great work, Jayster!

Many thanks for sharing this.

It may give more people courage to take the step!
fldave wrote on 6/16/2006, 4:12 PM
Jayster-
"I notice on your profile that your system apparently is a single core P4. If this is correct, you wouldn't have the same issue I observed."

Not necessarily. I verified that the Hyperthreading in my 3.2 Pentium does work similarly to my true dual processor P3. XP Pro is set up as 2 cpus, and I have the dual task monitor CPU monitors.

Thanks for your testing on this, makes sense about memory usage. Not sure if you saw my original testing page on this ====>Here
rmack350 wrote on 6/16/2006, 5:00 PM
Jayster,

Couple of things going on with address space in a 32 bit system. Yes, Windows is using memory, of course, and this may well limit what Vegas can get hold of to well below 2GB.

The other thing is that other hardware besides memory needs addresses. In a 32bit system those addresses start up at 4GB and start filling downward, using up addresses that your memory might otherwise use. So, when you install 4GB of RAM in a 32bit system much of the last GB can't be used because it can't be addressed. It's like having a whole subdivision of houses but you've run out of street numbers. No one gets any mail.

That other hardware isn't using up the memory, just the addresses. In a 64bit system there are tons of addresses available far above 4GB, so there's no competition. You can use all 4GB, and of course you're able to address and use more than 4 GB as well.

Now, I have no idea if Win32 puts Vegas in the first 2GB, the second 2 GB, or any old discontiguous bunch of memory, but it sure sounds like Win64 does more than just make the 4th GB of memory usable. And that's a good thing!

Rob
Jayster wrote on 6/16/2006, 10:54 PM
Coursedesign - thanks for the acknowledgement. It's nice to know it when people find value in the time and effort we spend on some of our research, testing, and posting... RMack - thanks for the additional info. I didn't know that!

fldave - yes, I did see your post a while back about your different results with different preview RAM settings. That and other posts made me really wonder about it and motivated me to conduct my own tests. I downloaded your test and tried it at two settings, 64MB and 0MB. My results were consistent with yours. With 64MB preview RAM I got a render time of 1:01:27. And with 0 MB preview RAM, it finished in only 27 minutes and 9 seconds! I had closed Vegas, deleted the output files, and started over. To me this is bizarre. Using your HDV render test says I should set preview RAM at 0. Using the ones that people in this forum use (the old render test and Spot's HDV render test) consistently show that I should never set preview ram to zero.

So it appears there is no blanket solution. What differences could I observe that might explain? None that I know of. When running Spot's HDV render @ 64MB preview ram, I saw both cores were always completely pegged at 100%. When running your veg file, it was always the same CPU / core usage, with waves centered probably around 98%. I would also note that the complexity of your project is a lot less than Spot's HDV test.

fldave, I noticed in your test data that setting your "threads" to two on your single core, hyperthreading machine was of little benefit. On my dual core system it is a big mistake to make it equal 1 instead of 2. I don't have any good theory to offer, but FYI I read somewhere an explanation of what hyperthreading does. Intel found that there is a bottleneck somewhere in their architecture's pipeline that causes the CPU to waste a lot of cycles waiting. They came up with a way to utilize these wait cycles for another thread of execution. This is apparently what hyperthreading is about. Maybe your lack of benefit from the threads=2 setting is because the particular veg file and/or Vegas's implementation of the rendering engine weren't fitting into the conditions that best make use of hyperthreading. But this theory is weak. Cineform states that they make great use and benefit from hyperthreading.

So what can we conclude from all of this testing and discussion? My thinking is that there is no blanket solution, no perfect answer of "set A to x and B to y." For a project you are working on, I guess it will pay off to just pick a small region, set a preview ram, then start rendering. Observe CPU utilization and render time. Then you can repeat at a different preview ram and compare. It's probably best to do this by sampling the most complex part of your project, or anyways the part which uses effects that have the most impact on rendering speeds (somebody else kindly provided a table with this data).

And it can be seen that not choosing optimal values for these things can double or perhaps even triple your rendering time. Yikes!
fldave wrote on 6/17/2006, 12:12 AM
Jayster,
Thanks for your persistence in this. I'm still scratching my head.

"Intel found that there is a bottleneck somewhere in their architecture's pipeline that causes the CPU to waste a lot of cycles waiting."

It figures. I'll just wait to upgrade to the Cell processor. 9 cores. Run 4 instances of dual XP with Vegas network rendering on the same machine??