2ND HARD DRIVE NEEDED FOR RENDERING?

yellowsubmarine wrote on 12/8/2009, 8:58 PM
Sony Vegas Pro 8 user - novice/noob.

I just got a new desktop. i7 quad cpu - decent pc but only one hard drive, SATA 7,200 RPM.

It has been a while for me so I need to ask whether it is still preferable to have a second hard drive for the rendering. I think back then, it was considered that the c drive would be for the program and the rendering would go to the d drive. Is still the current thinking? Thanks.

Comments

randy-stewart wrote on 12/8/2009, 9:07 PM
Yes, it's better not render to the same hard drive as your program is on. It's hard for it to read and write to the same drive without causing glitches.
Randy
Terry Esslinger wrote on 12/8/2009, 9:08 PM
Yup, you're welcome.
Mike M. wrote on 12/8/2009, 9:14 PM
I have a comment and question on this. I wonder if using a second hard drive would also include an external USB 2.0 drive. I've noticed a lot of lockup and freezing with Vegas when the source files are on a external drive.

Another value of having a second internal drive is that you can setup a paging file (virtual memory) on it as well which will improve performance overall. Source: http://www.theeldergeek.com/locating_the_page_file.htm
musicvid10 wrote on 12/8/2009, 9:37 PM
USB externals are not reported as being as reliable as Firewire externals for capture and render. There is a wealth of discussion available on this by doing a search of these forums.

The added value of having your swap file on a second drive is measurable, but negligible.

CERTAINLY I would not have my page file and render files on the same external drive. That is counterintuitive, considering the number of additional simultaneous read/write operations that kind of setup would create.
ritsmer wrote on 12/9/2009, 2:53 AM
Agree fully with musicvid.

Could also be a very good idea to split up some of your larger disks into logical disks as Windows does not really fancy reading from- and writing to the same logical disk at the same time.

If you use Windows 7 it has a fine Task Manager and an even better Resource Monitor which quickly show you where the bottlenecks are.

Im my machine (2 x quad Xeons and 3 internal disks) the bottleneck when rendering seems to be the writing of the rendered file - so I am into investigating faster disks (SSD's) and also do some tests with a 4 GB RAM-disk.
srode wrote on 12/9/2009, 3:37 AM
On paper this make sense but I've tested this theory multiple ways and have never seen a difference in overall render time regardless of the way I arranged my drives. That being said, I do have some fast drives with large caches (32mb), and the files I write are normally AVCHD. I think it might make a noticable difference when rendering to AVI or other uncompressed formats where the drive is writing almost continuously.

Personally,based on my testing I think using one RAID 10 array keeps things moving fine regardless of the format being written. Breaking a drive into multple partitions slows disks down generally speaking it doesn't speed them up, I wouldn't do that.

My set up has 1 raid 10 array, 1 Raid 5 array with 4 discs, and one single drive. I do all my work on the RAID10 because it's the fastest of my 3 disk sets.
Whaledad wrote on 1/1/2010, 9:41 PM
"Yes, it's better not render to the same hard drive as your program is on. It's hard for it to read and write to the same drive without causing glitches."
Misconception: If Vegas is rendering, it isn't also at the same time reading it's program code; Vegas is already in memory and sits there just fine. There will be swapping, so you want the swap drive (typically C, but can be moved) on a different drive. And it is reading the source files, and writing the destination files. Both can be enormous. It speeds things up if they are on different drives. So, a total of 3 drives (not being partitions or logical drives on the same hard drive or array): 1 for swap, 1 for sources, 1 for targets. It doesn't really matter where the program code sits, nor where the project file (VEG) sits.
rmack350 wrote on 1/2/2010, 12:27 AM
Whaledad sez:So, a total of 3 drives (not being partitions or logical drives on the same hard drive or array): 1 for swap, 1 for sources, 1 for targets. It doesn't really matter where the program code sits, nor where the project file (VEG) sits.

I'd refine this a little and say use C: for programs and swap, and then two more physical drives for sources and targets.

All of this starts to get really nit-picky but here are some points of reasoning:

--In the course of editing you may create lots of large files. There is a real risk of filling your boot drive if you use it for Vegas' media files, prerenders, and renders. When you fill your boot drive, or you fill the drive that contains your swap file, your system can get really sluggish and it becomes difficult to clear space (because the system is sluggish). This is a good reason to at least use a separate partition for your sources and renders. Ideally you'd use a second physical disk.

--All page (swap) file activity affects Vegas' performance. It doesn't matter if it's Vegas that's using the page file, the mere fact that it's happening has an impact. It's best to have lots of memory available and to avoid being extravagant with Vegas' preview RAM setting. And of course it's better not to have the page file located on the same physical discs that Vegas is using.

--USB drives and many RAID arrays (mostly integrated RAID) use more CPU cycles than PATA/SATA and firewire drives. Use of USB drives will most likely slow down a render. If the RAID array loads the CPU then this will also steal cycles from your render.

--Using two additional physical disks for sources and renders is a fine idea but you need to decide whether prerenders and intermediate files are "sources" or "renders". I'd consider them sources.

For practicality's sake I'd be looking at having at least one extra drive for Vegas data. It's nice to have a third drive for renders but it's most important to at least have the second drive and not use your system drive for media files.

My opinions.

Rob Mack
Chienworks wrote on 1/2/2010, 5:02 AM
"Yes, it's better not render to the same hard drive as your program is on. It's hard for it to read and write to the same drive without causing glitches."

What sort of glitches would this cause? Any at all? If this scenario caused glitches it could only be because of a flaw in the drive system so basic that the computer itself would be completely unusable. A system this flawed wouldn't even complete the boot process.

Is it possible you're thinking about dropped frames that can happen while capturing while the drive system is overloaded? If so, this doesn't apply to rendering at all. Rendering is not a real-time operation and will simply wait until it can proceed. It will never glitch because of drive activity.
Skuzzy wrote on 1/2/2010, 5:20 AM
You really do not want to use a USB connected hard drive for anything that requires a lot of access to it. The USB bus is a byte oriented bus which generates a ton of interrupts the CPU has to deal with.

The SATA bus is a block oriented interface which only interrupts the CPU at then end of the each block of data requested. It is far less dependent on the CPU than the USB bus is.

Just saying....
RalphM wrote on 1/2/2010, 6:05 AM
WD Caviar Black drives are around $100 for 1TB units. Suggest you equip your new PC with two additional drives - one for source files and one for renders.

While external USB drives can load the CPU, I've never seen any pattern of unreliability with reading and writing to them, and 25GB files are not uncommon for my renders. I've not yet had the opportunity to test an eSATA external, but any future externals I buy will have that capability.

RalphM
randy-stewart wrote on 1/2/2010, 9:36 AM
"What sort of glitches would this cause? Any at all? If this scenario caused glitches it could only be because of a flaw in the drive system so basic that the computer itself would be completely unusable. A system this flawed wouldn't even complete the boot process.

Is it possible you're thinking about dropped frames that can happen while capturing while the drive system is overloaded? If so, this doesn't apply to rendering at all. Rendering is not a real-time operation and will simply wait until it can proceed. It will never glitch because of drive activity."

Hey Kelly, thanks for the clarification. Yes, that's part of why I believed that. However, I based most of my comment on past experience and advice from others. Years ago when I first started out, I had only one drive and was experiencing stutters every so often in the rendered files. I bought/installed another drive as advised and the stutters went away. Hence my long time belief that it's best to have your program on a different drive. However, as several have itterated here, the stutters probably had been caused by some other technical issue.

From what I've learned here, just about everyone agrees that it's better to have the program files on a separate drive from your render and/or resources drive, right? Again, thanks for the clarification.

Randy
rmack350 wrote on 1/2/2010, 10:13 AM
The main point is to have your media files on a separate drive. You could store the Vegas program in the "My Programs" folder on the C: drive.

Some people like to install all their programs on a separate partition but I think the reasoning is that this makes the system partition smaller and quicker to backup and restore. It's not a performance issue for Vegas, I think.

When I first installed Windows XP I went out of my way to try to partition it like a Linux system because it seemed like a good practice. Turns out it was easier not to fight Windows about where the programs would go and where my home folder would be. Some programs are flexible about the locations of these folders, others seemed to have "C:" hard coded into paths. A few programs did it both ways because their programming team wasn't consistent about it. (All this may have changed by now but it was still the case for a few years after XP was released.)

The best rule is to at least use a second physical drive for your media and project files. Anything beyond that is extra complexity that might be helpful but also might make life more difficult if you work with other people or have lots of drives to use.

Rob Mack
Steve Mann wrote on 1/2/2010, 7:14 PM
"Could also be a very good idea to split up some of your larger disks into logical disks as Windows does not really fancy reading from- and writing to the same logical disk at the same time."

It's still the same disk. Partitioning is a relic of the days when the available hard disks were larger than the OS could address, and there's very few good reasons for using partitioning today.
johnmeyer wrote on 1/2/2010, 9:19 PM
Partitioning is a relic of the days when the available hard disks were larger than the OS could address, and there's very few good reasons for using partitioning today.I have to disagree pretty strongly with you on that one Steve. Partitioning is still VERY important, and every single person should do it.

Why?

Because you really don't want anything except your programs and O/S on the C: drive.

Why?

So you can do in image backup of just the programs and O/S, and then if anything should happen (user error, virus, bad program, etc.) you can very quickly and easily restore the entire partition and be back where you started, usually in less than ten minutes. The way image backup programs work, they have to create an image of the entire partition.

As for the original posting, it makes no difference whether the programs and data are on the same drive (except as I just noted above). Once the program is loaded, there is seldom a need for Vegas (or any other program) to read the EXE or DLL data from the drive again.

What DOES make a difference is having your source video files on one drive, and then rendering to a second drive. This is for the simple reason that a drive cannot read and write from the same drive at the same time. Try copying a 1 GB file from one folder to another folder on the same drive. Time it. Then, take a different 1 GB file (so there is no caching of the read) and copying it to a completely different physical drive. Time it. That difference is the minimum performance hit you will get. In practice, the performance hit will be worse because the head cannot do the read or write in a continuous manner like it is done during a simple read/write of the entire file all at once, but instead will have to bounce around between the read and write.

Another thing that can make a difference during editing is to have files from different cameras on different drives. This is especially true if your are dealing with large intermediate files (like Cineform) and if you are using standard-performance SATA drives. If you want fast timeline performance during multi-cam editing, put each camera on a different drive, if possible.
ritsmer wrote on 1/3/2010, 3:10 AM
"It's still the same disk."

Perfectly right - but having more partitions lets Windows do the disk-caching much more efficiently.
Skuzzy wrote on 1/3/2010, 4:58 AM
A single partition hard disk is just begging for long term performance and accessibility problems. It is bad enough Microsoft defaults all its temp storage to the C: drive along with the "Documents and Settings" folder.

You should also create a single sized page file, rather than letting Windows dynamically deal with it. This helps performance and further reduces filesystem fragmentation.

Further, for those of you using Vista or Windows 7 with UAC enabled, you will want to keep all your programs in a folder named something besides "Program Files".

"Program Files" is treated very differently, off of the root of the partition of the disk (does not matter what the partition is), when UAC is enabled. What Microsoft does there should be considered a crime.

Of course, the best thing to do is to disable UAC, but that is not an option for some people.
rmack350 wrote on 1/3/2010, 12:00 PM
Skuzzy, could you explain a little more how folders named "Program Files" are treated differently by UAC, and how UAC is a crime?

I'm writing from an Ubuntu system now where the Linuxian form of UAC requires a little more effort than what Vista requires with UAC. In Ubuntu you have to type in your password to make system changes, in Vista it seemed you only needed to click a button.

Rob Mack
Skuzzy wrote on 1/4/2010, 5:31 AM
You'll have to excuse me. I am more than a little harsh when it comes to a company making attempts to restrict how I use my computer.

UAC snoops on every access to the "Program Files" folder. One of the more nefarious things it does is to create a 'virtual store' for files and folders created after the initial installation of anything into the "Program Files" folder.

Even when the files and folders are in the same physical location on the drive, all accesses to anything in the "Program Files" folder goes through the 'virtual store' management.

In our application, we have measured up to a 15% decrease in performance when our application was installed in the "Program Files" folder location, when UAC is enabled. It averages at around 10% loss in performance. Memory utilitzation is also higher.

Once something is located in the 'virtual store' and you set the 'Run as administrator' flag, your application will lose all access to the files in the 'virtual store'. This requires you to reconfigure your application again.

Why set the 'Run as administrator' flag, after the fact? Vista/Windows 7 blocks any attempt for the application to update itself unless you do. Of course, if the update is for data that now resides in the 'virtual store' the update will fail as the software will not be able to find it.

For this and a few other minor reasons many application houses are moving the default installation folder from 'Program Files'. Of course, this will only work until Microsoft decides to include all folders under the UAC umbrella.

What I consider a 'crime' is placing data in folders the application did not create. This is the 'virtual store'. It is a path outside the installation path of the application. There is no way for the application to directly reference it.

I talk to around 20 people a day who cannot find their files on their hard drive because of this. All UAC did for me is to increase my workload and frustrate/anger users who are more than happy to take it out on me.
dogwalker wrote on 1/7/2010, 11:44 AM
Skuzzy, I've just discovered this same thing (the virtual store) when trying to determine why I couldn't locate configuration files I'd changed in a game (I wanted to copy the configuration file and send it to my brother as a simple way of ensuring we had the same key mappings).

And yep, I'm frequently fighting UAC on our computers - all Windows 7. My sons have limited user accounts (my choice), which makes it even worse.

I have a question for you. Would it be worthwhile to uninstall applications and reinstall them to another location, other than Program Files? If so, should they be installed to the Virtual Store folder, or to a folder I might create?

For example, I guess I could create a folder off the root, name it "My Installs"?
rmack350 wrote on 1/7/2010, 5:14 PM
You'd want to create your own folder. Don't install into the virtual store. (This is not from experience, it just makes sense based on what Skuzzy describes)

Rob
lynn1102 wrote on 1/7/2010, 5:35 PM
How about installing each program in it's own folder - would that work? I'm still on sp pro so I don't have that problem. What is the reasoning behind putting all programs into one folder.

Lynn
Chienworks wrote on 1/7/2010, 7:00 PM
All versions of Windows starting with 95 have had a "Program Files" folder and have encouraged all software installation in this folder. Each application is expected to make it's own subfolder inside "Program Files". This scheme has been used all the way up through Vista, including XP. Apparently 7 starts out with this but adds something else ridiculous to it.
ritsmer wrote on 1/8/2010, 12:31 AM
ridiculous?

“It must be considered that there is nothing more difficult to carry out, nor more doubtful of success, nor more dangerous to handle, than to initiate a new order of things. For the reformer has enemies in all those who profit by the old order, and only lukewarm defenders in all those who would profit by the new order, this lukewarmness arising partly from fear of their adversaries, who have the laws in their favour; and partly from the incredulity of mankind, who do not truly believe in anything new until they have had actual experience of it. Thus it arises that on every opportunity for attacking the reformer, his opponents do so with the zeal of partisans, the others only defend him half-heartedly, so that between them he runs great danger."
Niccolò Machiavelli 1513

...sorry, could not help...