hard drives and new VMS 10...

CurtA wrote on 6/22/2010, 6:23 PM
I've been using VMS 8 VERY infrequently over the past 2 years. I got a new Canon HF S100 almost 6 months ago and haven't used it, waiting for VMS 10 to be released and just being busy.

I have 2 hard drives on my computer, both are 350g and need to be relieved of photos and past video projects, as they both are almost full. I DO have the photos and projects backed up on DVDs and external drives.

Couple of questions...

1. If I remove almost 220g of data on 1 and 160 on the other, is it then important to do a defrag before beginning new video projects and installation of VMS 10?

2. I have read here that it is best to have the program installed on the main drive. When creating new projects, do you use the main drive for that also, or have the program on main drive and files on the secondary? ALWAYS wondered about that.... as I said, haven't been too involved in video editing, but hope to be. it seems I have video files scattered on both drives! lol

Thanks..
Curt

Comments

ggrussell wrote on 6/22/2010, 6:44 PM
In general, I think it is good practice to keep applications and data on separate drives. I also render to a different partition or separate drive. That way there isn't a bottle neck trying to read and write to the same one.

Oh yeah, and ALWAYS keep your drives defragged especially after removing that much data.
CurtA wrote on 6/22/2010, 7:37 PM
Thanks! I am such a dweeb with this lol. So, if you've got another minute... if I understand you correctly, you have the program/application on your main drive and load the "raw" video data onto a secondary drive, doing all the editing and rendering from/on that drive? Man... I've GOT to get a better understand of drives and cross functioning etc. lol I'm already feeling lost! lol

Chienworks wrote on 6/22/2010, 10:50 PM
Actually defragging is never important. It has no demonstrable benefit, and it's unnecessary wear and tear on the drive. Don't bother. There's no need for it at all and it's actually harmful.
drw wrote on 6/23/2010, 12:34 PM
I recently defragged the partition that contains my video files and my render times improved by almost 2x. I was amazed because I'd never seen much difference with any other application after defragging, so I wasn't really expecting it to make any difference. The thread about it is still in the first 20 topics or so in the forum if you're interested in the details. I had a relatively small partition, so that may have contributed to it getting too fragmented in the first place, therefore defragging might not always help, but it did for me.

To the original poster, its always a good idea to install your applications on the system C: partition, and use other partitions or additional disks for data. This allows you to easily backup your system partition with disk imaging software and easily recover from any virus attacks, human error mistakes, etc that screw up your OS or applications. By keeping your data on other partitions, after doing a restore of your system partition after trouble occurs, your data is still intact on the other partitions.

Its also more efficient to render video if your source files are on one disk and your rendered files are on another, that way you can simultaneously read and write data during rendering.
david_f_knight wrote on 6/23/2010, 1:17 PM
Rendering to separate partitions on the same hard drive doesn't avoid a bottleneck, it worsens it. Each partition on a single hard drive is physically separated from every other which will, in general, require greater seek distances and therefore greater seek times when switching from reading tracks from one partition to writing tracks in another partition. Every partition on a single hard drive shares the same buffer and communication bus so there is no avoiding bottlenecks with those simply by having multiple partitions on the same drive.

To minimize hard drive transfer times, you must minimize seek (moving the head to the correct cylinder) times and latency (waiting for the disk to spin until the correct track is under the head). This is where having a defragmented hard drive could be faster, by minimizing those activities. But it can only have an impact if the bottleneck actually is the hard disk (which is probably usually not the case), and if it had been horribly fragmented (which is usually not the case unless the disk is very nearly full of mainly very small files which have undergone a lot of revisions/copying/deleting).

By the way, there is a relatively fast way to defragment drives that causes virtually no additional wear to them, if you have more than one drive. Just copy all your files from one drive to another (obviously, the second drive must have sufficient free space), delete all those copied files from the first drive, then copy them back from the second drive to the first, finally delete the copies from the second drive.

This works because copying a file implicitly defragments it, to the extent that it is not being copied to a fragmented drive, and deleting files implicitly defragments that part of the drive where those files existed on it. However, you cannot copy protected operating system files with this technique, which is okay because they are rarely fragmented in the first place. The other issue is that folders will lose their original timestamps and instead be timestamped with the time at which each was created during the copy operation; individual files will retain their original 'Modified' timestamps but be given new 'Created' and 'Accessed' timestamps.

Obviously, caution should be exercised when performing this technique.
Chienworks wrote on 6/23/2010, 1:30 PM
dwalby, your speed increase almost certainly had nothing to do with defragging. In every case where a speed increase was reported, it was actually due to things like rebooting after defragging, or shutting down other programs while defragging. The defrag process itself really makes no measurable difference in drive access speed.
Chienworks wrote on 6/23/2010, 1:32 PM
David, yes, that's the method i've suggested to people who absolutely feel they *must* defrag, although in fact no one ever has to. However, i would usually only do half of what you do. I would copy files to a nearly empty drive and end there. I wouldn't feel any particular need to copy them back to the original drive.
drw wrote on 6/23/2010, 3:22 PM
dwalby, your speed increase almost certainly had nothing to do with defragging. In every case where a speed increase was reported, it was actually due to things like rebooting after defragging, or shutting down other programs while defragging. The defrag process itself really makes no measurable difference in drive access speed.

I had tested this particular render several times before, it always rendered at 6m15s to 6m20s consistently (I was comparing VMS to some other alternatives, so that's why I had some history with that render). After defrag it was 3m25s or something like that. I work on a laptop that doesn't get used much, so each time I edit I'm working from a cold boot, and VMS is the only thing running. I didn't change any VMS settings, only did the defrag. The disk analysis reported a fragmentation of 75%, even though the partition was only about 40% full, not sure how it got that bad in the first place. Its only a 60GB partition, because I don't really need anything bigger, and the laptop only has a 160GB drive in it. I only do very small projects, so it works for me. The small partition may be part of the problem as well, I don't know, but AFAIK the defrag was the only change between the 6m+ and 3m+ render times.

I don't really want to get into a big technical discussion on hard drives, but if a file can span one big continuous space on the drive that would seem to require less seeking, thus shorter access times than a fragmented file would require. The fragmentation may have to get pretty bad to have any significant effect, but in my case that appears to have been the case. I resized that partition a few times (made it smaller to install Linux), perhaps all the file shuffling associated with the resizing messed it up, because as much as I've used it I was very surprised to see how fragmented it was. You're probably right that a minor fragmentation may not be noticable, but in my case I'm 99.9% sure the defrag was the only difference.

Just remembered one other thing, I also used that partition occasionally for holding backup images of my C: partition, which typically run about 15GB each. Sometimes I may have had two images there, but eventually copy them to an archive disk over USB, so they get deleted over time. That may have also contributed to the excessive fragmentation issue.
drw wrote on 6/23/2010, 3:47 PM
Rendering to separate partitions on the same hard drive doesn't avoid a bottleneck, it worsens it.

If you are rendering files such that the source and destination files are on the same partition, but VMS is installed on another partition, does that reduce efficiency? Is the rendering algorithm pretty much loaded into main memory so it doesn't need to access the system partition of the hard drive for more program code during the rendering operation. I suspect in my case where I have very few effects and transitions that the s/w has to apply it doesn't have to access the VMS install partition for those algorithms very often, if at all.
david_f_knight wrote on 6/23/2010, 5:37 PM
Your intuition is correct.

Generally, the entire program code is loaded from disk to memory when you first start a program. That's why programs don't start up instantly. (With some programs, there can be instances where the first time a particular function (stored within a .dll file, such as some effects plug-ins) is utilized it will be loaded on demand from disk, but once in memory will remain there. The only other time program code needs to be loaded from disk is to reload it into memory after it has been swapped out to the pagefile. That is only necessary if you have insufficient memory to hold all the programs you are executing simultaneously. So, if you run Vegas by itself, there will be no need to swap it out to make room for other programs and it will stay in memory until you shut it down. In any case, the pagefile is an operating system file, not an application program file, and is usually located on the system disk regardless of where the application program files are stored.

Sorry for the long-winded explanation. In other words, it will have absolutely no effect on efficiency which partition the application program files are stored on as compared to where either the video source or destination files are, because there is no overlap in those disk accesses. When loading a program, no video files are accessed, and when rendering a video file, no application program files are accessed, so there is no resource contention to reduce efficiency.

If your computer does have to access the pagefile frequently while you execute programs, then your only solution for better efficiency is to have more memory, or to reduce the number of programs running simultaneously.
CurtA wrote on 6/23/2010, 6:18 PM
Interesting discussions... thanks! (I think) lol good stuff for future reference...