NTFS and Defrag Questions

yirm wrote on 11/15/2001, 2:07 PM
I've been using NTFS on my two drives dedicated to video (40 & 20 GB) so as to avoid having my files chopped into 4GB segments. What I'm noticing on NTFS under XP, though, is that you can only defragment when there is a lot of free space on the drive. Since the built in defragmenter doesn't tell you much, I downloaded Executive Software's full version to see if I could learn what was going on. It seems that it is reporting extremely high levels of defragmentation even though there may be only one file on the drive (say 12 GB). What is happening, is there is "reserved space" on an NTFS partition which is not insignificant in size. And the video file "wraps" around it, causing Diskeeper to report high levels of defragmentation. So I have two questions:

1) Is NTFS no longer prefered now that VF automatically splits the files into 4GB segments in a presumably seemless fashion?

2) In NTFS, what is all this "reserved space"?

3) When Diskeeper reports high levels of fragmentation even though there is only one file on the disk, can we pretty much ignore this as a technicality?

4) I'd think that if you capture DV via 1394, it writes it in one big chunk if possible, no? Or could there be fragmentation inherent in the file? Or is that a contradiction in terms, since fragmentation means parts of files interspersed throughout the disk physically?

Any insight would be much appreciated.

-Jeremy

Comments

wvg wrote on 11/15/2001, 4:42 PM
1) Is NTFS no longer prefered now that VF automatically splits the files into 4GB segments in a presumably seemless fashion?

NTFS is still a superior file system for many reasons.

2) In NTFS, what is all this "reserved space"?

Reserved space is used by the MFS (Master File Table) similar to WIN98's FAT or file allocation table.

Details: http://support.microsoft.com/support/kb/articles/q174/6/19.asp

3) When Diskeeper reports high levels of fragmentation even though there is only one file on the disk, can we pretty much ignore this as a technicality?

No. Regardless what version of Windows you use, files get written to hard drives in a way beyond human intervention. Over time files become fragmented because the files are not written across a block of consecutive sectors, rather Windows scatters files all over your hard drive. So files, especially large ones may reside in hundreds, even thousands of clusters, many of which may not ajoin each other. When it happens your hard drive is said to be fragemented. The more fragemented it gets the more performace suffers. It also makes your hard drive work harder, and it will fail faster. Anything over 2% fragemention is TOO MUCH!

4) I'd think that if you capture DV via 1394, it writes it in one big chunk if possible, no?

See above. There is no practical way to "force" a hard drive to write data to a specific area. Of course if you start with a severely fragemented drive you guarantee large files in particular will be fragemented more because there are few if any spans of "free" clusters for Windows to write to.

yirm wrote on 11/15/2001, 11:55 PM
Yeah, but I have a 40 GB drive. Put two 12 GB files on there, and it won't let me defrag because there isn't enough space for it to do it's thing. Furthermore, I formatted and defragged the drive before capturing the clips. So after doing nothing but capturing two tapes (one clip each), I should take seriously what Diskeeper tells me about fragmentation? It seems kind of crazy to me. And it says it needs at least 20% to do its work. But I'm finding that with large files like these it needs MUCH more than that. I wasn't able to defragment with as much as 50% free.

Check it out.

http://www.tiferet.com/defrag
wvg wrote on 11/16/2001, 12:31 AM
I looked at the chart. If red is what is suppose to be fragemented, it is possible that the utility you are using is actually the culprit. I once tried some non main stream product. I started with 2% fragemented, it ran for hours, then reported the disk was 98% fragemented. LOL!

Are you running XP?

Are you the guy that said it takes 18 hours to defrag?

Take a look at Norton Utilities 2002.
yirm wrote on 11/16/2001, 1:21 AM
XP Yes. Non-main-stream product? I wouldn't say so. Executive Software licenses made the first defragmenter for NT4 (Diskeeper). MS has licensed a stripped down version for Win2K and XP. I downloaded the Diskeeper trial for XP because the Windows defragger wouldn't even touch it, claiming there wasn't enough leftover space. I thought Diskeeper would at least give me some more info, which it did.

I still don't see how, if you have a freshly formatted disk, and you capture a tape (one big clip), it can be fragmented.

What I think is happening is that the file is so large, that the "reserved" space, which cannot be moved has to interrupt what would have otherwise been one continuous chunk. And since there is only one file, Defrag says 100% of the files are fragmented and gives all kinds of warnings. But in reality, it's not like having that chunk of reserved space in the middle of a (presumably) otherwise unfragmented file is really going to screw anything up. It's not like the computer repeatedly is searching all over the disk.

Agreed?

-Jeremy
wvg wrote on 11/16/2001, 7:56 AM
Under Windows XP, the NTFS file system is slightly different than under previous versions of Windows. Are you sure the version of Diskeeper you downloaded supports XP? That's why I suggested Norton's 2002 Utility, because I know for a fact it does support XP.

Briefly checking the Diskeeper site, I saw no mention that Diskeeper supports XP. If not, then that may explain why your drive is getting so massively fragemented.
yirm wrote on 11/16/2001, 11:25 AM
Aboslutely positive. If you go to the demo download section, you will see that they have multiple versions for the various platforms. I may try Norton Utilities. It will be interesting to see how their utility handles it. But like I've stated, I don't think the drive is really as fragmented as DK is reporting.

-Jeremy