Zipping files for longterm storage

LV studio wrote on 5/30/2014, 6:54 PM
1. Is there any quality loss to zip and re-expand (mts, mp4) or wave (16, 24) using the Windows zipped compressed folder? When doing each file separate? When zipping multiple files to a single zip file folder containing all the files?

2. Is there a maximum number of files that can be zipped into a single zip file?

3. Is there a maximum zip file size?

Thanks for the replies. :)

Wayne

Comments

musicvid10 wrote on 5/30/2014, 7:35 PM
1. nope
2. maybe depending on your utility
3. not that i know of
Former user wrote on 5/30/2014, 7:53 PM
If you are trying to group all of your files into one file, a zip file might be fine, but if you are trying to make the files smaller for storage, you probably won't see much gain in file size. Video files tend to not have a lot of overhead in terms of compression and as such, do not compress much more.
Chienworks wrote on 5/30/2014, 8:01 PM
You will find that .mts, .mp4, and most other compressed formats are already so densely compressed that zipping them will save very very little space. It's probably not worth the time it will take to zip & unzip such huge files. I wouldn't bother.

.wav will compress pretty well. However, since these are usually quite a bit smaller than video files to begin with, the space savings may be inconsiderable anyway.

.zip is always, *ALWAYS* lossless compression, considering that it was originally designed for office documents. A loss or change of data when compressing your company's budget spreadsheet really wouldn't be a good idea! Imagine having to explain to the auditors that "up to 5% of the numbers may have been lost and replaced with made-up data when decompressed".

I've created some 100GB+ .zip files consisting of 10's of thousands of files. If there are limits, i haven't reached them yet. That being said, i suppose it's possible that some .zip implementations and utilities might have smaller limits than others. It's conceivable that my 100GB .zip file generated under CentOS 5.5 might not be readable under Fedora 7 or Windows XP. So, the caution there is to make sure you can open and extract from your monster .zip files on a variety of computers before you delete the original files.

Considering all this advice, what would i do personally? I'd just keep the original files.
VMP wrote on 5/30/2014, 8:08 PM
1. Win Zip/ Winrar 'zipping' compression is lossless. When you extract the files in a zip file created by zipping the result will be exact, byte for byte duplicates of the original files (if the zip file is not damaged).

2. I have once had around 80.000 (80 thousand) files in a rar file.

3. Something to note. External & USB Drives that are formatted with FAT32 have a file size limit of 4 GB. That might be something to consider when creating large files.
If you are using NTFS formatted drives this is not a problem.

Also the bigger the zipped file the longer it takes to unpack.
Also double the file size could be needed if you add any file later on to the zipped file (for the program to make the changes it creates a temp file).

I burn my projects on Bluray discs that's a big relief having so much space.

Before I used to use Winrar to compress and split the larger files to DVD sized files. Later Winrar can unzip and combine the seperate files to single files.

As others have stated, zipping already codec compressed files would only make the files larger rather than compressing them any further.

Large uncompressed files like .BMP, DV Avi and PCM .wav files will be quite smaller after zipping.
Also text files, empty spaces and repeating patterns will be compressed by Zip.

VMP
ushere wrote on 5/30/2014, 8:46 PM
i could pop out and buy a 1tb usb portable hd in the time it takes to zip / unzip my average video project;-)

as pointed out above, not much point using it for video.

i usually zip my raw stills after a couple of years as that does save some space, but still leave 'active' ones in a folder.

storage is so cheap... that said, important stuff is always double backed up - just in case....
LV studio wrote on 5/30/2014, 8:56 PM
Thanks all for chiming in. I did notice that my mts and mp4 had maybe a 2% zip rate so I'm not doing them.

The waves range from 7 to 70% so I'll archive them as necessary.

Thanks again.

Wayne
musicvid10 wrote on 5/30/2014, 9:38 PM
wav and avi can be zipped efficiently, but the latter not always.
most other audio and video formats cannot.
farss wrote on 5/30/2014, 10:45 PM
Just a couple of thoughts.
Putting oodles of files into one zipped file does mean if something does go bad you've lost a lot of "stuff".
If you have many tiny files just putting them all into one file regardless of the compression does save some space depending on the disks cluster size.

Bob.
musicvid10 wrote on 5/30/2014, 10:55 PM
"Putting oodles of files into one zipped file does mean if something does go bad you've lost a lot of "stuff"."

Yes, absolutely true, and there is no better example of that than some of the zipped .exe files I've downloaded over time. Either it's perfect or it doesn't work.
Frans Meijer wrote on 5/30/2014, 11:25 PM
You could use FLAC for audio, compresses about 2:1, lossless, and can be dropped straight into a Vegas timeline
John_Cline wrote on 5/31/2014, 3:29 AM
You could also use the RAR compressor. You can specify file extensions in RAR's preferences to tell it just to store these file without compression. WinRAR also has recovery record and recovery volumes allows reconstruction of even physically damaged archives. WinRAR provides complete support for RAR and ZIP archives and is able to unpack CAB, ARJ, LZH, TAR, GZ, ACE, UUE, BZ2, JAR, ISO, 7Z, XZ, Z archives. RAR can do a pretty good job of compressing WAV files, although FLAC and Sony's own PCA lossless audio compressor create smaller files.
Gary James wrote on 5/31/2014, 8:50 AM
.zip is always, *ALWAYS* lossless compression, considering that it was originally designed for office documents.

I sure hope not. There are thousands, if not millions or programs distributed via Zip files. ANY lossless compression of a program executable would render it unusable. I'm sure you meant to say that .ZIP is *USUALLY* lossless compression for certain file types stored in the .ZIP container.
Former user wrote on 5/31/2014, 9:10 AM
"ANY lossless compression of a program executable would render it unusable"

Gary, could you explain. I thought ZIP was always lossless compression as well. If an exe file loses data due to compression, wouldn't that make it unusable or do I not understand compression.
Gary James wrote on 5/31/2014, 11:23 AM
That's exactly what I'm saying. Zipping up an EXE is NOT lossless. otherwise the program could not run. A Zip program can compress executables without it being lossy. It may not be as compressible as an image file, but it has to maintain all it's original bits. The same is true for data files like spreadsheets and word processor documents. I'd hate to loose that manifesto I was writing because WinZip dropped a few pages for size sake.
larry-peter wrote on 5/31/2014, 11:31 AM
Gary, are you saying that .exe files are NOT compressed, or that their compression under .zip is not lossless?

And a general reply to this topic: I use zipped folders as a redundant backup, but never as a primary backup, for the reasons mentioned earlier - potential loss of everything in them.
Former user wrote on 5/31/2014, 11:45 AM
Gary,

Lossless to me means there is NO loss (like flawless means there are no flaws). Lossy means there is loss. So I think we are just having an issue a semantics

LOSSLESS= no loss
LOSSY= loses data
Gary James wrote on 5/31/2014, 12:45 PM
You guys caught another senior moment of mine. I meant to say that certain file types are NOT compressed so they would loose data. They are NOT lossy. Now where did I put those Lithium tablets ....
musicvid10 wrote on 5/31/2014, 2:30 PM
The reason, I assume, that executables get messed up is because of index errors, which usually don't affect non-exe files the same way.
larry-peter wrote on 5/31/2014, 3:24 PM
And on the original topic, I wouldn't ever consider Windows zip utility as a real backup unless you go to the lengths to verify the backup by uncompressing and comparing files to the originals.
In my backup scheme, I use Backup4all to zip selected folders and archive to BluRay. It unzips the folders and does a bitwise verification after burning to disk. And I do that only after creating incremental disk image backups and uncompressed duplicates of footage folders on external drives.
Chienworks wrote on 5/31/2014, 9:57 PM
Gary, even executable files can be compressed by .zip and are losslessly compressed. Many of the .exe install files i download are 1/3 the size of the uncompressed version.

Of course, .zip is always lossless. This means that the amount it can compress is sometimes limited since there isn't always a lot of redundant or repetitive information that can be squeezed, however most .exe files can still be compressed a substantial amount.
Gary James wrote on 5/31/2014, 10:27 PM
"Gary, even executable files can be compressed by .zip and are losslessly compressed. Many of the .exe install files i download are 1/3 the size of the uncompressed version."

Correct. Which, as you can see, is what I tried to say in my original post; except for mentally tripping over the word lossless.

"A Zip program can compress executables without it being lossy. It may not be as compressible as an image file, but it has to maintain all it's original bits."

As to the original post. I guess .zip file suitability all comes down to what you are trying to store in your zip file. I've actually had zip files that were larger than their component files because the files stored in the zip were already highly compressed.

deusx wrote on 5/31/2014, 10:52 PM
>>>>.zip is always, *ALWAYS* lossless compression, considering that it was originally designed for office documents.<<<<<

Not true, not with video/audio anyway.

If you zip and unzip once you may not notice a difference. Do it 3-4 times you will definitely notice a significant drop in quality, and I mean really significant with some file formats ( .mov for example )
John_Cline wrote on 6/1/2014, 2:31 AM
"If you zip and unzip once you may not notice a difference. Do it 3-4 times you will definitely notice a significant drop in quality, and I mean really significant with some file formats ( .mov for example )"

That is simply not true. ZIP is always lossless no matter how many times you zip and unzip. Just because I had some time on my hands, I just made ten zip/unzip passes on an MOV file and the resulting file at the end was bit-for-bit identical to the source file.
Grazie wrote on 6/1/2014, 2:39 AM
@ deusx: Where did you get that info from?

I'm no IT expert, but zipping/unzipping has to be totally transparent to the process otherwise all hell would break loose.

Surely?

Grazie