Zipping files for longterm storage

Comments

John_Cline wrote on 6/4/2014, 4:38 PM
"I'm wondering if somehow what deusx has seen is a version of the original file pulled from thumbs.db or something similar rather than the uncompressed original file?"

deusx is talking about changes in an .MOV video file, Neither Windows nor WhereIsIt creates a highly compressed, lower resolution version of a video file to use as a thumbnail. Even if another program created a low-res version of a video thumbnail, it wouldn't just look "slightly different" than the original and it wouldn't have the same file name extension.
farss wrote on 6/4/2014, 6:15 PM
[I]"deusx is talking about changes in an .MOV video file,"[/I]

That'll teach me to go back and read the original post, somehow I had gotten it into my head we were talking about till images.

[I]"Neither Windows nor WhereIsIt creates a highly compressed, lower resolution version of a video file to use as a thumbnail."[/I]

Of the top of my head I think WhereIsIt can create thumbnails of video files via 3rd party extensions but that's irrelevant.

I've spent some time trying to figure out if there's anyway duesx could have been fooled but without success. I'm giving up before I wear out one of my disks. I've only persisted because I'm still recovering from something similar with one of my clients.

Bob.
larry-peter wrote on 6/5/2014, 9:08 AM
My earlier response was out of character for me, and I regret my harsh tone, but it’s extremely frustrating that anyone would refuse the best thing this forum has to offer: the ability to learn.

To deny all prior knowledge by saying, “I know what I saw,” is a bad start, but to defend a position by offering further untruths is doing the same thing one accuses Google or Wikipedia of. He may have seen a difference in his files, but were he to take the time to learn the structure of video files, both compressed and uncompressed, he would reach the conclusion that what he saw was not caused by zipping/unzipping.

Data can be randomly altered – from events like theorized quantum phenomena to alpha particle interaction. I believe you would find that such random alteration (or even errors from unzipping) would, in an uncompressed file, cause a luminance or color shift in a single pixel, a corruption of frame size, or a total inability to reproduce a frame (or the entire file, if the header was corrupted). In the case of a file compressed by a codec, the error limit for total corruption of the file would be much smaller.

I don’t believe there exists a bit or group of bits that governs “sharpness” or “better looking.” If there is it must be related to that “in focus” bit that my camera keeps randomly altering. ;-)
Terje wrote on 6/6/2014, 2:18 AM
>> always "see" everything therefore it can make mistakes

Here is the problem with your observation deusx - the zip algorithm doesn't know how to compress video at a reduced quality. That requires knowledge of video, and the zip algorithm doesn't have such knowledge. If the data going in to a zip file is different than what comes out, with video that would show up as a video file that the decoder was unable to decode, or able to decode but with significant errors, skipped frames, botched images etc. The zip algorithm simply doesn't have the ability, anywhere, to reduce the quality of your video. Either it comes out pristine, and identical to the way it went in, or it comes out as a mess you have to throw away. It is impossible that it can come out at reduced quality.

Also, after all these years we can be reasonably confident that zip utilities do nothing wrong except under special circumstance. It is one of the most broadly tested utilities out there, probably the most tested, and bugs are extremely rare.

Example: All your downloaded installers use zip. If there is a single bit wrong in that installer, nothing will work. The entire thing will blow up and not do diddly squat. Millions of installers are run every single day. Windows, when installed from scratch, is in a zip structure. A single bit wrong in the majority of files (.exe and .dll particularly) and Windows would not install.

Also, as I said, if you, magically, were able to get a bit wrong in a video stream when zipped/unzipped, it would not show up as gradually lower quality of that video it would show up as errors in the video stream, and the video decoder would either give an error, crash or show really bad frames with hick-ups, bad frrames etc.

Your observations can not be correct, it simply isn't possible, Your zip utility doesn't have the ability to degrade video quality, that would require knowledge of video in the zip utility, knowledge it doesn't have. Gradual degradation of video quality requires intimate knowledge of video. Zip knows about files and statistics, but knows nothing about video.
Terje wrote on 6/6/2014, 2:23 AM
>> but one insinuation that zip utilities also make mistakes

All software has bugs. Probably some zip utilities too. Problem is, zip doesn't know video. It can't gradually reduce the quality of video under any circumstance. No matter how buggy or error-ridden a zip utility cannot degrade video quality.

A bad zip utility that messes up your file in the compress/un-compress phase would totally and utterly destroy the video. The video would not have lower quality as such, it would be an un-watchable mess and it would only play in VLC because you can play almost anything in VLC. It would not resemble your original files though. Not because zip is too smart, but because zip is too dumb. It doesn't know anything about video.

I find it odd that you are continuing this deusx, you are simply completely and utterly wrong here. Move on.
Terje wrote on 6/6/2014, 2:26 AM
>> I think deusx's tests should be run again in a repeatable environment

There is no need to. We have a much higher probability of seeing flying pigs than seeing what he claims to have seen. Zip utilities knows nothing about video, so if they stuff up, which they can do, as all software, it would show up as a damaged file, not reduced video quality. Lowering the quality of video requires intimate knowledge of video, image size-reduction algorithms etc. There is no such thing in zip. What deusx claims to see is impossible since zip is too dumb (as it relatsd to video to accomplish this).
Terje wrote on 6/6/2014, 2:39 AM
>> Premis: Is it at all, AT ALL, possible to unzip and have pixels "appear"
>> in different places than they should, and STILL have the exact same
>> checksum?

It is theoretically possible to change a file, given very specific circumstance, and still get the same checksum. The problem is that the changes would have to be precisely mathematically correct, so it is impossible to change a file and get a usable output and retain the checksum.

Specifically, for a video file, if it is (theoretically) changed in a way that would retain the checksum the resulting file would not be a video file, it would be a mess.

This is the most damning part of this discussion. Of course bugs in software can mess things up. Given checksums in zips, that would normally mean that after zipping a file you would be unable to unzip it since the checksum was bad.

What is not even theoretically possible is that it could gradually create video with lower quality since that would require the zip has knowledge of video compression. It doesn't.

Even though we are talking about "compression". Compressing video (with resulting quality loss) is not the same as compressing a file with a zip utility. It is just our lack of language skills that makes us use the same word. What zip does and what a video compression tool does is "the same" in the same way that "apples" and "giraffes in captivity in Bosnia" is the same.

Compressing video to get smaller files at the cost of quality requires knowledge of the human system of vision, it requires knowledge of color, shape and images. The compressor needs to know that a large surface of the color blue can be "compressed" into a smaller surface of blue and a description of how you take that small one and expand it to the original size while retaining as much of the original visual representation as possible.

There are no zip utilities in the world with such knowledge. What deusx is reporting simply isn't possible. He's wrong, but to stubborn to even admit that possibility to him self. What he is saying is that Microsoft Word suddenly, and without warning, can edit video. By accident. It can't..
ushere wrote on 6/6/2014, 3:17 AM
Microsoft Word suddenly, and without warning, can edit video. By accident. It can't

but by golly, it can sure mess up script formatting when it gets a mind to ;-)