OT: Video images lost forever--Update From MainConcept!

Jay Gladwell wrote on 7/26/2004, 4:36 AM
Just as I was falling asleep last night, I had a thought regarding video compression. Not being the least bit technical (or mathematical), I have no way to confirm my “theory” one way or the other. So I’m throwing it out here in the forum to see if any of you “intellectual” types can tell me if it is feasible or not.

If I understand video compression correctly (and I may not), codecs, such as those created by Main Concept for example to compress video for DVDs, use a series of mathematical algorithms to compress the video. If they know what their codec does mathematically to alter (compress) the video, moving it from point A to point B, as it were, could they not create a codec that would reverse the process thereby expanding (reconstituting) the video back into its original form? It could/would be akin to zipping (compressing) a file then unzipping (expanding) the file to it’s original state.

The reason I ask this is because I read a post on Saturday, I believe it was, that mentioned DVDs may have a shelf life of 50 years at best. If that were true, and I don’t think we have the definitive answer on that yet, under the current system there does not seem to be a way to restore these compressed images to their original “pristine” condition. It is conceivable that all of the images transferred to DVD, in the absence of their original masters, could be lost forever!

Yes, no, maybe?

Jay

Comments

ScottW wrote on 7/26/2004, 5:20 AM
If the compression was loss-less, certainly you could decompress back to the original.

Unfortunately, MPEG-2 is not a loss-less compression, which is typical for most of the video compression available.

Actually the issue of loss has been around ever since digital was invented. You can never hope to take something from the analog world and represent it in digital format to 100% accuracy.

--Scott
Jay Gladwell wrote on 7/26/2004, 5:30 AM
If the compression was loss-less, certainly you could decompress back to the original.

That's my point! Could they not create an application that as it compressed the video it created a "log file" of what was done and then use that log to rebuild the original?

Scott, what I'm talking about is restoring a "digital file" back to its original state. Not worried about analog vs. digital, this is a non-issue within the context of this discussion.

Jay
Chienworks wrote on 7/26/2004, 6:25 AM
The log file would be (at theoretical best) the size of the original file less the size of the DVD file. If that's the case, why not just store the original file instead?

You could compress the video file with something like WinZip but you probably wouldn't achieve anything close to even 2:1 compression. This would be lossless, but it would take up almost as much space as the original. So, why bother? Why not just store the original file instead?

DVDs are compressed typically 5:1 from a DV source or 40:1 from an uncompressed source. The only way to get this much compression is to throw away data. If you create a log file of what was thrown away, then you're not really throwing it away since you have to keep that somewhere. And since that log file will be about the size of the original ... well, you know the rest by now.
farss wrote on 7/26/2004, 6:32 AM
Pretty well any encoder comes witha decoder (hence codec!). The issue isn't decoding the data, the issue is the loss in encoding. But bear in mind DV25 is already lossy, depending on many factors mpeg-2 can be far less lossy the DV25.

As to DVDs lasting 50 years, well the pressed ones maybe, DVD-ROM, 20 years with proper storage and handling for the very best media if we're really lucky, neither the dye layer or the metalisation layer is designed for archiving.
Best bet is to store the video as say AVI in short segments on archival grade CDs, expensive but it'll last at least 50 years, maybe as long as 100 years.
8x and 16x speed DVDs, well they might last a few years, too early to really tell.
Jay Gladwell wrote on 7/26/2004, 9:02 AM
I sent my question to Main Concept and got the following reply:

Hi Jay,

I'll try to give you a short and clear answer. I tried to simplify
everything, else we would need a book to explain it.

1) Codec mean *Co*mpression*Dec*ompression. The compression part is the
enconding process which is used e.g. during export, the decoding part is
used during playback (watch the clip on pc) or import to a editing program.

2) A codec can be lossless or not

3) Even dv is compressed (very little, but it already has less quality
than the original). 9min.dv material has a size of 2GB

4) Umcompressed video (RGB) has about 1,74 GB per minute. (a lot of
internal cpu parts do not have enough bandwith to handle this ratio e.g.
harddrive). RGB uncompressed is the only video which shows the
"reality". (This is only calculated it for normal tv resolution, HD or
cinema is far larger)

5) Dv is lossless, so you can compress/decompress/compress etc. and
there is no loss of quality.

6) MPEG (which is the codec of the DVD format) is not lossless. Here you
get every time a worse video you recompress it. MPEG works with the
insufficiency of the human eye as well as mp3 does. The next trick to
save space on the harddrive is to work with reference pictures (so
called I frames). Other frames so called P and B frames only store the
difference of the picture regarding the I frame. Let say there is a bird
flying in front of a wall, than only the bird "changes" in the picture,
so why store in each frame the whole wall, and not only the part where
the bird flew by. There are some other "tricks" of MPEG to save size. It
is right that it is a complex mathematical calculation, but sometimes
the a things done like:
1.000000000000000000000000000000000000000001234 x
0,99999999999999999999999999999999999999999845 is replaced with
1 x 1. The result is nearly the same, you won't see any difference, but
there is a difference. Do it several times, scale the video, or
recompress it for several times and you will see the loss. This is why
you cannot unpack the MPEG like you do it with a zip file.

7) The liftime of a DVD is short, but it is the same with film, VHS
tape, etc. There are storage devices which can handle the data longer
than this. But sure the records of our civilisation will be gone very
fast, if nobody takes care to refresh the records from time to time.

Bye
Oliver Steinhauer
MainConcept AG
apit34356 wrote on 7/26/2004, 10:20 AM
Jay, Main Concept answers 3 vs. 5 ?

the codec speed is usually the factor in the about of loss that is acceptable in the operation,(plus poor programming of alg). most Meg2 codec loss can be determined. The regeneration of loss data produces a data set of mutli values. the re-creation of the most likely choices is CPU intense! no real time viewing and still requires an operator/editor to approved the generated data.


AJP