I don't think we'll ever see a time when video will routinely be stored uncompressed. Video compression schemes are here to stay.

Your 13 GB/hour DV AVI is already compressed, just not as much as MPEG2 or 4.

720x480 29.97 fps, uncompressed video is approximately 105 gig/hour (720 x 480 x 3 x 29.97 x 60 x 60 = 111,862,425,600 bytes per hour (not including audio.)

John

Uncompressed SD is not 13GB/hour, that's DV. Uncompressed is about 104GB/hour.

Uncompressed HD can be about 625GB/hour.

If you can't see the difference between a DVD copy of the VHS original of your kids when they were little, then why care? MPEG2, for all its warts, does a remarkably decent job of compression. I would guess that wmv improves on MPEG2 compression by about a factor of 2, and the latest state-of-the-art probably improves on wmv by about another factor of 2. Nonetheless, if I carefully encode my DV avi and use a high enough bitrate, the results will be pretty much identical to the DV avi clip. In that case, all that Blu-ray gives me is the chance to archive 50 hours of DV on a single disk. Since that's a solution to a nonexistent problem (for me, at least) then Blu-ray is not on my current wishlist. It's a different story for HD, of course.

I have no doubt that MPEG2 will one day fade away, but that doesn't mean it will go away. Look at cassette--you can still buy C90 tapes, and that technology is at least 40 years old. MPEG2 is also the backbone of ATSC HDTV over-the-air transmission, as well as cable transmission.

So I guess my point is why one would care if it is getting the job done? Now, if you're looking at preserving raw 4:4:4 video, that's another issue.

"With these HUGE advances compared to our measley 4.7 GB single layer home burned DVD's, do you think that any type of compression scheme MPEG-2, 4 or whatever will eventually go by the wayside?"

Excuse me if I chuckle. SURE things will keep changing and improving. I can remember when the first PC came with a tiny capacity single floppy drive that hardly held anything, where hard drives were a luxury option costing well over a thosand dollars. You had what was considered a "hot" system if you had two 1.44 MB floppies a short time later.

AS the capicity of hard drives grew, then came tape and then "zip" drives, followed by CD's, then single layer DVD's, now double and soon multiple layer. What tomorrow holds is anybody's guess.

Lets go down memory lane on the history of communications:

3500 BC Phoenicians develop an alphabet
1270 BC The first encyclopedia found in Syria
900 BC first postal service in China.
776 BC first recorded use of homing pigeons in Greece
530 BC first library in Greece
500 BC first recorded use of messengers using horseback
100 AD first hand written bound book
1450 AD first newspapers appear in Europe
1455 AD Guttenberg in Germany, printing press metal movable type
1714 AD Hill in England gets first patent for eary version of typewriter
1835 AD Morse developes morse code
1861 AD Poney Express in America
1876 AD Edison in America patents the mimeograph a early office copying machine.
1876 AD Bell patents the telephone
1888 AD Eastman patents Kodak roll film camera
1906 AD Lee DeForest invents the electronic amplifying tube
1910 AD Edison demos first talking motion picture
1923 AD Vladimir Zworykin early work on television or iconoscope
1927 AD RCA starts two radio networks
1948 AD Vinyl phonograph record developed
1948 AD transitor invented, modern electronic era begins
1949 AD TV broadcasting begins in U. S.
1949 AD 45 RMP phonograph record introduced.
1951 AD First commerically sold computer
1958 AD Chester Carlson invents the photocopier machine
1966 AD Xerox developes first FAX machine
1969 AD ARPANET - the "pre-Internet" started
1971 AD First "floppy disc" and microprocesser
1972 AD HBO introduced pay TV for cable
1976 AD Steve Jobs, Apple 1 computer
1979 AD First cellular telephone network started in Japan
1981 AD IBM introduces "PC"
1994 AD U. S. government releases control of ARPANET, world wide web born

I had a post on an article about this a little while ago. In a nutshell their requirement was lossless storage. They settled on "MJPEG", storing each frame in JPEG2000 format. This allows 1 hour of content to be stored in about 25GB. Pretty reasonable idea, mho. After all, its not whether or not we compress, its whether the compression is lossy or not.

BB,

Thanks for a great list, really fascinating!

Three trivia addenda:

1971 AD First "floppy disc" and microprocesser
These two were unrelated for a long time. The floppy disk drive was created to store boot code in IBM mainframes. Only 8 inches across, using a floppy disc in a floppy "envelope", it held all of 242KB. Of course, at this time, most mainframes were lucky to have 1MB of memory for everybody to share. The technology was core memory, little ferrite cores that were actually knitted into a web by patient women (men were too sloppy).

It took many years before floppies got anywhere near microprocessors. I remember using floppy drives professionally in the late 70s, but they cost something like $2,000 each in today's dollars, and the drives were larger than today's ADS Datatanks. The first floppy drive interface cards cost $300 initially (perhaps $1,000 in today's money). At that time, somebody invented hard sectors (a series of punched holes around the hub area of the disc). This increased capacity to a rocking 320KB!

So what did microcomputers use for storage before floppies became affordable? Modem signals recorded on C-60 audio cassettes. Can you spell s-l-o-w? Modems in the 70s were 110 bps, before the hypersonic 1200bps/75bps modems became available (1200 bps to your screen, 75 bps from your keyboard).

Early microprocessors were not used by hobbyists. The 4004 is often (unfairly) thought to have been the first microprocessor, this was a real megastinker, very ugly design and a bear to program.

TV was active in England long before it became widely used in the U.S. I talked to one guy in Devon who built a very early TV set. Buying ready-made components cost too much, so he had to wind his own capacitors from aluminum foil, and resistors were made in various values by varying the width of a carpenter's graphite pencil stroke across a piece of wood.

1861 AD Pony Express in America
A few years ago, somebody dropped a First Class letter into a mailbox in Los Angeles, to be sent to an adressee in Phoenix, Arizona. Simultaneously, another letter was given to a Pony Express rider. You can guess which letter was delivered first...

RMD,

JPEG2000 lossless? This was news to me, I think there is some small loss at the compression mentioned.

There are a bunch of lossless video codecs that achieve compression ratios up to about 2.85:1 That's it. HuffYUV is perhaps the most popular one, and it comes in several variations, all based on Huffman encoding.

JPG2000 has visually no loss to the image, and that's why it's being chosen as an archive format. You'll hear some very cool applications for J2K very soon. J2K is scalable too, all the time, so that means that there are myriad benefits to it. The key these days is the "visually lossless" in that you can't see what it's cut, simply because the compression works on tricks of the human eye, not much different than how DV works with the strengths and weaknesses of the human eye.
that said, and understanding your argument, I'd be looking at DVD as a good intermediary. I've got a lot of old VHS tapes of various events, that I've been converting to DVD on a crappy Symphony VHS to DVD box, simply so that the media is no longer being degraded by sitting on a shelf. More importantly, while DVD might not be a format in 20 years, I know VHS won't be either. And, DVD won't degrade the image quality while sitting on a shelf, but VHS certainly does. So, I'll have a lot more opportunity to do something with this vid in 20 years if I'm still around, and it will be in the same quality in 20 years as it was the day I transferred it.

If I remember correctly some very early, mostly expertimental TV broadcasts happened in Europe around the mid to late 30's and was part of the "plot" in one of my favorite SF movies of all time, "Contact" with Jodie Foster who played the lead role of a ridculed radio astronomer looking to make contact with aliens. She got back a coded message the alliens re sent to us that included TV broadcasts of the first TV signals the aliens received which just happened to be of Hilter parading around. Talk about first impressions.

I remember those 8 inch "floppies" almost the size of dinner plates, they were just phasing out when I was starting my accounting career.

Just a note that "visually lossless" is fine for archiving where the video will only be "looked at" later.

If there is to be any post production on the footage (compositing, scaling, effects, etc.), there can be a large difference between visually lossless and mathematically lossless formats. The little artifacts the eye can't see can suddenly take on a new significance.

23GB per hour is good for visually lossless, but for about 50% more one can get mathematically lossless. Good where needed.

Of course for old VHS tapes, just about anything will be OK...

Agreed. However, I should have made the caveat/addendum that J2K also is capable of mathmatically lossless if you'd like to archive that, too. It's all dependent on the number of scans that you'd like the system to achieve. You can dial it down to say....20 scans, and you've got a visually lossless image, or you can dial it up to 90 scans, and you've got a mathmatically lossless image. That's the beauty of J2K. The codec and licensing are still very expensive, but there are two products at NAB that I'm aware of that are taking advantage of this technology. I just wish I knew the cost. :-)

1973---Xerox invented the GUI interface, ethernet, and applied the mouse to same which was invented by NASA in 1969. Then they gave it all to Mr. Jobs.

JJK

riredale: "If you can't see the difference between a DVD copy of the VHS original of your kids when they were little, then why care? MPEG2, for all its warts, does a remarkably decent job of compression."

Yes, true, with a decent bit rate one can make a DVD that is indistinguishable from the original VHS or HI-8, but I would love to someday see a lossless final presentation format for video. The whole point is to have one source for both archival storage, where one could always go back and edit it, and one source for presentation. The problem with DVD is, of course as you know, it's a terrible format for long-term archival storage. One could never go back and edit MPEG-2 without a huge hit in quality. That is my goal, to see one format to be used for both archival storage and final presentation!

Lossless compression is the holy grail, of course. There will be many "lossy" schemes on the way there. Audio storage in Hollywood is uncompressed or lossless because of the low bandwidth requirements, and video will follow. The biggest stakeholders in this quest are the news organizations, who want to archive and monetize everything they can. For now, it's tape and huge amounts of near-line & off-line disk-based storage.

>JPEG2000 lossless? This was news to me, I think there is some small loss at the compression mentioned.

As I think DSE is also saying, JPEG2000 can do "lossy", "visually-lossless" and "lossless". I haven't been able to check the spec for myself, but you'll find hundreds of articles all making the same comment: There is a lossless option. And supposedly its on the same 2-3 reduction as the "normal" lossless formats.

The following is speculation, but (1) what is stopping anyone from using png or lzw+tiff to save lossless frames? And (2) this means that we're abandoning temporal (in-between frame) compression. Why? Certainly there are lossless approaches that can take advantage of that as well. That might be able to give you another factor of 2.

I could be wrong, Rich, but at the conference at CES, they talked (or I thought they did) about the technology having a lossless ability too. I recall it only because of a product I'm aware of that's using this technology and I found myself thinking "damn, that would be a huge frame at high speed" if they could utilize the lossless in this new product that you'll hear about soon. (no, not a Sony product)

>I could be wrong, Rich, but at the conference at CES, they talked (or I thought they did) about the technology having a lossless ability too.

You're not wrong. Jpeg2000 (I guess we start practicing; read it as "J 2 K" and it has extension "j2k" :-) does support lossless. I've been looking at some good technicle articles and they confirm it. Haven't reached the math yet, though.

Did you know that jpeg also has a lossless mode? Neither did I. Here's the link saying that it does and also noting that it isn't a popular option. No kidding.

Thanks for that link. I was a little wiped at CES, I'd done 4 training sessions that day and was afraid I'd missed something in my notes. I'd not seen this paper, either. I've been pretty excited about j2k compression since I'd first heard of it at the NIST conference 4 years ago. It holds a lot of promise for video archiving and even potentially for editing.

I think a number of compression formats have the option for either lossy or lossless encoding. You know how each programming language that came along was hoping to be the end-all language that everyone could standardize on (hah!)? I suspect the compression people were following the same pholosophy, hoping that if they offered lossy AND lossless compression then surely everyone would jump on board.

Anyway, just playing around with Paint Shop Pro 8, which offers most compression formats under the sun including JPEG2000, I note the following:

(1) I start with a slightly-compressed (jpeg) photo of my daughter riding a horse in competition. The photo is 1200x1600 and the file size is about 740KB (or 3 bits/pixel). (Photo) (reduced to 480x640 for bandwidth reduction)

(2) I save a copy in jpeg such that the file size is now 120KB (0.5 bits/pixel). If I zoom in 8x so it's just a head-and-shoulders shot, there is a lot of 8x8 pixel blockiness in the face. (Photo)

(3) If I save the same original photo in jpeg2000 lossy format at 120KB, the 8x8 blockiness is gone, but the face is not much sharper. Still, a substantial improvement. (Photo)

(4) If I used jpeg2000 lossy at a higher compression rate and save the image at a file size of just 40KB (0.17 bits/pixel) then the face is still not 8x8 "blocky" but it is quite blurred. I would judge it to be roughly equally offensive as the 120KB jpeg version. (Photo)

So, my conclusion is that JPEG2000 is good for maybe a 2-3x improvement over regular old jpeg.

So, my conclusion is that JPEG2000 is good for maybe a 2-3x improvement over regular old jpeg.

That would be true for however PSP is utilizing the format. Keep in mind there are a LOT of variations of how this technology might be utilized. It would just depend on how they've implemented it, and who's version they've implemented. Mileage will vary.

Spot: I updated my response with photos.

I think JPEG2000 is nice, but I think the improvement is not enough for everyone do drop JPEG. Kinda like how mp3pro is better than mp3, but not by an order of magnitude, and people aren't rushing out (at least to my knowledge) to re-encode to the newer format.