OT media storage

Former user wrote on 3/14/2012, 11:27 AM
edit: I just found out it is mac only. bummer


Now that I have bought a flash based HD camera, I have a panic going on about original footage storage. I have always felt comfort in knowing that if all else fails, I have my digital 8 tapes that I can redigitize and recreate a project. Well of course now, I am at the mercy of multiple harddrive backups, optical disks etc.

I recently came across this software which would allow me to repurpose my digital 8 camera and I wanted to see if anybody else has used this and what they think. It makes your firewire digital camera a data tape backup unit.

http://www.coolatoola.com

Thanks for your input

Dave T2

Comments

[r]Evolution wrote on 3/14/2012, 11:56 AM
That does look interesting. Sounds kind of like using Digital Linear Tape (DLT).

If this would accurately back up HD Video then I could definitely see this being useful and familiar.
Former user wrote on 3/14/2012, 12:05 PM
From what I understand, it is like a DLT system. So it would back up any data. There are two versions, standard and lite. There is an FAQ that answers many questions and an email contact if you have others.

Dave T2
videoITguy wrote on 3/14/2012, 1:20 PM
To Davet2:
I recall that this has been discussed many times before...can't recall if it might have been at DVInfo.net.or here at SCS. Somehow, because I am heavy into Digital-8, I recall a negative about this...but can't remember why.

Storage has been discussed many times here...some say hard-drives, some say tape...my own preference is optical disc -Blu-ray- for the archival storage media...tapes only short term - 2-4years. I say people must be crazy to trust a spinning disc mechanical hard-drive. That may be just me.
Former user wrote on 3/14/2012, 1:31 PM
Yeah I know it has been discussed. This seems to be a very inexpensive way with equipment I already have and will not use for video anymore.
I don't understand why you say storage for tapes 2 - 4 years. Tapes will last several years longer than that. Especially digital tapes, because you are dealing with 0's and 1's as opposed to many variables. And with digital video tape especially, you can lose some bits and still not lose all of your video.

Anyway, I am trying to make the transition from tape to data floating and I stilll am secure in tape storage, not so much in optical and harddrives.

Dave T2
videoITguy wrote on 3/14/2012, 1:56 PM
My days of dealing with tape go way back - I built one of the first corporate video networks in the USA with 3/4Umatic. In those times we learned a lot about tape formulations and storage -like humidity controlled, spooled out, stand-on end kind of stuff.

Now to clarify - what I REALLY meant by tape storage is taking D-8 or HDV as well as other current video shooting formats and storing the original camera tape as a means of keeping the original archive video away from production processes. The problems that become apparent are trying to restore original head tracking and pliable tape handling with spooling not exercised periodically after say 5 years.

In the DLT format of 1's and 0's storage, there are issues with getting archives created on older machine technology to respond to playing out on machines newer /or different manufacturer.
larry-peter wrote on 3/14/2012, 2:45 PM
I am very wary of any "repurposing" of tape formats that are either obsolete or predictably short-lived - OK let's get clear, I'm wary of ANY video tape format today. Any others like me who still have Betamax PCM F-1 audio masters that will never be heard again? DLT I consider to be a proven format that I expect will be supported for our lifetimes, at least - but archiving on Dig8 or DV? There will come a day when you will be frantically searching eBay of the future looking for a player that will load that old archive that you need RIGHT NOW. And look at the data transfer speeds?

I dunno, Dave, if it was freeware, maybe I'd try it for kicks, but they want $50 for the full version. I would suggest digging in the couch cushions for a little extra change and getting a BluRay burner. More data per media, faster backups and your hardware's included as well! Just my jovial two cents worth.
Chienworks wrote on 3/14/2012, 2:49 PM
I recently started a restoration project for a bunch of old audio cassettes from 35 to 45 years ago, many of which were never played since they were recorded. Some were stored in boxes, some not. Some were very dusty, some had obviously been damp and gotten moldy. However, the only problem i've been encountering is that some of the cheaper cassettes didn't have a very good slip liner and the tape is sticking. In once case one of them apparently had some form of lubrication that has now solidified. I was able to put the tape from those cassettes into a new shell and they played fine.
videoITguy wrote on 3/14/2012, 3:02 PM
I am a very great supporter of Sony's great experiment launching the D-8 format several years ago. But there are some interesting features that are built-in to this format that make me wary of using the tape transport for anything but D-8 video.

As you no doubt know -you can take a Reg-8mm video recording and play it back on a D-8 machine. Try experimenting with a tape that has both Reg-8mm video and D-8 video recorded in near proximity of each other on the same tape. Now watch what your transport tries to accomplish. Well, it should auto-adjust to the formats via the tape servos in the transport. How successfully and seamlessly it does this - is up to the condition of the machine.

As your D-8 transport ages and is either stored or used frequently some noticeable D-8 system problems will become evident with this test. Your machine can quickly fail to play either media successfully. Whoops - no good for archiving process.
Former user wrote on 3/14/2012, 4:13 PM
videoITguy,

I can't say from experience with bluray, but I have had many more optical disks (cds and dvds) fail than I have tapes. They have just stop being readable. I can still read the D8 tapes that I recorded when I got my camera new. In theory, as long as the devices exist to play it, I feel the tape will play as well. Now optical disks will fade, they just do. So even if you have a player to play it, the disks itself will not work after a certain point. I know the best practice is to renew the recording at intervals.

I was just trying to think of some way to feel safe until I get used to the idea of total virtual video. I don't use this end for business, so I have nothing to lose there, just personal stuff. My films from 1950's are still viewable. Not as original quality of course, but still viewable. My videos from the 80's are still watchable. I have burned CD's from the 90s that are blank.

Dave T2
Former user wrote on 3/14/2012, 4:14 PM
Chienworks, I recently transferred some audio cassettes from the mid 70's. Still work, have lost some frequencies but I don't remember what they sounded like anyway.

Dave T2
Former user wrote on 3/14/2012, 4:17 PM
Atom12,

Yes, I understand that playing the tapes may be an issue some day. But you can still find places to play Quad Tapes from the 60's on if you are willing to search and pay for it. I just don't know that optical disks are the best answer. I prefer DLT tapes at least which I have access to here at work. It is just a matter of buying the tapes, which tend to be a bit expensive for storage vs. cost.

I think this dilemma will plague us all for a while until someone creates an storage system that does not erase itself and holds a lot of data.

Dave T2
larry-peter wrote on 3/14/2012, 5:27 PM
I can definitely understand all sides of this. I am a backup fanatic, and have become even more so with tapeless workflow. (I do have plenty of DVD backups from 1999 that I have have accessed recently and all were fine.) Everything now goes to a backup 3.5" HD and BluRay.
There just seems to be no permanent solution in the digital age. The fear I have most is not that my media won't hold up, but the technology will change so fast that I'm unable (at a reasonable price) to play back the media I've stored. I have analog tapes 25 years old I can play and digibetas from the late nineties that won't. The tradeoff in the "perfect" digital backup is that loss of a few bits can screw everything. Let's just all transfer our stuff to 35mm film neg and place in climate controlled storage. I bet someone will at least be able to look at it in a hundred years.

And on my wish list for years have been hard drives where the physical discs live in a removable enclosure that can be inserted into a new motor/electronics housing if it goes bad.

BTW, since you mentioned this was for personal archiving rather than paid work, it sounds like something worth trying if you've got the playback machines.
PeterDuke wrote on 3/14/2012, 7:54 PM
I think the best strategy is to have your two or more copies on different media types, e.g. hard disk AND Blu-ray disc. (You must have at least two copies or it is not a backup.)
Former user wrote on 3/14/2012, 8:35 PM
Of course, then again, will I really care to see this stuff in a few years. I have video of our wedding. We have watched it maybe 4 times. the first time, times with each in-laws and then when I made a DVD of it. At least with a still photo, you can frame it, put it on your wall and enjoy it the rest of your life. Video has a lot shorter interest/life span.

Just being moody now. :)

Dave T2
videoITguy wrote on 3/14/2012, 8:46 PM
A burned optical disc ejected from the disc tray is not an archival backup. And the only way I know to have a CD/DVD dissappear or evaporate -to turn the disc burned side up and place it on the roof of your house for a few sunny days- UVrays will do their job really super well.

Here are the steps I undertake to create an archival Blu-ray disc.
Choose good media - burn the disc in burner 1 (one). Perform check-sums of all files on disc compared to their original. In burner 1 (one) use a performance diagnostic to determine inherent disc burn errors and the success rate of error correction by the reader. Place burned disc in BDROM reader 2 (two) and do the same check-sum and performance diagnostics. Save the results of all tests on additional library index volume for later reference.

Make a second burned disc exactly the same way of the same files and follow through conducting the same tests.

After storage and thereafter for every two years in a dry moderate temperature place under disc covers that do not touch the actual burned surface - bring both discs out for the same tests and compare against results in test series made during initial recording. IF there appears to be a significant degrade in test performance from Tests Series Initial to Test Series Subsequent....rewrite the files to a new disc. And so on.

In the 16 years I have been doing this archiving to CD/DVD/Blu-ray I have yet to encounter one single significant degrade of a single disc.
Chienworks wrote on 3/14/2012, 8:51 PM
Dave, what amazed me was what a couple of gentle noise reduction passes and some mild EQ could do to bring new life and sparkle to them. Some of these tapes had probably less than 25dB S/N ratio. I was able to reduce the hiss and hum 30dB without any noticeable effect on the voices and music. They probably sound much better now than they did when recorded. In once case the recording of the music teacher introducing the next piece was so quiet that it was completely buried in the noise and i was able to bring it out well enough to not only understand the words, but even recognize his voice.
Chienworks wrote on 3/14/2012, 9:01 PM
Not sure if i can find it again, but a while back i was sent a link to a company that etches CD & DVD images on stone. The resulting disc is then coated with a hard scratch resistant polymer. They're a little thicker than regular CDs but still thin enough to fit in a drive. The projected life span if handled carefully is thousands of years.

If i recall, they charged something like $600 for a CD and $1500 for a DVD.

I think my preferred method will be RAID 6, where whenever a drive starts failing one simply hot swaps it for a new one and the image is rebuilt. Theoretically it will have an infinite life span. However, in that case it's not really the hardware holding the data that matters so much as the encoding of the data. I suppose it's possible that a few hundred years from now someone will mount an MPEG-2 file from now, read the file perfectly file, but have no idea how to decode the data into a video stream. None of the current coded algorithms will run on their CPUs and since the format had fallen into disuse no one bothered updating the codecs over the decades. I suppose that's a slim risk, but still possible. So for real archival storage you need to store not only the files, but enough information to allow their decoding as well.
PeterDuke wrote on 3/14/2012, 9:16 PM
@videoITguy

What diagnostic software and hardware do you use?

For CDs and DVDs I burn using my Pioneer burner and then check the soft errors (before error correction) using KProbe 2 and my LiteON burner. If the soft errors look too high, I burn another disc.

I have no similar test for Blu-ray yet. Some people use Opti Drive Control in conjunction with a LiteON Blu-ray Combo drive (Blu-ray read only).

I occasionally check old CDs and DVDs and if soft errors look too high I copy them before it is too late.
Former user wrote on 3/14/2012, 9:34 PM
videoITguy,

I have had several CDs stored in a case in a closet or dark area that have just faded to the point that they are not readable. could have been the media or the burner. These were not for archive, just music or data disks.

But your method makes sense. I will investigate this plan.

Dave T2
videoITguy wrote on 3/14/2012, 10:02 PM
To PeterDuke:
When I was close to the CDROM industry back in the mid-90s there used to be software available to the industry insiders to properly diagnose and performance test CDROM disks.

Since I have moved-on and times have changed I have looked to the only source code writer that I know of such disc diagnostics - the coder behind Nero disc utilities who also releases under the Opti-drive label. Both brandings do have the ability to performance test Blu-ray.
As you have noted, these kind of performance tests must be done in certain manufacturer series with the appropriate chipset on-board. That's because in order to compensate for reader error correction methods -you have to get in under the hood of the system to evaluate.

NOTE that burner software of many types and for many years have used specific software checks for creating file checksums. I do not rely on these alone, but use third-party software universally available to create checksums of my own accord. That gives me yet another level of quality assurance.
Former user wrote on 3/14/2012, 10:14 PM
Chienworks, that is cool. Nothing like reviving old audio.

Dave T2
Chienworks wrote on 3/15/2012, 7:41 AM
Not sure how useful checksums are for archival. They're very useful in the present for making sure that you got an exact copy of the data, and if you didn't you get another copy right now. But off in the distant future when trying to determine the quality of the archive, what good is knowing that the data is good or bad when you can't go get another copy? It would only be useful if there were more than one copy *AND* at least one of them still matched the checksum. Personally i'd rather try to preserve something with an occasional glitch in it than not preserve it at all because there might be glitch.

Kinda like, "a man with one watch always knows what time it is, but a man with two is never sure."
videoITguy wrote on 3/15/2012, 8:24 AM
To Chienworks:
Your point is well taken about archiving - checksums have to be managed carefully. They are used most everywhere in doing backup and retrieval. Did you know that the DLT tape process uses a form of checksum to internally verify a back-up working on the fly?
To elaborate on what I pointed out earlier...I don't want to rely on the internal software of disc burning to tell me that checksum is working - because the error correction that is taking place will mask any useful conclusion.

Take disc copy 1 of burn and reset the disc by eject and reload. With disc reloaded, do an OS file copy back to a new folder at new hard-drive location. Use a third-party software to verify checksums of burned disc 1. Eject and reload disc copy 2 of burn and verify against checksums of the disc 1. If all checks-out error free then I have strong assurance that archival copies 1 and 2 on disc are the same and identical to original.

This process cancels out the masking effects of error correction (happening alt the time with every burned-disc you create) and probable OS copy file errors. Not perfect perhaps but does give strong assurance that at least one of the two media will survive storage for another couple of years. In my experience of 16 years I have never seen an instance where a disc did not survive in-tact. I have done some limited (just to be...) testing to see if an initially recorded defective disc (that did not meet a good perfromance standard) - would be stable enough- that it would replicate relatively the same defective way two years hence...and it did.
PeterDuke wrote on 3/15/2012, 8:30 AM
I agree that the use of checksums doesn't tell you how close a disc is to failing - only whether it has failed or not, and only for that reader, and for now. A marginal disc may read OK on one reader and fail on another. It may pass today but fail tomorrow.

That is why I use the C1 and C2 errors for CDs and the PI and PO errors for DVDs. If these are low, then it is reasonable to suppose that there is still plenty of life left in the disc and vice versa.