Storage overload, what is the right strategy????

klaatu2 wrote on 7/18/2003, 10:02 PM
Please--my hard disks are crammed full and I fill like I am missing the point on this....

I got into this to convert family VHS and Super8 analog into permament media. I used Dazzle PVC II which changed analog into MPEG2, and edited it as well, somewhat successfully. Here is what I think I know:

After much struggle, I learned that most editors including Vegas don't like MPEG much and prefer AVI. But AVI (which also comes out of my new camcorder) seems to be about 4 gigs for about 20 minutes.

If one data DVD can only hold 20 minutes of AVI (is that possibly true???) What format do I store all these family memories in???

- data DVD with AVI--easily editable, but huge (6DVDs = 1 VHS tape)
- data DVD with MPEG2--small, but with compression picture quality loss
- movie DVD--which uses MPEG (and it's issues) but I understand is difficult to pull off and re-edit

Thanks.

Comments

musicvid10 wrote on 7/18/2003, 11:21 PM
With the huge hard drives (120-200 GB) costing about $.50/GB after rebates right now, your answer should be not so complicated . . .
Grazie wrote on 7/18/2003, 11:27 PM
If you are going to do a lot of multiple big file projects, you might also consider HD "trays" - more like a Hard-Disc-To-Go way of working. Billyboy has had much experience of this and would possible jump in with a few hints and tips. - However don't mention the "F" word - that's firewire. I believe he isn't a fan! ;-)

Grazie
BillyBoy wrote on 7/18/2003, 11:43 PM
I'm not a fan of firewire hard drives because my original and two Maxtor replacement firewire drives failed all in the same way. Not wanting to beg for a third replacement I decided to open up the firewire drive and wasn't really surprised inside was nothing but a regular IDE drive. The problem with firewire is the cheesy interface card. In my experinece if you move the drive from PC to PC a lot (I did) sooner or later the stress on the plug will kill it. Not the drive, just the inteface card. They are that poorly made.

However a regular IDE drive in a drawer or tray like Grazie says works great. I've got a bunch of them and never had a problem with any from them. That's one solution.

To avoid buying drives by the case, if I were you I would leave the source files on VHS tape until you're ready to complete a project that requires tape A B C, whatever. Then bring them over to AVI, render the project, make a copy on DV tape and burn a DVD. Save a copy of the COMPLETED project as MPEG-2 and it will take a fraction of the space the AVI file does and you can still use it to re edit if that need comes up later with no real loss in quality. Basically what I've been doing for a couple years on a similar mamouth project that seems to have no end.

riredale wrote on 7/19/2003, 1:06 PM
I am not familiar with the Dazzle product, so I don't know how good a job it does at converting analog video into MPEG2. In my case, I would take analog video into my system through the analog-in ports on my miniDV camcorder, do my editing in DV avi on Vegas, and then do a high-quality render to MPEG2 using Vegas. You could either store raw MPEG2 files on a DVD (playable only on a PC), or you could create an actual DVD-video disk with those MPEG2 files.

If you encode to MPEG2 using a fairly high bitrate (i.e. 8 MB/sec), you should not be seeing any difference between the MPEG2 video and the source, especially since you are starting with relatively crummy analog video. In fact, once you get the hang of it, you could experiment with very-low-bitrate techniques in order to get upwards of 10 hours on a single DVD, playable on any settop DVD player.
JohnnyRoy wrote on 7/19/2003, 1:47 PM
I agree with riredale. If you are concerned about saving your tapes then your best bet is to archive them to DV tape using your camcorder. Then you can capture it with no loss in quality from your DV camera at your leisure as you find time to edit and burn to DVD. I know you don’t want to hear this because it means your Dazzle isn’t getting used. Dazzle is good for archiving straight to DVD without editing but if you want to edit, MPEG is not the best format (as you have already seen).

~jr
musicvid10 wrote on 7/20/2003, 10:25 PM
This is silly. The audio takes up a miniscule portion of the file size, so differences in audio encoding don't make a significant difference.

The highest video quality results from the least compression/highest data rate. That means the biggest file size. The incessant search for the perfect picture with the teeniest file size is like searching for the holy grail.

If you want a good archive, buy some big drives or put it on dv tapes. If you want small files, go right ahead with your complicated mpeg/divx encoding schemes. It's a choice. This is not rocket science.
2G wrote on 7/21/2003, 12:01 AM
If I erase an AVI captured from DV that was used as an editing source, then recapture it at a later time, will it be guaranteed to be precisely in sync? In other words, can I recapture and pick up editing in my .veg file where I left off? I tried this with a competitor product, and it was not in sync. Is there anything I need to be aware of if I do this?

Thanks.
BillyBoy wrote on 7/21/2003, 12:11 AM
The VEG file is nothing but a window into your source files. If you delete the source files BEFORE you're finished with the project you will have managed to effectively blind Vegas.

In human speak the VEG file is just Vegas taking copious notes about what you've done editing. It it necessary for Vegas to render the file. Everything comes together during render. So if you haven't rendered, don't delete/rename, relocate* any source file used in the project. Or you'll be sorry.

*Vegas can find relocated or renamed files. The problem comes from YOU remembering what you renamed them and where you put them. <wink>
AFSDMS wrote on 7/21/2003, 8:38 AM
Been there, done that, still have the scars.

My first video card was a DVC II also. I finally pulled it out of my machine a couple weeks ago. As I think you have already found the capture utility and editing suite are trouble prone. This card seems very popular in Europe so you might want to check out the Unofficial DVCII site http://www.spawns.dk/svcd/Index.htm.

I found the MPG2s created by the Dazzle were actually very good, but the problem was doing anything with them. The problems with editing them, which Vegas could 'kinda' do seem to be with the highly complex nature of using a NLE with an MPEG compressed video stream.

I got a Sony Digital video camera and decided to get one that had Firewire and video pass through. Never intended to use it the way I am. As mentioned by someone else, it really seems to be a good idea, if your camera supports, to put the camera in VCR mode and just copy your analog material to MiniDV (or Digital 8) tapes. That way the analog to digital conversion is done. You are now digital and there should be no further degradation. Just keep the machines heads clean and store the tapes in a stable environment.

From that point, whether tomorrow or in months from now, you can use your video capture software in Vegas to choose only the clips you want off the originals. Save then as AVI clips (yup, they are large) and create your edited product with Vegas. You can set chapter points if you want some sort of navigation within longer projects and then render the final project to MPEG2 with Vegas. Follow the Help and make sure you render to a format compatible with DVD Architect and things will move faster.

I don't remember if it was mentioned, but AVIs don't go on DVDs. It is always MPEG (compressed) so you can get much more content on a DVD than 20 minutes.

Good luck.

Wayne
kameronj wrote on 7/21/2003, 9:00 AM
I agree with Wayne (on mostly all the points).

AVI's don't go on DVDs....just MPEGs.

It may be the Dazzle product you are using and have to tweak some of the settings. I have a Dazzle product (an earlier USB one that compresses to MPEG1). I have had absolutely no problems with the files I have saved. They work just peachy in Vegas.

However, since I also have a Sony Digicam...the footage I shoot with it I just dump via firewire into Vegas.

It is true that an AVI capture takes up MONDO SPACE!! I captured a ten minute (13 actually) presentation I did...that I dumped to tape...and it was something supersilly in size (like 4 or 5 gig for ten minutes!!!).

Naturally, I edited and rendered back to MPEG to save to disc (VCD at the moment....I should have a DVD burner in a month!! Yippeee!!).

But when saving or wanting to work on older VHS, I would either capture directly to MPEG1 with my Dazzle, or resave to digitape via the Sony RCA plugs...and this way, I have an archive to digitape and can firewire and edit when ever I would like.

Here is a tip if you don't have a great deal of space to work with.....pre-edit your video as you capture. That is....trust me...you don't need to capture EVERYTHING from a VHS tape that Grandpa shot at the graduation last year.

You can fast forward through the "...Is this thing on? Say something, Jimmy!! Who's foot is that? Oh, it's my foot!! Well where is Jimmy, damnit!!"

And you don't have to capture all of the wild panning and zooming that Aunt Ethel did while trying to find the band and the section of tuba players to get that shot of Cousin Ed at that school play.

Lastly....and I forget the name of the software...but...there is a NLE software for digicams hookup via firewire that allow you to edit on the fly from the tape (it just records the timecode) and you don't actually do any capturing until the editing is done. It then goes back and reads the timecode of the tape, and starts/stops/starts where you put in your edit points.

So the long and short of that process is - you only save to your HD your "finished product".

knawhaI'mean?
mcgeedo wrote on 7/21/2003, 10:06 AM
You can use a DVD to store avi's, but it is in DVD-ROM format, i.e. a data disk. I've done this a couple of times (using R/W disks) when I had a piece of footage that I was getting a number of clips from. The DVD loads faster than a DV tape. Also, the rule of thumb for DV is 4.5 minutes per Gig, more or less.

Good luck,
-Don
JohnnyRoy wrote on 7/21/2003, 11:01 AM
> This is silly. The audio takes up a miniscule portion of the file size, so differences in audio encoding don't make a significant difference

Not totally true. The default PCM encoding is huge because it’s uncompressed. Before I had DVD Architect, I would always use MPEG audio on my DVD’s (even though its not “officially” part of the standard for NTSC) because I could get more video on a DVD that way. The same is true for AC3, its way smaller than PCM will allow more video on a DVD. At least this is what I've experienced.

> If I erase an AVI captured from DV that was used as an editing source, then recapture it at a later time, will it be guaranteed to be precisely in sync?

Yes, if you use Advanced Capture using the Timecode In and Timecode out and then save the .sfvidcap file. SPOT discusses how to do this in his Vegas 4.0 Editing Workshop DVD. He says he can edit a project, then come back to it months later on another machine and recreate the project by using the same sfvidcap session file. So save those session files.

> Here is a tip if you don't have a great deal of space to work with.....pre-edit your video as you capture.

The Advanced Capture in Vegas will let you do this. You can watch the tape and log in and out points. Then you can tell it do go back and capture just those logged points.

~jr
2G wrote on 7/21/2003, 12:08 PM
I understand how vegas uses the files. That's not my question.

My situation is this: I finish a project and need to erase the source AVIs to free up space for the next client. But then for some reason, the original client wants a tweak to some segment. I can recapture the AVI from the DV tape (give it the same name, etc.) My question is whether two separate captures of the same DV tape will be truly identical? Assuming both captures start from fully rewound, can I guarantee that the frame in the AVI from the first capture at, for instance, 2 min, 16 sec, frame 8 is truly the same frame I'll see at that time offset in the recaptured AVI. Or is it going to just be "close" with no real guarantees?

My delimma is simply that I can't keep 11 hours of raw video on the hard drive after I finish a project for a client. But I also don't want to be in a situation where I absolutely cannot recreate the environment and make a change if a paying customer needs it. The ideal situation is to archive the veg and all other support files, and erase the AVI files. Then recapture to AVI from tape if absolutely necessary, assuming obviously that it will guarantee the same results.

Is identical recapture a realistic expectation?

Thanks for the help.

P.S. Is a more reliable alternative to convert all the AVIs to MPEG2, archive, then convert the MPEG2s back to AVI if necessary? Is that more reliable? What are the pitfalls to this approach (I'll accept any minor quality loss) Are there scripts available that'll convert a whole directory of AVIs to MPEG2 in batch?
riredale wrote on 7/21/2003, 12:29 PM
2G:
That's an interesting question. I had tried it back when I used Studio7, and as I recall, the recapture was not frame-exact. This should be easy to test, however, and if I get some spare time I'll try it myself, both with Vidcap and with ScenalyzerLive, which I use almost exclusively. In theory it should work perfectly. Perhaps some other user already has a lot of experience with this.

Marquat:
There is something strange going on in your case. The basic idea is that you can pick any bitrate between about 3 and 9Mb/sec for a legal DVD. VBR should always give better results than CBR because the encoder is then allowed to allocate more data for the complex scenes, taking bits away from the simple scenes that don't need them anyway. To figure out your bitrate, simply divide 600 by the number of minutes of video you plan to put on a single DVD-5 disk. For example, to hold 90 minutes of video, you would choose 600/90=6.7Mb/sec as the overall rate. Audio takes up about .2Mb/sec (for compressed audio like AC-3), leaving 6.5Mb/sec as the target VBR "average" bitrate. I personally would set my minimum at 0, my maximum at 9, and my average at 6.5.
JohnnyRoy wrote on 7/21/2003, 12:48 PM
OK perhaps my answer wasn’t clear. (sorry)

> Assuming both captures start from fully rewound,

If you just rewind the tape and hope for the best, then No you will not get an exact frame-accurate copy. BUT, if you use Advanced Capture and you tell it to start the capture at timecode 00:00:15.00 and save that as an sfvidcap file, you can come back at any time and reload the tape, and reload the sfvidcap file and the DV camera control will roll your camera to timecode 00:00:15.00 and capture the exact same AVI file all the way through (assuming no dropped frames). You can also mark several in and out points with advanced capture and get the same results every time.

The reason I chose 00:00:15.00 is to give the tape transport time to stabilize. You should probably get into the habit of recording 15 – 20 sec of black onto the front of your tapes anyway because you probably will not be able to capture from the very beginning with any accuracy.

~jr
Chienworks wrote on 7/21/2003, 12:53 PM
I would say that "fully rewound" is definately a wild card in this situation. Whenever i rewind a DV tape with video recorded on it, the time display will indicate anything from 00:00:00 01f to 00:00:00 07f. Maybe it's just because i bought a cheap camera, but it would assume that any mechanical device could show this sort of variation to some degree. So, probably the first scene captured from the tape could have a variable starting point depending on how well the tape rewound.

This doesn't help for tapes you've already shot, unless you're not using the first scene, but for future shoots i would always tape a couple of seconds of fluff at the beginning of the tape, then pause or stop before shooting scenes. This way there will be a hard timecode change at the beginning of the first scene no matter how well the tape is rewound. After the first scene i would thing that the rest would be accurate to within a frame, or maybe two at most. That's probably good enough for most purposes. The few that need to be adjusted on the timeline would be easy enough to handle with the Alt-drag method.
2G wrote on 7/21/2003, 5:37 PM
Thanks for all the good info. Got some things to try now.

2G