Vegas in RGB and vuy

CowDoc wrote on 6/20/2006, 12:05 AM
I recently had some home movies converted to Quicktime .mov files using a wetgate Rank system. They used a 4:2:2, 8 bit BlackMagic 2vuy codec. I would like to archive the individual movies. So, I need to split the 1.5 hour compilation back into short clips. Mechanically, this is an easy task. However there is a hidden dilema.

Vegas operates in RGB color space. The movie is in vuy color space. Using a vectorscope, I verified that just splitting the movie and resaving the shorter clip resulted in a color shift. The problem is the vuy to RGB to vuy conversion. RGB has a smaller color space than vuy. Here's a link that explains the problem.

http://developer.apple.com/quicktime/icefloe/dispatch027.html

I read that Vegas only operates in RGB when you add effects. If you just split and save a video, it should stay in vuy space. However, that doesn't appear to be the case. Would someone please recommend a way to trim the movie into short clips without color loss?

Once I begin editing, naturally I will be stuck with RGB limitations. However I would prefer to archive the backups before any loss.

I checked VirtualDub, but it also operates in RGB. I am currently looking at AviSynth as it claims to operate in vuy. Unfortunately, it's not user friendly. I'm still hoping for a Vegas solution.

Thanks,
Barry

Comments

farss wrote on 6/20/2006, 12:33 AM
How did you use a vectorscope?

Were these 8mm or 16mm movies. Only reason I ask is there's probably many other issues you might need to factor in such as frame rate.
Marco. wrote on 6/20/2006, 1:56 AM
When you import a Quicktime file in Vegas and do output it - it will be rerendered in any case even if you don't apply effects. And if a file will be rerendered it will be decompressed to RGB.

Marco
CowDoc wrote on 6/20/2006, 12:19 PM
farss,

Vegas has a plugin that displays a vectorscope, waveform, histogram and RGB analysis. This was an analysis of a single frame of a colorbar pattern. Input, project and rendering file parameters were identical (frame rate, field order, aspect ratio, codec, etc.). So I don't think film size matters. However, it was 8mm.

Thanks
Barry
CowDoc wrote on 6/20/2006, 12:21 PM
MarcoB,

If it were an .avi file instead of Quicktime would it still be re-rendered in Vegas? Any ideas on how to split a Quicktime file other than in Vegas?

Thanks
Barry
Former user wrote on 6/20/2006, 12:24 PM
You could get QT Pro, about $29.99 US.

Dave T2
johnmeyer wrote on 6/20/2006, 12:26 PM
I verified that just splitting the movie and resaving the shorter clip resulted in a color shift.

Ah, but saving using which codec? That is the real question here. What codec are you using in Vegas to save?

If you save as a DV AVI file, you will obviously be using the Sony DV codec, and you will be changing color space as well as your other issues. However, if you select "Save As Type:" Video for Windows in the Render As dialog, and then click on the Custom button and then click on the Video tab, you will find a "Video Format" drop down in the middle of the dialog. Choose the appropriate codec, such as the Sony YUV codec.
CowDoc wrote on 6/20/2006, 12:41 PM
I used the same BlackMagic codec that it was originally stored with (8 Bit, 4:2:2 BlackMagic 2vuy, a Quicktime .mov codec) and with the same exact settings.
I stay away from DV files because they're 4:1:1 and color corrrection suffers.

Can anyone confirm that Quicktime Pro won't cause the colorshift as well. From what I've read almost all editors work in RGB space.

Barry
Marco. wrote on 6/20/2006, 1:35 PM
>> If it were an .avi file instead of Quicktime would it still be re-rendered in Vegas?

The only AVI codecs I'm aware of which aren't re-rendered in Vegas are DV and a certain type of Cineform HD. Not sure if there are others.

Marco
farss wrote on 6/20/2006, 3:36 PM
So you put Vegas generated media color bars on the T/L, looked at these with scopes then rendered to ????, bought that back into Vegas and looked with the scopes.
Or,
There's bars in the QT files that change when rendered out to the same codec?

My question re film format relates to how it was telecined and to what frame rate it was telecined. 8mm ran at a number of frame rate and as far as I'm aware there's no way to transfer it to PAL or NTSC frame rates without producing merged frames.

Also from my experience with transferring 8mm unless it was recently shot then the color in the film has shifted so much from what it was originaly that I wouldn't be loosing too much sleep over minor color shifts from color space conversion.

Not to say that I don't admire your goal, sort of been there. I only used to use a pretty crude 8mm telecine and one of the transfers I did was for broadcast, we got the 8mm and the S8 went through a Rank. Watching the final product off air (in PAL) neither I nor the client could pick which was done on what. This was fairly old film that had been shot on good cameras for the day by a competant cameraman with good processing.

Of course if this was recently shot on Pro8 then it's another matter!

Bob.
CowDoc wrote on 6/20/2006, 5:33 PM
Bob,
CowDoc wrote on 6/20/2006, 5:33 PM
Bob,
CowDoc wrote on 6/20/2006, 5:33 PM
Bob,
CowDoc wrote on 6/20/2006, 5:33 PM
Bob,
CowDoc wrote on 6/20/2006, 5:33 PM
Bob,
CowDoc wrote on 6/20/2006, 5:33 PM
Bob,
Spot|DSE wrote on 6/20/2006, 5:37 PM
Boy, you must really have wanted Bob's attention, eh? ;-)
farss wrote on 6/20/2006, 5:54 PM
OK, ok, already!

You've got my attention. Please start talking!
Or is your tongue stuck in the gate?
CowDoc wrote on 6/20/2006, 7:04 PM
Bob,

I hoped to avoid a long post, but perhaps I should explain more fully.

I've been working on archiving these films off and on for about six months. There is about 5000 feet of 1950-60's 8mm di-acetate and tri-acetate film. There are several film types and, as expected, each color shifted differently according to the dye type used. There was also some minor dye bleeding and acetate scratches. However the emulsion itself was not cracked.

I sent the same sample film, to four film transfer houses. One used an older telecine method, one used a Sniper Pro with a high end 3CCD camera, one used a Rank and the last used a Rank with a wetgate. They all used different color correction equipment. All used the same pulldowns (16 and 18 fps). Unlike the old 24 fps pulldown, 16 and 18 fps pulldowns preserve all of the original frames. Between the original frames there certainly are frames which have mixed fields. However, none of the original frames are lost to blending as with the older 24 fps 3:2 pulldown. Hence, I could produce a progressive output if I wanted.

The difference in transfer quality (and color correction) was VERY noticible. I went with the wetgate Rank transfer.

They compiled the first batch of films and provided a 1.5 hour Quicktime .mov file using the 4:2:2, 8 bit, BlackMagic 2vuy codec. Their color correction was excellent and I would like to fully preserve that for future generations. However, for storage and editing, the file needs broke into smaller clips. Since one purpose is archiving, I would like to do that without losing quality up front. It's sort of like buying a shiny new car and immediately knocking the mirror off just so it will fit in the garage.

The goal is simply to split the original file into smaller clips with no loss. I placed the file on the Vegas timeline and insured the project format was exactly the same as the input file. The beginning of the video has static color bars. Since there is no motion, all of the frames and fields are static. Frame rate, merging fields, etc has no effect. I'm looking at one single frame. I used Vegas and took snap shots of the Vectorscope, waveform, RGB and histogram analysis. I then trimmed the file to 30 seconds, leaving only the color bars, and re-rendered it using the exact same Blackmagic codec and with the exact same output settings. Ideally, the new file would be an exact duplicate of the color bars portion of the original file. However, when I ran the vectorscope and other anaysis on the new file I found the colors had shifted.

Technically, the shift is considerable. However practically, I admit it would not be noticible unless you saw the images side by side. I sure won't lose sleep over it, but it's still silly to throw away color information.

As Marco noted, the problem is that Vegas and perhaps all editors work in RGB color space. If you make any changes to the file, Vegas converts it to RGB does it's duty then reconverts it to vuy (if asked). So no matter what you do, you loss quality if you start with a vuy file.

Since I'm archiving these films, I'd prefer not to throw ANY info away. So I'm looking for a way to simply make clips of the original film, with no loss. Sounds pretty easy, but apparently not.

All of this being said, I agree, no one will notice the color shift in the final product. However, we should be smart enough to avoid the shift in the first place. I'm just irritated we aren't. I may have to take the mirrors off the car after all.

I'm still checking into the possible Quicktime Pro solution.

Thanks
Barry


CowDoc wrote on 6/20/2006, 7:06 PM
Sorry about the odd post. I kind of hit the wrong key and then got side tracked a while. Sure got your attention though. :-)

Barry

GlennChan wrote on 6/20/2006, 8:10 PM
You can use compression programs like winRAR to simply split the file into multiple files. However (I think) if you lose one part of the file, the entire file may be difficult to read back.

With digital archiving, you'd still have problems like:
-Digital formats becoming obsolete.
-Degradation of the media - if you use a winRAR-like method, you might lose everything instead of 1 file.

It may be that the film will outlast the digital files. With digital, you could of course make multiple pristine copies and check your backups every 5 years or so. Or even re-backup the backup onto fresh media.

2- It seems like Vegas + the Blackmagic codec aren't playing well together. I believe this was fixed by Sony using the SonyYUV codec instead of the Blackmagic codec... when Vegas got support for the Blackmagic cards.

3- studio RGB versus Y'CbCr ("vuy"):
In terms of color gamut, Y'CbCr is bigger than studio RGB, which is bigger than normal RGB ("computer RGB" in Vegas terms).

StudioRGB uses the 16-235 range for legal video, instead of 0-255. The extra bits are used for over/undershoot of the signal... i.e. if the white balance is off, you'll still have all the information. But if the white balance is too off, then clipping of the colors can occur.

I think you also get slight rounding error when converting from Y'CbCr to 8-bit studio RGB... I haven't looked into this that deeply though. It probably won't be noticeable... especially if the footage will undergo compression (where the compression noise is going to swamp out the rounding error).

4- FCP operates in Y'CbCr internally I believe (they call it YUV, but they probably mean Y'CbCr). It sometimes has some bugs in it pertaining to color, although I don't think it'd be an issue.

CowDoc wrote on 6/20/2006, 10:08 PM
Thanks Glenn. That is a good summary of the predicament.

WinRAR does need all of the files. The file has to be completely reassembled to function. Unfortunately, I need to be able to play each individual file.

I have thought about going back to film long term. In 100 years there will still be some form of optical projector around, but there won't be computers as we know them. Yet, if I chose a stable digital medium and update as that medium becomes obsolete, I can achieve even better longevity. DVDs should physically last over 100 years. With multiple copies, I'm hoping that will be a safe medium... much safer than a hard drive or any magnetic tape at least.

We are certain to be software and hardware limited as compared to our decendants. So I'd like to preserve the most info possible, even though you and I might not be able to use it effectively. Eventually, someone might. As you say, I'm sure the small errors won''t be noticed today, but then again, I didn't notice TV lines until I switched to a HD TV either.

I actually considered buying FCB because of the YUV issue, but finally chose Vegas. As Bob said, RGB is not something I'll lose sleep over.

I'd like to thank everyone for their comments and I agree with everything everyone has said here... but doggone it, I'm stuborn. So it's back to the real question. How can I split a Quicktime (2vuy) file with no loss? Surely, I can't be the first person who would like this capability.

Thanks,
Barry






farss wrote on 6/20/2006, 11:47 PM
A couple of thoughts here.

1) I'm no expert on color space conversion, those matrixes are a bit beyond me however if the problem is simply a shift i.e. the constants are wrong it might be possible to correct for this in the process.

2) A much bigger concern that I'd be thinking about is how readily available will the codecs be to read these files even in say 20 years? Both the QT and Sony codec could be very problematic so an archivist would face the task not only of possibly moving the files to new media but chnaging codecs to keep up.
A better approach might be to archive as an image sequence, I'd imagine jpegs will be readable for a very long time to come.
Problem then is that's a HUGE amount of data.
Your other option is to print back to Digital Betacam, that seems to be the tape format of choice for archiving as in includes a HUGE amount of error correction. The stuff is so robust trying to bulk erase a tape is virtually impossible.
Conventional DVD ROMs have a very limited life even when stored in archival facilities, the aluminium layer and the azo dyes just will not last. You can get archival grade DVDs, a bit more expensive and very slow to burn at max 4x although I burn them at 2x, these should last around 100 years. They almost certainly will not play in STB players as the gold layer is less reflective than aluminium.

The very best archival media is gold / phthalocyanine CDs, good for hopefully 300 years but they're around the same price as the DVDs and obviously hold very little data.

Bob.

CowDoc wrote on 6/21/2006, 1:24 AM
Bob,

Or should I say Bob, Bob, Bob, Bob :-)

The RGB color space is smaller than vuy (like a little box in a big box). So the color values have to be rescaled during conversion. No clipping occurs if the colors are within the "legal" vuy ranges. However, almost all video has segments that overshoot into the "illegal" range. When legal vuy color is rescaled to fit in RGB space, the illegal stuff simply gets clipped. Once it gets clipped, the original information is lost and can't be recovered. I don't think there's any way around it... except to stay in the "legal" range. Unfortunately, that often doesn't happen.

I have wrestled with the long term storage problem for some time. As far as I can tell, there isn't an easy solution or a clear cut choice. A DigiBeta tape exists but magnetic media, including hard drives, has a shorter shelf life than optical media (i.e DVDs). Going back to film may be best, but it is costly and there are still dye problems. DVDs with periodic updates (media and codec) seems to be the best compromise (considering cost, changing technology, accessability, etc.), but your argument is well taken.

Regardless, that isn't my dang question. How can I split the %^$#@ .mov files? :-)

Barry







farss wrote on 6/21/2006, 2:14 AM
If I get a chance tomorrow I'll ring BMD in Melbourne and see what they can suggest.

If you had a Decklink card you could PTT and capture short clips using a DB deck but that's a lot of kit to have to use. Even doing the tape trick is not entirely lossless though.

Bob.

EDIT: There's a BMD forum over at Creative Cow, that might be a good plce to ask?