What is a "generation"

MRe wrote on 11/5/2006, 12:34 AM
Hi,
every now and then, especially when discussing about HDV and Cineform -projects, the term "generation" pops out.

What is the "official" specification in generation, when it is stated that m2t-format is not good for multiple generations? Or that Cineform intermediary is better?

I understand if I take a take ;-) and add color correction, some animation, possibly some zooming, etc. and render it as new (m2t?) file, I then create one generation. And when I insert it to another project, do some color corrections again, and rerender it, its the second generation. Or?

What if I just include the aforementioned modifications directly to master timeline and lets say make two color corrections using two color correction FX's one after another. Would this now be the first generation after the rendering?

What I'm after here is that (when talking about m2t) is it so that "unnecessary" intermediate renders should be avoided at any cost? Or will the end-result be the same regardless on the number of actual intermediate renders?

I do not know whether I explained this clear enouhg for you to understand what I'm after?

Comments

PeterWright wrote on 11/5/2006, 12:59 AM
In the old days, a generation was whenever you copied from one tape to another, so it was camera tape = first generation .... edit master = 2 nd generation .... distribution dubs = 3rd generation.

Now in the digital era things have improved a lot - for a start, :capturing" is now a data transfer, so if you capture, don't apply effects and output DV for instance, the edit master is still 1st generation.

But, if you have to Render to a new file, particularly of a different type, you create a new generation. The amount of degradation is still far less than in the old analogue tape days, but its there, especially if you go through multiple generations of rendering. So the thing is, to minimise the number of generations, to keep quality at maximum.

HTH
farss wrote on 11/5/2006, 1:18 AM
The internal pipeline in Vegas is 8 bit uncompressed.
In you first example the video is compressed in the camera, then decoded, FXs applied and encoded. Then it's again decoded, FXs applied and encoded again.

In you second example you loose a whole decode / encode step.

The encoding used with HDV can be quite lossy. It's bandwidth constrained. If it runs out of bandwidth it ditches data. Your FXs may add more data that causes more data to get lost during the encode. Worse still the way mpeg-2 handles running out of bandwidth can be visually quite noticeable, only parts of the frames maybe degarded (macroblocks) and they have hard edges.

Wavelet compression schemes such as used in the CF DI degrade the whole frame when they have to. The result is less visually offensive.

Unnecessary intermediate renders, well it depends what codec is used. Uncompressed HD will be lossless but require large amounts of disk space. The CF DIs are so close to lossless it probably isn't an issue with them. HDVs m2t files might also be OK, or not. For example if you were to add some 'film look' FXs like grain the loss could get really bad. A static image would probably suffer very little loss if any with no FXs added, even some CC should have no impact.

MRe wrote on 11/5/2006, 1:46 AM
Thanks. This explained it to me and helped a lot. Most of the cases I can avoid using any intermediate renders. I'm using those only to keep my timeline clean (SD avi-editing only so far).

So in the future I probably investigate the veg-nesting in more detail.

BTW. Here's two examples of heavy editing/compositing of HDV-material (two music videos of Finnish group "Poets of the Fall"):
Carnival of Rust
Lift

They were shot using Sony HVR-Z1E, which can be seen from these "behind the scenes"-shots:
Behind the Scenes

I do not know which NLE (or codec) was used for this but I can find out if someone's interested enough.

And no, I do not have anything to do with those videos. Unfortunately this kind of work is well beyond my capabilities (artistic and technical).

---
Links edited
Chienworks wrote on 11/5/2006, 9:15 AM
Lossless isn't always lossless ... when effects or processing are being applied. There will then be rounding errors which may alter the image slightly more than intended. These can build up bit by bit (pun intended) with each generation. For a trivial example, one could imagine that adding 1% brightness, rendering to a new file, then adding -1% brightness to the new file and rendering it again would return you to the original state. Not so. The top 1% of the brightness curve has been flattened to white on the first render, then reduced to 99% on the second. So there is no 100% white left in the image any more. On top of that, every other step in the brightness curve has had to be recalculated and then rounded to 8 bits, so some of the values may end up 1 off from where they were originally.

So, even with uncompressed, it's desirable to do as many things as possible in as few renders as possible to keep these rounding errors from accumulating.
GlennChan wrote on 11/5/2006, 1:03 PM
In my opinion, the quantization error is not that big a deal since you won't notice it. More noticeable problems would be color space clipping, improper color space conversions, and generation loss from heavy compression (i.e. decoding + re-encoding).

Vegas inherently picks up some rounding error when it converts from Y'CbCr to studio R'G'B'. This conversion is one source of rounding error. The Y'CbCr color space is also much larger than R'G'B' color space, so some illegal values will get clipped off. Vegas is a little better than the Adobe products in that it keeps these out-of-range values around, as it uses Studio R'G'B' color space (where legal video is in the 16-235 range, with headroom and footroom for out-of-range values). i.e. Practically every camera records values above legal white (superwhites).

In practice, the rounding error isn't noticeable. Clipping of illegal values is.

2- Chroma subsampling (i.e. 4:2:2, 4:1:1) is also a source of generation loss. You can think of it like this:
To save space/bandwidth, video uses the Y'CbCr color space where brightness is separated into the Y' (luma) component and color is separated into the chroma components (Cb and Cr). *This separation of brightness and color isn't perfect, but it's not the issue in this case.

What many formats do is keep the chroma at a lower resolution than the luma. This works since our eyes aren't as sensitive to color as they are to brightness. However, when you want to process the image, Vegas only works in R'G'B'. When converting from Y'CbCr, you have to "resize" the chroma. The codec has to stretch the chroma out... much like changing image size in Photoshop. When encoding back to Y'CbCr, it has to do the opposite. In this stretching and squeezing process, you pick up some generation loss. Vegas actually doesn't apply any filtering when it resizes the chroma, so it incurs aliasing artifacts (i.e. try encoding some titles in DV; the edges on red text on black end up jagged).

Ideally, your intermediate format would be 4:4:4 (no color subsampling). In practice, most people find 4:2:2 uncompressed (without filtering) acceptable- this is what Final Cut Pro does.

Overall, quantization/rounding error ranks very low on my list:
Quantization error- typically not noticeable; not objectionable

Chroma subsampling artifacts- noticeable (if you know what to look for); not considered objectionable

Superwhite clipping- noticeable, objectionable

Generation loss from lossy compression schemes (i.e. DCT / MPEG2, wavelets)- sometimes noticeable and objectionable. Some schemes are unnoticeable. At very low bitrates, the artifacts are weird and objectionable (they stand out).

Incorrect color space conversions - pretty noticeable; usually objectionable. In Vegas, this can happen easily when doing web encoding (I would nest your project and apply the studio RGB to computer RGB preset in the color corrector on the nest). This is probably the most common and noticeable form of avoidable generation loss.
MRe wrote on 11/6/2006, 10:12 PM
BTW. this gets more interesting. The videos I mentioned above were edited in Vegas and special effects were done with AE.

The video "Lift" is also mastered as HD; "Carnival of Rust" only as D1 PAL because of the timing of the project.

The project was done by Mr. Erkki Halkka