"Purest" SDI 10-bit uncompressed solution?

Kevin R wrote on 5/1/2010, 8:36 PM
Sony makes much "to do" about Vegas' 32-bit processing:

New! Enhanced 32-bit floating point video levels mode designed to work like 8-bit mode but with the precision needed for 10-bit and higher sources and render formats

But, just what the heck does this mean?

I am looking for transparent 10-bit SDI pass-through. Is this possible? AJA makes some talk about 4:2:2 support, but NO WHERE can I find any good explanation about what is the true quality retained by Vegas.

What is the purest SDI in/out path that anyone is aware of? What hardware? What codec?

Thanks.

Comments

farss wrote on 5/1/2010, 11:21 PM
If you want pure SDI passthrough you've got to start at the VCR as SDI doesn't always provide a lossless copy of what's on the tape. To get a bit copy of what's on the tape you need a VCR with a SDTI interface and something at the other end that can capture it. There's only two systems around I believe capable of that and they cost, big time.

If you simply want to cut something captured over SDI then you can capture to the BMD codec which is the same as the Sony YUV codec. Both come in 10 and 8 bit variants. My tests sometime ago showed that feeding 10 bit YUV into Vegas 32bit float pipeline produces true 10bit YUV out the other end. Would be nice to have a 10bit pipeline like other NLEs offer but keep in mind its decoded to RGB and back again anyway, well as far as I know.

Bob.
Kevin R wrote on 5/1/2010, 11:50 PM
Let me be more specific.

I am capturing from DigiBeta, which I believe is a 4:2:2 source. Currently, I am using a Mac with older FCP and AJA Io hardware. I can capture on the Mac into 10-bit uncompressed.

Mac with FCP is a known, proven solution.

Vegas, on the other hand is a bit undefined in its internal handling. I'd like to know if any hardware/codec has been proven to be transparent in its handling of a 10-bit signal. If so, then I can consider using Vegas for capture, color-correction, and output of my broadcast video clips.

Otherwise, I will buy the latest Mac/FCP and continue to use Vegas for pro-sumer and Internet video only.

Kevin
farss wrote on 5/2/2010, 1:27 AM
1) Yes Digital Betacam is 10 bit 4:2:2.
2) I've captured it via SDI using a BMD Decklink card. Best to use the Decklink capture utility that comes with the card.
3) As I said the BMD codec is the same as the Sony YUV codec that ships with Vegas, both can be either 8 or 10bit 4:2:2.
4) Render using a 32 bit float (video) pipeline. You can use the normal 8 bit mode for editing and switch to 32bit float for the final render.
5) As I found out only a few days ago a surprising number of Australian TVCs and dramas are cut using Vegas.

I don't know what you mean by "undefined", a 32bit floating point pipeline is as good as it gets, certainly better than a 10bit pipeline. Just be aware thet not every plugin supports 32bit float but the color of the icons tells you which ones don't. The so called 10bit modes in other NLEs are NOT transparent at all, everything has to convert from YUV to RGB and back again. Even in 8bit Vegas does a best of class job of this, in 32bit it gets even better.

You're missunderstanding "uncompressed". 4:2:2 is already compressed and recorded to Digital Betacam using a proprietary mpeg-2 codec with a 2 frame GOP. If you don't believe me watch what you get out of a DB deck when you spool the tape. When the tape is captured the VCR decodes the mpeg-2 stream from the tape. When you record back to tape it gets compressed again. Not what anyone would equate with "uncompressed". As I said before its almost impossible to get that data from the tape as a bit copy into any computer, Mac or PC or to make a cloned dub.

Bob.
Coursedesign wrote on 5/2/2010, 1:40 AM
I've been shooting 10-bit uncompressed 4:2:2 via SDI since many years (and DB is true 10-bit 4:2:2 also).

Used a BMD card initially, into Vegas with straight cuts only in the BMD uncompressed codec.

Got tired of this after a while for the same reasons you seem to be facing, and got an AJA Io.

Later when Vegas got 32-bit handling, I considered that but was wondering how to live with all video files taking up three times as much space for what was in regular use not making any difference in PQ.

When ProRes came, I saw no reason to use uncompressed. Amazing performance and PQ out of a true 10-bit codec that was also compact. See this for competent info on ProRes.

Today, if you want to truly work in 10-bit, you need to work in one of the three A's NLEs (Apple, Avid, Adobe).

Of those three, Apple has the largest ecosystem by far, the upgrade to a full Final Cut Studio from whatever version you are using is $297 or less, and you presumably already know the basics. The Apple Pro Training Series books by Diana Weynand are incredibly valuable, because they don't just teach you how to do things, they teach how to do things in the best way that saves time. Really A+++.

Avid is a better editor for just cutting, especially MC 5 coming this summer. Still, it is nearly $2.5K, and the difference is to some extent a matter of your adaptability. Their DNxHD codecs are comparable to ProRes, except afaik there isn't a 4:4:4:4 codec which has real uses for working against After Effects for example.

Adobe's Premiere Pro: well, mine (from a suite) has gone unused. I never liked it, and neither do most others.
PP CS5 has some potential, too early to tell. They've got some CineForm codecs, but in practice you probably need to buy more from CineForm (which comes with a number of advantages).

Avid and Adobe are cross platform, and run equally well on both nowadays.

Apple of course runs only on its hardware, but that hardware is really nice, and I prefer OS X over Windows.
You could run the AJA Io into a 27" iMac, that's been a very popular choice recently, or get a Mac Pro for expandability (4-5 hard drives inside and slots for PCIe cards, etc.).
Coursedesign wrote on 5/2/2010, 1:56 AM
...recorded to Digital Betacam using a proprietary mpeg-2 codec with a 2 frame GOP

No, that's Betacam SX.

DigiBeta is DCT-compressed I-frame only 10-bit YUV.

It is true that Vegas does a best-in-class job with 8-bit, because it works in RGB which in many situations is more information dense than YUV.

Cameras today still mostly record in YUV though, so there will be conversions somewhere anyway.

The higher ProRes codecs also come in RGB (even RGBA with alpha channel).

It could be argued that 10-bit is a compromise compared to 32-bit, but it is a very good compromise, as we don't have infinite hard disks and infinitely fast CPUs yet. Rendering 10-bit is quick, 32-bit isn't.
willqen wrote on 5/2/2010, 2:19 AM
to Kevin R. -

be aware that CourseDesign is some kind of Apple promoter. I've read lots of his posts and most of the time his solutions to our Vegas needs and wants are to go get an Apple product such as FCP.

As I said, I read a lot in these forums, and don't post much. I'm not an expert, but I have read the Vegas Manual and some of Spot's (Douglas Spotted-Eagle) books where it and he explain the use of a 10-bit timeline and 32bit floating point processing. One reason Sony developed 32bit floating point processing was to allow us to run 10-bit media on the timeline, and render with better codecs such as the 10bit Sony YUV codec, and you can also do the same with some of the MXF codecs.

If you want to use 10bit media then by all means go ahead and do so, on the editor of your choice.Just be aware, there is a lot of junk written about Vegas both here in the forums, and outside, even in major magazines by people who either have other agendas or should know better.

I myself personally have had no problems importing and running 10bit media, that is after I read Spot's books . . . .

Will

PS: concerning 8bit plug-ins; Vegas will run 10bit media on a 32bit timeline until it runs into an 8bit plug-in or other 8bit process, where it will change the video to 8bit run it through the plug-in and then change back to 10bit (32bit) and go on . . . .
Coursedesign wrote on 5/2/2010, 3:09 AM
I've been accused many times of being an Apple promoter in this forum (only).

Yet, I don't own any Apple stock, or have any other kind of financial or personal relationship with Apple.

I am just relating what worked for me, and what choices I had to make for my own needs.

It's up to each reader to see what of this, if anything, applies to their needs.

Funny how this is the only NLE community I know of where people don't want any information about other tools.

The extreme opposite can be found in the Final Cut Pro User Groups, where people from Avid and Adobe make presentations nearly every meeting, and nobody is offended, nobody feels it is an imposition, they just see it as information that may be valuable to them for various professional needs.

Many pros use different NLEs for different clients or where the strengths of a particular tool stands out.

Vegas is a particularly wonderful NLE, but like the others it is not the best tool for every need.

So it's good to have choices.
willqen wrote on 5/2/2010, 3:26 AM
Now if you had but said that or some abbreviated version of it in your previous (to mine) post, I never would have said a thing. And you are absolutely right.That's why I haven't commented before when others have castigated you. This started as a question about Vegas, then the OP went on about FCP so comments concerning FCP were warranted, I would have just liked to see more comments about Vegas. That's all. I didn't mean to imply what you do or say is wrong, just to inform an obviously new OP that you lean towards apple a lot. to call you a promoter was wrong, I see that now and you have my heartfelt apology.

Will
Coursedesign wrote on 5/2/2010, 4:21 AM
No problem. As you may have noted above, I also said that Avid was the best NLE for just cutting (because its trim modes are still unsurpassed, although FCP7 can get close with some minor hacks, for example for four-point editing). So I expect to be accused of being an Avid shill also :O).

Vegas is clearly the best NLE for many if not most people editing established 8-bit formats, where it's basically rock solid in my experience (which spans Vegas 3 through 9). Vegas makes many things jawdroppingly easy compared to industry-standard NLEs, and having a good DAW inside is just heaven (I have done much audio-only work in Vegas too).
farss wrote on 5/2/2010, 4:40 AM
You could do like around 80% of the FCS users down here do, capture it over firewire :(

I don't know why people still see DB as so difficult to do right. The average PC or Mac handles it just fine with a basic RAID array, probably easier than HDV from my experience. A basic SDI card from BMD or AJA is pretty cheap.

Bob.


Coursedesign wrote on 5/2/2010, 6:40 AM
You could do like around 80% of the FCS users down here do, capture it over firewire :(

??? You mean they use a capture device with SDI in and firewire to the computer?

That works fine for SD uncompressed 10-bit which is 216 Mbps for NTSC (and about the same for PAL).

AJA made a good living selling those boxes.
farss wrote on 5/2/2010, 1:34 PM
"You mean they use a capture device with SDI in and firewire to the computer?"

Nope, J30, use firewire interface to capture as DV.

Bob.
Laurence wrote on 5/2/2010, 3:03 PM
You can get scary close to the quality you guys are talking about with 10 bit color and uncompressed video just by using Cineform Neo HD and color correcting in First Light instead of Vegas directly. Not only that, but you can do it with off the shelf dual core PC and average USB 2 hard drive.
kairosmatt wrote on 5/2/2010, 4:17 PM
To come back to something that was said earlier by Course:

"I've been shooting 10-bit uncompressed 4:2:2 via SDI since many years"

I was under the (presumably mistaken) understanding that 4:2:2 was-by its nature-already compressed.

Or to put it in a questions some pros can answer:
Shouldn't uncompressed be 4:4:4??

kairosmatt
Coursedesign wrote on 5/2/2010, 5:09 PM
No, the color sampling is figured separately.

Uncompressed 4:2:2 means that every Y luma sample is stored as-is, and every other U and V chroma sample is stored as-is (this is the standard terminology, even though it isn't scientifically correct to the nth degree).

There is a variety of compression methods, but they deal with how to compress the image for a given chroma sampling.

For example, there are different profiles for MPEG-2, using different compression algorithms, and even different chroma sampling such as 4:2:0 or 4:2:2.

There is even 4:4:4:4, which is very common in Hollywood for compositing (the fourth number is for the alpha channel, and in this case it's RGBA instead of YUV above).
apit34356 wrote on 5/3/2010, 3:14 AM
"There is even 4:4:4:4" ---- is not compressed in any form or wrapper.

"Uncompressed 4:2:2 means" this explanation is very weak for a newbie. Pixel lost is still ..... pixel lost.

robwood wrote on 5/3/2010, 6:23 AM
a) generally (and somewhat inaccurately) speaking, 444 is RGB, 422 is YUV (DB, DVCPRO50, ProRes, etc)

b) 444, 422 indicates the number of color samples taken for each luma sample
(eg, 422 = 4 luma samples have 2 red and 2 blue samples)

c) uncompressed is referring only to the compression format used to store the video

d) 10bit is common for good-for-broadcast: the extra 2bits per channel allow more color detail and subtlety in the highlights/gradients.

soooo... uncompressed and 444/422 don't relate. for instance:

if we were working with DVCPRO50 footage or DNxHD for instance, chances are good the footage is 4:2:2... this isn't lossless/uncompressed even tho the quality could be excellent.

conversely, i've been given JPG image-sequences for title sequences... this is 4:4:4 in spec but the format is still compressed and lossy.

and in between, PNG image-sequences are usually 4:4:4, but PNG is compressed, tho lossless.

~

sigh, i thought i'd explained this well, but having re-read now feel i've just muddied it up... lemme know if it needs to be more clear.
rs170a wrote on 5/3/2010, 6:28 AM
From Adam Wilt's website:
http://adamwilt.com/DV-FAQ-tech.html#colorSamplingWhat are 4:2:2, 4:1:1, and 4:2:0 anyway?[/link]
http://adamwilt.com/pix-sampling.htmlDV Pix - Sampling Methods[/link]

Mike
Coursedesign wrote on 5/3/2010, 7:04 AM
"Uncompressed 4:2:2 means" this explanation is very weak for a newbie. Pixel lost is still ..... pixel lost.

Sorry to have confused. The luma resolution is 100%, which is what the eye is most sensitive to. It is "the black & white" picture part. The chroma, well, the human eye is practically ecstatic when it gets color information for every other pixel, because it can fill out the rest during the brain's processing.

When you watch HDTV, you don't get even that. You get 4:2:0, which means the brain only gets one fourth of the "color pixels."

And when you shoot NTSC DV (4:1:1) and render to 4:2:0 for a DVD, you get only 12.5% of the original "color pixels" (the pro term is chroma resolution).

That's why that looks so mushy compared to the 4:2:2 broadcast production standard, which was always 10-bit and still is in many places.
kairosmatt wrote on 5/3/2010, 8:42 AM
Thanks everyone,
Robwood, it made sense to me. I also found Barry Green's article about it here:
http://dvxuser.com/articles/colorspace/

I'd actually read that before, but forgotten about it.

I was confused because I thought that -by its very nature-uncompressed would be 4:4:4. Its interesting that it doesn't have to be. I understand that uncompressed is really referring to the lack of codec compression, but I just thought it would default to 4:4:4.

kairosmatt
Kevin R wrote on 5/3/2010, 9:43 AM
I'm very pleased to see such great responses! Especially from Coursedesign who really defined DigiBeta's compression scheme and made an ideal recommendation of a 27" iMac and AJA Io. Why is this ideal? Two reasons:

(1) I already have an AJA Io used with an older Mac.
(2) I am now set on ProRes 422 HQ for broadcast delivery which requires FCP to produce. ProRes is industry standard, perfect for post, and is equally accessible in Windows with the decoder download. (Anyone know if the Windows decoder provides all 10 bits to Vegas, or just 8 bits?)

Coursedesign provided very good technical information, along with the link to Apple's ProRes document (which I have previously read several times). And, I didn't know there was an upgrade version of FCS for only $300!

Thanks to wilgen and farss for partially clarifying the somewhat mysterious 32-bit floating point question. I recognize the far greater dynamic range and reduced quantization induced in processing that 32-bit offers, but it seems questionable if 10-bit will be passed though exactly untouched after going through RGB and 32-bit conversion, then back. It seems that Vegas could be in good shape to process 12- or more bit video as well??? Perhaps Sony will add YUV and other color spaces in the future (though this would break all existing effects).

So what about poor Vegas? I will be using Vegas as well. After creating clips on the Mac I will load them into Vegas to prepare them for Internet delivery. I will be writing a custom plug-in for Vegas (something that FCP certainly cannot do). The custom plug-in will automate my workflow.
Laurence wrote on 5/3/2010, 10:45 AM
Anyone know if the Windows decoder provides all 10 bits to Vegas, or just 8 bits?

Vegas only sees 8 bits. This is a function of it's still using VFW which only works at 8 bits. You can use 10 bit codecs with Vegas, but Vegas will only see and process the 8 most significant bits.

That is one of the reasons I like color correcting with Cineform FirstLight. That way the color correction is all done in ten bit precision and only truncated once at the end as you render.
farss wrote on 5/3/2010, 1:48 PM
"Vegas only sees 8 bits. This is a function of it's still using VFW which only works at 8 bits. You can use 10 bit codecs with Vegas, but Vegas will only see and process the 8 most significant bits"

Try this simple test. Create an 8 bit gradient, Vegas's Gen Media only does 8bit. Render it to the 10bit YUV codec. Bring that back into a 32 bit project. Drop it down two stops. Render out. Bring it back and bump it back up two stops.
Now if Vegas was only reading the MS 8 bits you'll see the truncation on the scopes but you don't. Vegas does read 10 bits from a 10bit AVI into the 32bit float pipeline. It'll even read 12 bits from R3D files.

Bob.
farss wrote on 5/3/2010, 2:20 PM
"Perhaps Sony will add YUV and other color spaces in the future (though this would break all existing effects)."

YUV isn't a "color space" at all. Rec709, Rec601 are color spaces. YUV is a way of encoding to save space. Cameras actually read RGB, digitize that and then convert to Y"CbCr.
To the best of my knowledge all NLEs decode that to RGB for internal processing. Vegas does an exceptionally good job of this and encoding it back to Y'CbCr. Tests done almost a decade show zero loss after 100 generations. Avids systems show significant loss after 1 generation dur to their unavoidable use of chroma smoothing.

Here's a tip when testing codecs and loss. I see this mistake all over the web. Don't use a difference composite. 5 - 6 != -1 as negative values become 0. Instead use Vegas's Difference Squared comp mode. Personally I use a different method that yields 50% grey as then its much easier to see any differences.

Using ProRes as an intermediate may be OK but it is a lossy codec. Given that you wanted "purest" that would be quite contrary to your initial goal. If you want to use an intermediate codec I'd always go for something that's cross platform compatible such as Cineform or Avid's DNxHD.

ProRes is not an industry standard. DNxHD is, SMPTE 2019, recently qualified by SMPTE. That means that Avid must licence it to anyone and for the same fee, as it so happens, for free.

Bob.