{quote}
Then I think it would be great to have an option: codec restricting the signal to PAL gamut or working in full range.
{/quote}

full range of yuv is more than rgb24, there is rgb with negative values and values more than 255, so while 8 bit per channel it's not so easy.
restrict output rgb to yuv (PAL/NTSC) - that all codecs do in practice

DaLIV,
why are you asking these question here when as you already know the faulty lies in the camera, why don't you try asking the camera manufacturer?
As the good engineers here have also pointed out many of the Hollywood produced DVDs also produce illegal levels out of the DVD player.
To answer your question (again) almost all consumer cameras can produce illegal colors and levels, in fact I haven't seen one that does produce legal levels.
I'd also have to point out that apart from your footage having out of gamut colors you're very lucky you didn't destroy the camera. Pointing a camera anywhere near the sun is the quickest known way to destroy it.

sun - there is well seems that effect, but in nature you can see other well luminated objects :)
your final suggestion can be translated as :"you must sit in studio in dark room and luminate that with dozed lights and no come out to nature ...." :)

Yes, agree. But this is a workaround only. I don't want to correct afterwards, I want the codec and the system to preserve original levels and colors. Look at any Canopus system. Them all do that job very nice. Add an option to restrict colors to PAL gamut if it is meant for broadcasting and everything is fine.

Marco

Could somebody from Sony please make some comment about all this. It seems there is a problem here that needs addressing. This discussion has been going on for some time now and it seems like finaly everybody is agreeing THERE IS A PROBLEM. As I said in an earlier post, how can you do color correction if the dv codec itself is changing the color again when it renders !? Perhaps this problem only occurs under special circimstances but it is still a pretty big problem. I shouldn't have to go out and buy a new camera (perhaps the VX2000) so that I can use Vegas properly. I guess if there are products out there that don't have this problem (like Canopus) It should be possible For Sony to fix.

living in hope

Berti

Don't worry too much here. As DaLiv and others pointed out earlier - this is not a "real" Vegas problem. It is something almost any DV codec does. And it is something which does only happen to very certain kind of colors. And it is not a bug. It is the way color transformation works if you use RGB color space for decoding.

Marco

I haven't tried the test but in principal there could certainly be cases where one set of colors in xyz colorspace at 123 compression didn't map perfectly to another colorspace in another compression scenario.

This is true in any type of translation- Perform "Peter and the Wolf" in English and the piece loses some level of subtlety no matter how good the translator. Render a killer perfect photoshop graphic to uncompressed HD in YUV colorspace or output as 70mm on an ArriLaser and...wow, looks a little different than it did on my Dell 17". BUG?????????

No!

The right thing to do is to learn how to work with your tools- understand the limits, the strengths, what you need to do when importing or exporting. You can do amazing, world changing, life affirming art with DV, with Vegas,with uncompressed HD, with charcoal on a cave wall.

I don't know that everybody is finally agreeing that there is a problem. that YUV doesn't convert everything exactly to RGB is an issue, but not one that can be easily fixed, or one that needs to be fixed, IMO. It's rare that issues would creep in. Converting from avi to mpg, 4:1:1 to 4:2:2, 4:2:2 to 4:1:1, 4:1:1 to uncompressed, 4:2:0 to Mpeg, PAL to NTSC, 29.97 to 24p, mpeg to avi, avi to mpeg, 4:1:1 to uncompressed.....All conversions cost something. Nothing, nothing at all can be done about that. We used to deal with this a lot in BBD digital delays. Every time the delay repeats, a little bit of the signal is lost. That's why they are called Bucket Brigade Devices, because like a bucket, every time the audio changed "hands" a little bit of the signal was slopped out.
god, I wish I had a nickel for everytime something was 'easy.' If this was all so easy, everybody would be doing it, right?
BTW, look at any FCP forum on this same issue. People are no where near as freaked, and FCP is FAR worse at the transcode than most anything out there.

Spot, I tend to disagree here from a customer perspective. No, I do not disagree about the point that every conversion costs something, and also not about the point that the conversion is limited by the fact that not all colours can be transformed into another colour space - that is quite clear.

But I disagree that high-sofisticated tools like Vegas should accept such flaws - the solution would be to switch from RGB to YUV rendering (as we know it from the Canopus DV codec and so "cheap" tools like LetsEdit). As far as I understand that now, this makes the tools not only faster, but avoids also such conversion errors.

Kr
Wolfgang
PS: at the end we could order your book also via the German Amazon in Germany and Austria - I never gave you that feedback.

wshmid,
LOL, as I understand the trend is to ditch YUV in favour of RGB, that gets around the conversion problem very nicely.
Any any case the maladjustment of the average TV will have a much greater impact than odd quirks in a YUV>RGB>YUV conversion anyday. If you live in NTSC land, well just tweak that knob labelled "Hue" for whatever color scheme pleases your eye.

Actually I find this whole discussion pretty wierd, the eye adjusts to quite large shifts in color and doesn't see them as being wrong or of very limited resolution, this is the whole basis of color television transmission, both PAL and NTSC and it's also the reason why trying to color grade to an absolute standard using your eyes is so difficult.

If you really want something to worry about how about the limited temporal resolution of film and video? It is so for off what is really natural it's a joke. Of course Vegas will happily cope with 100 fps, just don't try to find a camera or a display system that'll go along with that idea of achieving a natural look.

Well dispite what I've said I do think Vegas is a good product, and I can understand that the "problem" is not strictly speaking with Vegas but with the dv codec ( or at least RGB\YUV conversion ). However am I correct in understanding that it is TECHNICALLY IMPOSSIBLE to make a dv codec that can convert these colors without an OBVIOUS change.I know that every conversion or compression causes some kind of degredation to the original but the question is really whether or not this degredation or change can be seen or not. We are not talking about recoding a DVD with just 125 Kbits/s, I agree you will definately see a difference. I want to stress again that these color shifts are quite large, and I am not really satisfied with "oh your eyes will get used to it" or "adjust the color on your TV". I know that every piece of software has it's "limitations" but in this case it means that Vegas cannot be used at all (atleast not the DV codec) to edit the film I have because it introduces such strong shifts in skin tone. I'm not saying that my original HI8 film could have been broardcast in cinemas and now it's ruined, but after rendering backout to dv there is a definate change for the worse. If Vegas wants to compete with other "high end" products is it not atleast worth considering what Canopus have done if that avoids the problems we are talking about ?

Berti

so.... when tools that will can help us to check incorrect conversions will be available?
and also may be possible preserve color information - in this case in color corrector also may be added work with native yuv and presets to limit yuv to normal range ...

codec is possible - but not to rgb24 - need other rgb (12 bit per base color is enough) - and software must work with this ....
but while soft works in rgb24 ...

if you review conversation before - you can find -
1. we will need tool that can help find out theese incorrect conversions and mark out them on timeline
2. may be created manageable codec - from vegas color corrector(add capability to work with yuv) set limits to decoder (by presets, functions, values, ...)

>> However am I correct in understanding that it is TECHNICALLY IMPOSSIBLE to make a dv codec that can convert these colors without an OBVIOUS change.

Canopus DV can do that. I tested it within Canopus Let's Edit.

Marco

Firstly I think your issue is a quite different one.
How did you digitise your Hi8 footage?
Whatever did the conversion is most likely what caused the shift in skin tone. Canopus codecs (in the AVC 100, 300 etc) are well known to push the reds a bit higher. This could explain your problem, nothing to do with the YUV>RGB conversion. Also you should be easily able to balance it out using color correction.
To the best of my knowledge Canopus do not make any high end systems, that is the realm of systems like Avid and again I'd point out that the trend now for high end end systems is away from working in YUV color spaces and going to RGB systems. I think you'll start to see more cameras that are fully digital with the entire chain working in RGB. YUV is a carry over from the antique analogue television system, I can't see any place for it in the digital world.

YUV is a compressor but we don't give it a hard time probably because its behaviour isn't unlike the human eye. There is always going to be a place for it in uncompressed pictorial representation. If you care to quote 4:2:2, 4:1:1 or 4:2:0 when you state your equipment capability, you are stating a YUV characteristic. Tapes (amateur and pro), DVDs and a good percentage of digital disc recorders live and breath in YUV planar space (as opposed to RGB linear space).

TVs don't have phosphor, LCD or plasma in anything other than RGB emitting or illuminated materials. CCDs (camcorders, scanners) also do the RGB thing (through a convoluted band pass filtering).

<tongue in cheek>
So it would appear that to properly excite the visual cortex. We would need that to be swapped out for an RGB one to suit our modern art?
</tongue in cheek>


I suspect the real truth is that RGB pipelines (for transitions/DVEs, filters, 2d/3d manipulations etc) are the natural programmers choice. YUV would be a saving to those who can get their heads around how to program in that "plane". Most sources and targets are carried as YUV data. So either argument stands.

I suspect that either swapping or adding a YUV mentality to the RGB paradigm that Sony currently has with these products would take a major overhaul. It wouldn't buy enough progress for the time involved. I suspect that all those built in effects and 3rd party plug-ins would also be sub-optimal. I must admit, when working in Vegas, I tend to layer the Sony filters far more often than I layer tracks of video (which need converting). So I would be happy that things stayed the same and hit my directx display adapter with RGB data without a CPU or silicon matrix making use of YUV data.

If a YUV pipeline is important to me, I'd save up and buy a NewTek VT3 (or DVStorm2). Which then I'd have to apply 10bit RGB generated media for to get a half decent YUV representation for without contouring.

We simply can't look at mother nature or informatics to get the perfect answer. I believe you can get a "proper" conversion, if you add the bits into the dynamic range of your target format. So it comes down to when this is practicable to engineer so either the conversion doesn't cost CPU cycles and quality issues. Or when the application development slows down in Vegas so that every angle of data processing approach can be written into the underlying fabric. (HDTV progress makes the latter unlikely to me)

We could always blame our tools.

I would personally rather have the timeline search out all the blips in vision and sound so that I can hide my mistakes. Especially in multi-cam edits. 8bit RGB would be just fine for me to live with whilst I hide my amateur errors. I'm sure there are other smarts people can think of that they'd have before they'd complain that the fabric of Vegas is flawed because it is RGB8 based. (I'd wager that it'll already be higher than your project res and probably higher than 8bit for many of the things you can do in Vegas. At least whilst the data is rattling around inside it).

Are we nearing the end of this thread? Probably not!

It is not a question if we blame our tools - it is simply a question of comparison and fact finding. The SoFo DV codec - or now Sony DV codec - is one of the best available on the market - similar to the Canopus DV codec. However, we learned in this thread is, that in some special cased the Canopus DV codec has advantages. This colour changes are new findings. That is something that is a fact - full stop about that.

Another fact is, that you will not see the colour shifts in Lets Edit - what is not a Storm but a cheap product for 150 Euro. I do not need a Storm at all, not for my hobby-videos that I tend to do. Nor do I wish to change to Lets Edit or other Canopus products.

The point is more that you cannot be sure if such colour changes takes place or not in the material you cut - since you do not know if you have such colours in your material as we had in the sunset video. And - a second issue is that you do not see that flaw - if you have tools activated in Vegas like the vectoscope (that was the reason why we had difficulties to reproduce the findings in the beginning).

So, what I would like to see is a solution for these issue identified here - even if I admit that it seems to be not significant for 98% of the videos we cut - otherwise people would have detected that earlier.

I'd have to strongly disagree that there's anything natural about YUV, it was simply an engineering trick to take advantage of the inability of our eye / brain to easily recognise variations in chroma.
The use of RGB or CYMK is not an electronic or digital age invention unlike YUV, I've yet to see a painter work in YUV, I've yet to see film that uses YUV.

I don't know how it can be suggested that any system is superior becuase it works in YUV, certainly it'll have less risk of problems working with material that originated in a YUV space but as time goes on there'll be less and less such material. As we enter the age of true digital video cameras that have no analogue path, YUV will fade back into oblivion.

Perhaps a zebra pattern could occur just in Vegas preview (not unlike the pro cameras do when they see levels over ITU-R.601 for over illuminated sections). This zebra could be detected in the same phase as when Vegas checks the audio levels on media import. (ie a Vegas thing timeline phenomenon).

Instead of the zebra, the valid transcode gamut (YUV to RGB) could be drawn as a graticule ("splat" like) on the Vegas vectorscope. This would warn us of an impending crush or a mis-lookup-registration problem.

Then we can colour correct as best we can within our cramped RGB colour space. This afterall is sort of a companding error in signal processing and transmission terms.

I would like a project setting to be able to mimick the input media with regard to matching the file's associated codec's preferential colour space, resolution, frame rate and fielding.

Without further conversion back and forth within the timeline. I just believe that a lot of the provided and 3rd party codecs would break if they go to a mixed pipeline mode. Or they will be compromised by each conversion. OK Sony are to blame for the SDK that 3rd parties write to. So if there is no provision for YUV (8,10,16bit etc) then it isn't the 3rd parties fault exactly.

RGB8 is a choice, perhaps a poor one given that DV and D1 formats being so much in favour of YUV storage representations.

In my last few projects I've matched up the colour and luminance latitude to the overall look of a programme. I don't think I've gone out of gamut when it has hit the target media. Same way that I've used an audio compressor filter to sweeten by clarifying the sources and finally maintain about the same volume throughout the programme through normalisation.

New cameras are now being touted as being 12bit dynamic (the tape remains 8bit YUV). So it might figure that when computers have the extra grunt, that these extra dimensions will also be used within NLEs, until they to finally output to tape/disc/network. Same goes for sub-sampling. However Sony haven't yet qouted whether their pipeline is actually RGB8 and whether all or any of the filters or 3rd party implementations use sub-pixel sampling to get us the good results we see today.

This thread still has life in it. However I'm not convinced that we have the engineers at Sony singing from our hymn sheet. I think they are still very proud of what they achieve in however many bits they currently support.

Perhaps RGB-8bit is semi pro and definitely not pro. I'm comfortable that it is pro, especially in this all too often degraded values digital world we now live in.

Farss, I glad to read your healthy disagreement to my personal conclusions.

Now, I've not seen a painter knock out 50 pictures per second, nor a film stock that I can use and re-use without buying new. Yet I do take what you say seriously though and value your steer.

My reasoning has left me settled for now. Stills digital pictures are typically JPEG oriented (GIF is slightly RGB oriented in how the Look Up Table works). This again is YUV domain. DVB/MPEG/DVD is YUV oriented. The informational content (entropy) of YUV is equal to RGB over a given time, so it isn't so much a trick as a representation. Data volumes are the reason for the change of representation but the information is nominally the same.

Life is simple when we buy/sell oil worldwide in prices that are listed all in US$ and when we normalise the super tanker's volume to what it would be at an agreed temperature. Simplification doesn't work for everything, so we translate and adjust for almost every other trade that we do around the world. (sorry - that was a bit off the wall)

Generated media (digital paintbox, compositor, rotoscoper, 3d animator etc) tends to be RGB but the middle ground is usually capable of more than 24bits per pixel (excluding the optional alpha channel's additional bits) to these can look great in MPEG or DV sometimes by virtue of this extra latitude that hits the compressor.

Our eyes respond to wavelength and amplitude on that wavelength (or particles if you like). RGB gives you three stakes in that spectrum that can be excited and mixed to give you a fantastic range. Excited material light emision is different to chemical absorption/reflection of ambient light. We don't need to do a physics lesson.

This Cruelty to monkeys might have the unlearned reader conclude that our eyes are Y,Cred-green,Cyellow-blue, or at least our monkey friends might be. Vector relationships are however quite a natural way of measuring our worldly phenomenon. Not that there is no value in those frequency stakes of RGB that mix so well when you excite them alone or together.

Bored yet? (and I've not even done my bit on poles-and-zeroes)

I can't argue that RGB is of prime importance and that we'll see things behaving that way across the board when we move away from compression technology for our home viewing. (this includes the storage formats and the socketry on the devices we use becoming RGB oriented worldwide).

I know a chief engineer at a software company who has puzzled over whether he has done the right thing with having a YUV pipeline for all the internals of his company's computer video implementation. Whilst nobody is asking him to go RGB or even concess to it being on the roadmap. It is sometimes a processing headache for the software libraries or even just the programmer to think and craft in YUV so one day he might change it all.

It is fair to say that NLEs that were RGB are either still RGB or are now YUV, or can be made to equate to a YUV equivalent.

Where I wholeheartedly follow you is with the tools that we use. I don't want to see YUV controls on too many filters. I'd rather wiggle in RGB.

We have no influence on what color space is used for the dv format. We simply have to use YUV for input and we have to use YUV for output. So I think this is not topic.
But if we are forced to accept YUV on both sides - why should we accept RGB in the middle if this means a loss of quality compared to the original signal? Vegas is made for working on DV and DV is YUV.
Using YUV surrounding would help to keep the color just like as it was in the (YUV) original. Quality most often means keeping as much of the original's properties as possible. Working in RGB though anything else around is YUV means accepting a loss in quality.

Marco

Thinking about this some more of course the main advantage of YUV is that it permits compression in a way that corresponds to how we percieve the world. Like all such 'tricks' the real world can at times fool the system and certainly trying to convert from one system of representation to another is going to throw up some oddities, that's just the way things are.

I take you're point on currency, i worked on a SAP implementation a few years ago and the one thing that through a total spanner in the works was the Euro, the system couldn't cope with being able to use two currencies in the one country.

I have to admit although I come from an engineering background all this YUV stuff is a bit alien to me and just when I think I understand it I find all those little subscripts so I'm never certain just which YUV anyones really talking about or if I can feed YUV from one box into another and it'll be OK. At least RGB seems to be just RGB!


But what I was talking about were cameras like the HD1 and the Viper, the HD1 uses a full digital path, RGB out of the CCD into the DSP chip and then into the encoder. The Viper records raw RGB straight to disk l. I know these are at very different ends of the spectrum but I see that as less and less of the media is going through the traditional transmission chain the need for YUV diminishing.

As I undestand it the native capture devices for video are RGB and so are the display devices, so it would seem logical to keep the whole chain working in the same color space.