DV codecs errors while YUV->RGB recompression. (to sonyEPM: please stick this post)

Comments

taliesin wrote on 1/30/2004, 7:49 AM
> could it be the digital to analog converter on my video camera

Yes, might be.

> Could it even be my TV ?

I would say "yes" if the original file "16_235_Sun.avi" also had the same display. If it doesn't it is more likely it is not the TV, but I'm not sure about it.

Another point sometimes mentioned here is the PAL - NTSC difference. If you have NTSC surroundings I would say it is almost sure you must have color differences after rendering that file which is PAL in original. Proofing a signal's color if the original is PAL but proofing happens on NTSC is kind of not legal. It is impossible to compare color signals if one is PAL and the other one is NTSC. In this case it is very likely there's a difference noticable.
If you have NTSC surroundings your original test file must be NTSC as well as your rendered file.


Marco

Berti wrote on 1/30/2004, 8:24 AM
Thanks for the response.
All my equipment is PAL and as far as I can tell the files depicting the sunset ( the original "16_235_Sun" and the rendered "chroma_test_DV" ) are also pal. I guess this means there is no issue with NTSC and PAL conversion etc. If this is not the case I can definately say that the very similar problem I had with skin tone changing, was all PAL. All the equipment is Sony, the original HI8 footage was shot with a semi-professional sony camera, the conversion to digital was done using a stand alone sony HI8 player with S-VHS out going into my Sony PC105 mini DV camera which then passes it to the pc via firewire. The capturing was done with Vegas 4, no NTSC, all the settings in Vegas are PAL and on "best" for rendering etc.

Thanks again.

I'll keep looking

Berti
taliesin wrote on 1/30/2004, 8:46 AM
Yes, in your case everything seems to be fine.

Just to be sure: There differences you see are when you compare a file which was captured with Vegas with that file resulting from rendering within Vegas? Captured DV AVI against rendered DV AVI?
Or did you compare your original Hi8 file with the resulting DV file?

Marco
Berti wrote on 1/30/2004, 10:37 AM
I first noticed the difference in my video when I was playing off the time line and viewing on my TV via firewire. I was really just experimenting because I am new to the program, I had a single frame from the video being fed through to the TV ( the video was paused and so displayed the frame where the cursor was on the time line). Because It was only a single frame I could turn the setting in the "Preview Window" dropdown menu up to "Best Full". I loaded the Contrast and Brightnes plugin on the track FX, and this is when the color shift happened. Even when I load the preset "reset to none" ( i.e. the plugin is in the chain but has zero value) the color was different. I can literally tick and untick the check box of the plugin and watch the color flip back and forth on my TV screen. I can get exactly the same results with other plugins (the normal Sony plugins that come with Vegas). Even just inserting time code causes the natural pink of peoples faces to change to a yellowish color ! Any time a plugin is added to the Video Track FX the data must be recompressed to be viewed via firewire, even if the plugin has a value of zero, vegas apparently has to "go through" the plugin and render it.

This effect is also visible on fully rendered files with all the settings set to "Best". Audio plugins have no effect (seems obvious but I thought I'd mention it anyway.

If I have no plugins in the Video Track FX, the rendered film doesn't change and looks virtually identical to the original HI8 footage (it comes out a tiny tiny bit brighter) By connecting my HI8 player and the computer to different S-video sockets on the TV, I can compare the rendered (using print to tape) and original film live by simply switching channels back and forth.
I would love to hear that I can fix this because I can't apply any effects to the video track without significantly changing certain colors. Again, it only appears to affect the skin tone, not all colors or the whole image.

Perhaps I'm just really unlucky and have a strange combination of hardware that creates this problem.

Berti
taliesin wrote on 1/30/2004, 11:13 AM
What you describe is rather clear - as well as curious in the results.

This is very strange actually because some of the folks over here can't reproduce this result whatever we try to do.

Are you sure you are using the Sony Pictures DV codec? In "Options/Preferences-General" have you checked "Ignore Third Party Codecs" and unchecked "Use Microsoft DV Codec"?

What result do you get if you don't render to DV-AVI but to uncompressed AVI, to M-JPEG (4:2:2) or to DVCPro50 (if you have those codecs available)?

Marco
DaLiV wrote on 1/30/2004, 11:29 AM
berti - they can't explain your problem
while no recodings (as peoples have ntsc devices - and recode file to ntsc /very possible that recoding done via rgb/ so they can't see problem in this file. i'm while have no the same type of recordings in ntsc format [as have no capable devices]) they can't explain to you this problem.) you can see this effect. After recoding - which is done for ntsc - they see no difference (conversion not capable to handle overluminated matherials) - and can not see orange on tv - but yellow as on monitor - so they will talk's about your codec faults, your hardware faults, but they realy works normally.

also may be they render to see this file - but need preview direct from timeline

their hardware also do this - but they while don't want to search that type matherials.
no matter ntsc or pal as they says. difference in this is ntsc send color information for each 4 pixels horisontally oriented , pal send for 2 pixels horisontally but once per 2 lines (you can see that in many specifications and noone of them don't tel that ntsc give you more applicable colors - so both use 8 bit per each value Y,U,V /we are speak about consumers devices/ ).
---
Best Regards DaLiV

so wait answers from other then taliesin and farss
DaLiV wrote on 1/30/2004, 11:43 AM
i'm knew mine english not so good - so:
may be i'm cant to explain what they must be done to see that effect - you can try do it....
Spot|DSE wrote on 1/30/2004, 12:19 PM
K. I've just borrowed a PAL monitor from my friends at Thomson.
Previewing out to PAL from my PAL camcorder via firewire, I'm still not seeing this difference that this nearly 80 comment thread propounds.
Sorry, DaLiv, I've tried to figure out what you are seeing, all I can see is the illegals measured by both a tektronix 1740 and the scopes in FCP.
I've spent a lot of time trying to understand the problem, and frankly, don't see it but what I see is a matter of opinion. What the scopes show in all three instances, Vegas, FCP, and the 1740 don't lie. Rendering to PAL uncompressed, rendering to the Matrox DV50 codec, or rendering to DV-Vegas all provide very similar results. It's an illegal signal, it is scaled to what it best can be scaled to, and that's it. But the chroma values you are claiming just don't hold. Berti's problem could be entirely something different. I've not looked at that. Either way, it doesn't make sense. It's pretty clear from your posts that you play with a lot of video stuff, for all I know there is something else playing with your system. I don't know. I just know that myself and others can't repro your problem, and before you go off saying that none of us WANT to see your problem, realize that a lot of people have read your posts, tried to repro, and spent a lot of time trying to help you. I've gone as far as getting a PAL monitor to see if there was a difference in image even though the scopes show a consistency. The international head of Thomson Broadcast technical support couldn't find accuracy in what you are saying either.
So the only thing I can say at this point is: Don't shoot illegal video.
edit[forgot to mention, 95% of Thomson's work is in PAL, so this guy works with PAL all day, every day. All SD or HD only. he's forgotten more about color value in his 28 years with Philips/Thomson/Grass Valley than most of us will ever know]edit
Berti wrote on 1/30/2004, 12:22 PM
I have "Use Microsoft DV Codec" unchecked, and "Ignore Third Party codecs" checked. About a week ago I tried using the Microsoft DV codec and even download a demo of the Mainconcept DV codec but the results were the same (infact with the Microsoft codec it was worse).The problem is I can't see any difference when I view within Vegas, only on the TV using the Mini DV camera as the D\A converter. If I render to another file format I can't see that on an external monitor, it always has to be in the DV format to get it out through the firewire. This means I can't check if the problem also occurs using other codecs. I was on the verge of buying a new capture card with analog inputs thinking I could capture using another codec and then play straight back out to the S-VHS recorder or HI8 recorder using the analog out on the capture card. To my horror I have discovered that even on Canopus cards (in the 400 to 500 Euro range) with analog in, it always converts to DV format, you can't choose the capture codec !

Thanks to everyone still trying to resolve this problem

Berti
Wolfgang S. wrote on 1/30/2004, 12:35 PM
I have been asked by taliesin to test the files on my PAL-System - simply to try different systems.

Optical assessment: shows no difference between the files before and after conversion. I do not see any difference on the PC-Monitor, but also no difference on the external control PAL monitor.

Analysis with histogramme: shows a small increase in red with the histogram, in the range of 244 to 255 - tiny small in bleu and no change in green. As said, nothing what you see on a TV oder monitor.

Analysis with videoscope: shows a slight decrease in intensity after rendering, but again nothing what you see in the optical control on both TV or monitor.

Conclusion: there seems to be no significant difference at least in these files before and after YUV-RGB-YUV conversion.

Kind regards,
Wolfgang
www.vegasvideo.de

Desktop: PC AMD 3960X, 24x3,8 Mhz * GTX 3080 Ti (12 GB)* Blackmagic Extreme 4K 12G * QNAP Max8 10 Gb Lan * Resolve Studio 18 * Edius X* Blackmagic Pocket 6K/6K Pro, EVA1, FS7

Laptop: ProArt Studiobook 16 OLED * internal HDR preview * i9 12900H with i-GPU Iris XE * 32 GB Ram) * Geforce RTX 3070 TI 8GB * internal HDR preview on the laptop monitor * Blackmagic Ultrastudio 4K mini

HDR monitor: ProArt Monitor PA32 UCG-K 1600 nits, Atomos Sumo

Others: Edius NX (Canopus NX)-card in an old XP-System. Edius 4.6 and other systems

taliesin wrote on 1/30/2004, 1:45 PM
Are you trying to fool us?
First you tell us to render your test file because there would be a difference between rendered and original file only. And now after a long term of stating there is no noticable difference between rendered and original file then you tell us we must not render but view the unrendered timeline fx!!!???
At the same time you go back and edit your postings where before you talked about the rendered file. Now this is false and must be unrendered instead.

If you really talk about fx-applied but unrendered file preview output: I don't care because I never use unrendered but fx-applied files in the end. I think no one does that. I really did not test it this way and you know I didn't because I reported in details how I tested it and I even sent my rendered file. You could have said 3 days before: "Do not render", but you didn't. Anyway - if you are right here - then it is a preview problem only. Nothing which would be serious in the end. Not a codec render problem. Rendering means: decoding and encoding.

Now it is rather obvious you are creating something which shouldn't care us. Here the dv codecs we use work (render) fine. Not ony visual. If you succeeded in creating a strange way of preventing a codec from working the way it usually does: go for it. Waste of time to care about that.
So you think our fault was to render the file?
We didn't see the difference because we rendered the file???
- Do you know what the solution to your problem is: Render your file!!!
Enough said.

Marco
DaLiV wrote on 2/2/2004, 2:21 AM
{At the same time you go back and edit your postings where before you talked about the rendered file. Now this is false and must be unrendered instead.}
in that posting i'm add only lines - that can be helpful to other (main context not changed) .... that idea was in our missunderstandings result. (language barrier - in past time you can't understand me how you must see this effect)

{Rendering means: decoding and encoding. }
i'm always you trying to tell don't decode ... that will false your results ...

problem not only preview - if you already have done this and see difference - then see on tv - and apply FX - after that on tv you will see the same picture as on pc (with color shift)...

{- Do you know what the solution to your problem is: Render your file!!! }
after file rendering - it's no more original, and seems different.

P.S. one more time sorry for bad english.
taliesin wrote on 2/6/2004, 1:40 PM
I think it's time to say I'm very sorry for what I wrote before.

Today it seems like I did something wrong 10 days ago. It's a bit early for details but I have to admit today I am not able to have same results I reported before. Anything I do now causes exactly same codec behaviour as you described. I'm still testing to verify my earlier results. Don't know at all why it is different now. But I don't want to leave this thread without giving infos about my latest results which are same like yours. Now it is very obviously to me to see the orange area around the sun change to yellow - output on external monitor.

So I'll be back once again - probably stating all you said about Sony Pictures DV from my side. Not sure about it now.

The only thing left I would have to correct you: Canopus DV.
Canopus DV renders in YUV, not RGB. And actually if I do that test with your sunset file in Canopus Let's Edit then the result does not show the differences. Sony Pictures now does.

Back later Marco
taliesin wrote on 2/8/2004, 12:00 PM
O.k. Daliv, we did some more testings here and meanwhile I am sure you are absolutely right.

Remember that I told you several folks over here tested it and no one could see a differnce between the two files (original AVI and filtered AVI)?

Remember that I told you I could not even see a difference if I use an external vectorscope?

We did a great big mistake when we did that tests two weeks ago!!!

We had the internal vectorscope activated while outputting the signal onto the external dv-device and the external monitor. And having the internal vectorscope (or histogram, waveform, RGB-parade) active means all the files in the timeline are always recompressed.
So what we saw actually were two identical files. Your original AVI looked absoluty identical to the AVI with timecode filter set on. And this was because the activated internal vectorscope forced even the original AVI to be recompressed. So in our test BOTH files had the color shift. We did not the the difference because both files had the shift!!!

From the moment I close the internal vectorscope I can see the color shift in the filtered AVI file only. A certain part of organge becomes yellow then. This is clearly different to the original AVI which is not recompressed then.
To be absolutely sure I took your original file and my rendered file (with timecode) and printed it to dv tape. Watching the dv-tape output also shows the difference.

But nevertheless - one question: Are you sure your camera works correctly? - Is that kind of orange which turns to yellow when being recompressed a color which a dv camera is allowed to produce?

Anyway - after I could clearly see and reproduce the color shift after recompression I would like Sony Pictures to think about if they could change their dv codec to work in YUV instead of in RGB. Canopus DV shows the way to go. This color shift does not appear when working in Canopus software using Canous DV for that one works in YUV.

DaLiv - once again - sorry for my misbehaviour. I was completely mislead.

Marco
DaLiV wrote on 2/11/2004, 3:42 AM
{qote}
But nevertheless - one question: Are you sure your camera works correctly? - Is that kind of orange which turns to yellow when being recompressed a color which a dv camera is allowed to produce?
{/qote}
yes works correctly. most cameras will produce this effect (if camera have zebra effect - then this will activated on this overluminated objects [as you can decide in life you can't film all without zebra - that often when luma have bigger range that itu601 allow {need dark object in cadre and well luminated at once} ]), and as you knew much of consumer cameras also have no "zebra", that can help to users.

{quote}
Anyway - after I could clearly see and reproduce the color shift after recompression I would like Sony Pictures to think about if they could change their dv codec to work in YUV instead of in RGB.
{/quote}
in addition also all plugins must works with yuv, so you ca't use boris red, after effects, ...
while yuv not present in most programs - you can't safely works in them on any of matherial

in russian forum now we discuss about how to find overluminated places in matherial.
here is copy of dll that allow to find thats
http://www.bokova.lv/~daliv/video/down/taviinfo_dll.zip
this is plug and script for avisinth 2.5.

P.S. don't worry.
taliesin wrote on 2/11/2004, 5:10 AM
I think there are two different things. Over- and underexposure does not seem to cause this color-shift problem. Otherwise simply using the broadcast-color filter of Vegas should help overcoming the color shift. I tried broadcast-color filter as well as other color-correction tools but reducing the luminance and color level does not help here.

I didn't succeed yet in analyzing what colors exactly are affected by a shift. The only one where I saw it till now is this very certain kind of orange that was shown in your example. Analyzing this orange tells me there are R-values (out of RGB) of 255. But in the outer surroundings there are other color-tones also having R-values of 255 but these ones are not affected by a shift.

To me this is a very strange behaviour because I can't see a dependency of the levels itself. I will do some more researches ...

Marco
farss wrote on 2/11/2004, 5:43 AM
The answer to this as I and several other very learned gents have pointed out is very obvious to anyone who knows anything about video.
What isn't so obvious is why DaLIV refuses to accept the obvious.

The camera has recorded out of gamut colors. This is NOT the same as illegal levels. The only way to check this is with a vectorsope on the cameras composite video output while playing the tape out.

What happens to those colors when a DV codec tries to recompress them isn't an issue, maybe it should just stop and put a warning message into the stream.

I can create exactly the same situation within Vegas using generated media, just try moving the color sliders around, see the little yellow triangle pop up? It's warning me that the color I'm creating might be fine on a computer monitor working in RGB but I'm going to have trouble with it in a TV system working in YUV. Why is that? Well it's very simple, not all the possible values of Y,U and V are legal. Doesn't stop the camera from recording them. It SHOULD stop the camera from recording them in the first place but for a number of reasons consummer cameras don't seem to worry too much about the finer points of video.

taliesin wrote on 2/11/2004, 3:36 PM
>> The camera has recorded out of gamut colors. This is NOT the same as illegal levels.

Yes, this is exactly the point which confused me.

I tried that method using the Generated Media function of Vegas and I noticed the warning triangle which does NOT appear on illegal levels but just on a certain color hue. O.k. - this must be the colors out of PAL gamut.

What worries me once again then is that the color of DaLivs sunset AVI which have the color shift after recompression seems NOT to be one of the colors which are indicated with a triangle within the Vegas Generated Media function.

In the end I realized this is one of the very advantages of Canopus systems and Canopus DV which work in YUV. This will not make me switch from Vegas to Let's Edit or Edius but this makes me think about whether there ain't a slight improval potential for Vegas ... ;-)

Marco
farss wrote on 2/11/2004, 3:58 PM
So what you're saying is one of the advantages of the Canopus system is it will not correct out of gamut colors. That sounds like a real plus to me.
That is until you try to get a TV station that knows what it's doing to broadcast your material. You will not even get anyone to look at it, the gamut nazis that guard the doors will just put a big red "rejected" sticker on it and send it back.
taliesin wrote on 2/11/2004, 4:24 PM
Mmh, I'm willing to learn here. Broadcasting is one thing. But there are different uses besides. Speaking of projects except of broadcasting - why not perserving the original colors?

Maybe I do mix things again. Regarding illegal levels - the broadcasting station I work for uses a limiter on the transition way. Illegal luminance and color levels are simply cut off. Though I usually take care of my signals by doing precise color corrections - even if I wouldn't do that, no harm would be done to my video signals (except of being cut off in the highlights). Almost any broadcasting station in my area use limiter to avoid illegal levels (but ony to high levels).
I always thought same would happen to signals which are out of gamut colors. But I must admit I never asked about what happens to signals out of PAL gamut when transmitting. If it really causes problems, o.k., then for broadcasting it is better to have a color switch sometimes but legal colors then.

Then I think it would be great to have an option: codec restricting the signal to PAL gamut or working in full range.

Marco
farss wrote on 2/11/2004, 5:00 PM
I'll admit that I'm not only at the limits of what I know but also what I want to know on this!
Probably most broadcasters don't care about these things. It's just that the only people I know who work in television work for a public broadcaster who's much the same as the BBC. They striclty enforce standards. Not just in terms of how they transmit but what they even allow into the place.
What makes it more confusing is that the color gamut is slightly different for PAL and NTSC.

I think this is probably a bigger issue than we realise. Years ago when video only existed for television the situation was much different but now with all sorts of programs creating images that end up in video and video being presented by so many different means of delivery I suspect there's many 'cans of worms' out there.

I know from comments I get from people in the television industry that graphic artists are one of the main culprits when it comes to creating illegal things.
DaLiV wrote on 2/12/2004, 2:30 AM
if you relist our conversation - you can find measured yuv values and rgb that converted from them. they is out of range 0-255 (rgb).

overexposure in this case have cause this effect. / i'm say about over- and under- exposure as trying to say about all consumer cameras and filming technique of this effect /
DaLiV wrote on 2/12/2004, 2:44 AM
i'm not refuse. i'm want any helpful solution for this problem (let this how it is - is not solution).
ok. not all legal - but we must knew where this is illegal in matherial - so if not possible to make direct conversion to rgb and back - let programs will make markings of regions that must be reviewed on color changes (this may be done by control values from codec) . i'm think : when you produce - you must knew where you need make corrections, which can't be seems directly on monitor.
DaLiV wrote on 2/12/2004, 2:59 AM
About this triangle - this is colors of rgb that can not be converted to safe yuv values.
but we at now talks about yuv->rgb parts, we can't process our sources - not generated media