Color space conversion

Red Prince wrote on 12/13/2016, 12:20 PM

Hello,  I have been working out the math of converting a video from one color space to another. And by color space I do not mean converting from RGB to something else, but rather converting from one RGB type to another.

I have created two LUTs, which should, in theory, take a video shot in the Rec. 601 space and convert it to a video in the Rec. 709 space, or vice versa.

I consider these two LUTs to be a Beta version. I would like them tested on different footage, so I can know whether I did them correctly (or not, of course). If I get enough confirmation I’m on the right track with this, I would like to create an OFX plugin that will be able to convert not just between these two spaces, but any RGB color spaces, including custom created spaces (for special effects). I think such a plugin would be nice to have under Vegas.

If you want to help, or even if you just want to grab the two LUTs for whatever reason, you can download them and then just use them with the VisionColor LUT plugin.

Thanks,
Adam

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Comments

Musicvid wrote on 12/13/2016, 3:21 PM

But the differences are so slight --- what will this accomplish?

Red Prince wrote on 12/13/2016, 3:38 PM

They may be slight but they are quite visible. Additionally, if I go and make the plugin, the differences created by it will be fully configurable, from slight to drastic.

For now, I am just requesting that people try it on an actual 601 fotage and watch the result on a 709 monitor, or if anyone still has a 601 monitor, try the opposite and let me know how it worked.

Adam

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Jam_One wrote on 12/13/2016, 3:42 PM

Checking "LUTs against real-world footage" is suboptimal way.

For the reason it's all bout mathematics. Pure math it is.
And any (non- precisely generated) footage is the thing that requres proof of being correct.
(You surely remember the fun with Canon's videos in times of 5Dm2 - they were in bt601 but every NLE was decoding them as bt709.)

======================================================================================
 

-- Naming convention: which display space is supposed to represent which source space.

Jam_One wrote on 12/13/2016, 3:43 PM

You just can't get the confirmation you're on the right track from random footages.

Jam_One wrote on 12/13/2016, 3:49 PM

To my mind, Vegas does really deserve it's native bt.709 <--> bt.601 converting plugin, since this operation is so... fundamental, to say so.

Red Prince wrote on 12/13/2016, 4:04 PM

You just can't get the confirmation you're on the right track from random footages.

No, I suppose I can’t. But it was worth trying before getting into developing a plugin, which is much more involved than producing a LUT. At any rate, I’m not asking for random footage. I’m asking for people who have some traditional 601 footage and have both a 601 and a 709 monitor to view the footage on the 601 monitor, and to view the same footage processed by the Rec601ToRec709.cube LUT on their 709 monitor and tell me if they look the same (within reason), and perhaps do the opposite with their new 709 footage. I unfortunately moved around the world a lot and most of my footage is permanently lost 🙁, as is any of my old 609 equipment. Otherwise I’d just do the comparison myself.

As for the naming convention, Rec601ToRec709.cube takes a 601 source and converts it to 709, and Rec709ToRec601.cube does the opposite. At least the math says they do.

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Musicvid wrote on 12/13/2016, 4:47 PM

601 is standard def.

 

Red Prince wrote on 12/13/2016, 5:06 PM

601 is standard def.

You might say that. The actual reason for the different recs is different monitor technology. Rec. 601 is based on the color response in the phosphors in the 1953 TV monitors. Rec. 709 is based on the color response of present-day monitors. So, even for just watching an old SD video on a modern HD monitor, converting the video from 601 to 709 should be beneficial, at least theoretically.

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Musicvid wrote on 12/13/2016, 5:40 PM

Well, the primaries in many dslrs, including all the early Canons and Nikon's, were flagged "incorrectly" as B.t. 601. No one made much of a fuss then.

Not sure converting SD to anything is beneficial.

Red Prince wrote on 12/13/2016, 7:01 PM

Not sure converting SD to anything is beneficial.

😕 But I’m not pushing it on you. All I am asking is if someone properly equipped for it could test if it works the way I think (from the math and the theory) it does.

So far, I have only been able to test it on several photographs and on a screen cap. The 601 to 709 LUT makes everything warmer, the other one makes everything colder. That actually surprised me, so I’m wondering if that is the expected behavior, or if I’m doing it wrong. Once I have a plugin (assuming my health allows me to make one, as ever since my heart attack I’m just living from one day to the next, generally taking it easy).

What the 601-to-709 LUT does essentially, it converts an RGB image to YCbCr with the Rec. 601 factors (0.299, 0.587, 0.114), then converts that back from YCbCr to RGB with the Rec. 709 factors (0.2125, 0.7154, 0.0721). Except it does it all in one step using matrix math (well, the LUT is just a set of results precomputed that way, of course, but that is how the plugin would do it if anyone finds it useful).

I’m not even sure this technique should be properly thought of as the conversion from one standard to another, or if it is just a color effect. The plugin would then allow you to choose the three source factors and the three destination factors (even totally made up ones), and calculate the right matrix to convert from RGB to YCrCb using the source factors and from that to RGB using the destination factors, and then use the matrix to change the colors accordingly, and even animate it so the change can happen gradually over a number of frames.

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Red Prince wrote on 12/13/2016, 8:28 PM

Well, it seems I have found the answer to my question, and it is a no. That is, no, the algorithm I used was incorrect, so the LUTs do not do what I thought they were doing.

I figured it out by using a different video editor, one that allows you to convert RGB to YCrCb 601, and also allows you to convert YCrCb to RGB 709. I tried that and the result was much more subtle, just as Musicvid suggested (though I can still see that as beneficial).

So, there goes my theory, which was that if I multiply the source matrix by the inversion of the destination matrix, the resulting one matrix would allow me to do the conversion in one step. I am still puzzled why it does not work, but the observation has shown me it does not.

Thanks for everyone’s input.

Adam

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Red Prince wrote on 12/13/2016, 8:29 PM

http://www.belle-nuit.com/test-chart

Thanks, Nick!

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Red Prince wrote on 12/13/2016, 10:00 PM

OK, I think I have it figured out now, even though I find it confusing. If I swap the order of the multiplication of the matrices, that is if I multiply the inverse of the destination matrix by the source matrix, I end up with a single matrix, which seems to be doing the right conversion between the two color spaces (and yes, it is very subtle).

Here is the zip with the two new LUTs.

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

wwaag wrote on 12/14/2016, 12:26 AM

Here is a post I did from last year that details how to do 601 to 709 conversions. My interest grew out of the fact that Mercalli was using 601 coefficients rather than the correct 709 coefficients. AFAIK, it's still a problem, but as others have said, it's not that noticeable unless you look for it. Regardless you can use the channel converter in Vegas to make the corrections.

wwaag

AKA the HappyOtter at https://tools4vegas.com/. System 1: Intel i7-8700k with HD 630 graphics plus an Nvidia RTX4070 graphics card. System 2: Intel i7-3770k with HD 4000 graphics plus an AMD RX550 graphics card. System 3: Laptop. Dell Inspiron Plus 16. Intel i7-11800H, Intel Graphics. Current cameras include Panasonic FZ2500, GoPro Hero11 and Hero8 Black plus a myriad of smartPhone, pocket cameras, video cameras and film cameras going back to the original Nikon S.

Red Prince wrote on 12/14/2016, 3:48 AM

I have just compared that to the matrix I calculated for my latest 709->601 LUT (included in my second Beta mentioned above), and it is:

0.9135,   0.0785804,  0.00791956
-0.105163,     1.17244, -0.0672766
0.00959559,  0.0323044, 0.9581

So pretty much the same, except not rounded.

P.S. I have just entered my matrix values into the Vegas Channel Blend. And even though it showed the numbers rounded as shown in the image here, internally it used my own values in the full precision as I entered them.

How do I know? I applied the Channel Blend to an entire track, then my (Beta 2) LUT on an exact copy of the same track (before applying the Channel Blend, of course), and I set the top one’s compositing mode first to subtract, then to difference, and finally to difference squared. In all of those cases I got a completely black composite output, indicating no difference between the two tracks, and, therefore, no difference between the use of the Channel Blend and the VisionColor LUT. But since Channel Blend is internal to Vegas, I saved it under the Rec. 709 to Rec. 601 label, so I can use it in the future.

It is 4 am here, so I’d like to go to bed. But in the morning I will do a similar comparison for the 601->709 matrices.

Last changed by Red Prince on 12/14/2016, 4:12 AM, changed a total of 1 times.

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Jam_One wrote on 12/14/2016, 3:59 AM

The more digits the better! 🙂🙂🙂

So, I did comparison within Vegas and I can positively confirm the process goes in the way I believe to be the right one 🙂
 

Last changed by Jam_One on 12/14/2016, 4:02 AM, changed a total of 1 times.

Win 7 Ultimate | Intel i7-4790K @ 4GHz | nVidia GTX 760 4GB * 2

SSD | 32 GB RAM | No Swap file | No Overclock | GPU-in-CPU OFF

t.A.T.u. F.o.R.e.V.e.R.!

 

Jam_One wrote on 12/14/2016, 4:45 AM

Tested your LUTs again.
I think your computations are perfect.
Well done, Red Prince!

Red Prince wrote on 12/14/2016, 9:07 AM

And here is my 601->709 matrix:

1.0865,  -0.072433,       -0.014067
0.0966428,       0.844833,        0.0585243
-0.0141401,      -0.0277599,      1.0419

Again, it is a match, except mine has more decimal places.

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Jam_One wrote on 12/14/2016, 12:32 PM

Looks like you can roll out your 'Final Release' version ;-)
It seems to be not in a 'Beta' stage any longer ;-)

Cheers, Red Prince!

Red Prince wrote on 12/14/2016, 3:08 PM

Thanks! I will be rather busy for the next seven days, but I will do it after that.

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

Red Prince wrote on 12/14/2016, 8:21 PM

OK, you can download the final release from here.

He who knows does not speak; he who speaks does not know.
                    — Lao Tze in Tao Te Ching

Can you imagine the silence if everyone only said what he knows?
                    — Karel Čapek (The guy who gave us the word “robot” in R.U.R.)

NickHope wrote on 12/14/2016, 8:41 PM

Thank you Red Prince. FYI, the readme.txt file opens for me as a large block of unformatted text.