Color banding & ghosting on text

drrohle wrote on 11/22/2011, 1:45 PM

Just upgraded to Vegas Pro 11 and am still getting this color shading/ghosting effect (as in Vs. 10e 64bit) on all colored text, usually near the bottom of the characters (Except Black, White or primary colors like Red, Green, Blue). It happens when I render the clip using any compression like mpeg-2, avi, Quicktime etc. If I render it uncompressed it looks great. Doesn't matter if it has a background or not, matches the project settings, clip settings or not. This also happens using the Prototype Titler too. All in standard def.

I don't have this problem using the 32 bit version, just the 64 bit.

I ran this problem through the support center and after waiting 3 weeks for a reply (unacceptable), Sony finally said it was fixed in the "e" release (it wasn't). Now I've upgraded to VP-11 and have the exact same problem. Except now it takes everything 20% longer to render than in Vs. 10.

Can anyone help fix the color banding/ghosting on generated text? I have a weekly television program to produce and can't afford the delays & problems. Also any "tuning" hints on quicker renders would be appreciated too.

Running HP machine with i7-970 6 core processor, 12GB RAM, GEForce GT440 vid. card. Latest drivers.

Comments

Laurence wrote on 11/22/2011, 2:49 PM

I have seen this sort of thing with odd frame dimensions. What is the frame size of your video?

farss wrote on 11/22/2011, 4:06 PM

"I don't have this problem using the 32 bit version, just the 64 bit."

Now that is very strange. 64bit code is generally not a complete rewrite of 32bit code.

Bob.

drrohle wrote on 11/23/2011, 9:18 AM

Frame size is 720 X 480 SD. I have noticed some color remnants in the text in 32 bit vs. but not nearly as bad as in the 64bit vs.

Also noticed this problem with NO underlying video track, just a single text on background will produce it.

drrohle wrote on 11/23/2011, 9:23 AM

To give a better description...Orange text on black background will have some cyan streaks and blocks in the text with a red shadow on the bottom stroke. Like when you check the "shadow" box and instead of a black shadow it's red on the bottom.

musicvid10 wrote on 11/23/2011, 12:13 PM

Can you post an example? From you description it may be chroma subsampling errors.

drrohle wrote on 11/23/2011, 1:34 PM

Sure, how do I post a .jpg image?

drrohle wrote on 11/23/2011, 1:54 PM

OK, if you go to: http://vimeo.com/32590009 in about 30 minutes from now you will see an example of this.

musicvid10 wrote on 11/23/2011, 2:00 PM

"Sure, how do I post a .jpg image?"

Upload to your own site or Dropbox, Photobucket, Mediafire, etc. and follow the markup instructions at the top of the forum topics.

drrohle wrote on 11/23/2011, 2:09 PM

Thanks, but just cut/paste this into your browser and you will find it there as soon as vimeo converts it in about 15 minutes:

http://vimeo.com/32590009

amendegw wrote on 11/23/2011, 2:10 PM

"Sure, how do I post a .jpg image? "The short answer is: [img=http://dl.dropbox.com/u/20447760/Jazzy4.jpg]

The longer answer is in the sticky

The even longer answer is here:

...Jerry

drrohle wrote on 11/23/2011, 2:27 PM

Here ya go: http://vimeo.com/32590009

The cyan banding is quite evident on the crossbars of H's, E's, B's etc.

farss wrote on 11/23/2011, 3:09 PM

It's hard to tell for certain but I would tend to write that off to chroma subsampling as mentioned previously. Certain color combinations do not fare well with 4:2:0 chroma sampling. You can confirm that by rendering to an uncompressed AVI file or a 4:2:2 codec. If uncompressed looks clean then you can rightly attribute the problem to the codec you're using and its chroma subsampling. I'm a bit nervous about blaming this on that because you're saying it is different in the 32 and 64 bit versions of Vegas and support seem to have acknowledge there's some problem as well.

On top of that there was an acknowledged issue in V10 that might well have not been fixed in V11. I'd also add that Vegas has never had best of class anti-aliasing when it comes to text however from my experience the chroma subsampling is one of the biggest determing factors on how text will look when you've got extreme chroma excersions between the color of the text and background.

Bob.

MacVista wrote on 11/23/2011, 3:40 PM

Text that small with saturated colours will never survive heavy compression.

Also having a text colour that is almost the exact opposite of the background does not help.

Things you can try:
Make the text bigger (I know, it doesn't fit :-)
Lower the saturation of the colours.
Use white text
Use a black background
Add a thin outline of white or black to the text
Add a drop shadow 0,0 offset and a small amount of blur

Give these a try

I would be very surprised if you managed to get that text to look OK from the 32bit version of Vegas, it should be no different to the 64 bit version.

The issue here is the compression throwing away colour information (chroma subsampling)

amendegw wrote on 11/23/2011, 3:40 PM

Since there is no media associated with this problem, I bet you'll get a quicker solution if you get a free Dropbox account, copy the .veg to the public folder & post the URL back here.

...Jerry

drrohle wrote on 11/23/2011, 3:42 PM

Thanks Bob, rendering to uncompressed looks pristine, no problems there. But in every other compression it does look bad.

I remember in Vs. 10 this was reportedly fixed in the "e" release but it never made a bit of difference on my machine.

I know I can render the clip to uncompressed then nest it back into the project but that's just too clunky & wrecks my workflow for such a nice app. like Vegas.

So what can I do to fix it? Since I'm guessing that it's not happening to anyone else or in any 32 bit version. I recall in Vs. 6 (32bit) this was not an issue at all. Everything looked great. It does not seem to be much of a problem when rendering text with primary colors (Red Green Blue) or white either.

farss wrote on 11/23/2011, 4:08 PM

"Thanks Bob, rendering to uncompressed looks pristine, no problems there."

That pretty much says to me the problem is not within Vegas itself but in the chroma subsamplig that the codec has to use.

"I remember in Vs. 10 this was reportedly fixed in the "e" release but it never made a bit of difference on my machine."

The problem that was fixed might not be the problem you think you're having.

"I know I can render the clip to uncompressed then nest it back into the project but that's just too clunky & wrecks my workflow for such a nice app. like Vegas."

Doing that would almost certainly achieve nothing if the problem really lies in the chroma subsampling. You feed the pristine text from the uncompressed video into the same encoder and it should go down in quality just the same.

BUT it would be something to test. If it makes any difference THEN yes, Vegas has a problem or there's some issue in your workflow or something.

Bob.

drrohle wrote on 11/23/2011, 4:20 PM

Yes, pasting the rendered, uncompressed text clip back into the project does work quite well and looks perfect. Just wrecks the workflow.

So what do you think now?

musicvid10 wrote on 11/23/2011, 6:35 PM

The chroma subsampling errors are being compounded by the fact that your colors are way out of gamut. If you are rendering to a YUV codec (such as the .mpg I downloaded) your colors must be constrained to 16-235. It won't eliminate the problem completely, but will help.

Also, putting complementary colors next to each other is fertile ground for subsampling errors. You won't get around it except, as you have discovered, it is somewhat less using an RGB codec.

farss wrote on 11/23/2011, 7:48 PM

"So what do you think now?"

I'd be inclined to totally agree with what Musicvid says above except if as you say:

"Yes, pasting the rendered, uncompressed text clip back into the project does work quite well and looks perfect."

If you do that and then render to the same YUV codec as before and don't get the same problem something else has to be going on. When Vegas renders the text for the encoder to encode it is, by my understanding, sending it RGB. Adding in the extra step should yield exactly the same result.

Whilst I'd certainly encourage you to avoid out of gamut issues because they can bite you hard when you take your eye of the ball at the same time something additional seems to be going on.

Perhaps you can try again keeping your colors within gamut and see what happens?

I'm also still perplexed by the difference between the 32bit and 64 bit version of Vegas. You may well have uncovered an issue that SCS needs to persue. Most of us don't use out of gamut colors and I'm very, very wary of combining colors from opposite sides of the color wheel. There's still people around with DVD players connected to TVs via composite video leads and it you think that looks bad on Vimeo you ain't seen nothin.

Bob.

musicvid10 wrote on 11/23/2011, 8:08 PM

"Most of us don't use out of gamut colors and I'm very, very wary of combining colors from opposite sides of the color wheel. "

That's a byproduct of experience, and one that we've all gone through. Same as for using thin serif fonts in titles for interlaced DVDs.
;?)

farss wrote on 11/23/2011, 8:11 PM

"That's a byproduct of experience"

And more so for those of us who lived through the dying days of VHS <shudder>.

Bob.

drrohle wrote on 11/23/2011, 8:22 PM

Wow that was a very good catch you guys! Now I pay much more attention to the yellow bang light on the titler.

I did try changing the gamut by clicking on the "fix" button which shifted the colors into range but not within the 16-235 as you suggested. I then adjusted them manually to be within 16-235. I think I'm doing this properly?? Anyway the renders DO look a little better now just by virtue that the colors don't "pop" like before. However the problem is still there and much more pronounced on the 64 bit render.

As soon as vimeo lets me upload more video examples I'll post the links here.

Thanks to all for the wonderful input!! Happy Thanksgiving~~

musicvid10 wrote on 11/23/2011, 9:31 PM

"However the problem is still there and much more pronounced on the 64 bit render."

This is something that needs to be pinned down, as Bob pointed out. Are you sure everything BELOW the line in your project properties is the same, i.e., 8-bit or 32-bit??

drrohle wrote on 11/23/2011, 9:48 PM

Not sure I know what you mean by "Pinned Down" but here is another example with the text color as 234-129-0 rendered to mpeg2 on the 64 bit app. Note the red shading on the bottom and under the dot in the "i"s.

http://vimeo.com/32606539