Sort of OT - speckles & shimmer

Arthur.S wrote on 5/8/2013, 5:25 AM
On a Blu-ray disc, HDV 50i I'm getting a lot of shimmering and speckles. here's the really odd thing; If I play the 00000.m2ts file in the stream folder (which is the main video) it's fine, so something is going awry in the burn stage. I've tried ImgBurn, DVDA, and TAW5 to burn. Same results with all. Even odder (and exasperating) is that the DVD file rendered from the SAME HDV file with TVMW 5 to a progressive file looks fantastic when burnt to a DVD! Any advice out there?

Comments

Arthur.S wrote on 5/8/2013, 8:22 AM
Well, stranger yet. Tried a different brand disc. Same. Tried 2 different players. Same. Burnt at slowest speed. Same. Played with PowerDVD....perfect! How can the hardware players make such a mess of it? Aren't software players supposed to be vastly inferior?? I'm baffled.
corug7 wrote on 5/8/2013, 9:21 AM
Sparkles could be caused by the bitrate being too high for your player. What bitrate are you topping out at? Also, if you are watching on a plasma or led display the shimmer is likely being caused by poor deinterlacing by your player, display, or both. Media and burn speed usually have little to do with the problems you described.
Arthur.S wrote on 5/8/2013, 12:24 PM
Thanks for the reply. The bitrate is 25bps CBR which is as the original HDV files captured. I've used this for BD many many times without problems. What you say makes sense though, so I'll try knocking that down to 20 and see if any improvement.
Arthur.S wrote on 5/8/2013, 5:30 PM
I've cracked it - but still baffled to be honest. I've tried every different bit rate and HD file you can think of. Then I noticed that it was actually shots from one camera. So, checked out the original camera files. Phew! They're fine. Back to the project file. On each shot with the problem was a sharpen filter at zero. Removed that and Bob's yer uncle. Why that doesn't show up at all on Vegas preview or PowerDVD, God only knows. Or on Standard DVD discs for that matter.

So now I need to pore through a 3hr project looking for anything with a sharpen filter. Looks like being a long night. :-(
Arthur.S wrote on 5/9/2013, 5:07 AM
Interested if anyone else has had this problem with V12? The sharpen filter has changed a bit from V9, and I've used the 'sharpen at 0' a lot with V9 and BD. Is this another bug in V12? it seems every time I give V12 another try I get bitten in the a**se by something completely unexpected. :-(
farss wrote on 5/9/2013, 8:26 AM
[I]"On each shot with the problem was a sharpen filter at zero. Removed that and Bob's yer uncle. Why that doesn't show up at all on Vegas preview or PowerDVD, God only knows."[/I]

The Sharpen FX adds edges that have levels outside legal video levels.
Played on a STB BD player into a HDTV they could look like "sparkles".

PowerDVD may very likely handle these much better or actually your PC's monitor will.

Bob.
musicvid10 wrote on 5/9/2013, 10:16 AM
Any "sharpen" runs the risk of blowing pixels due to high pass gain.
One blown pixel can propagate to neighbors during normal decoding processing.

If your camera levels are high to begin with, don't sharpen, but use unsharp mask instead. It will give a light sharpening effect without blowing pixels.

Or, if you're an adventuresome matrix-head, play with convolution kernel. Does the same thing (the USM presets do what I want).

We recently had a good discussion of this here:
http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=857179

Arthur.S wrote on 5/10/2013, 1:44 PM
Appreciate the advice folks. All interesting options to 'sharpen'. I've posted a sample of sharpened and unsharpened as I still find it hard to believe that such a small amount of sharpen could do such damage.

https://dl.dropboxusercontent.com/u/94686734/Sparkles-SHARPENED.m2t

https://dl.dropboxusercontent.com/u/94686734/Sparkles-UNSHARP.m2t

As always, I appreciate any thoughts.
Apologies for the 4:3 shape, dropbox has done something strange to it. On my laptop monitor I can't see a single sparkle!!
farss wrote on 5/10/2013, 6:35 PM
[I]"All interesting options to 'sharpen'. I've posted a sample of sharpened and unsharpened as I still find it hard to believe that such a small amount of sharpen could do such damage."[/I]

Vegas always has incorrectly assumes it is working in Computer RGB (Y' = 0 to 255). Any effects which add new elements, which is what any form of sharpening will do, will use that colour space.

Looking at your first sample with the Waveform monitor it shows tiny but high frequency signals that go from 0 to 255 which could indeed cause HDTVs and other
devices that do video signal processing to create sparkles.





Of some interest is the same video viewed using the Histograms:





One could easily enough assume this video is Computer RGB and apply the incorrect correction to fix the problem.

Bob.
Arthur.S wrote on 5/11/2013, 4:11 AM
So, what/how should I set up Vegas to work with HD files Bob? How is the Histogram of interest? Please bare in mind I'm an ignoramus as far as this stuff goes. :-(
farss wrote on 5/11/2013, 5:37 AM
[I]"So, what/how should I set up Vegas to work with HD files Bob?"[/I]

I know of no way to stop Vegas from doing this :(
All we can do is fix the problem after it has happened.
To do that at least as a test to confirm it is the errant levels that are causing the sparkles and shimmer try applying the Broadcast Colours FX AFTER the sharpen FX or else on the Video Bus, render it out as before and check it on the device that was displaying the problem. Use the Conservative Preset with the Broadcast Colours FX.
If you apply it on the Video Bus then you only need to do it once.

TBH after thinking this through some more it could be your problem has nothing to do with the levels but let's get that theory out of the way first. Are you rendering this out as a SD or HD???

[I]"How is the Histogram of interest?"[/I]
Only because it shows how the Histogram is not a reliable tool for judging levels,

Bob.
Arthur.S wrote on 5/11/2013, 7:23 AM
I'm rendering out as HDV Bob - from HDV.

OK, so I've played with Convolution kernel & Unsharp mask too. The default setting of CK still produced sparkles etc. Rocketeer's setting improved it, but still showed far to many. I pushed that setting in column 2 to 43. Only a slight improvement. Tried Unsharp mask at a setting between light and medium (light's sharpen was too subtle). Still sparkles and shimmering.

Here's a short clip showing them all briefly:
https://dl.dropboxusercontent.com/u/94686734/CK%20test.m2t

Q: Does Vegas actually output HD via preview??
farss wrote on 5/11/2013, 5:19 PM
So you haven't tried correcting the levels?

[I]"The default setting of CK still produced sparkles etc."[/I]

Downloading and looking at your latest sample I am not seeing any "sparkles".

[I]"Does Vegas actually output HD via preview??"[/I]

I'm not certain what that question means.
In general though when set to Best / Full what you see on the preview monitor is sort of like what you're going to get in the rendered output with three exceptions:

1) It will not correctly display interlaced video, no computer monitor can display interlaced video correctly. Your video is Interlaced. HDTVs can display interlaced video correctly.

2) Vegas's preview monitor does not correctly display video levels. To get a correct display of how your video will look you either need to add an FX to the monitor to adjust the levels or else use the external preview monitor.

3) You don't see the effects of whatever compression the codec used to render your HD video will introduce.

What I do see in your video is:

1) Noise. Quite noticeable in the background audience.
2) Aliasing / moiré. Most noticeable in the bride's veil. No doubt on a HDTV correctly displaying interlaced video it would look worse.

Adding any sharpening will make both of those issues worse.
As the noise is most noticeable in the blacks and your blacks are now outside legal, legalising all of your video before I watched it on the external preview monitor helped a little. No doubt if you'd done this before rendering it out it would have helped a lot, the mpeg-2 codec used for HDV doesn't like noise.

Bob.


musicvid10 wrote on 5/11/2013, 8:03 PM
I don't think I would never sharpen a bride's face.
She doesn't want to see fine lines and pores!
The last one (no sharpening) is by far the best.
farss wrote on 5/11/2013, 8:43 PM
[I]"The last one (no sharpening) is by far the best."[/I]

Indeed!
I turn the Detail (Sharpening) down in my EX1 regardless of what I'm shooting.
The ONLY time I add any sharpening in post is when I'm delivering standard definition from high definition. Event that is rarely and with some trepidation.
To further compound this, many HDTVs by default add sharpening.

Bob.
Arthur.S wrote on 5/12/2013, 7:37 AM
I use an external preview via HDMI to a 23" LCD. I would have thought that would show up the problem at Best/Full? I only see sparkles when burning to BD. (Luckily I still have a BD-RW :-) )

Reason I added the sharpen is because the Groom's slightly off focus - not critical, but noticeable.

farss wrote on 5/12/2013, 8:40 AM
[I]"I only see sparkles when burning to BD. (Luckily I still have a BD-RW :-) )"[/I]

And do what with that disk??

If you copy the file from the BD disk back to your HDD and look at it with Vegas do you see the problem??



I think our biggest problem here is we don't have any clue what your problem is.
So far you've called it "sparkles". "speckles" and "shimmer". Each of those denotes a different visual thing.

I thought I'd found a "sparkle" but that was the bride's ear ring visible through her hair.
"Shimmer" sound to me like the moiré patterns in her veil and "speckles" could be the noise in the guests in the shadows. We need more clues to where in the frame the problem is happening, help us help you.

Bob.
musicvid10 wrote on 5/12/2013, 11:08 AM
Bob went to some length to point out the levels issue in your clips. I don't see where you've acknowledged or done something about that.

If you will set your luminance in the Levels Filter like this, you won't get blown whites in the output.
--Output Start = .008
--Output End = .922

Here is a slightly softer, warmer treatment of your clip, with the highlights restored. Sharpening lines and pores in the bride's face in order to bring dubious benefit to the groom's is a big mistake afaiac.





I assume it's the bride's family that is paying your bill, correct?

Arthur.S wrote on 5/12/2013, 12:08 PM
I really appreciate the advice on the levels, but that's not an issue for me here. The only way I can think of for you to see what I'm talking about is to burn the file to a BD and play it via a set top BD player to a HDTV. As I said, all other ways I've tried of seeing it on my work station have failed. That's how I got caught out with it. And that's really the problem - if you can't see the effect when editing, it's a trap door!
musicvid10 wrote on 5/12/2013, 12:40 PM
"
It's not only an issue, it may be your only issue here, judging by your thread topic.

OK, one last try.
The standard luminance levels for BluRay are defined in ITU-R BT.709
That means [16,235]. Your video levels are nominally [14,255].

That means the whites are being blown in your output. This can be seen quite clearly in the bride's cheeks and forehead (above). It is also "likely" the cause of your "speckles" when you apply a sharpening filter, which is actually a high-pass boost. And yes, I quite understand it is a BluRay! We just don't know what's being done in playback.

I hope you will spend some time trying to understand and learn about this. Here is a properly conformed histogram (I know, Bob) of your clip at legal [16,235] luminance. Note that the black level is set just below the knee of the curve. Since Bob and I have repeated ourselves a number of times, I wish you the best of luck.





farss wrote on 5/12/2013, 2:31 PM
Over the years I've had people describe DV tape dropouts and macro blocking as "sparkles".
Given that the problem only shows up when the video is burnt to a BD disk and I assume played back in a set top BD player it could be something as simple as an ailing player.
Adding sharpening puts more strain on the mpeg-2 encoder and might just make the video too hard for the ailing player to handle without errors.

Again here that's a very long shot.

So much time could be saved by a better description.
If all else fails point a camera at the HDTV and let us see it.

Bob.
Arthur.S wrote on 5/13/2013, 5:12 AM
Yes you're correct musicvid, we are going around in circles a bit. Let me just list what I've tried to now.
Tried the conservative broadcast colours FX in the way Bob described above.
Tried the levels setting as described by musicvid above.
Tried all of the various ways of sharpening as above.
Tried 2 of everything, including: Set top BD players, types of Blu-ray disc, HDTVs, Disc authoring programmes (DVDA and TAW5). APART FROM burners - I only have one.
Copied the file back from the disc to my workstation. This doesn't show the problem. It ONLY shows when played from the disc in HD.
This doesn't show at all when rendered to an SD file.
I've also tried rendering it (to HDV as per source) with V12, V11, and V9.

Below I've added shots taken of my TV set showing the problem. Also, 2 screen shots of the histogram/wave form. I couldn't find a way to view exactly as yours above. The best result is from an unsharp mask, but you can still see it. The worst - by far is the default CK sharpen.

https://dl.dropboxusercontent.com/u/94686734/Sparkles-TV%20view.m2t

https://dl.dropboxusercontent.com/u/94686734/Sparkles-Waveform.jpg

https://dl.dropboxusercontent.com/u/94686734/Sparkles-Histogram.jpg

Not sharpening in future for anything going to BD isn't going to be a problem, but I've been merrily editing HDV for 5 years at least without (to my knowledge anyways) coming across this.
farss wrote on 5/13/2013, 6:00 AM
OK, NOW I can see what you're talking about.

Based on your tests I think we can eliminate the burner, the media, the BD player and the HDTV.
There seems to be [I]something[/I] in the file that is upsetting [I]something[/I] but what and how eludes me.

Give me 24 hours, I'll try burning your file to a BD disk and try it in my player and LG HDTV. I'd like to see those artefacts first hand to exactly correlate where they are in the frame and then check back with the original file and look very closely with the scopes.

I'm rather surprised that legalising the video levels made no difference.
Really all that leaves is something in the temporal domain and they're difficult problems to diagnose.

Bob.
musicvid10 wrote on 5/13/2013, 7:31 AM
Silly question -- does the problem go away if you turn turn down sharpen controls and turn off all enhancements in the player / TV?