A question about deinterlacing?

farss wrote on 8/11/2011, 2:06 PM

"I give up. I obviously have completely failed to describe the problem. I guess it is why I was never tempted to go into teaching"

Don't give up, I'm fairly certain I know what you meant to say.

Perhaps what you meant to say was "which pixel should be sampled".?

The answer is no pixel from the original holds the correct value. This applies regardless of the source being a field or a frame. To solve the problem as well as it can be (it'll always be an imperfect solution unless scaling is by an integer value) then interpolation is used.
Now with interlaced footage there's an additional problem compared to progressive footage, every second row of pixels is missing. How to deal with that and the need for interpolation.

1) None. Ignore the problem and process as a half height frame.
2) Blend. Simply mesh two fields togther.
3) Interpolate. Fill in the missing rows by interplation from what does exist.

1) Creates major problems with aliasing.
2) Can will have issues if there's motion, we are using pixels that are spatially offset because they were taken at a different time.
3) Is foolproof but we are working from an image of only half the possible vertical resolution.

The argument for using ME is to get the best outcome by combining 2) and 3). If you dig back around this forum I think I was the first to suggest this idea. The only suitable de-interlacer available at the time was Mike's and I just couldn't get it to play nice and/or the results were no better than just using 3).

So why am I now waving my arms in the air when I was one of the proponents of using ME in the first place?

There is a problem. Interlaced video is not the same as taking full raster frames at double the frame rate and then throwing away half the alternate lines. This is the way it is generally described, that's what is taught and repeated by many. It is wrong, bad teachers not telling the whole story. I hate such teachers, spent a good part of my life undoing what such teachers had taught me. That's why I never became a teacher despite ever so many people telling me I should have been a teacher.

Now I've got that off my chest, back to the topic :)

Let's see if I can describe this correctly.
Imagine we're taking images of a fine horizontal line that'll resolve as only one pixel high and it lies between two rows of pixels on the imager. We can get a result where it is seen in one field and not in the other. When viewed correctly on an interlaced display it will appear to "twitter". Very visually distracting. This is a problem the designer of a camera that shoots interlaced video has to resolve.

Cheapest solution happens if the lens simply cannot resolve that line fully. Next cheapest solution is to specify an OLPF that achieves the same outcome. That is probably how it was done in the Z1. Best solution is to use line pair averaging. Buffer a field and apply an averaging algorithm between pairs of lines so no line can appear as previously described.
Line pair averaging allows the camera designer to design a camera that shoots very good progressive video and interlaced video. When shooting progressive the line pair averaging is switched off and very high vertical resolution is achieved. In interleced it is switched on and vertical resolution drops. An added bonus is when shooting interlaced, noise is also reduced. If you dig into the test results for the EX1/3 you'll find all of this applies. It is to the best of my knowledge still the cheapest camera that does both interlaced and progressive video by the book. As an aside just thinking all of this through again I realise that the designers of the first HDV cameras must have breathed a sigh of relief that it was 1440x1080 and not 1920x1080.

I've got to get ready for my paying job, I'll post some more thoughts about all this later today. The important thing to grasp is that interlaced video is not just half vertical resolution progressive video, the pixels in each field may already contain information from the other field.

Bob.

GlennChan wrote on 8/16/2011, 7:25 PM

That problem applies as much to progressive scan video as interlaced video. This is a problem with all digital sensors that the designers try to improve on.
John Meyer is talking about the bizarre interaction between interlacing and rescaling the video. It's a problem that interlacing introduces.

EDIT:
Suppose you have 1920x1080 60fps progressive video and you wanted to rescale it to 720x480 progressive.

You would take a 1920x1080 image and rescale that. It would be preferable to using only every other line in the 1920x1080 image to make the 720x480 image.

Now suppose you wanted to convert the video to 720x480 INTERLACED. You would still take the 1920x1080 image and do your rescaling from it. Then throw out half the lines (or don't make them in the first place).

Really high quality rescaling of interlaced video involves converting it to 1920x1080 60fps, then rescaling from that.

I've never been slow to put the boot into the SCS coders but on this one I strongly suspect they have known what they are doing.
Um... no. To do it well takes more effort. To do it well AND at reasonable speed... that requires a piece of hardware like the various standards conversion boxes out there that cost five to six figures (e.g. Shibasoku, Teranex) and custom hardware.

To be fair... I don't think any of the NLEs available do it well.

As an aside just thinking all of this through again I realise that the designers of the first HDV cameras must have breathed a sigh of relief that it was 1440x1080 and not 1920x1080.
Rescaling from 1440x1080 <--> 1920x1080 repeatedly is something that does not work very well. It you do it properly, you will (effectively) get generation loss. It's the same idea as trying to resize an image in photoshop from 1440x1080 <--> 1920x1080 repeatedly. You can try it yourself to see what happens.

explanation:
http://www.glennchan.info/broadcast-monitors/scaling-artifacts/scaling-artifacts.htm

The reason why people don't notice the problems is because the image is low resolution and this solves other problems (aliasing/moire). We don't notice this as long as enough sharpening is applied to give the image contrast; and partly because we aren't very good at noticing low resolution. And it's also partly because many monitors and TVs sometimes have low resolution in practice. A 1920x1080 LCD panel is capable of excellent resolution (BETTER than a grade 1 CRT), but sometimes the image is resized to get rid of the overscan area and some awful deinterlacing method is used.

farss wrote on 8/16/2011, 9:42 PM

Suppose you have 1920x1080 60fps progressive video and you wanted to rescale it to 720x480 progressive.

I've actually tried doing exactly that, shooting 720p50 to produce 576i25 and the result has a major problem with aliasing and line twitter. You don't need a camera to see this problem, just look at the number of posts over the past decade from people having the same issues with high resolution stills with fine horizontal detail being rendered to interlaced video.
Your article explains the problem quite well.

To do it well takes more effort. To do it well AND at reasonable speed... that requires a piece of hardware like the various standards conversion boxes out there that cost five to six figures (e.g. Shibasoku, Teranex) and custom hardware.

If it can be done in hardware it generally can be done in software. De-interlacing is a problem that has no perfect solution. The same applies to low pass filtering. It would be good to have a configurable low pass filter in Vegas to better wrangle the aliasing and line twitter problems for those who can tolerate the render times.

The reason why people don't notice the problems is because the image is low resolution and this solves other problems (aliasing/moire). We don't notice this as long as enough sharpening is applied to give the image contrast; and partly because we aren't very good at noticing low resolution

Indeed and I suspect this is where a lot of the problems a lot of us think we're having with Vegas comes from.
Take an image with 600 lines res in 1080 lines and compare it to an image with 400 lines res in 480 lines, both at the same size. It's a no brainer the HD image will look better.
Only rough numbers here: Now downscale the HD image to 480 lines and it's only got 300 lines res (at best) and it looks worse than the SD image. I've spent years trying to get my head around this and wondering why do we always specify resolution in lines when lines per full raster could make a lot of sense. Just recently I found some do do this e.g. 600 lines in a 1000 vertical pixel image is 0.6.

Bob.

PeterDuke wrote on 8/16/2011, 9:46 PM

"A 1920x1080 LCD panel is capable of excellent resolution (BETTER than a grade 1 CRT), but sometimes the image is resized to get rid of the overscan area and some awful deinterlacing method is used."

I have often wondered whether so-called full HD TVs actually have 1920x1080 pixels and then magnify 1920x1080 video to give overscan, or do they do the simpler thing of only providing say 1728x972 pixels on the panel in the first place. I have mused about counting the pixels on my TV but not done so (yet!).

When I tried to use my TV as a HD monitor on my computer, lower resolutions were OK but 1920x1080 was cropped.

GlennChan wrote on 8/20/2011, 10:52 PM

Take an image with 600 lines res in 1080 lines and compare it to an image with 400 lines res in 480 lines, both at the same size. It's a no brainer the HD image will look better.
Not necessarily... :D

http://www.glennchan.info/presentations/idcf/idcf-sharpness.htm

The perception of sharpness depends mostly on contrast and only a little bit on how much resolution there actually is. (!!!)

If it can be done in hardware it generally can be done in software.
What's being done in the Shibasoku box would be brutally slow on a desktop computer because they are doing motion estimation. If you have used optical flow-based technologies (e.g. the one in Shake or 3d motion tracking programs) then you know that this stuff is really slow.
*Unless somebody invested the R&D into doing it with GPU acceleration... but THEN it might only work on newer Nvidia cards so people with ATI video cards, older Nvidia, and integrated graphics would still have the same problem. And it may not even be practical in the first place.

In any case, there are simpler techniques that wouldn't be that much slower than what Vegas is doing and would do a better job. It's just that somebody at SCS would have to spend time understanding it and coding it. (Hmm this sounds like a feature request? :D)

When I tried to use my TV as a HD monitor on my computer, lower resolutions were OK but 1920x1080 was cropped.
Personally I think cropping makes more sense. To resize the image by a very small percent can cause a lot of problems.

I don't think anybody makes a panel that is a few pixels short of 1920x1080. If you did, you could not sell the panel for use in computer monitors. Plus it's not a 1080 monitor, etc. etc. It's simpler just to crop the overscan pixels and replace them with black if you want that.

PeterDuke wrote on 8/21/2011, 6:13 AM

"I don't think anybody makes a panel that is a few pixels short of 1920x1080. If you did, you could not sell the panel for use in computer monitors."

My TV is 50 inches. Do they make computer monitors that big?

farss wrote on 8/21/2011, 8:30 AM

"The perception of sharpness depends mostly on contrast and only a little bit on how much resolution there actually is. (!!!)"

Indeed but I fail to see what that brings to the discussion here. The image in the page you provided a link to contains very low detail, there's little to best lost to start with.
Obviously actual resolution is difficult to judge by looking at images, the only certain way is to use test charts.

Perhaps a more revelant question would be what happens when a low MTF image with high detail enhancement is downscaled and then upscaled.
I raise this because in this thread we have two groupd of people. One who say they are relatively happy with how well Vegas performs scaling interlaced video and another who say they are not.
The difference between the two groups appears to be the fidelity of the images they are wokring from. The people shooting with expensive ($10K +) cameras are happy, those shooting with lesser cameras not so happy.
Those working with footage from the top shelf HD cameras maybe somewhat missled by the effects you're talking about but that still does not disprove my supposition.

On the other hand those who a proposing the use of methods outside of Vegas could be equally missled, maybe they're not getting a better result, they only think they are because of edge / contrast enhancement.

Personally as I know I cannot trust my eyes just looking at footage the only thing that will answer all of this for me is by shooting charts and measuring the results.

Bob.