TRANSFORMING 4:3 VIDEO TO 16:9 VIDEO

Jøran Toresen wrote on 8/26/2005, 6:48 PM
A resent question in this forum was: How good is Vegas’ 16:9 stretch? I’ll try to contribute to the ongoing – and repeated – discussion of this topic, with some clarifying points and my own experiences on this topic. In this note I exclusively refer to dv-avi and other formats with the same resolution. I use PAL video as the example for my exposition, since I live in Norway. But exactly the same principles apply to NTSC video, althought the explicit figures / numbers are different. First of all, it is useful do make a distinction between loss of information and loss of quality. And loss of quality can be evaluated / compared to a video shot in “true” 16:9 mode and to a video shot in standard 4:3 mode. But first some definitions and important relations.

A. The difference between standard 4:3 video and “true” widescreen 16:9 video.

The display aspect ration DAR is the ratio between the horizontal width and the vertical height of the physical display (TV). The relevant numbers, for standard and widescreen TV, are:

[1] DAR(4:3) = 4 / 3 = 1.3333

[2] DAR(16:9) = 16 / 9 = 1.7778

There are 720 horizontal and 576 vertical pixels in each frame (PAL video). As a result the fraction between the width and the height of each frame – which I call FAR – is:

[3] FAR = 720 / 576 = 1.250

As you can see, the proportions between the width and the height of the physical display – whether it be a standard or widescreen TV – and each frame in the video, is not equal. Therefore the width of each pixel must be adjusted to mach the dimension of the TV screen. As a consequence, each pixel is not a square. The pixel aspect ratios PAR for 4:3 and 16:9 PAL video, respectively, can be calculated as shown below. These are not the exact same numbers as reported by Vegas, but this is an issue that is of no importance for the problem discussed here. (In my opinion the numbers used by Vegas are not correct, but that’s another discussion.)

[4] PAR(4:3) = DAR(4:3) / FAR = 1.333 / 1.250 = 1.0667
[5] PAR(19:9) = DAR(16:9) / FAR = 1.7778 / 1.250 = 1.4222

Conclusion: The only relevant technical difference between a video shot by a camera in standard 4:3 mode and “true” 16:9 mode, is the pixel aspect ratio: Each pixel is wider in a 16:9 video than in a 4:3 video. Hence the horizontal resolution (quality) is lower for video shot in 16:9 mode than in 4:3 mode. This has (almost) nothing to do with the problem of resizing a video; it’s just a physical fact. However, it’s important to understand this difference when discussing the potential loss in information and / or quality from a transformation of standard video to widescreen video.

B. Resizing (Cropping) 4:3 video to the 16:9 format.

In order to transform a 4:3 video to 16:9 video, there are – in principal – three steps. Each pixel has to be stretched horizontally, the top and / or bottom of the 4:3 video must be cropped, and the cropped video must be (vertically) up scaled / enlarged (by interpolation) to restore the original proportions of each object and to fit into the frame size of the final (output) video.

The three steps I’m describing in what follows are the principle actions that Vegas has to perform to transform standard video to widescreen video. Actually, the only thing you have to do in a widescreen project, is to select the 4:3 event, click on the Event Pan Crop icon, right click the display in the Event Pan Crop window and choose “Match Output Aspect”. At the same time, there are some important project / event settings you have to specify, see below.

B1. Vertical adjustment: Cropping.

Compared to the width, the height of a 16:9 TV is 75 percent of the same proportions on a standard 4:3 TV. The mathematics goes like this:

[6] (4:3) / (16:9) = 0.75

Consequently, Vegas has to reduce the height of the original 4:3 video to 75 percent to get the proper size for a 16:9 video. Therefore the number of vertical pixels (lines) must be reduced from 576 to 432. This is the number reported for Height in Position in Event Pan Crop after right clicking and choosing Match Output Aspect in a PAL widescreen project.

In other words, Vegas crops 576–432 = 144 lines. By default, Vegas crops 144 / 2 = 72 lines on both the top and bottom of the original 4:3 video. This relationship can be manually changed, but you always have to remove 144 lines from the original video.

Cropping, seen isolated, implies no loss of quality – only a loss of information. (In a way, it’s just like using the Cookie Cutter, placing black bars on the top / bottom of the event. The analogy is, however, not exact because transforming standard video to widescreen video also implies changing the width of the video (the pixel aspect ration), not only the vertical portion of the original video displayed on the TV – see below) The problem is up scaling, see beneath.

The point is: cropping in itself isn’t sufficient to transform a standard video to widescreen video, because – after an isolated cropping – both the width and the height is too small to fit into the full widescreen format. The reasons are that the pixel aspect ratio is still 1.0667 and that the cropped video only contains 432 lines, while the pixel aspect ratio is 1.4222 for widescreen video and there are 575 lines in the final video.

So, in addition to vertical cropping, both the width and the height of the cropped video must be expanded by 33 percent to fit into the physical display of a widescreen TV. Therefore, two more steps are required to fulfil the task.

B2. Horizontal adjustment: Pixel stretching.

In order to fulfil the transformation of a 4:3 video to the 16:9 format, Vegas has to stretch the horizontal length of each pixel (by 33 percent) such that the pixel aspect ratio PAR becomes 1.4222 instead of 1.0667. Actually, this kind of stretching is exactly the same thing that my video camera does when I switch from 4:3 to 16:9 mode. Consequently, the horizontal resizing of a standard video to a widescreen video implies (in principle) no loss of “horizontal quality” compared to a video shot in 16:9 mode: Each pixel becoms wider, but he number of horizontal pixel are constant / the same as in the original 4:3 video. In addition, there is no loss of “horizontal information”.

However, the horizontal resolution is reduced compared to the original 4:3 video (since the pixel aspect ratio increases). The reason is that the same number of horizontal pixels (720) must fit into a wider (16:9) TV screen than a standard (4:3) TV screen. But, as mentioned above, compared to a video shot in “true” widescreen format, this horizontal stretching of each pixel implies no decrease in quality or information. (An analogue: the quality of the video is perceived as worse the bigger the TV screen is, because each pixel is getting bigger and bigger, the larger the physical display is.)

Conclusion: The change in pixel aspect ration represents no technical reduction of the “horizontal quality” or “horizontal information” when standard video is transformed to widescreen format.

B3. Matching the finale output size: Vertical interpolation.

The final PAL video, for example when you render to dv-avi or mpeg2, must have 576 lines (vertical pixels), not 432 lines as the cropped video, and 720 horizontal pixels with the correct pixel aspect ratio. Vertical cropping and horizontal pixel stretching has two relevant consequences:

1) Vertical cropping implies that you are loosing “vertical information” since some parts of the original picture are removed from the top and / or bottom of the video when the standard video is cropped to fit the widescreen formal. This is unavoidable. But cropping in itself does not reduce the quality of the cropped part of the video.

2) Horizontal pixel stretching implies that, as an example, people get thicker than they really are. So each frame also has to be “stretched” vertically, to restore the original proportions of the objects (a circle must be perfectly round – not an oval).

Therefore, to restore the correct proportions (distorted by pixel stretching) and to fit the transformed video to the correct number of lines in the final video (distorted by cropping), Vegas has to “fill in” the 144 missing lines with information that is not contained in the vertically cropped 4:3 video (containing only 432 lines). Or stated in another way: Vegas has to blow up the vertically cropped video to get it fit into the number of vertical pixels in the final video and to restore the correct proportions of each object – distorted by the stretching of each pixel in the cropped part of he original video. In principle this is exactly the same thing that a program like Photo Zoom Pro does when it enlarges still photos; the exception being that Vegas only has to enlarge the vertical part of each frame by interpolation. Exactly how this vertical up scaling / enlargement is done in Vegas, I do not know. But certainly there is some form of interpolation that performs this task.

What about the “vertical quality”? In theory, up scaling / enlargement of the vertical cropped part of the original 4:3 video, implies that the “vertical quality” of the relevant part of the original video is reduced by 33 percent (144 / 432) since Vegas has to “guess” the content of the missing 144 lines. Or, as a theoretical maximum, the cropping implies that the “vertical quality” is reduced by 33 percent because Vegas has to “guess” the content of 144 missing lines out of a total of 432 lines in the vertically cropped video.

But, if the “guessing” / interpolation is good, the “vertical quality” of the final video is not reduced by 33 percent. In fact, there is no theoretically correct answer to this problem. The actual – as opposed to the theoretical – quality loss depends on the specific algorithm used for interpolation and probably also the nature / content of each frame. And as an example: many users report that Photo Zoom Pro does an excellent job in up scaling / enlarging photos with a low resolution. And my experience is that Vegas does an excellent job in up scaling / enlarging video when transformed from standard to widescreen format.

C. Important settings and a short tutorial.

These are the most important settings and a short tutorial explaining how to best transform standard video to widescreen video:

C1. Important project settings:

1. Project Template: PAL / NTSC DV Widescreen
2. Full-resolution rendering quality: Best
3. Motion blur type: Gaussian
4. Deinterlace method: Blend fields (or Interpolate fields)

C2. Important event settings / a short tutorial:

1. Select the 4:3 event you want to transform to widescreen format.
2. Click the Event Pan Crop icon on the event you want to transform.
3. In the Event Pan Crop window, right click the display and choose “Match Output Aspect”.
4. Adjust the vertical cropping up or down if necessary: move the cropped part of the video up or down without changing the size / form of the cropped area.
5. Back on the time line, right click the event, choose Switches and check / enable “Reduces Interlace Flicker” (very important!). Be sure that “Maintain Aspect Ratio” and “Smart Resample” or “Force Resample” is enabled.

D. Main conclusion and own experiences.

1. The “horizontal quality” and the “horizontal information” are not reduced when transforming 4:3 video to 16:9 video.
2. Standard PAL video must be vertically cropped by 144 lines to fit into the widescreen format.
3. The “vertical information” is reduced due to the fact that the original 4:3 video has to be vertically cropped when transformed to 16:9 video.
4. Because a 4:3 video has to be vertically cropped to fit into a widescreen TV display, the “vertical quality” of the transformed standard video must be reduced compared to a video shot in “true” 16:9 mode.
5. The degree of the loss of “vertical quality” depends of the algorithm used to interpolate missing lines in the vertically cropped video
6. The project and event settings are crucial for the quality when transforming 4:3 video to 16:9 video.
7. My own experience is that Vegas does an excellent job in guessing / interpolating the content of the missing 144 lines, and when stretching the video to obtain the correct pixel aspect ratio. In short: I’m very pleased with the quality of the standard 4:3 video events that I have transformed to fit into my widescreen projects.

I’m not an expert on these topics, but pure logic and mathematics tells med that these are the principal relations and steps necessary to transform 4:3 video to 16:9 vide. I hope my contribution is informative, despite the technical nature of this post. Please correct me if I’m wrong. (And please excuse med if may English isn’t as good as it should be.)

Joran

Comments

Laurence wrote on 8/26/2005, 8:52 PM
Pretty good summary except for one point. In interlaced video, the fields must be separated before resizing, then folded back together after resizing. In Vegas, if the deinterlace method is set to none, it will resize the interlace pattern and the resulting 16:9 footage will have a horrible vertical combing effect on motion. Setting the deinterlace method to either "blend fields" or "interpolate fields" will let Vegas know that the footage is interlaced and it will separate the fields, resize the separated fields and fold them back together every even and odd line. I have examined frames resized to 16:9 using both deinterlace methods and am pretty confident that there is no difference. In this case I believe that the deinterlace method is more of a flag telling Vegas what kind of resize to do than it is an actual method being used.

I'm pretty sure that the in-camera 16:9 resizing misses this step and instead does a hardware deinterlace before resizing. That would explain why stretch mode looks so bad on an interlaced camera like a VX2000 and so good on a progressive mode camera like a DVX100.

I know the subject has been examined quite a bit in the past and I don't want to keep rehashing old stuff. On the other hand, I simply can't afford a new camera right now, and yet I feel that 16:9 output is really important. I want to get as good a 16:9 image as I can.

With the Vegas stretch, I feel that I can get a 16:9 image that fills a widescreen TV and looks as sharp as regular 4:3 footage. I may be missing out on the extra vertical resolution that a native 16:9 camera would give, but I can live with that for now.

It also seems to me that shooting 4:3 PAL might give a better Vegas stretched 16:9 picture, even when rendering to NTSC. This option is especially attractive now that Vegas's frame rate conversion is so much smoother than it was before version 6.

The one thing I still don't understand is if supersampling will help this process. I don't see how it could, but recent posts suggest that it might.
Jøran Toresen wrote on 8/27/2005, 7:20 AM
Hello
Thanks for the response. But under C1 I did say that you must set the deinterlace method to Blend fields (or Interpolate fields)

Joran
Laurence wrote on 8/27/2005, 9:38 PM
Well some more experiments and I know that the VX2000 is not doing a hardware deinterlace before resizing. Maybe the interpolate algorythm is just more confused by motion than the Vegas one is.