Smart Upscale

shanedk wrote on 10/17/2016, 9:30 AM

Running the 30-day trial of VP14. I'm having no luck with the new Smart Upscale FX. I have a 1080p video which I've loaded into a project that's set to 2160p, but whenever I try to apply it it makes no visual difference whatsoever. There is no apparent upscaling of the picture at all, even if I set the preview quality to Best (full) and look at it zoomed in.

Comments

NickHope wrote on 10/17/2016, 10:33 AM

It requires GPU acceleration to work. What's your GPU and driver version, and is GPU acceleration enabled in your video preferences? But I don't think you should expect a big difference. From 1080 to 2160 it should be a subtle difference.

shanedk wrote on 10/17/2016, 10:45 AM

It's a GeForce GTX 650 with the latest drivers (373.06). Yes, it's enabled in preferences. I'm not even seeing a subtle difference; it really is not making any visual difference at all!

NickHope wrote on 10/17/2016, 11:10 AM

What happens to your histogram when you enable and disable the effect? This is 1080 to 2160 on my machine with AMD HD6970 GPU:

shanedk wrote on 10/17/2016, 11:37 AM

Very subtle differences, but none that can actually be seen in the preview.

NickHope wrote on 10/17/2016, 11:48 AM

You should of course only see a difference if you are you viewing full screen preview on a 4k monitor, or in the preview window with "Scale video to fit preview window" unchecked (right click on the preview window to get that menu).

In the smart upscale FX try setting "noise sensitivity" to zero for a slightly sharper result than the default 200.

Upscaling is difficult. I have tried fancy upscalers in AviSynth and the result was always underwhelming.

shanedk wrote on 10/17/2016, 12:34 PM

Yes, that's exactly how I'm looking at it. Even zoomed in, there's no discernable difference.

shanedk wrote on 10/18/2016, 5:57 AM

So, is this all there is to it? That's what's to be expected from this filter?

Musicvid wrote on 10/18/2016, 6:50 AM

Even zoomed in, there's no discernable difference

The goal of upscaling by any means is to make the picture larger, not better. If you see no difference in the scaled output it suggests the "smart" algorithm is doing a pretty good job; comparing the results with bicubic-upscaled material, not the source, will give you a fairer test of that theory .

You will not turn HD into 4k by upscaling, alchemy, or another known means. Play your HD movie on a 4k set and you will see better upscaling than by any software, period. Unless you just want larger files conveying the same usable information, there is no point.

shanedk wrote on 10/18/2016, 7:09 AM

If you see no difference in the scaled output it suggests the algorithm is doing a pretty good job

Even when comparing it to the normal upscaling? What's the point of having the filter if it's no better than what Vegas normally does? And it's disappointing, given how much Magix has been hawking this feature. I'm not expecting miracles, but I'd at least expect SOME kind of sophisticated anti-aliasing which is most decidedly not in evidence here!

Musicvid wrote on 10/18/2016, 7:30 AM

I didn't say that. Vegas' "normal" upscaling is bicubic; if you see no difference then use that because it's faster.

I think you may have been affected by market-speak; "smart" in this context means better than the old way, NOT better than the original!

 

NickHope wrote on 10/18/2016, 10:00 AM

Sounds like some comparative testing is needed. Would be interesting to see what happens to some moving, geometric, clean-edged generated-media shapes, as well as real world sharp footage, upscaled from 1080p to 2160p by these methods:

  • Bilinear without integration (Vegas "good")
  • Bicubic with integration (Vegas "best")
  • Vegas smart adaptive
  • AviSynth ResampleHQ (and here)
  • AviSynth (or VirtualDub) Lanczos 3/4, Spline 32/64, Blackman
  • Anything else appropriate

I suspect Vegas' smart adaptive upscaling is actually not that fancy, although I suspected that about the new smart deinterlacing and it actually turns out to be pretty good. Maybe someone from Magix can tell us what the smart upscaling is actually doing under the hood.

Some interesting background:

shanedk wrote on 10/18/2016, 11:15 AM

Maybe someone from Magix can tell us what the smart upscaling is actually doing under the hood.

And maybe also if there's a way to do it where it's more effective: like, it has to be a Media or Track FX, has to go before or after other effects, etc.

Musicvid wrote on 10/18/2016, 5:12 PM

Scaling, using an encoding filter, is done last. It is not like an effect that occurs upstream of the encoding engine. Head over to doom9 if you want to learn how it really works, and just how limited in application upscaling really is. Honestly, that's what players are for.

Robert Johnston wrote on 10/20/2016, 11:22 PM

And maybe also if there's a way to do it where it's more effective: like, it has to be a Media or Track FX, has to go before or after other effects, etc.

Open Help in Vegas and search for "upscaling." There you'll find topics where Magix tells you how to use Smart Upscaling. Smart Upscaling does need to be used as Media FX, or if used as Event FX, it has to be before Pan and Crop and be first in chain. I didn't see anything mentioned about using it as a Track FX.

Smart Zoom is a little different.

I made the mistake of turning the noise suppression down below 200. But that made it look worse. You need to really clean up the noise to the point where artifacts along lines disappear and you can see the edges come into focus. That's especially true if the video has any noise to begin with.

Without the effect, upscaled video will be soft (blurry) and noisey (depending on how great a camera you have).

Intel Core i7 10700K CPU @ 3.80GHz (to 4.65GHz), NVIDIA GeForce RTX 2060 SUPER 8GBytes. Memory 32 GBytes DDR4. Also Intel UHD Graphics 630. Mainboard: Dell Inc. PCI-Express 3.0 (8.0 GT/s) Comet Lake. Bench CPU Multi Thread: 5500.5 per CPU-Z.

Vegas Pro 21.0 (Build 108) with Mocha Vegas

Windows 11 not pro

NickHope wrote on 11/8/2016, 10:23 AM

I got a little more information from Magix about how Smart Upscale works. It apparently uses area adaptive processing, which means that the image processing applied is controlled by the local content of the image. For example, an area containing texture (like leaves) will get scaled up differently than an area containing smooth surfaces and an object edge.

Robert Johnston wrote on 11/9/2016, 4:27 PM

Here's a thought: There's a third-party plug-in that you may have installed in Vegas already, and that is Neat Video, which has adaptive noise processing. If you aren't happy with the Smart plug-ins, try Neat Video. I use Neat Video followed by Smart Zoom. While it takes long to render, Neat Video creates a sort of "super-resolution" especially when you set it to look 5 frames before and after, and turn on sharpening in the Y channel.  It also helps if you have a good noise sample.

Intel Core i7 10700K CPU @ 3.80GHz (to 4.65GHz), NVIDIA GeForce RTX 2060 SUPER 8GBytes. Memory 32 GBytes DDR4. Also Intel UHD Graphics 630. Mainboard: Dell Inc. PCI-Express 3.0 (8.0 GT/s) Comet Lake. Bench CPU Multi Thread: 5500.5 per CPU-Z.

Vegas Pro 21.0 (Build 108) with Mocha Vegas

Windows 11 not pro

NickHope wrote on 1/10/2017, 4:07 AM

I've been asked for a 1080p version of one of my 576i PAL DV videos, which is a big ask but I'm giving it a shot. Planning to give them a 1080x1474-50p AVC file.

Firstly, smart adaptive deinterlacing is doing a fine job, so no complaints there.

I'm not even sure if Smart Upscale is supposed to support a 576i>1080p conversion. All the online help refers to upscaling HD to UHD/4K. The help also says "The Smart Upscale plug-in should be applied as a media effect or a pre-crop event effect. Upscaling is not applied in any other context." Well, adding as a media effect is long-winded in this case, since there are lots of media and there is no way to add a media effect to multiple media at once. And adding as pre-crop event effect is also impossible other than one event at a time [Edit: because I have lots of existing FX on the events, and "Paste Event Attributes" removes those].

But actually in this project there is no vertical cropping, so I simply added Smart Upscale as a Video Output FX and set the scaling to 1.88. It's incredibly difficult to see what this plugin is doing that a simple sharpen effect doesn't, and to do comparative analysis, but as far as I can tell it works and the result is pretty good, and it looks very similar to adding it as a media effect. Plus it doesn't mess up my compositing, which the media effect does. So I'll go with it.

One observation... When I change the settings I have to play or scrub the timeline a bit to make the smart adaptive deinterlacing take effect in the preview. It can look horrendously jaggy otherwise.

Marco. wrote on 1/10/2017, 5:56 AM

You could use a script or Vegasaur to apply the fx to several media clips in one go.

NickHope wrote on 1/10/2017, 6:49 AM

You could use a script or Vegasaur to apply the fx to several media clips in one go.

I thought it might be scriptable, but with a quick look I couldn't see how to do it as a Media FX in Vegasaur, nor as a pre-pan/crop Video FX. I'll look again when my render's done.

Marco. wrote on 1/10/2017, 7:32 AM

You could also use the given Timecode script and modify the fx GUID and preset name (though I don't know how to get the Smart Upscale GUID).

In Vegasaur for target you can select "Project Media" and "Selected" (just in case you would not want to apply to all the media). "Video Events (Media FX)" is selectable also (all/selected).

Jam_One wrote on 1/10/2017, 9:00 AM

...how to do it as ... a pre-pan/crop Video FX. ...

1) Choose some Event.
2) Choose your Effect.
3) Add this Effect to desired Event. Open the FX window to tune the FX and place it before pan/crop.

4) Right mouse button click on that very Event.
5) Click Copy in the context menu.

6) Select all of the Events on the whole timeline.
7) Right mouse button click on any Event.
8) Click Paste Event Attributes in the context menu.
9) (optional) Open Champagne.

 

Jam_One wrote on 1/10/2017, 9:17 AM

... how to do it as a Media FX in Vegasaur ...

NickHope wrote on 1/10/2017, 9:44 AM

...how to do it as ... a pre-pan/crop Video FX. ...

1) Choose some Event.
2) Choose your Effect.
3) Add this Effect to desired Event. Open the FX window to tune the FX and place it before pan/crop.

4) Right mouse button click on that very Event.
5) Click Copy in the context menu.

6) Select all of the Events on the whole timeline.
7) Right mouse button click on any Event.
8) Click Paste Event Attributes in the context menu.
9) (optional) Open Champagne.

Thanks but unfortunately this removes all the existing effects on the target events. I forgot to mention those previously and have just added that to my previous post.

Thanks for the Vegasaur guidance. That works nicely. In this case I'm still going to apply it as a video output fx because I don't want to mess up my existing compositing.