Side by Side comparison test

Cliff Etzel wrote on 5/8/2018, 11:11 AM

I'm still in the middle of a project for a client but took some time out inbetween edit revisions to run a side by side comparison of the same snippet of interview material and trying my best to get the same results in the 3 NLE's I have at my disposal.

As you can see in this video clip, as much as I tried, each NLE rendered out a different looking clip - no matter how much I tried to get scopes, levels, etc to match up in each NLE.

Each clip has a power window/clipping mask applied to the subjects face, the same LUT, a vignette and Noise reduction applied to mitigate the less than stellar recording bitrate of my Olympus EM5 camera.

Render times were pretty fast even with NR applied except for Vegas since there's no real way to pre-render the timeline and to utilize that like you can in Premiere Pro and Resolve.

I'm unsure what to make of this test tbh - What surprises me the most is the marked differences in render quality - the Vegas render looks the smoothest, yet with a flatter look, where as the Premiere Pro clip has the most banding in those gradiated areas. Resolve seems to fit inbetween the other two in that regard. Overall color my nod goes to Resolve with Premiere not too far behind. I'm not sure what happened with the Vegas Pro clip with the more muted color palette as no saturation was applied to any of the clips. The shadows also look a little smokey in the Vegas Pro clip

What I will say is that of the three - Vegas was the one I spent the most time in trying to get the look I wanted and still couldn't get there. Once I understood how Resolve works for grading, that took the least amount of time given it's excellent motion tracker for power windows - I didn't have to spend time adding key frames as compared to the other 2 NLE's.

Given I'm still using a measly GTX-660ti 2GB card, it shows the hardware utilization with Resolve using the most followed by Vegas and Premiere Pro coming in a distant third when it came time to render.

https://vimeo.com/268617856

Comments

Vliegvisser wrote on 5/8/2018, 11:50 AM

How and with what program did you make and render this Side by side video for upload to Vimeo?

Cliff Etzel wrote on 5/8/2018, 11:59 AM

How and with what program did you make and render this Side by side video for upload to Vimeo?

I quickly created inside Vegas Pro 14, nothing else done other than to put each clip side by side and title graphics - rendered out using the MAGIX h264 internet codec - VBR target 12,000,000 and max 24,000,000 encode quality and using the PC output on the chain for the preview window

By all means this isn't standardized test - This is more about observing the differences in how each program works in post and the output quality. I should try to do a more standardized test - Ideally doing the Vegas Benchmark test would be the benchmark, but I have no idea how to go about converting that to test in Premiere Pro and Resolve. Export an XML??? (color me clueless in this regard 😕)

Former users wrote on 5/8/2018, 12:07 PM

I would have to see the original media to come to a more technical conclusion. But of the three images, the one I liked the most was that of Resolve 14. It is well balanced. The vegas was the second best, it's only a little clearer than Davince's Rasolve. The one in Premiere is apparently a more saturated and also darker compared to the other two. It would be nice to have the original also placed for comparison purposes.

 

 

Nick Hope wrote on 5/8/2018, 12:13 PM

To make this meaningful, you need to put a standard test chart through the same render process in each of your 3 options, perhaps once with the LUT and once without, but with no other effects. Then you'll know how each program fundamentally treats levels and colorspaces. I suggest the 1920x1080 chart from here: https://www.belle-nuit.com/test-chart

Former users wrote on 5/8/2018, 12:20 PM

To make this meaningful, you need to put a standard test chart through the same render process in each of your 3 options, perhaps once with the LUT and once without, but with no other effects. Then you'll know how each program fundamentally treats levels and colorspaces. I suggest the 1920x1080 chart from here: https://www.belle-nuit.com/test-chart

Correct.

Cliff Etzel wrote on 5/8/2018, 1:56 PM

Thanks to all who have chimed in so far - I'm trying to figure out how to create a benchmark test that can be utilized in more than just Vegas Pro or Premiere Pro (Theirs is the PPBM test). I dont' believe Resolve has a benchmarking test right now.

@Former users - I agree with your assessment of which I prefer based on this simple comparison - Resolve looks cleaner with Vegas less saturated but coming in second. I was pretty shocked by the drastic difference from Premiere Pro which has been my go to editor for a number of years now. And given my version doesn't support AMD cards for the MPE, it now becomes less appealing for that very reason, and I'm sure not going to start renting my software from Adobe. Looks as though an AMD card is in my future later this month along with Retesting Vegas and Resolve side by side - determining if I should use both or just settle on one or the other based on the results when I upgrade the graphics card.

I did the simple render of the test chart - It's clearly apparent from this that Vegas crushes the blacks from a straight render. So it seems I need to correct for this by applying levels within Vegas in order to get the same results as Resolve and Premiere Pro

Former users wrote on 5/8/2018, 5:00 PM

@Cliff Etzel

Thanks to your test I was able to figure out a way to solve this problem with Vegas rendering. For you to achieve the desired image you will have to go to Video FX and in LEVELS apply the Computer RGB to Studio RGB effect. After that the rendering of Vegas does not change the blacks of the image. Take a test on your machine and see if that works for you too.

I had also realized this difference for some time but had never really gone into it. Thanks to your initiative I was able to make this discovery. Thank you!

Cliff Etzel wrote on 5/8/2018, 5:06 PM

@Cliff Etzel

Thanks to your test I was able to figure out a way to solve this problem with Vegas rendering. For you to achieve the desired image you will have to go to Video FX and in LEVELS apply the Computer RGB to Studio RGB effect. After that the rendering of Vegas does not change the blacks of the image. Take a test there on your machine and see if that works for you too.

Is that applied at the preview window FX? I have the SeMW plugin installed and set the preview window to PC which I believe is the same as applying the Computer RGB to Studio RGB - am I mistaken?

fr0sty wrote on 5/8/2018, 5:31 PM

If I'm not mistaken, that plugin just makes it so you see the image at studio RGB levels, not that the rendered output will be at those levels.

Cliff Etzel wrote on 5/8/2018, 5:36 PM

If I'm not mistaken, that plugin just makes it so you see the image at studio RGB levels, not that the rendered output will be at those levels.

So now I'm confused... I thought the whole point of the SeMW plugin was to allow you to compensate for what Vegas does to the levels/contrast of the output video so when you render - WYSIWYG in the final rendered video... If that's not the case then it's another tick against Vegas for me as it means I have to remember yet another step before rendering and as humans beings are prone to doing - we can forget...

Maybe @Nick Hope can clarify this...

OldSmoke wrote on 5/8/2018, 5:44 PM

@Cliff Etzel don't forget that Vegas treats still images differently from video and you need to apply the FX despite using the SeMW extension.

Last changed by OldSmoke on 5/8/2018, 5:44 PM, changed a total of 1 times.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)

Former users wrote on 5/8/2018, 5:44 PM

No. The SeMW extension is only for preview. Is not applied in render.

But it's simple to apply Studio RGB to Computer RGB throughout the project to render it. You only have to apply this effect to your project as OUTPUT FX. This causes it to be applied throughout the project.

I agree with you that it is disappointing to have to remember one more step before rendering a file in Vegas. Hopefully that will change someday.

Former users wrote on 5/8/2018, 5:54 PM

This is the test I did.

Former users wrote on 5/8/2018, 6:19 PM

Maybe this is not Vegas problem. Maybe it's a display player issue. I tested the rendered file in vegas where theoretically there is a change in the blacks of the image and to my surprise in the Power DVD 15 player the file is normally displayed. I do not know the causes of it. Then it's with the technical people.

Cliff Etzel wrote on 5/8/2018, 6:28 PM

Maybe this is not Vegas problem. Maybe it's a display player issue. I tested the rendered file in vegas where theoretically there is a change in the blacks of the image and to my surprise in the Power DVD 15 player the file is normally displayed. I do not know the causes of it. Then it's with the technical people.

And yet - The Premiere Pro and Resolve test clips show normally in any player I try - this is the sticking point I have with Vegas - nothing is as it should be... I need consistency, not little detours in troubleshooting the technical side of things... Guess I'm getting irritated with Vegas again...

Former users wrote on 5/8/2018, 6:45 PM

I understand you better than you can imagine. But keep in mind that all software has problems, so there are updates. I could list several that exist in Premiere and Resolve, but that's not the point.

I like vegas for simplicity and objectivity in editing. For me this has a very big weight when it comes to choosing a video editor. I am hopeful that one day Vegas will become stable and have the recognition it deserves.

But as a professional I will not think twice if one day find software that is better than vegas in the matter of simplicity and objectivity at the time of editing and that above all is functional.

Resolve is what pleases me the most. If Blackmagic continues with the work you've been doing, you'll surely have one more customer on your list. But it's too early to decide. While Vegas meets my work needs I will continue with Vegas.

Cliff Etzel wrote on 5/8/2018, 6:56 PM

I get what you're saying about there's no perfect NLE out there - if there were, everyone would be using it ;-)

Having said that - I see Vegas the same as you do - it is simple and really is an AIO solution for video and audio - but it comes at a price of having to unlearn the way just about every other NLE works - and that's a difficult thing to do. Having said that - Resolve is powerful - as long as you have the horsepower to run it, something my laptop is unable to do at this point (even Resolve 12.5). I dont' need much for laptop editing but given I like to stay in one app and hand off without the challenges of XML's not reading correctly back and forth between different apps (XML's from Vegas show tracks in reverse order) I'm thinking the hardware for editing in Resolve will probably usurp my sticking with Vegas or Premiere Pro. I'd rather become really efficient in one app than trying to wrangle between different NLE's. I truly love the outside the box paradigm of Vegas for editing, I just haven't gotten it to work for me the way I want no matter how much I try.

Former users wrote on 5/8/2018, 7:27 PM

I've been editing in Vegas for ten years. But I'm always up to date with other softwares. Because it's always good that you have a second alternative not to be a slave to one. Premiere is very good and the operation and enjoyment of Nvidia's GPUs are infinitely superior to that of Vegas that is still crawling in that area. My problem with the premiere is that it makes a simple editing task difficult like putting a simple transition between events. It annoys me.

fr0sty wrote on 5/8/2018, 9:43 PM

Took the words right out of my mouth about premiere... Its workflow sucks, otherwise it would be great. Vegas' "your transition is how far your clips overlap, just drag one onto that space, otherwise it's a fade" method is so much more intuitive.

Cliff Etzel wrote on 5/8/2018, 9:46 PM

@Former users I've gotten use to it over time as it is the way the majority of NLE's work. My issue with Premiere is the amount of resources it takes just to run, add to that project files that can be corrupted pretty easily (a known bug that has yet to be fixed and one I've experienced on more than one occasion) and the move to ransomware are the only reasons I haven't moved forward with newer versions with added features (Lumetri color technology in Premiere Pro now and the death of Speedgrade). TBH, every time I start Premiere to continue editing, I wait to see if the dreaded error box comes up saying the project file is corrupted and unreadable. I've even had that happen with auto saved projects saved every 5 minutes (20 latest versions no less!)

@fr0sty - The same paradigm also exists in Resolve so all the others reside in a fairy traditional paradigm while Vegas does things differently - which is one of the biggest appeals for me and why I've tried to make it work as my primary NLE.

Vegas in principle is how I think and edit - my ongoing challenge is getting all things to align like they have for users like yourself, @BruceUSA, @OldSmoke and others... Again I still come back to the idea that the weakest link for my computers are the graphics cards which will soon be dealt with for my desktop and will be looking to replace my aging laptop with newer old technology. The question is to I stick with Vegas Pro 14, upgrade to Vegas Pro 15 or wait it out and see what MAGIX does with Vegas Pro 16? And then the question comes back to - nVidia or AMD? It seems more convoluted now than before MAGIX added nVidia support to Vegas 15.

Kinvermark wrote on 5/9/2018, 12:13 AM

OK, this thread is a little strange for me - but I am trying to understand :) If I render a mp4 from Vegas without using effects it will look identical to the original. Which is what you want; sure don't want it changing without being told to!

So rather than taking some sample footage and applying cc/levels effects in three different NLE's to try to match and compare why not just go "bare." i.e. Don't apply ANY effects, just render (please use same codec and bitrate as far as possible) and compare. Original / Vegas render / Premiere Render / Resolve Render. Now, besides a slight quality loss from the render, all the images should look the same. Do they? Never mind differences from media players - judge compared to the original. Result?

 

PS Just tried this (again) to confirm I am not going crazy: Yes, Vegas render = same as original. Note however, that different media players DO change the colours (etc.), but of course that applies to both clips so they still remain identical WITHIN each player.

Nick Hope wrote on 5/9/2018, 3:25 AM

Cliff's 3 renders are exactly as I would expect for AVC renders.

Level 16 (very dark grey) in the Vegas Preview remains 16 when the rendered file is viewed in the Vegas Preview, but becomes 0 (pure black) when the rendered file is viewed in (most) other players. Likewise, level 235 (very pale grey) in the Vegas Preview remains 235 when the rendered file is viewed in the Vegas Preview, but becomes 255 (pure white) when the rendered file is viewed in (most) other players. Non-default settings in the "Video" section of graphics card software might change that behaviour.

Premiere Pro and Resolve appear to be doing an internal "levels squeeze" (compared to Vegas) somewhere in the chain so that original image, timeline preview(?), and ultimate rendered-video playback all look the same.

Personally I like how the Vegas preview window works because, although it's not WYSIWYG by default, I can see the variations in my superblacks (0-16) and superwhites (235-255), not just flat black and white, so I can see if there is any detail there that I can retain by reducing the contrast. If I want to see how the rendered video will look in a video player then I can do any of these 3 things:

  1. Set SeMW Extensions to "PC" and view in the Vegas preview. (I agree that it's only for preview, and not applied in render).
  2. Look on my external monitor, with Adjust levels from studio RGB to computer RGB checked in my Preview Device Preferences. This is what I normally do on my desktop.
  3. Temporarily add a VEGAS Levels FX with the Studio RGB to Computer RGB preset.

A straightforward workflow for "getting the levels right" for a Y′CbCr render such as AVC or HEVC is not rocket science:

  1. Open the video scopes and refer to them constantly.
  2. In the video scopes settings, check "Studio RGB (16-235)".
  3. Using whatever FX, set your blacks to 0 on the waveform / 16 on the histogram, and your whites to 100 on the waveform / 235 on the histogram. Crush blacks/blow out whites as much or as little as desired.
  4. Preview using one of the 3 methods described above.
  5. Render.
  6. Use an app like Just Color Picker to check results.

A real expert like Glenn Chan might still say there are untruths in the above, but anyway it works for me in my 8-bit workflow.

There is more on this subject in this FAQ post.