Rendering 1080p for YouTube - 2020 Survey

john_dennis wrote on 3/25/2020, 3:41 PM

I prepared four videos for YouTube using different encoders with Vegas Pro 17-421. I wish to learn which encoders look the best to viewers. The playlist in the link below has four options, A-D. Each option has three sections that are clearly delineated by titles and sync tones within the video.

The first section of each option is High Complexity. High Complexity has much action with a great deal of fine detail. Examples: Swimmers splashing in water while the camera pans with crowds in the background. Camera panning across a grass lawn in a hail storm. You get the picture. I included everything except strobe lights and confetti.

The second section of each option is Medium Complexity. Medium Complexity consists of a fairly well lit stage play with the camera locked in one place while multiple subjects move around the stage.

The third section of each option is Low Complexity. Low Complexity consists of a white on black credit roll. While the text moves on the screen, the movement is uniform and might be predictable to the encoders.

From my YouTube site, select Play All on the playlist Encoder Comparison at the following link to watch all options:

https://www.youtube.com/playlist?list=PLSV9x8Yqu0m0so7mpvYxnFmnvpLM6I0DU

****** or ******

The individual Youtube options are here:

Option A

Option B

Option C

Option D

Please watch and state your preference by option letter.

Edit: 2020-03- 25 Changed links to be more direct.

1659 Added Graphic.

Comments

Dot wrote on 3/25/2020, 3:50 PM

All I see is Option A

walter-i. wrote on 3/25/2020, 5:48 PM

All I see is Option A

@john_dennis Confirmed

Last changed by walter-i. on 3/25/2020, 5:48 PM, changed a total of 1 times.

Camcorder: Pana HDC SD909, Sony FDR-AX53
Hardware: CPU i9-9900KS 4GHz, 32 GB Ram, GTX 1660 SUPER, ASUS TUF Z390-PLUS Gaming, SSD 970 EVO Plus 500GB M.2, SSD 860 EVO Series 1TB, Toshiba 3TB SATA, WIN 10 Pro
NLE: Vegas Pro11-15+17, Heroglyph4, RespeedR

vkmast wrote on 3/25/2020, 5:58 PM

Try the individual YouTube options that @john_dennis added to the Post.

Dot wrote on 3/25/2020, 6:04 PM

He fixed it after my post, thanks.

john_dennis wrote on 3/25/2020, 6:07 PM

@walter-i.

Added screenshot to explain Play All from the Encoder Comparison playlist. The four 1:27 videos will play with the Option title displaying occasionally in the lower left of the screen for each video. YouTube will also the title if you move the mouse.

vkmast wrote on 3/25/2020, 6:36 PM

@john_dennis in (my) Chrome (80.0.3987.149) the link turns into this https://m.youtube.com/playlist?list=PLSV9x8Yqu0m0so7mpvYxnFmnvpLM6I0DU and files don't play. My Edge shows it as in your screenshot.

walter-i. wrote on 3/26/2020, 3:57 AM

@john_dennis Thanks for the instructions.

I can't really see (with my eyes) a difference between A, B, C and D on Firefox.
In my opinion, everything in very good quality - only slight jerking when playing the roll title in all versions.

Where I can see a clear difference is between Firefox 74.0 and Chrome 80.0.3987.149
In Chrome - especially "High Complexity" - is rendered very mushy - almost "unsightly".
The grass is partly "green sauce" See screenshot.
But may also be due to an incorrect browser setting from me.

Camcorder: Pana HDC SD909, Sony FDR-AX53
Hardware: CPU i9-9900KS 4GHz, 32 GB Ram, GTX 1660 SUPER, ASUS TUF Z390-PLUS Gaming, SSD 970 EVO Plus 500GB M.2, SSD 860 EVO Series 1TB, Toshiba 3TB SATA, WIN 10 Pro
NLE: Vegas Pro11-15+17, Heroglyph4, RespeedR

fifonik wrote on 3/26/2020, 4:59 AM

D is the worst.

C is much better.

A and B slightly better than C and are very similar.

P.S. If someone would like to do pixel peeping, here are mp4 files in archive that I downloaded from youtube (~120MB)

Last changed by fifonik on 3/26/2020, 5:17 AM, changed a total of 4 times.

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

john_dennis wrote on 3/26/2020, 10:56 AM

@walter-i.

I was also taken back at the apparent damage to the lawn side of the video.

@fifonik

Thanks for your observations and the link to the downloaded files.The intent of the survey is to do a double-blind perceptual test, but I don't really mind if you and everyone else pokes and prods at the individual pixels. If you're wondering how I could have a double-blind test, 1) the encoders are not identifiable by the participants and 2) I stripped all the identifying titles from the encodes when named them Option A-D. I can't remember which is which. I have the actual titles in a folder which I'll ignore for now.

lenard-p wrote on 3/26/2020, 6:44 PM



Where I can see a clear difference is between Firefox 74.0 and Chrome 80.0.3987.149
In Chrome - especially "High Complexity" - is rendered very mushy - almost "unsightly".
The grass is partly "green sauce" See screenshot.
But may also be due to an incorrect browser setting from me.

i was able to replicate your extremly undetailed grass by choosing the 720p option. I don't see that in any of the 1080p versions. Also that shows why if you have a 720p source you would be better to upscale to 1080p before uploading to youtube. The grass should not look like that merely due to the lower resolution. I think it's higher compression (could be wrong) With that thinking also upscale 1080p to 1440, where the vp9 compression of 1440p is much better. Someone here was talking about doing that

Musicvid wrote on 3/26/2020, 7:12 PM

My eyes are too old to even make a stab at a subjective complexity/motion comparison.

I have looked at the files and individual frames, blown them up, and yes, there are some differences. The flag with grainy blue sky is "better" with one encoder, while the splashy swimmers are "better" with another. Proves not a thing, because "better" is a qualitative, not objective judgment. A few stray souls may not even agree with me, so for that reason, I have not identified the "obviously" "better" clips.

One thing that can be shown is that the 900 lb. elephant in the studio is not the encoders, not the settings, but Youtube's unforgiving stingy processing. Stingy, because they are focused on bandwidth and speed, not compromising a single bit to the notion of quality.

If @john_dennis would upload the presumptive "best" original clips for comparison, this would probably be the only thing we "could" agree on. Sorry for the excess "" marks, but they are there to make a point, not a hope for a consensual outcome. We will see.

 

lenard-p wrote on 3/26/2020, 7:26 PM

 

If @john_dennis would upload the presumptive "best" original clips for comparison, this would probably be the only thing we "could" agree on. Sorry for the excess "" marks, but they are there to make a point, not a hope for a consensual outcome. We will see.

 

It is still an interesting test as it is. If a lower quality encoder is still better than the low compression of Youtube for AVC, then potentially youtube may handle that better. The logic has always been to upload the highest quality version to youtube, but what if that's not true. Maybe you want your own encodings to soften things or reduce sharpness or detail in specific areas (where it's not critical) to improve the output of youtube's avc encoder.

Musicvid wrote on 3/26/2020, 7:51 PM

Youtube's unforgiving stingy processing.

Really, this comparison has nothing to do with pixel-peepy, touchy-feely, or streetcorner jabberwocky.

One glance at the file sizes should pretty much clear up that notion. So whatever looks the best to you, is the best!!

A previous, related example:

john_dennis wrote on 3/26/2020, 11:33 PM

At this point in the thread, I'll share some facts about the files I uploaded to YouTube. None of these facts should influence anyone in selecting their preference, if he or she has one.

  • Three of the files have identical bit rates.
  • The remaining file is 2.8 times the bit rate of the other three.
  • Played locally these files would impress 100% of all over-the-air network TV watchers in the known world. 
  • Played locally these files would impress 100% of all satellite and cable TV watchers in the known world.
  •  The 720p files encoded by Google and made available for me to download were the same file size within 100 KB in 17.8 MB.
  • I have a long history of skepticism about YouTube.
fifonik wrote on 3/27/2020, 12:03 AM

I perfectly understand your reason for doing this. I thought about the same one day, but as I do not upload on YT I decided to spent my time to something more interesting to me.

However, I'd use objective comparison instead of subjective. Upload files to YT, then download recompressed back (as I did) and compare (I'm using the MSU VQMT Free, you could use wwaag's tool, Vegas diff, ffmpeg ssim, etc)

I can do the comparison with VQMT for you if you provide 'reference' file that should be the best possible quality (and not damaged by YT)

Last changed by fifonik on 3/27/2020, 12:06 AM, changed a total of 1 times.

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

john_dennis wrote on 3/27/2020, 12:23 AM

@fifonik

"However, I'd use objective comparison instead of subjective."

I did that. I'll keep the results to myself for now.

Howard-Vigorita wrote on 3/27/2020, 2:05 AM

I've only had a chance to study the stage sections and focused on the frame at 0:40 where the girl holds up the pencil. I grabbed it off the screen of a 4k laptop display... I like the detail in A best with C a close 2nd. C seemed to have a little more contrast than A. The girl's hand looked smeared in B. I also grabbed other frames @ 0:10, 0:25, and 1:10 but I haven't had a chance to study them yet.

ps: Here's a jpeg render of the blowups I did in Photoshop of the frame at 0:40 of the pencil...

Last changed by Howard-Vigorita on 3/27/2020, 9:44 AM, changed a total of 2 times.

Cameras: Canon XF305, JVC GV-LS2, Canon 6D w/L-glass line. (Mar 2020: testing Zcam E2)
Laptop: Dell XPS15-9570; i7-8750h 32gb (integrated Intel UHD-630 & Nvidia GTX-1050Ti)
Road: Intel NUC i7 8809g 32gb (integrated AMD VegaM 4gb graphics and Intel HD630)
Workstation: i9 9900k 32 gb (Sapphire AMD Radeon VII 16gb graphics and integrated Intel UHD630)
Workstation2: e5 1650v4 128 gb (Sapphire Nitro+ RX5700xt Navi 8gb graphics)
Workstation3: i7-980X 24gb (Sapphire RX480 4gb graphics)
currently Vegas 17.452

john_dennis wrote on 3/30/2020, 2:07 PM

@Howard-Vigorita

Thank you for taking the time to study this and present your analysis.

[Slightly skeptical musing]

"Even a blind pig gets an acorn once in a while."

[/Slightly skeptical musing]

With encoders where the index frames fully represent a per frame snapshot of the source within the bit rate limitations, wouldn't you expect that any of the encoders would have a pristine frame somewhere in the video while the other encoders did less well with that particular frame?

If yes, then an assessment of the whole video is more appropriate than looking at results for individual frames.

Here is a snapshot of the frame-by-frame quality measurements using Wayne Waag's RQM2 tool. The data is in a comma delimited file for each Option in the form:

Frame Number, Mean Squared Error, Peak SNR, SSIM

Forty seconds is somewhere around frame 1200.

The largest differences between the encoders can be seen in the High Complexity section of the video. Here is a sample of the frames around 24 seconds.

set wrote on 5/2/2020, 4:47 PM

admit it - difficult to notice the differences...

thanks for sharing