Nvidia App Beta: AV1 Recordings cannot be imported - VP21

Howard-Vigorita wrote on 7/7/2024, 2:26 PM

@RogJ Here's another data point: the original SVT libs released by Intel to ffmpeg had a bug that was later fixed. The bad libs had a signature of Lavf60.3.100 or earlier. Those av1 renders are not read properly by Vegas. The fixed libs have a signature of Lavf60.10.101 or later. Those read and play fine. Suggest you at least do a signature-check with MediaInfo to see if the metadata records anything like that.

RogJ wrote on 7/7/2024, 3:25 PM

@Howard-Vigorita
I am on Vegas 21 Build 315

So I used media info, in short the Nvidia made AV1 has a different codec id and is missing the encoder info.

This is the AMD made AV1:

Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/av01/iso2/mp41)
File size : 1.21 GiB
Duration : 1 min 38 s
Overall bit rate mode : Variable
Overall bit rate : 106 Mb/s
Frame rate : 30.000 FPS
Writing application : Lavf60.3.100

Video
ID : 1
Format : AV1
Format/Info : AOMedia Video 1
Format profile : Main@L5.0
Codec ID : av01
Duration : 1 min 38 s
Bit rate : 105 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 30.000 FPS
Minimum frame rate : 15.000 FPS
Maximum frame rate : 30.059 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.424
Stream size : 1.21 GiB (100%)
Color range : Limited
Codec configuration box : av1C

And the Nvidia AV1:

Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42 (isom/mp42/av01)
File size : 181 MiB
Duration : 12 s 232 ms
Overall bit rate : 124 Mb/s
Frame rate : 30.000 FPS
Recorded date : 2024
Encoded date : 2024-07-07 20:19:30 UTC
Tagged date : 2024-07-07 20:19:30 UTC

Video
ID : 1
Format : AV1
Format/Info : AOMedia Video 1
Format profile : Main@L5.2
Codec ID : av01
Duration : 12 s 232 ms
Bit rate : 124 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 30.000 FPS
Minimum frame rate : 18.446 FPS
Maximum frame rate : 80.501 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.498
Stream size : 181 MiB (100%)
Title : VideoHandle
Encoded date : 2024-07-07 20:19:30 UTC
Tagged date : 2024-07-07 20:19:30 UTC
Color range : Limited
mdhd_Duration : 12232
Codec configuration box : av1C

Howard-Vigorita wrote on 7/7/2024, 5:31 PM

I just tried the Rigaya NVencc64 encoder on my 4090 with a simple command line to render an av1 clip and vp21 b315 and b208 both recognized and played it fine. I used the latest Rigaya encoder built on ffmpeg 6.1 libs. The good news for you is that there doesn't seem to be a conflict with Nvidia 4000-series hardware encoding firmware. Whatever the trouble is, maybe a rewrap with ShutterEncoder might fix it. I've had mixed luck with .mkv and suggest you stick with .mp4 or .mov containers.

Here's the script and MediaInfo on the clip I just tested successfully which yielded a max gop of 300.

nvencc64 -i 'hevc-lossless-an.mov' -c av1 -o nvenc-av1.mp4

General
Complete name                            : D:\test ffmetrics\nvenc-av1.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (mp42/av01/iso2/mp41)
File size                                : 362 MiB
Duration                                 : 30 s 731 ms
Overall bit rate                         : 98.8 Mb/s
Frame rate                               : 29.970 FPS
Writing application                      : NVEncC (x64) 7.21

Video
ID                                       : 1
Format                                   : AV1
Format/Info                              : AOMedia Video 1
Format profile                           : Main@L5.0
Codec ID                                 : av01
Duration                                 : 30 s 731 ms
Bit rate                                 : 98.8 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 0.397
Stream size                              : 362 MiB (100%)
Language                                 : English
Color range                              : Limited
Codec configuration box                  : av1C

Btw, if you tried OBS before the Nvidia beta app, try revisiting it and selecting ffmpeg encoders. I noticed when I installed the Nvidia app that it said that it optimized OBS and DaVinci.

Todd-AO wrote on 7/7/2024, 5:40 PM

@RogJ Here's another data point: the original SVT libs released by Intel to ffmpeg had a bug that was later fixed. The bad libs had a signature of Lavf60.3.100 or earlier. Those av1 renders are not read properly by Vegas. The fixed libs have a signature of Lavf60.10.101 or later. Those read and play fine. Suggest you at least do a signature-check with MediaInfo to see if the metadata records anything like that.

@Howard-Vigorita The Nvidia app did have a bug where AV1 encodes could not be encoded by YouTube. This was fixed, it may still have a bug where it has the wrong levels and you need to manually choose video levels over full. So both of you could be on to something, not enough meta data for Vegas to work. Capcut and Resolve work fine though.

Also are you now on team AV1? The VMAF don't lie

Todd-AO wrote on 7/7/2024, 5:45 PM

Btw, if you tried OBS before the Nvidia beta app, try revisiting it and selecting ffmpeg encoders. I noticed when I installed the Nvidia app that it said that it optimized OBS and DaVinci.

@RogJ I also noticed that, and how Vegas doesn't exist as far as Nvidia app is concerned. So I am wondering if it's optimizing Tensor cores and Vegas doesn't use tensor cores or it's doing something else. When you run Resolve for first time it spends 5 minutes optimizing tensor cores

RogJ wrote on 7/7/2024, 6:32 PM

I did the simple rewrap with Shutter Encode, and that was good enough to import Nvidia made AV1 into Vegas. Plays back fine looks great.

Its still annoying that in the world of Nvidia, everything is supposed to "just works!" in comparison to AMD.

It sounds like a simple, bad metadata.

Howard-Vigorita wrote on 7/7/2024, 6:46 PM

Also are you now on team AV1? The VMAF don't lie

@Todd-AO Ha, ha, the numbers don't lie but statisticians do all the time.

If you're trying to evaluate encoding from a camera, your reference has to be from the same source... lossless from the sensor just like the encoder gets it. All the analysis I've done has been from camera sensors because that's my thing. And I've never seen av1 do better than hevc yet... but the gap does narrow as frame size and bit-rate go down. However the gap was still significant at 24 mbps when I measured it. I also measured it at 50, 60, 100, and 150 mbps which covers what I shoot and the divergence gets steadily wider and goes through the roof at lossless. Rgb monitors being fed generated media from games have more color info than consumer camera sensors. So I'd expect a wider gap for the required higher data rates. Screen capture data rates shown above are 124 and 151 mbps but the color must have been down-sampled at capture for 4:2:0.

john_dennis wrote on 7/7/2024, 7:04 PM

@RogJ said: "I did the simple rewrap with Shutter Encode, and that was good enough to import Nvidia made AV1 into Vegas. Plays back fine looks great."

That appears to be a low-effort workaround or short-term solution while you wait for Nvidia and/or Vegas Creative Software to provide a more elegant fix.

Todd-AO wrote on 7/7/2024, 8:40 PM

Also are you now on team AV1? The VMAF don't lie

@Todd-AO Ha, ha, the numbers don't lie but statisticians do all the time.

If you're trying to evaluate encoding from a camera, your reference has to be from the same source... lossless from the sensor just like the encoder gets it. All the analysis I've done has been from camera sensors because that's my thing. And I've never seen av1 do better than hevc

@Howard-Vigorita I"m transcoding 1080P60 Prores, not sure how it was generated but I've always used it as a constant, at higher bitrates I still see AV1 in the lead. Considering the negative things I've heard about AV1 which you briefly touched upon, trying to create a codec that is patent free, and resisting using methods for highest quality/efficiency if that could present a legal problem in the future.