Nvidia Nvenc data rate limit

Former user wrote on 2/2/2019, 12:03 PM

I don’t expect there will be much of an interest in this issue, but I thought it should be put out there anyway, in case in the future someone else runs into it.

Theoretically its possible to set say 135/100 data rate in the Magix Nvenc encoder render template but in reality there appears to be a cutoff, based on input file type.

Above the cutoff you get either a zero byte output file or an output file with no video and an audio stream in its place there instead.

I came across this issue while attempting to get a high data rate using VCE on a mates laptop.  I got enough to update the Happy Otter Render Quality Metrics for the “BruceUSA” comparisons, which was 24-36 data rates, but couldn’t get anything close to 135/100 Mbps that I needed for the “General Purpose” HORQM comparison testing.  At the time I assumed it was only a VCE issue.

On my laptop (Nvidia 1070) I found the Max render setting I could get was 46/45 Mbps.  Anything above that produced an unusable output file.

The source material was typical 4K uhd avc from Pana or Sony cams.  Output was set to 4K uhd. avc using Magix Nvenc, all 25fps.

If I used an intermediate source file made from the original source clips, I had no issue rendering to the higher data rate from the intermediate file.

If I used my PC (Nvidia 2080) I had no issue at all rendering from the original 4K uhd clips to a 135/100 Mbps file.

So I changed the graphics drivers from what I had to two different ones, later and earlier, on the laptop to what was working on the PC, no difference to the result.

I tried a few changes in the render template, no luck.  I watched the memory usage while rendering, no issues there.  I disabled SO4 ... .dll, I have Quicktime 7 istalled on both machines but it never appears as used, only SO4 and Compound ... .dll.

Of course it is possible to render the same input project clips to the high data rate by using Magix Mainconcept, ignoring the Nvenc encorder.

I use the term high data rate above, but thats a relative term as source and output are 4K, 100Mbps is what cams often shoot at.

Eventually I found the culprit, the first clip in the project was a gh5 .mov clip with .pcm audio.  I converted to .mp4 as per Nick's FAQ, see link below.  It didn’t fix the problem because as mentioned in the FAQ, it uses .pcm audio.

https://www.vegascreativesoftware.info/us/forum/faq-how-to-convert-mov-video-e-g-iphone-to-mp4-for-vegas-pro--106154/

Whats weird about the whole thing is that I can understand now why I had the issue, GH5 .mov clip, but that it wasn’t an issue on the PC, only the laptop.

To summarise, on my laptop... the Nvenc encoder ( “SO4 ... .dll” limit, bug? ) only works up to a certain data rate and no further, (45/46 setting) if a GH5 clip with .pcm audio is used, and the render template is Nvenc hardware encoding.

So maybe a bit more work to do on the SO4 ... .dll.

 

 

 

 

Comments

john_dennis wrote on 2/2/2019, 1:25 PM

NOT NVENC    NOT NVENC    NOT NVENC    NOT NVENC    NOT NVENC    NOT NVENC    NOT NVENC

I'm equally interested in knowing the limits of AMD VCE as implemented in Vegas 15-416.

Using UHD XAVC 100 Mbps 29.97p source files, I was not able to push the maximum bit rate above 100 Mbps without causing the render to hang... (I killed the process from the Windows Task manager.)

  

...or producing and error message...

I was successful at rendering 100/100 Mbps with the following results:

The average bit rate was ~85 Mbps.

One oddity that I noticed was that even though the render template had Profile set to High by default, Mediainfo reported Format profile:  Main at 5.1.     

I found that curious. I'm willing to learn.

Former user wrote on 2/2/2019, 2:32 PM

Well @john_dennis 5.1 is pretty high. In my Nvenc case, as mentioned above the issue is limited to a GH5 clip. I haven’t yet tested higher than 135/100 but I can render to that once its not GH5.

It looks to me that it’s maybe a SO4 ... .dll issue, it’s fixed ok for non hardware encoding but may need more work for Nvenc and Vce, haven’t tested Qsv.

Ok, just checked, found that raising the data rate higher hits a bar of about 156 Mbps, using non GH5 clip, thats ok.

Former user wrote on 2/3/2019, 6:05 AM

I checked the max data rate on the output rendered clip for Qsv and I also redid using Nvenc, results might depend on source?

QSV ... 242 Mbps

Nvenc ...174 Mbps

You can enter a higher value than 240 but it resets always to 240 for both Qsv and Nvenc.

Of course this is something you would never do as the source is only about 100 Mbps.

Source and output was 4K h264 avc 25fps.