HEVC decoding errors

chris_lopez wrote on 12/23/2016, 5:45 AM

This applies to Vegas14pro build 211, I have not tried previous versions.

There is a decoding error(s) when an HEVC video is put on the timeline: frame 29 is repeated and frame 30 is dropped -

correct sequence: ...28, 29, 30, 31...

vegas14 build 211: ...28, 29, 29, 31, 32... ...54, 55, 55, 57, 58... etc...

clip and timeline are at 29.97

any ideas ?

 

 

Comments

NickHope wrote on 12/23/2016, 6:52 AM

If I render HEVC files out of VP14 and bring them back to the timeline, I'm not seeing that problem. I tried with UHD 29.97 footage from my GH4 and 1080-59.94p footage from my RX100. However in both renders I am seeing the very first frame of the clip repeated, which is bad. And in the UHD render, the end of the clip is not quite quantized to frames.

What's the source of your file? Can you share MediaInfo and Vegas File Properties, and maybe a sample?

Peter_P wrote on 12/23/2016, 7:57 AM

I can reproduce this problem with UHDp30 HEVC material from a Samsung NX1 from the dkamera site at the bottom for download :

http://www.dkamera.de/media/testberichte/samsung/nx1/6_beispielaufnahmen/video/video02.MP4

Currently running Vp14 B201

Former user wrote on 12/23/2016, 8:07 AM

In build 211 I have also this first frame duplicate in the beginning of each clip (NX1 HEVC UHD30p).

Peter_P wrote on 12/23/2016, 8:09 AM

To be sure that this is not caused by the NX1 material, I just rendered an UHDp30 clip from a Sony RX10 II to Intel HEVC with 40Mbps and this shows the same repeated frames in Vp14 B201. The original RX10 II has no repeated frames.

Peter_P wrote on 12/23/2016, 8:12 AM

In build 211 I have also this first frame duplicate in the beginning of each clip (NX1 HEVC UHD30p).


This seems to be new with 211. I'm not having repeated frames at the begin of this NX1 clip nor on my RX10 II clips.

Peter_P wrote on 12/23/2016, 8:22 AM

No, this seems to be a wrong assumption. In B201 with a Panasonic G70 UHDp30 clip from dkamera (also video02) the first frame is also repeated. No problem with the Sony RX10 II clips.

chris_lopez wrote on 12/23/2016, 9:54 PM

further info to my post: the HEVC footage I am importing is from a Huawei phone. I can tell that Vegas is introducing the errors because I can play it back ok without repeating frames using Windows10 movie player. The repeating frames occur throughout each video clip.

NickHope wrote on 12/25/2016, 9:14 PM

Which Huawei phone? Can you share MediaInfo and Vegas File Properties, and a sample?

NickHope wrote on 1/5/2017, 11:27 PM

This same repeat-frame issue was reported by GJeffrey in this post. I have done some testing today with encoding and decoding ProRes and HEVC and I can confirm that they share exactly the same issues with repeat frames, audio sync, and final-frame-quantization. This is not surprising as they both share the same codec. My results are summarized in this table:

If you have a repeat first frame, you can remove it on multiple clips on the timeline using this script which dates back more than 10 years to when this exact same issue was happening with HDV.

Really hoping the devs finally nail this issue in the next update as this is the 3rd release with different repeat frame issues for this codec.

chris_lopez wrote on 1/6/2017, 7:42 PM

Ok Nick this is the media info for that Huawei mate9 clip, I don't really have anywhere to upload it. Note that its not just the first frame that repeats but every 29th frame which makes it pretty much unusable.

General
Complete name                            : rt.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 2
Codec ID                                 : mp42 (isom/mp42)
File size                                : 84.6 MiB
Duration                                 : 23 s 422 ms
Overall bit rate                         : 30.3 Mb/s
Encoded date                             : UTC 2016-12-23 02:32:26
Tagged date                              : UTC 2016-12-23 02:32:26
com.android.version                      : 7.0

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main
Codec ID                                 : hvc1
Codec ID/Info                            : High Efficiency Video Coding
Duration                                 : 23 s 422 ms
Source duration                          : 23 s 433 ms
Bit rate                                 : 29.8 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Frame rate                               : 29.788 FPS
Minimum frame rate                       : 22.562 FPS
Maximum frame rate                       : 35.686 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 8 bits
Bits/(Pixel*Frame)                       : 0.121
Stream size                              : 83.3 MiB (98%)
Source stream size                       : 83.3 MiB (98%)
Title                                    : VideoHandle
Language                                 : English
Encoded date                             : UTC 2016-12-23 02:32:26
Tagged date                              : UTC 2016-12-23 02:32:26
mdhd_Duration                            : 23422

Audio
ID                                       : 2
Format                                   : AAC
Format/Info                              : Advanced Audio Codec
Format profile                           : LC
Codec ID                                 : 40
Duration                                 : 23 s 360 ms
Bit rate mode                            : Constant
Bit rate                                 : 192 kb/s
Channel(s)                               : 2 channels
Channel positions                        : Front: L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 spf)
Compression mode                         : Lossy
Stream size                              : 548 KiB (1%)
Title                                    : SoundHandle
Language                                 : English
Encoded date                             : UTC 2016-12-23 02:32:26
Tagged date                              : UTC 2016-12-23 02:32:26
mdhd_Duration                            : 23360

Marco. wrote on 1/6/2017, 7:45 PM

According to this info that clip has a variable frame rate spanning from 22 to 35 fps.

GJeffrey wrote on 1/6/2017, 8:05 PM

Variable frame rate video files are well known to give problem to most of the NLE out there and not only Vegas.

If possible, record in constant frame rate.

NickHope wrote on 1/6/2017, 8:48 PM

Ok Nick this is the media info for that Huawei mate9 clip, I don't really have anywhere to upload it. Note that its not just the first frame that repeats but every 29th frame which makes it pretty much unusable.

Thanks. Well that sucks. I was about to buy one of those!

I guess Vegas may be interpreting your footage as 29.97fps constant frame rate but, because it's variable frame rate and the average fps is actually 29.788, it's getting messed up.

If you send me your email address by personal message I'll give you upload access to a Google Drive folder where you could upload a sample file. I'll see what I can do with it. I could share it with the developers too, and even post a link on the forum, if that's OK with you.

According to this info that clip has a variable frame rate spanning from 22 to 35 fps.

Variable frame rate files from phones can be a bit strange. In my experience MediaInfo sometimes reports a variable range of frame rates but in reality they are at or close to 29.97p. I made a post a few months ago on how to quantize each frame of such footage to make it more NLE-friendly. But the methods there won't work if your footage really is continuously varying that much. There's a suggestion on there from JohnnyRoy for using FiLMiC Pro instead of the phone's native camera, to get a constant frame rate. Might be worth a try.

Peter_P wrote on 1/7/2017, 3:04 AM

This HEVC file (), with a constant frame rate, from the new DJI Phantom 4 pro can not be loaded into Vp14, but can be played with MPC-HC.

https://pbdownloads.wetransfer.com/downloads/7cc7eae9acc510561eca50b895888f5820170107005848/27c65e

Media Data:

General
Complete name                  : P4pro_DJI_0121.MP4
Format                         : MPEG-4
Format profile                 : JVT
Codec ID                       : avc1
File size                      : 1.01 GiB
Duration                       : 1mn 28s
Overall bit rate               : 98.9 Mbps
Comment                        : DE=None, Mode=M

Video
ID                             : 1
Format                         : HEVC
Format/Info                    : High Efficiency Video Coding
Format profile                 : Main@L5@High
Codec ID                       : hvc1
Codec ID/Info                  : High Efficiency Video Coding
Duration                       : 1mn 27s
Bit rate                       : 100 Mbps
Width                          : 4 096 pixels
Height                         : 2 160 pixels
Display aspect ratio           : 1.896
Frame rate mode                : Constant
Frame rate                     : 25.000 fps
Color space                    : YUV
Chroma subsampling             : 4:2:0
Bit depth                      : 8 bits
Bits/(Pixel*Frame)             : 0.452
Stream size                    : 1.01 GiB (100%)
Title                          : DJI.HVC
Language                       : English
Color range                    : Limited
Color primaries                : BT.709
Transfer characteristics       : BT.709
Matrix coefficients            : BT.709

Other
Type                           : meta
Duration                       : 1mn 28s
Bit rate mode                  : VBR

GJeffrey wrote on 1/7/2017, 3:46 AM

his HEVC file, with a constant frame rate, from the new DJI Phantom 4 pro can not be loaded into Vp14

This problem has already been discuss somewhere in the forum.

The problem is the codec ID. For HEVC file it should not be avc1 (which is for h264 normally) but hvc1 or hev1.

You can try to rewrap it with ffmeg (no reencoding)

And submit a request to the Vegas developpers.

Peter_P wrote on 1/7/2017, 4:15 AM
The problem is the codec ID. For HEVC file it should not be avc1 (which is for h264 normally) but hvc1 or hev1.

I have seen this also and assume this is caused by the early stage of this camera and might soon be changed by DJI. However only a wrong codec ID should not prevent Vegas from reading a file.

Peter_P wrote on 1/7/2017, 8:20 AM

Thanks for the link and hint.

NickHope wrote on 1/7/2017, 10:33 PM

This is chris_lopez' file from the Huawei Mate 9: https://drive.google.com/file/d/0B8ftVT5A1FqQUG1RREJsc3JLN0k/view?usp=sharing

As I suspected, it appears to be a constant frame rate, as far as I can tell by stepping through it, despite what MediaInfo says. But it really is 29.788p, so you have to set the playback rate to 0.994 to quantize the frames boundaries. And the last frame isn't quantized in either case.

The first frame is repeated, and then, as per the first post, every 29th frame is repeated and the 30th is skipped.

My only suggestion would be to try one of the converters in section 6 of this post and see if they fix the problem. I suppose there may be an FFmpeg command line that would remux the file (keeping it as HEVC) and set the frame rate to 29.97 fps, without resampling the frames themselves, and that might then open in Vegas without the repeat frames each second. I think it would still have the first frame repeated though. Any FFmpeg experts care to suggest a command line for that?

Also, I'd be interested to know if this file opens in other NLEs without those repeat frames, if anyone could try that.

GJeffrey wrote on 1/8/2017, 12:46 AM

As I suspected, it appears to be a constant frame rate, as far as I can tell by stepping through it, despite what MediaInfo says. But it really is 29.788p, so you have to set the playback rate to 0.994 to quantize the frames boundaries. And the last frame isn't quantized in either case.

I disagree here Nick.

Loading this file to avidemux gives me:

1st frame @ 00:00:00.044s

2nd frame @ 00:00:00.078s

3rd frame @ 00:00:00.111s

4th frame @ 00:00:00.144s

So for me it's really a variable frame rate file.

 

It's posible to convert to constant frame rate using mp4box.

1- Demux using

MP4Box -add input.mp4#video -raw 1 -new output

This will create a output_track1.hvc

2- Remux using

MP4Box -add output_track1.hvc:fps=29.97 -new output.mp4

Audio might need a resample as well but I haven't tried.

NickHope wrote on 1/8/2017, 3:17 AM
So for me it's really a variable frame rate file.

Well, apart from the 1st frame, the gaps between your numbers are 0.034, 0.033 and 0.033, which could all easily be rounded from the same number.

I didn't do anything as scientific. I just set the project frame rate to 29.788, added the clip at the project frame rate, and then stepped through it frame-by-frame with the right arrow. Every frame lands exactly (at least visually) on a frame boundary:

So I believe even though it isn't "officially" a constant frame rate, in practice it is or it is near enough. It certainly doesn't fluctuate between the minimum and maximum rates stated in the MediaInfo report.

Anyway, your mp4box commands eliminate the skipped frames, apart from at the start. Great work! The file seems to play more smoothly too.

So I remembered my batch file from here and adapted it to fix a batch of these types of files. It works really fast. Install mp4box, paste this code into notepad, save it as fix-vfr-HEVC.bat in the folder with the media in it, and double click it. I'm not sure if mp4box gets added automatically to the system path when it's installed so you might need to do that.

@echo off & setLocal EnableDelayedExpansion

REM Extract hvc and aac streams
for %%a in (*.mp4) do mp4box "%%a" -raw 1 -raw 2

REM Remove "_track1" from the hvc filename
for /f "tokens=* delims= " %%a in ('dir /b *.hvc') do (
set F=%%~Na
set F=!F:~0,-7!
move /y "%%a" "!F!%%~Xa"
)

REM Remove "_track2" from the aac filename
for /f "tokens=* delims= " %%a in ('dir /b *.aac') do (
set F=%%~Na
set F=!F:~0,-7!
move /y "%%a" "!F!%%~Xa"
)

REM Mux with 29.970 frame rate
for %%a in (*.hvc) do mp4box -add "%%~na.hvc:fps=29.970" -add "%%~na.aac" -new "%%~na_fixed.mp4"

REM Clear up the streams
del *.hvc
del *.aac

pause

p.s. I didn't look into resampling the audio either yet. It could always be ungrouped and stretched if necessary.

GJeffrey wrote on 1/8/2017, 3:57 AM

Great script Nick.very useful for cellphone video.

So I remembered my batch file from here

You already had the solution 😁. You are the ffmpeg expert here.

It certainly doesn't fluctuate between the minimum and maximum rates stated in the MediaInfo report.

Mediainfo gives the frame rate by getting the timecode of the video frames.

The 1st frame tc gives 22.73fps, not far from mediainfo actually.

 

chris_lopez wrote on 1/8/2017, 9:00 PM

thanks for the info

Peter_P wrote on 1/19/2017, 10:51 AM
If you have a repeat first frame, you can remove it on multiple clips on the timeline using this script which dates back more than 10 years to when this exact same issue was happening with HDV.

I have just rendered an UHDp30 Clip with Vp14 to UHDp30 XAVC-S and via Handbrake 1.0.1 to H.265 to compare the image quality on two tracks within Vp14. To get the frames fitted, the first frame of the H.265/HEVC file need to be trimmed and then both clips are aligned to the begin of the tracks. But in this case, where frames are matching, there is a slight shift of the audio data, as you can see here.

So in many cases taking away the first HEVC frame will work but you get a very slight audio shift.