FFMetrics -- Yet another program for quality metrics calculation

Comments

fifonik wrote on 9/18/2020, 5:40 PM

Version 0.8.2b published. Changes:

- Change:    mkv, mts and mxf were added on video file extensions list
- Change: Scan type (progressive|interlaced) provided in metadata is honoured
- Bugfix:    'Clear All' button is not clearing files list in plot
- Bugfix:    Files list in plot not updated instantly when file added/removed
- Bugfix: VMAF values/graphs not displayed correctly when system number decimal separator is non a dot
 

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

jason-b wrote on 9/23/2020, 2:52 PM

@fifonik I think I've figured out the issue. First off, it's not HDCAM, it's XDCAM as I stated in my previous post. Here's a link with a bit of info.

It seems to be the SMPTE 436M track containing captions that's the issue. The issues have been addressed for ffmpeg (I haven't verified this entirely yet) with some libavformat libraries that can be enabled at compile-time.

My work-around for the task of checking quality has been to use ffmpeg to simply copy the video track into another mxf container without mapping the 436M track and your tool works perfectly. Thanks for looking into this.

Here's an output of one of those files with 436M:

Input #0, mxf, from 'sample.mxf':
  Metadata:
    uid             : 6c301b00-6388-05c1-01dd-080046020130
    generation_uid  : 6c301b00-6388-05c2-01dd-080046020130
    company_name    : SONY
    product_name    : Opt
    product_version : 1.22
    product_uid     : 060e2b34-0401-0103-0e06-012002030300
    modification_date: 2020-01-15T15:57:00.000000Z
    material_package_umid: 0x060A2B340101010501010D43130000002F421A006388058008004602013001DD
    timecode        : 00:57:06;38
  Duration: 00:33:53.97, start: 0.000000, bitrate: 60741 kb/s
    Stream #0:0: Video: mpeg2video (4:2:2), yuv422p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 50000 kb/s, 59.94 fps, 59.94 tbr, 59.94 tbn, 119.88 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:1: Audio: pcm_s24le, 48000 Hz, 1 channels, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:2: Audio: pcm_s24le, 48000 Hz, 1 channels, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:3: Audio: pcm_s24le, 48000 Hz, 1 channels, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:4: Audio: pcm_s24le, 48000 Hz, 1 channels, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:5: Audio: pcm_s24le, 48000 Hz, 1 channels, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:6: Audio: pcm_s24le, 48000 Hz, 1 channels, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:7: Audio: pcm_s24le, 48000 Hz, 1 channels, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:8: Audio: pcm_s24le, 48000 Hz, 1 channels, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:9: Data: none
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
      data_type       : vbi_vanc_smpte_436M
Unsupported codec with id 0 for input stream 9

fifonik wrote on 9/23/2020, 3:42 PM

Thanks for the media info provided.

It is possible that this is what causing the issue: "Stream #0:9: Data: none"

Will check my code when got home.

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

fifonik wrote on 9/24/2020, 3:58 AM

Could you provide full output of the following command with your file:

ffmpeg -hide_banner -i sample.mxf -vframes 1 -t 1 -f null -

Thanks.

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

jason-b wrote on 9/24/2020, 11:06 AM

@fifonik Yes, I think that libavformat.mxfdec might address the 436M track that isn't otherwise properly read, but I've yet to have time to validate. Here's the output you requested:

Input #0, mxf, from 'sample.mxf':
  Metadata:
    uid             : 6c301b00-6388-05c1-01dd-080046020130
    generation_uid  : 6c301b00-6388-05c2-01dd-080046020130
    company_name    : SONY
    product_name    : Opt
    product_version : 1.22
    product_uid     : 060e2b34-0401-0103-0e06-012002030300
    modification_date: 2020-01-15T15:57:00.000000Z
    material_package_umid: 0x060A2B340101010501010D43130000002F421A006388058008004602013001DD
    timecode        : 00:57:06;38
  Duration: 00:33:53.97, start: 0.000000, bitrate: 60741 kb/s
    Stream #0:0: Video: mpeg2video (4:2:2), yuv422p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 50000 kb/s, 59.94 fps, 59.94 tbr, 59.94 tbn, 119.88 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:1: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:2: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:3: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:4: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:5: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:6: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:7: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:8: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
    Stream #0:9: Data: none
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
      data_type       : vbi_vanc_smpte_436M
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> wrapped_avframe (native))
  Stream #0:1 -> #0:1 (pcm_s24le (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
  Metadata:
    uid             : 6c301b00-6388-05c1-01dd-080046020130
    generation_uid  : 6c301b00-6388-05c2-01dd-080046020130
    company_name    : SONY
    product_name    : Opt
    product_version : 1.22
    product_uid     : 060e2b34-0401-0103-0e06-012002030300
    modification_date: 2020-01-15T15:57:00.000000Z
    material_package_umid: 0x060A2B340101010501010D43130000002F421A006388058008004602013001DD
    timecode        : 00:57:06;38
    encoder         : Lavf57.83.100
    Stream #0:0: Video: wrapped_avframe, yuv422p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 59.94 fps, 59.94 tbn, 59.94 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
      encoder         : Lavc57.107.100 wrapped_avframe
    Stream #0:1: Audio: pcm_s16le, 48000 Hz, mono, s16, 768 kb/s
    Metadata:
      file_package_umid: 0x060A2B340101010501010D431300000030421A006388058008004602013001DD
      encoder         : Lavc57.107.100 pcm_s16le
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.01 bitrate=N/A speed=1.58x    
video:1kB audio:2kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

fifonik wrote on 9/24/2020, 6:50 PM

The command above is executed by FFMetrics to get ref's media info.

I supplied your output to the routine that parses such output and everything worked as it should. No errors, crashes, lockouts and returned parsed properties as they should be.

Without having problematic media, I can only try to add more debug output, unfortunately.

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

fifonik wrote on 9/26/2020, 7:56 PM

@jason-b Sorry, I copy/pasted wrong frmpeg command.

To get media info the FFMetrics should execute the following:

ffmpeg -hide_banner -nostdin -i sample.mxf -frames:v 30 -filter:v idet -f null -

Could you check if this works and provide its output?

BTW, i was able to re-produce "locking" issue with one of file (mkv) I have on my PC.

Fixed that and PMed you a link to new build. Please check it and let me know if this issue is fixed for you.

Thanks.

Last changed by fifonik on 9/26/2020, 8:29 PM, changed a total of 1 times.

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

fifonik wrote on 9/28/2020, 12:55 AM

Version 0.8.3b published. Changes:

- Change: Long file specs on the list truncating with ellipses in the middle of path (full file specs still can be visible as hints on mouse hover)

- Change: On getting media info now 60 frames analyzed (changed from 30)

- Bugfix: Program freezing while getting media info when some files added as ref #4 (deadlock while redirecting process strout/stderr)

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

fifonik wrote on 10/9/2020, 1:05 AM

Version 0.8.5b published. Changes:

- New: Colour format displayed in Ref's media info

- New: Media info displayed not only for Ref, but for processed files as well (full info available as ToolTip)

- Change: When Ref changes, calculated metrics/graphs are resetting

- Change: Calc button does not reset already calculated metrics/graphs

- Change: Colours are tied to file numbers. When you tick/untick files, lines on graphs will not change colours

- Change: '20 seconds' and '2 minutes' entries added to dropdown with processing lengths

- Change: FFMpeg version detection routine improved

- Bugfix: Manual editing Ref's path does not work correctly

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17

fifonik wrote on 10/24/2020, 2:29 AM

Version 0.8.6b published. Changes:

- New: Program also get & display (in ToolTip) min, max, avg, std. dev. pop values based on frames' info.

- Change: webm added on video file extensions list

- Bugfix: version detection fail for some ffmpeg builds

Last changed by fifonik on 10/24/2020, 2:30 AM, changed a total of 1 times.

Camcorder: Panasonic X920

Desktop: MB: MSI B450M MORTAR TITANIUM, CPU: AMD Ryzen 3700X (not OC), RAM: G'Skill 16 GB DDR4@3200 (not OC), Graphics card: MSI RX580 8GB (factory OC), SSD: Samsung 970 Evo+ NVMe 500MB (OS), HDDs: Seagate & Toshiba 2TB, OS: Windows 10 Pro 1909

NLE: Vegas Pro [Edit] 11, 12, 13, 15, 17