Sorry, you are correct. I meant "vmaf-models" folder (not "vmaf-metrics"). So you do actually have the folder (with metric files in it) in the program folder?

Do you have any non-English characters in program path?

You can try to close the program, delete ffmetrics.log file and run the program with option "--log-level=debug". In this case more information will be written into the log so it might give some clues.

You can also archive the program dir with ffmpeg.exe you using (no any media files needed), upload it onto dropbox or similar web-site and send me private message with the link. I should be able to download and investigate what is going on then.

Thanks.

Fwiw, I got that same warning message in the log myself about my ffmpeg not supporting vmaf. The vmaf-models disappeared from the ffmetrics screen and the box unchecked itself. Also wasn't getting the usual preview of the reference clip. This was after it was running fine just the night before. Turned out it was because my laptop was on battery and running slower in power-save mode. Response timeouts were apparently the culprit. Plugged it in and all was well again.

If I remember correctly, in this situation an error message should be added in log file.

Situation when there is just no 'vmaf-models' folder -- error message not added as this is considered as a setup for people who are not interested in VMAF at all.

Hi fifonik, thanks so much for creating this program and help me to solve the disable of VMAF.
Situation I met is that VMAF is disabled in my FFMetrics program，and I changed a new ffmpeg 4.4 download section on ffmpeg web site ( http://ffmpeg.org/download.html ).

Thanks again~😀

Version 0.9.8b published
-    New:    Percentile 1, 5, 10, 25 calculated and displayed in metric value Tooltip
-    New:    Json VMAF model files are checked now that they are actually in json format (invalid models not shown in dropdown)
-    Change:    Improved enabling/disabling logic for buttons "Save results" & "Extract bad frames"
-    Bugfix:    Values in metric value Tooltip are highlighted for unticked files
-    Bugfix:    Message in metric value field is truncated (only up to 5 digits shown in messages like "Frame: nnnnnnn")
-    Bugfix:    Crash under some circumstances while extracting bad frames
-    Bugfix:    FFMpeg command to extract bad frame fails if system decimal separator is not point
-    Bugfix:    VMAF model is not restored on program restart as other options

Version 0.9.9b published

-    New:    The best and the worst values per metric highlighted in the grid
-    New:    New project option: "Skip" -- a duration that will be skipped at the beginning of video stream. Note: only a few small values are in dropdown as the skipping process is slow
-    New:    Metric calculation time is measured and displayed in ToolTip
-    New:    Files on the list can be re-arranged using Drag & Drop
-    Change:    Timeout for process of getting FFMpeg version info increased from 2 to 5 seconds
-    Change:    Timeout for process of getting supported VMAF model types increased from 2 to 5 seconds
-    Change:    Options "Scaling method" and "Write FFMpeg Commands in log" are program options now, not project options. So they are not stored in project file any longer.
-    Change:    Option "Analyze" renamed to "Duration"
-    Change:    Values in "Skip" and "Duration" dropdowns that will go out of ref's file duration marked in red
-    Bugfix:    Frames' Axe on plot not reset if calculation cancelled and no data gathered

Version 1.0.0 published
-    Change:    Plot line ToolTip shown on mouse hover instead of mouse click
-    Change:    Plot line ToolTip contains file name
-    Change:    Graph colours adjusted
-    Change:    While Drag & Drop distorted files, dragged file moved to target position and other files are shifted (previously dragged file was swapped with a file on target position)
-    Bugfix:    "Scaling method" and "Write FFMpeg Commands in log" are still saved to project file

Hello. I'm currently trying to batch file PSNR comparaison between files. I would lie FFMetrics to write the results in a file created in the same file directory as my bat file. The analysis works fine but no file is created to save the results. What is my error?

This is what I wrote in my BAT file :

FFMetrics.exe -metric=PSNR -save-results -run %1 "%~dpn1_001.mp4"

I've also tried :

FFMetrics.exe -metric=PSNR -save-results-file=%cd%\file.csv -run %1 "%~dpn1_001.mp4"

But no file is created.

Looks like a bug.

Unfortunately, my current dev version is far ahead and there is no reason to investigate what was wrong with the version 1.0. Too many changes. Sorry.

P.S. PM me if you're interested in testing new version.

Hey guys, i am not sure i can understand the results.. i have outputted the same video file with magix NVENC accelerated codec at 12 Mbit (youtube recommended) and with voukoder at similar settings.

So do the results show the voukoder variant is much better than magix or just marginally better..? Thanks !

You should not use the program like that.

Take original file (from camcorder) and encode it twice -- with nvenc and with vouk.

Then add original file into the program as ref and 2 encoded files and then calculate metrics. This way you will be able to say that encoder A with settings XYZ encoded better/worse than encoder B with settings ZYX

Version 1.1.0 beta 1 published

-    New:    OxyPlot library upgraded to version 2.1.0. Much faster renderer used (vital when many long files analyzed in one go)
-    New:    Plots moved to separate window that opened manually or automatically when metrics calculation started. So more room for distorted files list that is handy for someone who do not care about graphs and only looking on per file metrics (#44)
-    New:    Plots can be saved as SVG (smaller file size and better quality when upscaled)
-    New:    Plots update frequency can be adjusted using menu: "Options" | "Plot update speed" or using command line parameter "-plot-update-speed=HIGH|NORMAL|LOW|OFF"
-    New:    Plot line styles can be configured (see "FFMetrics.conf.example") (#71)
-    New:    New project options "Auto-save results on completion" and "Results auto-save file" (#18)
-    New:    A number of distorted files displayed in program main window title
-    New:    New command line parameter "-temp-dir=C:\Path\To\Temp". The program will try to create the folder if required (#75)
-    New:    Command line parameters can be specified in *nix style with double dash: "--param=value"
-    Change:    Maximum number of distorted files increased from 12 to 24 by default. User may further increase it by manually configuring plot line styles (see "FFMetrics.conf.example") (#3)
-    Change:    While extracting bad frames, program also extracts and save the same frames from reference file (#26)
-    Bugfix:    Files are incorrectly updating on plots while re-ordering
-    Bugfix:    Extracting bad frames is not accurate (#78)
-    Bugxif:    Unable to drop file dragged from Windows Explorer to ref input (#77)
-    Bugfix:    Crash while dragging item from distorted files list over blank Ref field (#79)

Version v1.2.0 beta 1  published

- New: Libvmaf options that should be used for checking supported vmaf models can be defined (see "FFMetrics.conf.example") (#81)

- New: Built-in ffmpeg vmaf models can be used (see "FFMetrics.conf.example") (#81)

- Change: FFMetrics.conf format is changed. Case sensitive metric properties 'Filter' and 'Lavfi' must be used instead of 'LavfiTemplate' (see "FFMetrics.conf.example"). You need to adjust your FFMetrics.conf is you use one

- Bugfix: Metric model property "default" was not correct parsed while using in FFMetrics.conf

Version v1.2.1 beta 1 published

- New: Temp dir can be specified in FFMetrics.conf (see "FFMetrics.conf.example") (#83)

- New: FFMpeg colon escape characters can be specified in FFMetrics.conf (see "FFMetrics.conf.example") (#84)

- Change: With command line parameter "-log-level=debug", option 'Write FFMpeg commands into log' is turned on automatically and disabled in UI (#82)

- Change: model_path and log_path enclosed in single quotes separately

I'm getting some pretty different vmaf results depending on whether I have ffmpeg v442 or 511 in my path. I'm selecting the same "vmaf_4k_v0.6.1.json" model for both but the vmaf reported by 511 is significantly higher. The 442 vmaf numbers are the same as I've measured in the past. There's also another model listed for 511, "vmaf_4k_v0.6.1" (no .json), and that reports an even higher number. Are the numbers reported by the 511 version more meaningful, in your opinion?

@Howard-Vigorita when you look at the graphs between the 2 versions, are you seeing any of the pulsing up and down as seen in the pictures @dape shows. Is it evident only in v442?

Unfortunately, I cannot comment on different values.

I was playing with ffmpeg5.0.x a little, but did not notice any metrics differences on my sources. Probably you need to run the ffmpeg commands manually (you can copy them from log) in console and double check results.

I can only guess that the differences related to colour range (discussed earlier in the thread already) that version 5 is handling differently.

As for vmaf model without file extension: you probably using the most recent version of ffmetrics that is detecting vmaf-models in-build into ffmpeg.exe. However, I have no idea why the values are different. They should be the same, unless VMAF developers modified these models. They were the same in my tests when I compared in-build models vs json models with ffmpeg5.0.x.

P.S. For encoders comparison I'd use 'neg' vmaf-model. Do not remember if it is available for '4k'.

Thanks. Psnr and ssim metrics come out the same on all versions. Only vmafs are different. I'll just stick to using ffmpeg v4 w/ffmetrics which gives vmaf measurements on the same scale and weighting making them directly comparable with all the measurements I've taken previously.

@Howard-Vigorita when you look at the graphs between the 2 versions, are you seeing any of the pulsing up and down as seen in the pictures @dape shows. Is it evident only in v442?

@Former user No, the charts I get do have ups and downs but the range is more narrow. But I don't do it the way he did. I do most of the measurements using a pure lossless source as a reference and use that same source as the input for the transcode. One thing I do notice is that the 1st 90 frames of each chart are pretty funky. Probably computational anomaly.

The latest beta (v1.3.1 beta 2) is released as v1.3.1 stable.

vmaf is a marketing tool, not a benchmark imho.

Hey fifonik,

I stumbled across your FFMetrics on my research for approaches to spot differences between two video files.

FFMetrics appears to be a great piece of software which I could use off-label. The only thing I am missing is an individual input for the trim command in ffmpeg. Since I need to compare files with different pre-rolls it would be helpful to have the ability setting a trim for each file (original & distorted). E.g. the original video has a 30 second pre-roll (black, technical lead, etc) and the distorted video has a 10 second pre-roll.

Would be nice to have this extra input field, so the videos don't need to be trimmed externally for comparison.

Thanks a lot, and keep going!

As a current workaround you can try to install avisynth and create a single .avs file for the ref to cut first 10 seconds. Then you can ask FFMetrics to skip first 10 seconds. That should be easier then trimming all files externally. However, there is always a trick: as the metrics require frame accutacy, you will need to find out your exact trim duration.

As for implementation of your feature request -- unfortunately, adding this into UI will make the program much more complicated and as a "visual guy" who would like to keep the UI as simple as possible, I would like to avoid this. I will never add the skip options per "every distorted file". Sorry. I may consider separate skip durations (one for ref and another for _all_ distorted files), but will need to think about this.

P.S. I do have a ticket on github for implementing auto-sync. This would be better approach that is also help in your situation.