Blacklist for So4 - Includes fix for GoPro, DJI & Xiaomi Yi AVC lag

NickHope wrote on 6/6/2018, 10:14 AM

Edit (April 2024): Note that the So4 blacklist feature was removed in VEGAS Pro 21, as the newer mxcompoundplug became the default reader for AVC files. The feature is available in versions 15 to 20.

Short Version:

To solve sluggish playback of AVC files recorded by GoPro, DJI & Xiaomi Yi cameras in VEGAS Pro 15 build 361, move this file:

C:\Program Files\VEGAS\VEGAS Pro 15.0\FileIO Plug-Ins\so4compoundplug\so4_blacklist_vp15.xml

...to this folder:

C:\Program Files\VEGAS\VEGAS Pro 15.0\

Then restart VEGAS.

If it doesn't help, you almost certainly haven't lost anything.

This does not apply to HEVC files.

Long Version:

VEGAS Pro 15 has a new decoder, so4compoundplug.dll, that reads AVC and XAVC-S files. More details in this post.

Peculiarities in the structure of some AVC files, in particular those shot by GoPro, DJI & Xiaomi Yi cameras, mean that they currently perform poorly with this new decoder. Playback is sluggish and thumbnails build slowly. It is hoped that will be resolved in the future, but in the meantime a "blacklist" was hardcoded into VEGAS Pro 15 update 2 (Build 261) to force these formats to be decoded by the older compoundplug.dll, which enables them to play more smoothly.

In VEGAS Pro 15 update 5 (Build 361), this blacklist has been made into a separate xml file, C:\Program Files\VEGAS\VEGAS Pro 15.0\FileIO Plug-Ins\so4compoundplug\so4_blacklist_vp15.xml, containing this text:

<?xml version="1.0" encoding="utf-8"?>
<filter_expr type="OR" comment="exclude IDR/I files where M = 1">
<video Title="DJI.AVC" />
<video Title="GoPro AVC" />
<video Title="XiaoYi AVC " />
</filter_expr>

Unfortunately this file is not being read in its default location. For it to be correctly read, it should be moved to C:\Program Files\VEGAS\VEGAS Pro 15.0\ and VEGAS restarted. It is intended to correct this in the next update/upgrade.

Note that AVC formats from these 3 camera brands will not be blacklisted if the Title attribute in the file's metadata is missing or does not match the description in the xml file. Others may play fine with so4compoundplug anyway.

Of course, you also have the option of completely disabling so4compoundplug.dll, but then you will lose native support for some formats such as iPhone/iPad AVC, Panasonic GH5 10-bit 422 AVC, and JVC YUV 4:2:2 AVC.

Blacklisting or Un-blacklisting Formats:

Now that the so4 blacklist has been made into a separate, editable file, it means that formats can be added or removed from it. THIS SHOULD BE DONE WITH GREAT CARE. You will probably need administrator privileges to edit the file, and you may need to copy it to a folder such as your Documents folder, edit it there, then copy it back over the original file.

There is probably no need to remove one of the 3 standard blacklisted formats, but to do so simply delete or comment out the appropriate line. Take a backup of the original file first in case you need to restore it.

To add a format to the blacklist, look for the Title or other differentiating metadata using MediaInfo on a representative file. Then add an extra line. As an example, here is a version of the file that blacklists XAVC-S (but not XAVC-L or XAVC Intra, which are decoded by mxfxavc.dll). This could be very useful for someone who is having trouble with playback or slow loading of XAVC-S files in VP15, but does not want to disable so4compoundplug completely and so lose the support for extra formats:

<?xml version="1.0" encoding="utf-8"?>
<filter_expr type="OR" comment="exclude IDR/I files where M = 1">
    <video Title="DJI.AVC" />
    <video Title="GoPro AVC" />
    <video Title="XiaoYi AVC " />
    <!-- Formats below added by user -->
    <general Format="XAVC" />
</filter_expr>

It's a good idea to run the edited file with a tool like XML Lint or any other XML validator (e.g https://www.xmlvalidation.com/ ) to check the edits are safe.

Comments

Red Prince wrote on 6/6/2018, 11:13 AM

But do you need Raymond Reddington to access The Blacklist?

Former user wrote on 6/7/2018, 2:27 PM

Thanks to this post I was able to finally understand why in Vegas 15 Build 321 the images from my Phantom were running fine and in Build 361 it was crashing.

As I understand it, in Build 321 these Blacklist formats were encoded natively by Compoundplug and not by so4compoundplug and you could not change that.

In Build 361 these blacklist formats are now encoded by so4compoundplug natively, but Magix gave us the option to switch to compoundplug using Blacklist. I get it right?

NickHope wrote on 9/2/2018, 11:05 PM

From Vegas Pro 15 build 384, so4_blacklist_vp15.xml is placed by default in C:\Program Files\VEGAS\VEGAS Pro 1X.0\ where it can be read, so there is no longer any need to move it.

In VP16 it still has "15" in the filename.

NickHope wrote on 10/9/2019, 12:53 AM

Hardware decoding of AVC requires the so4compoundplug decoder, but the blacklist disables so4compoundplug, even with hardware decoding enabled (Preferences > File I/O).

Some users of VP17 (e.g. and another 2 below it) have reported improved decoding of DJI/GoPro/XiaoYi AVC on NVDEC-capable NVIDIA GPUs by removing C:\Program Files\VEGAS\VEGAS Pro 17.0\so4_blacklist_vp15.xml.

If you want to try it, I recommend you rename that file to something like so4_blacklist_vp15.xmlDISABLED instead of removing it completely.

It's possible that decoding of DJI/GoPro/XiaoYi AVC is also improved with some QSV hardware decoding, and that has been available since VP15. Certainly worth a try if you use those formats and have a CPU with a good QSV capability. Besides the File I/O tab preferences in VP17, you need to check "Preferences > General > Enable QSV encoding and decoding (where available)" in VP15/16/17.

The developers are aware of this and may consider changing the default blacklist behaviour in a future version.

Please comment with your findings below if you try it.

David Johns wrote on 10/22/2019, 11:31 AM

Nick, I hope this isn't a stupid question (and please delete if it is or if it's inappropriate) but can you clarify the connection between the File I/O "Enable Hardware Decoding" setting and the Video "GPU acceleration of video processing" options.

For example, must the latter be set to something other than OFF in order for the former to have any practical use? Or indeed vice versa?

If there is no connection betwee them then what does the File I/O setting do that the GPU acceleration setting doesn't?

And, finally, do both of these relate to timeline playback only or to rendering as well (in which case, how does the choice of hardware encoder in the render template come into play?).

It's very confusing but it could be that I'm just thick.

Thanks

David

NickHope wrote on 10/22/2019, 12:29 PM

Nick, I hope this isn't a stupid question (and please delete if it is or if it's inappropriate) but can you clarify the connection between the File I/O "Enable Hardware Decoding" setting and the Video "GPU acceleration of video processing" options.

@David Johns The File I/O "Enable Hardware Decoding" setting uses the NVDEC technology in supported NVIDIA GPUs, or the QSV technology in supported Intel processors, to decode supported formats. Those formats are currently AVC and HEVC (but perhaps not all variants of those).

"GPU acceleration of video processing" uses OpenCL to accelerate playback of the timeline and GPU-enabled FX. OpenCL is available on AMD and NVIDIA GPUs, but AMD's implementation has generally worked better with VEGAS.

For example, must the latter be set to something other than OFF in order for the former to have any practical use? Or indeed vice versa?

I don't think so. I assume both can be ON if you have the technology to support it. I don't have a GPU capable of NVDEC or QSV so I can't test it. Maybe others who do can share their experience on that.

And, finally, do both of these relate to timeline playback only or to rendering as well (in which case, how does the choice of hardware encoder in the render template come into play?).

They apply to rendering inasmuch as the timeline has to be decoded as the video is rendering, although neither setting applies to the renderer itself. That setting is controlled in the "Encode Mode" on the Render As "Custom Settings" dialog. You can more or less think of that as separate from the 2 decoding-related settings, although I suppose there might be some performance disadvantage to using either NVIDIA or Intel for both decoding and encoding.

It's very confusing but it could be that I'm just thick.

It is indeed very confusing and I don't have a complete grasp of it myself.

j-v wrote on 10/22/2019, 1:40 PM

Maybe others who do can share their experience on that.

It is indeed very confusing and I don't have a complete grasp of it myself.

I showed my experiences here many times.
Previewing and rendering goes much faster and better with both options on.
I also had never the need to switch that so4compoundplug off, always did its job more than sufficient.

Here are my screengrabs with OBS during playing of previewing both possibilities on and off and the rendertimes of such an 1 minute 4K testproject with heavy GOPro 7 4K HEVC files, previewed in preview full 50p 4K project and rendered to a Magix AVC 4K 50p file with the default settings

Previewing

Rendering

Former user wrote on 10/22/2019, 4:55 PM

Hardware decoding of AVC requires the so4compoundplug decoder, but the blacklist disables so4compoundplug, even with hardware decoding enabled (Preferences > File I/O).

Some users of VP17 (e.g. and another 2 below it) have reported improved decoding of DJI/GoPro/XiaoYi AVC on NVDEC-capable NVIDIA GPUs by removing C:\Program Files\VEGAS\VEGAS Pro 17.0\so4_blacklist_vp15.xml.

From what I've been able to deduce XiaoYI action cameras were added to davinci Resolve Hardware decoder blacklist due to File Size - MAX crashing it. If file size was 4GB or lower did not crash. To fix problem people re-wrap h.264. Can anyone test file size max with vegas after removing blacklist?

NickHope wrote on 10/23/2019, 12:31 AM

Hardware decoding of AVC requires the so4compoundplug decoder, but the blacklist disables so4compoundplug, even with hardware decoding enabled (Preferences > File I/O).

Some users of VP17 (e.g. and another 2 below it) have reported improved decoding of DJI/GoPro/XiaoYi AVC on NVDEC-capable NVIDIA GPUs by removing C:\Program Files\VEGAS\VEGAS Pro 17.0\so4_blacklist_vp15.xml.

From what I've been able to deduce XiaoYI action cameras were added to davinci Resolve Hardware decoder blacklist due to File Size - MAX crashing it. If file size was 4GB or lower did not crash. To fix problem people re-wrap h.264. Can anyone test file size max with vegas after removing blacklist?

@Former user Do you mean just a re-wrap from MPEG-4 to MPEG-4? (for example using ffmpeg)

I've been told that what the 3 formats in the default blacklist have in common is that their I-frames are not marked correctly in their MP4 tables. i.e. The information in their STSS box is not accurate. This may be because they are using the same (flawed) library.

I can confirm that the following ffmpeg rewrap command does not improve so4 playback of a DJI AVC file, even at less than 4GB:

ffmpeg -i original.mp4 -c:v copy -c:a copy rewrapped.mp4

Former user wrote on 10/23/2019, 4:34 AM

Thanks for the correction. earlier Davinci Resolves would crash when importing some action cam footage, unrelated to hardware decoding. The fix was to rewrap as you did. I had thought that is why Hardware decoding is still turned off on resolve for certain cameras such as Yi 4k , and it was to pevent crashes but you have explained the real reason. This is thread about rewrap solution

https://forum.blackmagicdesign.com/viewtopic.php?f=31&t=62073