The slow-loading Matroska bug has been fixed by the incredible developers on the ffmpeg team (see the link in the post above for ticket details). This fix changes the entire discussion. MKV would now be the recommended export preset and Edit Friendly format for all lossless codecs like HuffYUV and Ut Video in order to preserve all color metadata. This would require the next release of ffmpeg to be bundled with Shotcut in order to work, of course.
I realize I’m not a developer and I’m in no position to call any plays. I’m just raising awareness to what is possible now that ffmpeg is working as expected.
EDIT: I reviewed the thread from the top and realized D_S should be added to the at-list as well.
But the newest release of ffmpeg literally just came out a month ago and Dan just upgraded to it for the upcoming Shotcut release. From what I can see on their main page, ffmpeg don’t release frequent updates often. It seems to be 2 a year. When will this actually take effect? A year from now?
Major releases are 6 months apart on average. Point releases are more frequent. This commit would most likely make it into the next point release. So a couple months at most probably.
Okay. Please post when that point release is out so that Dan can upgrade to it when it happens. As you said in the thread, that fix is a game changer. You might have to make a separate thread for it because threads usually automatically close after a couple of months or so here. In the meanwhile, is there anyway you can test that fix for yourself before the next point release just to see how it runs?
The fast way to verify whether it works is to pull a snapshot of their source code, compile it myself, and try it out.
The lazy way to verify whether it works is to wait for the next nightly build to appear at Zeranoe and then try out their pre-built executable. The last build was August 26 and new releases happen approximately every two weeks. https://ffmpeg.zeranoe.com/builds/
I will most likely go the lazy route. Because life. And because the Zeranoe build is more representative of the final ffmpeg product than any custom compilation I do on my own.
By “try it out”, I mean run the “Steps to Reproduce” that I mentioned in the bug ticket. I might also overwrite ffmpeg in my Shotcut folder with the new version to see if Shotcut plays nice with it.
That is awesome! I cannot wait until the new release. Typically, I stick to FFmpeg releases just to avoid surprises, but I am willing to switch to a pinned rev on git master until its next release.
The ffmpeg 20190909-9d1e98a shared nightly build is available at Zeranoe. I re-ran the “Steps to Reproduce” from the ffmpeg ticket I submitted and MKV now has zero load time. I also copied the ffmpeg DLLs and EXEs into the Shotcut 19.08.16 folder and Shotcut previewed the 50-minute MKV videos instantly. Previously, they took 40+ seconds to load.
I am not lobbying for or against the immediate inclusion of a nightly build. I’m just saying that the ffmpeg fix does indeed resolve the slow-loading Matroska issue, and future export presets could be designed accordingly.
For completeness, I toggled the -write_crc32 and -reserve_index_space options directly from ffmpeg while creating MKVs out of some hour-long videos, then created Shotcut MLT projects with multiple hour-long MKV videos in them to test load times. When comparing the project that reserved index space to the project that didn’t, both projects loaded in under two seconds, and the difference between reserved and non-reserved was sub-second, as in not worth the hassle to reserve it. I deduce that moving the cues to the front of the file is only of benefit to streaming environments.
Also, the amount of reservation space is way more than the ffmpeg documentation estimates. It required 2,283,309 bytes of index space for a one-hour long 30fps MKV video. If the video was 60fps, then the space required was double, and so forth.
I found no significant playback performance difference with -write_crc32 on or off at up to 1440p resolutions using Ut Video (about 2% CPU utilization difference). However, I continue to turn it off simply because I don’t need it.
Long way of saying… the MKV defaults are totally usable now. Export presets should not need any muxer shenanigans to get top performance anymore.
The first obstacle was already listed… getting color space and color range captured in metatdata for the lossless codecs.
The second obstacle (albeit related) was transcoding YUV video with an alpha channel to a lossless codec. There are only a handful of codecs that support yuva420p or yuva422p, most of which are lossy and not very desirable, like VP8. The one lossless codec that supports those pixel formats is ffvhuff, but I can’t use it without color metadata. MagicYUV supports yuva444p, but I don’t want to haul around 4:4:4 in a proxy.
Fortunately, the fixes to ffmpeg clear all of these obstacles. It’s smooth sailing from here.
