And it’s always a relief to know someone took time to read them. Yikes, they get long. 
Thank you.
Makes a lot of sense. I recently finished some PowerShell scripts that walk through MLT playlists for another purpose, and detecting stacked video with filters is not terribly complicated.
Yeah, that’s pretty sophisticated and really cool. It’s amazing that Shotcut can do what it does with 0.00037446% the developer resources of the FCPX team. Says a lot about how good Dan is.
Unless the data rate of the cache file maxed out the bandwidth to the external drive, which can happen at 4K over USB to a magnetic drive given the right (or in this case, wrong) codec.
It would be cool to have all those options. It would scale well to whatever the capabilities of the computer were, from low end to high end.
The lossless codecs I’m referring to are Huffyuv, Ut Video, MagicYUV, and FFVHuff. They are some of the fastest codecs that FFmpeg offers. But they generate huge files that stress the storage bandwidth, and your bottleneck may have happened there. The other lossless codec, FFV1, is wretched slow but that was already specifically mentioned.
That would certainly work and be foolproof, but it would lose a tremendous opportunity to speed up the export of a large project. We’re all dreaming here anyway, so keeping the cache if the MLT is unmodified is a way of dreaming even bigger.
I attempted hacking this into Shotcut a couple years ago and ended up not being a fan of it. The issue was that “actually used proxy segments” is a very fluid concept during editing. I would extend a clip here, delete one here, add a brand new section over there, then had to wait for a proxy to build… the initial editing phase was too random and chaotic on the proxy segments. Having a full-length proxy meant I could add any part of any video at any time with zero slowdown for transcoding. It was really hard for me to give up that instant workflow once hooked on it. Other people might be okay with it, though.
The file descriptors for chunks in the active playback area would need to be kept open for performance reasons. If a chunk needs to be played but isn’t already open, then FFmpeg has to open a file descriptor, detect the media type (this is not fast), hand it up to Shotcut which has its own frame cache currently, then playback can begin. It’s just like opening a large MLT file… Shotcut sits for a little while detecting all the media and registering video dimensions and other metadata before letting a user edit. This same delay would chew up playback of the chunk system unless the file descriptors were held open in the active playback area to avoid re-detection.
I 100% agreed with everything you wrote after that line, so it sounds like we’re on the same page. I think @DRM and I were being extra optimistic about what the cache could do if given time to fully compile and if it persisted between editing sessions.
Important point. Good catch.
Indeed, he has been notably silent so far. 
Is now a good time to remind everyone where the Donate page is?