Hi pdr, it’s so good to know you’re still on here. Thanks for tracing through my results and asking some great questions.
ffmpeg itself is the only convenient tool I have for testing this. I lifted a section of video from the 50 minute MKV transcode I made earlier:
ffmpeg -ss 00:45:00.000 -i BigHugeVideo.mkv -t 10 -c copy [output file stuff here]
The ffmpeg command acted just like the Shotcut preview player… a very long pause (40+ seconds) before transcoding began. So ffmpeg appears to be the bulk of the problem.
This would be my guess. I further think it is specific to MOV in MKV. As a total last ditch resort, I transcoded the 50 minute video to VP8 in WebM in Matroska. The VP8 file loaded instantly and seeks instantly. So apparently the Matroska format works fine with WebM. But DNxHR, HuffYUV, and Ut Video do not work in Matroska, and the MOV container is the common thread.
Yes. And they are both MOV-in-Matroska. MOV is automatically selected by ffmpeg/Matroska when the file is written. Is there a way to force Matroska to choose raw VfW instead of MOV? I tried the -allow_raw_vfw muxer flag and it made no difference.
Great question. I took the 50 minute DNxHR-in-MOV transcode I made earlier and -c copy-ied it into Matroska. The original MOV worked brilliantly, so maybe slipping it pre-built into Matroska would retain its performance. Nope. It was right back to slowness once in the MKV. I wonder if the Matroska muxer is trying to read/write the MOV itself, or if it’s doing a handoff to the “real” MOV muxer in ffmpeg and the handoff is slow.
That’s the million dollar question. AVI files don’t have this delay, so that somewhat rules out audio peak generation and other tasks.
Excellent question. I’m familiar with mkvmerge in concept, but not enough to do serious work. This would be a brief science project for me that I’d have to attempt a little later. The difficult part is that small MKV files load and play just fine. The performance drop-off increases drastically as a function of video length, meaning it takes a 30+ minute video to really notice there’s a problem in the first place. Once the video gets to 50+ minutes, the performance dramatically gets worse. So every change of settings requires a 30+ minute transcode to really know if there was an improvement or not.
Since VP8-in-WebM-in-MKV works as expected, it’s hard to know if this is a MOV-in-Matroska format issue or a ffmpeg parsing issue.
Interesting issue with the checkerboard pattern. I’m not too alarmed by it because I accept that any format short of lossless is going to have some kind of flaw. I am comfortable with the compromises that DNxHR makes when it comes to working in the ffmpeg/MLT/Shotcut environment. The encode time is 3x faster than ProRes on my computer, which is very appealing. If I were in a different environment, I would be a bigger fan of ProRes instead.
I use swscale’s sws_dither=ed in my scripts, but that’s for lack of a better alternative. It just occurred to me that I might be able to pass that line in an export preset as a global option and trick all of Shotcut’s scalers into using error diffusion. I’ll have to try that.
Thanks for the background on the avisynth options.
I do love x264. It’s a workhorse. But the intra option is just too slow on my old hardware. I can do one track of intra x264 just fine, but two simultaneous tracks (for a dissolve transition or a multi-cam scene) brings the preview to a halt. I also have a trade-off between quality and playback speed. If I get speed and lose quality, then my shadows are all blocky with dancing noise and I can’t color grade it. I do wish my hardware could handle multiple streams of x264 because that would solve everything, like you said.
Dan has outlined what it would take to make this happen. It sounds complicated. 
Thanks again for all your input.
For anyone wondering why I’m even researching all this, the intent may have gotten lost in all the details. The OP talked about changing the export preset for HuffYUV from MKV to AVI. Given how bad the performance of MKV is, my conclusion would be that AVI is the way to go for HuffYUV, with the caveat that color space and color range issues could happen if not careful. After all, without the performance of AVI, the MKV file is useless on its own.