Is it possible to add custom video formats to MLT framework?

I’m interested in using the MLT framework for a professional project. We would need to ingest various RAW formats: RED, Arri, Canon, etc. If I have the SDKs for those, can those formats be added into MLT as producers? I haven’t seen any docs on adding camera SDKs or new formats but I’m probably just missing them. Any pointers appreciated!
Note that these are mostly 10-12bit log, if that makes any difference.
Ideally I’d like to propagate the camera metadata into MLT, and that certainly does look possible.

Sounds like a cool project! For clarification, are you wanting to use Shotcut as well, or just MLT Framework for some batch tasks or other external usage? Using Shotcut may impose some extra limitations that MLT itself may not have. (Shotcut is essentially the GUI and MLT is essentially the engine. You may already know all this… I’m just covering all bases.)

The first example to come to mind is the pipeline bit depth. Shotcut is 8-bit from start to finish. Bringing in log footage would mean using a LUT or some hard grading curves to get into Rec.709, which if done in 8-bit space, can result in some severe banding.

An alternative is to batch-convert the ARRI footage from log to 8-bit Rec.709 using a LUT in an FFmpeg script. DNxHR and ProRes are available as intermediate formats. Then hopefully minimal grading would be needed in Shotcut to prevent banding. (ProRes is 10-bit of course, but scales cleanly to 8-bit since the encoding is gamma at this point.) Any banding that did appear could probably be countered pretty well with the Deband filter available in Shotcut.

The other question would be whether your project needs anything more than Rec.709, which is the most Shotcut supports. There is currently not direct support for HDR or BT.2020 or P3 or other modern color spaces.

As a more specific answer to your question… Shotcut relies on FFmpeg for file decoding. So the raw readers would need to be added to FFmpeg rather than MLT or Shotcut directly. It’s technically possible to add the raw readers to MLT outside of FFmpeg, but it would be substantial effort.

Thanks very much! That is super useful.
Yes, we’d only be using MLT for batch + interactive processing, outside of shotcut.
Unfortunately if MLT is 8bit only, it won’t work for our project. We definitely need a minimum of 12 bit, including HDR. (Ideally float or half float, but fix point is OK). We’re going fully color managed, OCIO, ACES, etc.
I’ll look into adding our codecs to FFMpeg; that sounds like the way to go.

This topic was automatically closed after 90 days. New replies are no longer allowed.