A Scripting engine for Shotcut?

Here is what I have been thinking about so far. Like I said, parts of Shotcut are already scripted–in JavaScript, mostly within QML. Today, one can write a custom filter UI to do whatever–either upon selection or behind a button–and put it into the correct install directory. You must specify some filter from the engine, and if you use brightness with no arguments/properties it does nothing. Now, what can you do? That is what I meant above with “There is no documented and stable interfaces it uses, which is intentional still.” :slight_smile: It can be gathered from the existing Shotcut code, and I am sure an AI would help with that. But it is limited. For example, you could not write something to find all the subtitle items and split long text into multiple lines because that is all internal C++ data. Also, you cannot modify the menus or toolbars; it could only make its own within its designated UI rectangle. Scripting can only do what has only been exposed to it through interfaces. We might be able to easily add all of the actions you see in Help > Actions and Shortcuts in addition to everything else Timeline, Filters, and Keyframes has access to today. Plus more added over time as seen fit.

Another place there is scripting today is in File > Export > Markers and EDL menu items. That is a little different than the QML and has very little internals exposed to it. Today, those scripts are based around the idea of processing the project XML file. Now, a new script in that manner could do what Shotcut MLT Toolbox v1.0.0 – Subtitle & Simple Text Filter Line Breaks is doing, except in JavaScript instead of Python. However, I am not going to add a script like that because it is a poor UX. It would be better to provide something in the Subtitle panel’s menu to split long items.

Not all operating systems and app package formats make it possible–or at least very convenient–to add files to the app bundle or install directory. Also, you do not want to worry about losing custom work on uninstall or reinstall. So, I have thought there could be an extensions/filters subfolder in the app data directory and scan those on startup in addition to the installed share/shotcut/qml/filters folder. Perhaps there can be another folder to hold general scripts, but where is the UI for them? Maybe we add a new Scripts panel that is much like Filters except it does not include a picker menu. It simply lists the scripts, you pick one, and it shows its optional UI and an Execute button.

With respect to scripting effects (filters and transitions), I am generally opposed to it for reasons of performance, reliability, and maintenance. It is OK in GIMP because it only needs to deal with one image, and it is OK in Audacity because it is limited to audio (much less data) and non-realtime destructive effects. There is a reason none exists today in FFmpeg or Frei0r, two of our primary sources of effects. However, there is one exception that is common in video: custom shaders. See https://www.shadertoy.com/ for example shaders. There are non-standard extensions for FFmpeg to handle custom shaders. However, they rely on some GPU integration, which makes it tricky cross-platform and cross-GPU-vendor: see the FAQ. These extensions (as well as OpenFX GPU plugins) raise performance issues due to multiple bandwidth-hungry uncompressed video transfers between CPU and GPU memory over a bus, often PCIe. I will put on my to-do list to look into a way to run custom shaders within Shotcut’s existing GPU Effects framework that is better managed as it groups things on the GPU.

3 Likes