A Scripting engine for Shotcut?

Suggestion
1

Hi there, my apologies for being such a PITA, but I’m curious as to whether there are any plans to enable users to extend Shotcut’s functionality or prototyping features that can later be ‘hard-coded’ (including the UI) by way of a scripting language like Python/Python Qt or JavaScript.

DaVinci and Blender use Python, Premiere JavaScript; Kdenlive is planning to implement a Python scripter; Inkscape, Krita and Scribus are using Python, whereas GIMP uses Python alongside its own “Script-Fu” engine.

I know that this would require a lot of work (perhaps as an MA project in CS), but it would be worth it, considering how many great extensions the scripting engines have brought to the fore in these products/projects (e.g. the “Resynthesizer” plug-in for GIMP).

2

I do not like to talk much about big future plans. Some things are already scripted within UI components. See share/shotcut/qml. There is no documented and stable interfaces it uses, which is intentional still.

Do you really want to script the UI? You can script the engine already. See Flowblade, which uses the Python API for the common engine.

3

Hi Dan,

My apologies for being too aspirational (if that’s the appropriate word), but I can’t read other people’s minds, so I couldn’t know what you like or don’t like. I just want this great project to succeed and improve.

However, I do know a few other things. One of them is that, in my experience, Shotcut is for most people the best and easiest way to learn professional-grade video editing without having to pay licensing fees or for expensive training. I also know from experience that the UI is generally well thought-out, and the learning curve is much lower than for, e.g., Avid, Premiere, Kdenlive or DaVinci. Now that you have added Bins and Subtitle Editing, it’s finally in the professional realm. In addition, it’s incredibly stable – I haven’t encountered a crash despite heavy CPU/GPU/memory use in the last two or three years.

From casual conversations I learned that, when I mentioned the virtues of Shotcut, one question was always: Can it be scripted for automation and other tasks? It turned out that advanced users wanted two features: 1) a scripting console for quick tasks, with the option to save the script (very important) and 2) a scripting interface for plug-ins. The latter is important, because ‘native’ plug-ins written in C, C++ or whatever language need to be compiled for each platform separately, whereas a Python or QML script works without compilation, as you know. In short: scripting console and easy plug-in options via scripts (Python seems to be the favourite).

As I’ve written in another post, this is not a challenge for you or any other current developer, so you don’t have to feel obliged to implement my suggestion. My hope is that someone picks this up and creates the functionalities I described, perhaps as a subject of an MA degree project in CS, or just because it’s an interesting challenge. There may also be commercial sponsors who need these functions and are willing to spend some money on it. Who knows!

I don’t know enough about the architecture of Shotcut, which seems to be a bit complicated (ffmpeg, MLT, Qt …), but I’d love to see some outsider sink his teeth into the whole thing and offer a workable solution for inclusion into Shotcut. It would then be up to you to decide whether this will become part of Shotcut or not. That’s the beauty of Open Source development. :slight_smile:

As I’ve written before: sowing seeds to be reaped later and perhaps unexpectedly. :slight_smile:

1 Like
4

I assume these same scripting hooks could be used to host or implement an MCP server. And maybe replace AutoHotKey for @jonray. His head would explode with happiness.

1 Like
5

Ha ha! My beloved Auto HotKey!! :grin:

6

Here is what I have been thinking about so far. Like I said, parts of Shotcut are already scripted–in JavaScript, mostly within QML. Today, one can write a custom filter UI to do whatever–either upon selection or behind a button–and put it into the correct install directory. You must specify some filter from the engine, and if you use brightness with no arguments/properties it does nothing. Now, what can you do? That is what I meant above with “There is no documented and stable interfaces it uses, which is intentional still.” :slight_smile: It can be gathered from the existing Shotcut code, and I am sure an AI would help with that. But it is limited. For example, you could not write something to find all the subtitle items and split long text into multiple lines because that is all internal C++ data. Also, you cannot modify the menus or toolbars; it could only make its own within its designated UI rectangle. Scripting can only do what has only been exposed to it through interfaces. We might be able to easily add all of the actions you see in Help > Actions and Shortcuts in addition to everything else Timeline, Filters, and Keyframes has access to today. Plus more added over time as seen fit.

Another place there is scripting today is in File > Export > Markers and EDL menu items. That is a little different than the QML and has very little internals exposed to it. Today, those scripts are based around the idea of processing the project XML file. Now, a new script in that manner could do what Shotcut MLT Toolbox v1.0.0 – Subtitle & Simple Text Filter Line Breaks is doing, except in JavaScript instead of Python. However, I am not going to add a script like that because it is a poor UX. It would be better to provide something in the Subtitle panel’s menu to split long items.

Not all operating systems and app package formats make it possible–or at least very convenient–to add files to the app bundle or install directory. Also, you do not want to worry about losing custom work on uninstall or reinstall. So, I have thought there could be an extensions/filters subfolder in the app data directory and scan those on startup in addition to the installed share/shotcut/qml/filters folder. Perhaps there can be another folder to hold general scripts, but where is the UI for them? Maybe we add a new Scripts panel that is much like Filters except it does not include a picker menu. It simply lists the scripts, you pick one, and it shows its optional UI and an Execute button.

With respect to scripting effects (filters and transitions), I am generally opposed to it for reasons of performance, reliability, and maintenance. It is OK in GIMP because it only needs to deal with one image, and it is OK in Audacity because it is limited to audio (much less data) and non-realtime destructive effects. There is a reason none exists today in FFmpeg or Frei0r, two of our primary sources of effects. However, there is one exception that is common in video: custom shaders. See https://www.shadertoy.com/ for example shaders. There are non-standard extensions for FFmpeg to handle custom shaders. However, they rely on some GPU integration, which makes it tricky cross-platform and cross-GPU-vendor: see the FAQ. These extensions (as well as OpenFX GPU plugins) raise performance issues due to multiple bandwidth-hungry uncompressed video transfers between CPU and GPU memory over a bus, often PCIe. I will put on my to-do list to look into a way to run custom shaders within Shotcut’s existing GPU Effects framework that is better managed as it groups things on the GPU.

3 Likes
7

I can’t think of any commercial program that provides this much insight and consideration for a user request. Very nice.

4 Likes
8

I concur!

9

Hi,

Thanks for your detailed reply!

As regards your third paragraph, there is an easy (?) solution that can be found in Scribus. If you edit a document in this software there’s an active menu bar item called “Script” with a sub-entry called “Execute Script …”. Click on it, and a file dialogue opens that lets you select and load a Python script into Scribus. If it doesn’t work (e.g. if it was written for another application), you’ll get an error message. There’s also another useful entry, called “"Recently Used Scripts”, which does exactly what it says. This could be a way to circumvent the current limitations of packaging formats.

What you didn’t mention directly, though, is the possibility of a scripting console. I’m not a scripting guy, but from what I’ve learned during conversations is that it is mostly being used for the automation of basic functions, like, e.g., colour correction of clips from the same camera, resizing etc. – basically batch processing. The scripting engines of programmes like Avid, DaVinci and Premiere are also limited to certain tasks.

As I said, I’m not familiar enough with the architecture etc. So these are just my thoughts on this topic.

1 Like