Command line rendering


#1

I’m not really sure if this is a “How To” or a “Suggestion”. But I can’t find the option so…

I’d like to batch render a bunch of videos as on a headless machine (remote command line access).

As it stands I can:

  1. Start exporting on my laptop
  2. Copy the /tmp/shortcut-*.mlt file
  3. (On the remote machine) call: LD_LIBRARY_PATH=Shotcut/Shotcut.app/lib Shotcut/Shotcut.app/qmelt -progress $HOME/shotcut-N27177.mlt

The massive pain here is kicking off the rendering process to get the .mlt file (steps 1 and 2) as they grind my machine to a standstill.

Is there / could there be an option just to export a “render-ready” .mlt file complete with rendering options?


#2

Save the project. It saves .mlt files.


#3

They’re portable too as long as the folder structure remains the same, I’ve been able to save a .mlt in the folder with my source files and copy it to a different system to handle rendering(even a linux system)


#4

Thanks. I spotted that both are .mlt files and my first test test was to compare the .mlt file from /tmp with my saved project file. They don’t match. By eye there are at least four differences:

  • Export settings (format and rendering options) are added
  • Relative paths are swapped for absolute paths.
  • I see at least one other difference that I think has no effect (an added attribute to the black background).
  • some abbreviated tags are expanded (<foo /> is replaced with <foo></foo>)

I might be able to hand craft the changes if I knew for certain what they all were. But I have no confidence that what I produce will match what shortcut would otherwise export. The differences are slightly illogical and I have no complete list of changes.

That’s what I meant by a “render-ready” .mlt file as opposed to a project .mlt file. I want a copy of the one produced by the export writer.


#5

I see.

Looks like a year ago no support was provided on a possibly related question: Exporting using the commandline

running qmelt -h makes me notice

-profile name Set the processing settings

Perhaps you can save an export preset as a text file and feed it to this parameter to input encoding settings


#6

On second thought, this might not work or only work if the profile is saved in shotcut’s folder.

I just noticed

-serialise [filename]                    Write the commands to a text file

Perhaps you can use this on one of the /tmp/ .mlt files to find out what kind of commands get created and then copy them into a script?

For reference, qmelt or melt help page ( like manpage perhaps)

$ ./qmelt.exe
Usage: qmelt.exe [options] [producer [name=value]* ]+
Options:
  -attach filter[:arg] [name=value]*       Attach a filter to the output
  -attach-cut filter[:arg] [name=value]*   Attach a filter to a cut
  -attach-track filter[:arg] [name=value]* Attach a filter to a track
  -attach-clip filter[:arg] [name=value]*  Attach a filter to a producer
  -audio-track | -hide-video               Add an audio-only track
  -blank frames                            Add blank silence to a track
  -consumer id[:arg] [name=value]*         Set the consumer (sink)
  -debug                                   Set the logging level to debug
  -filter filter[:arg] [name=value]*       Add a filter to the current track
  -getc                                    Get keyboard input using getc
  -group [name=value]*                     Apply properties repeatedly
  -help                                    Show this message
  -jack                                    Enable JACK transport synchronization
  -join clips                              Join multiple clips into one cut
  -mix length                              Add a mix between the last two cuts
  -mixer transition                        Add a transition to the mix
  -null-track | -hide-track                Add a hidden track
  -profile name                            Set the processing settings
  -progress                                Display progress along with position
  -remove                                  Remove the most recent cut
  -repeat times                            Repeat the last cut
  -query                                   List all of the registered services
  -query "consumers" | "consumer"=id       List consumers or show info about one
  -query "filters" | "filter"=id           List filters or show info about one
  -query "producers" | "producer"=id       List producers or show info about one
  -query "transitions" | "transition"=id   List transitions, show info about one
  -query "profiles" | "profile"=id         List profiles, show info about one
  -query "presets" | "preset"=id           List presets, show info about one
  -query "formats"                         List audio/video formats
  -query "audio_codecs"                    List audio codecs
  -query "video_codecs"                    List video codecs
  -serialise [filename]                    Write the commands to a text file
  -silent                                  Do not display position/transport
  -split relative-frame                    Split the last cut into two cuts
  -swap                                    Rearrange the last two cuts
  -track                                   Add a track
  -transition id[:arg] [name=value]*       Add a transition
  -verbose                                 Set the logging level to verbose
  -timings                                 Set the logging level to timings
  -version                                 Show the version and copyright
  -video-track | -hide-audio               Add a video-only track
For more help: <https://www.mltframework.org/>

#7

Yeah that fits with something I was just reading. I think that qmelt is an extension or modification of melt from the mlt framework and may work in a similar way.

melt would accept those settings in a .dv file. It’s unclear to me if qmelt does too.

If it does then I would copy those settings into a profile file (for example name: h264-1080p-hq.dv) And then call:

LD_LIBRARY_PATH=Shotcut/Shotcut.app/lib Shotcut/Shotcut.app/qmelt -profie=./h264-1080p-hq.dv -progress $HOME/my-project.mlt

I can only hope that (a) this works and (b) there is nothing else done by the export xml writer.


#8

No, a .dv file is something else entirely different.

LD_LIBRARY_PATH=Shotcut/Shotcut.app/lib Shotcut/Shotcut.app/qmelt …

No need to supply LD_LIBRARY_PATH as the wrapper script at Shotcut.app/qmelt already does that.

-profie=./h264-1080p-hq.dv -progress $HOME/my-project.mlt

profile is spelled wrong, and the .mlt XML file already includes profile information.

there is nothing else done by the export xml writer.

The export XML writer adds a <consumer> element that contains the export settings. You can alternatively supply a -consumer command line option, but that will take some time to teach you. Rather, you can right-click an export job and view the XML it generated, and save that to a file to run with melt or qmelt. qmelt is simply a wrapper to melt that initializes Qt on the main thread, where it prefers. This is needed on some platforms when using WebVfx (HTML filters) and/or Movit (GPU processing).


#10

You can alternatively supply a -consumer command line option, but that will take some time to teach you.

I don’t need to be taught, but I need to see a working example :slight_smile: If you could provide one that’d be great. All my other attempts produced segmentation faults.

… And just like that this turns back from a feature request to a “how to” … although a feature request to make this less convoluted still stands.

  1. Make sure the Jobs pannel is open (View --> Jobs)
  2. Press the “Pause” button on jobs.
  3. Export the video as required Where you export to is important. The command-line will use this location
  4. Right click the (“pending”) export job in the jobs panel --> View XML
  5. Press the save button and pick a location (use the .mlt extension not .txt)
  6. Again right click the job in the jobs panel --> Remove

To Render the resulting XML file:

Shotcut.app/qmelt -progress saved-xml-file.mlt

Hope this helps future readers.


#11

For what it’s worth, there are a bunch of examples on https://www.mltframework.org/faq/ - just ctrl+f search for -consumer on that page.

PS: could you explain what the difference is between rendering the /tmp/ .mlt and a saved .mlt? Is it just the render settings which one configured in the export ui?


#12

@nwgat built a working example with his benchmark here https://nwgat.ninja/thefireescape/ I used it in my testing to try rendering jobs on different systems.


#13

I just heard about headless computers. I understand you need a “Displayport Dummy Plug” in the graphics card? They are pretty cheap on Amazon.


#14

I’ve never bothered because I’m a linux user. The machine will run fine without a monitor plugged in. But in this domain there may be some hitches since we’re discussing graphics processing. Not least, you still need a graphics card for it’s GPU even if you have no monitor attached.

I know of at least one problem which causes qmelt fall over because something in it expects a display system, even if it’s not showing anything. I discovered this when I tried to run it on an Ubuntu server (virtual machine). I’ll be getting my new box home tomorrow so I’ll experiment then. One possible way round may be to run qmelt using xvfb: xvfb-run -- qmelt. I’ll experiment with this before I install a linux desktop.

If that doesn’t work, I’ll install the desktop packages and try again (without monitor plugged in).

I’ll keep those dummy plugs as a backup idea if everything else fails.


#15

You don’t need those anymore. It was a thing for some older display drivers if you wanted to use the gpu but I haven’t used one in years with gpu’s in servers.


#16

I was able to get it to run before by installing ubuntu(desktop) followed by the required packages and xvfb without too much issue. completely headless was a separate issue and something that could be addressed with changes to qmelt in the future(I’d love to see client/server render as an option for business uses like my office)


#17

@shotcut

I am just started using shotcut from After Effects and I was wondering whether shotcut supports JSX like templating engine through which you can create dynamic videos.

It will be great if you can guide me to some documentation/code for the sam


#18

No. It uses MLT and MLT XML. https://mltframework.org/