Shotcut proxy videos auto creation auto replace tool for fast smooth editing

Resources
1

After suffering hours of editing GUI lagging/crash/drag-drop non-responses… I made this tiny tool.

image
image814×361 21.3 KB

This tool has 2 buttons:

Button 1, Batch create proxy_videos in a sub-folder named “fastproxy” (proxy videos are 360p/720p i-frame only smooth editing format)

Button 2, “1-Click” automatically restores the proxy_videos back to the original_videos in your .mlt project file (after you finish editing with the proxy_videos and are ready to export the high-quality final movie).

Your workflow should be:

Make Proxy(button 1) ==> Do Shotcut editing using Proxy ==> Replace Proxy with original(button 2) ==> Export.

Options:

360p is for old computers with 1080p monitors, or you are going to add lots of filters on video clips and still want the GUI to be super smooth.

720p is for more powerful computers, or you want to see some details during editing(eg. zooming in)

1080p is slower, bigger but much finer than 720p. You can use it as a proxy, or you can use it as an “intermediate” to edit/export your 1080p video directly (without using button 2)

Full-Resolution is similar to the Shotcut build-in “edit-friendly conversion”. It is faster and the file it generates is much smaller (high quality lossy, qscale=3).

This little tool made my life so much easier, so here I share.

For people who don’t know what the proxy-editing means: If your video is too big or your computer is too old then your Shotcut GUI will be choppy and laggy. So we use a same-content but low-resolution videos (called proxy) to do the same editing job. Once finished the editing, we hack the project .mlt file to restore to the original high quality source videos for the final “export”.

Changelog:

1.01 – 1.02:

  • Change the output proxy_videos into a sub-directory named “fastproxy” ;
  • Allowing mixed editing with proxy_videos and original_videos;
  • Fixed some dialog bugs.

1.02 – 1.03 final

  • Add support for .mp4, .mov, .m4v, .mkv files;
  • Add file number and modify number displays so users can spot if something goes wrong;
  • Include ffmpeg so this tool is fully portable now;
  • Change the encoder profile from superfast to ultrafast ~ another 10% speedup;
  • Cosmetic changes, more tips and notices;
  • Now this project is marked as final ~ unless major changes are needed in the future.

1.03 – 1.041 final plus

  • Change the proxy audio codec to ac3, as reports saying its better for smooth editing;
  • Add a wrapper around ffmpeg to handle the incomplete output proxy files, meaning: now you can close/kill/shutdown the conversion process without any trouble. You can continue the conversion anytime later (simply run it again). The existing converted proxy_files will be skipped, the unfinished/interrupted/new files will be converted again.
  • v1.041 Improve the proxy video quality. (The quality was super low with no deblock so the proxy video was full of tiny blocks losing all the details, unable to preview the result of colour grading and adjustments properly). The proxy video is now 2 times larger but it is much clearer and useful.
  • v1.042 change the included ffmpeg from nightly version to stable version
  • v1.043 now ffmpeg runs in low priority mode
  • v1.043 fixed a file-name-lock issue when generating proxy directly on slow sd cards.
  • v1.044 fixed the “width not divisible by 2” conversion issue. thanks @DRM @Austin

1.04 – 1.10 final plus max

  • adding the 360p/720p options;
  • adding the lossless audio option;
  • remember the last used path;
  • v1.101 cosmetic changes.
  • V1.102 adding notification sounds when conversion finishes

1.10 – 2.020 Happy New Year!

  • Major proxy codec changes, using “MPEG-4 Part 2” instead of “H264 i-frame-only”, explained in post 70; Proxy_videos are ~ 1/3 of file size comparing to earlier versions, faster and smoother.
  • Add 1080p intermediate and full-resolution format options
  • Pass-through audio streams, remove the “lossless audio” option.
  • Because of the codec changing, now add .avi support.
  • 2.021 add video formats support to:".avi .ts .trp .m2t .m2ts .mts .mpg .mpeg .m1v .m2v .mp4 .m4v .mov .mkv .vob .3gp .wmv .asf"
  • 2.021 increase the quality for 360p/720p a little bit.

Download here:


(Windows only, work with .avi .ts .trp .m2t .m2ts .mts .mpg .mpeg .m1v .m2v .mp4 .m4v .mov .mkv .vob .3gp .wmv .asf files, open source, includes ffmpeg)
I don’t reserve any rights, feel free to use the file and code whatever you like.
Credits to: FFmpeg multimedia format and codec libraries under the GPL v3 License
13 Likes
2

Thanks! Please consider to work on something directly in Shotcut. There is a SDK to make it easier:
https://shotcut.org/notes/windowsdev/

1 Like
3

Hi,

can you tell what CPU you are using?

4

Hey, Thanks for the advice! I wish I have the skills to contribute to Shotcut project (If I can I would!). But currently making this script-based automation tool is the most I can do. (I am a script writer but not a coder)

There is a plan of making a build-in proxy-editing in shotcut roadmap I read. This tool is just to temporarily helps some before it is properly made.

5

Hey, this tool will call ffmpeg to do the quick proxy_video conversion.

I have a 8 years old dual-core i3 CPU, conversion reaches ~60fps on 1440p source videos. (50% of actual video time, libx264 profile is 360p+superfast+gop1+fastdecode)

I believe anyone with a newer generation i5 / i7 quad-core CPU can easily reach 200+ fps proxy_video generating speed on most of video sources.

This tool is made to run in batch. Pick the folder full of videos, click run, then go make yourself a cup of coffee. When you come back, all videos are proxy converted with a _fastproxy_xxx.mp4 name prefix.

6

HI @KKnBB, wow, this looks brilliant. I just tested it with a couple of video clips and the converted video responds to the J key (reverse play) like a dream. This will be a tool I will use a lot I think. Thank you so much for creating it and posting it!!:smiley:

A couple of points for feedback - clicking the “create proxy videos” button didn’t seem to have any effect for me. I eventually got to convert my folder of mp4s by opening the .bat file itself. Maybe that’s how it works (ie pressing the button just creates the bat file) but it wasn’t clear to me from your instructions. Otherwise your instructions are very clear.

Also - small typo in your documentation. The sentence “If you want to rule out come of the videos in the directory… should read " If you want to rule out some of the videos in the directory…”

Thanks again - great work :+1::+1:

7

Hey jonray thanks for trying and reporting.

I fixed some “dialog handler” things, now when you click the button, it should popup a dialog asking do you want to run it now, or edit the batch.

The 1.02 version updates:
Change the output proxy_videos into a sub-directory named “fastproxy”(clearer view for folders with lots of videos);
Some bugs fixed.
This is a major change, as from now the proxy files are in a sub-folder, and the “replace back” function will handle the sub-folder only.

The next version I will try to add multiple formats supports, like mov, mkv, m4v etc, then I will consider this little project as complete.

8

I wasn’t able to get the program to replace the proxy files with the original files in my test. I know I missed something in the directions.

The program doesn’t ask you to identify specific files it seems just folders. So I specified I created a new folder and put a 4K video file in there. When I clicked to create the proxy it created a new subfolder with the proxy video in it. I left the proxy video in that newly created sub folder and left the original 4K file in the folder just before the subfolder. I did some editing with the proxy. Then when I hit to replace the proxy video with the original file it created a new mlt file, the back up, but when I opened Shotcut with the original mlt project file it did not replace the proxy with the original. I’m not sure what I did wrong.

I don’t know if I was supposed to take the proxy video out of the newly created subfolder and put it in the folder together with the original 4K file since your instructions say to “make sure the proxy-file and the original-file are in the same folder” but since the program creates a subfolder for the proxy I just left it as is. So I’m confused.

9

@DRM Hold on, you maybe download it in the middle of the updating (program updated but instruction not updated yet)

FYI, If you are using the 1.02 version, you should leave the proxy_video inside the fastproxy folder. Don’t move it.

Using some short clisp give it a try, if it still not work, would you mind send me the .mlt file let me have a look please. (maybe different shotcut versions save different .mlt formats I have never met. My email is kylechen@@@@live.com.

10

@jonray @jonray I spent some time tonight polishing it to a “final” version. You should use this new version since many tiny bugs are fixed, just let you guys (early birds) @DRM know.

1 Like
11

Hi @k, report and feedback FYI. I tried out v.1.02 - and it works fine. Dialog told me it found 42 files, which was correct - there were 42 x 3-minute mp4s in my folder. Not realising this would take a while, I selected start batch and it began the process. Tried to abort (q) but it didn’t stop. No worries, I was able to continue working on computer whilst it carried on in the background.

Encoded OK at 10x - 12x speed. However afterwards I noticed that the proxy file-sizes were about 14x or 15x larger than the originals! Is this normal or to be expected?. I expected them to be smaller…

Not had time to test further yet. HTH…

Good idea to have proxy files saved to separate folder. Helps with file management!

PS I posted this just before I noticed you have created v1.03…

12

@jonray
CTRL+C is the standard way to terminate command line processes…And It should be closed when you simply click the [x] in the corner…but anyway, hope it didn’t overheat your computer too much.

The “proxy” files are low in resolutions, but come with a terrible terrible compression ratio (exchange for faster processing speed), and it is I-frame only video (every frame is a individual picture, no differential compression) for the smooth editing reason, making the size even bigger…It can not even compress the video with only still images!

So if you have low-resolutions well-compressed source videos, the “proxy” can be bigger (didn’t save much from lowering resolutions, but grew a lot from the bad compression).

But if you have 4k / 8k videos, the proxy size will be less than 1/10 of the original (saved a lot from lowering the resolutions)

In my own testing, generally speaking, the proxy_video will eat like 1MB/seconds (8000kbps), much smaller than the source videos from my DJI camera (100000kbps)

Another user reported the “replace proxy back to original” function was not working, made me worried that may there be any secret formats inside the .mlt file I haven’t found out (it works for myself so well so far). Let me know your result.

This supposed to be a 20 minutes project assisting my own video editing, modify some codes of my earlier “fetch all files in a folder then rename” project.
Then I found that I like it, it works so well, and you said it is very useful too. So now I have spent like 4 hours on polishing it into a proper tool.

13

I downloaded and my AV tagged this

image

14

I’m on a mac so decided to download the source code to perhaps mod it for MacOS.
Once the zip file was extracted, the contents had a .au3 2 extension, very weird.
Never seen this for any source code files.

22%20PM

15

I think .au3 is the scripting wrapper. I’m sure the author will confirm/advise

16

Okay, so I used 1.03 and this time the proxy files did get swapped for the original 4K file but the resolution of the project did not change to 4K. It stayed with the resolution of the proxy. When I did the test, I had the video mode in Shotcut on to “Automatic” and since your instructions say to start with the proxy files that’s what I did.

17

@KKnBB, version 1.03 worked perfectly.

Created a new project 1920x1080. Added the proxy files to the project. Created a few transitions using GL transition html files. These transitions are usually very laggy with HD video. Using the proxy files made everything go very smoothly.

Used the tool to replace the original mlt. Opened the project in SC. The video was HD. Exported perfectly.

:+1::+1:

18

@KKnBB, thanks for your detailed explanation of file-sizes. Makes perfect sense now. Yes, in my case the source video was low-resolution and well-compressed, explaining the higher proxy size.

I appreciate your work and will use this tool. Like @sauron says, certain transitions/filters can make the preview laggy, and also the reverse play (J key) is often slow, so this will help greatly. Thanks!:+1::+1:

19

@PaulusMaximus hey P, this program has these 2 behaviours:
Unzip an .exe (ffmpeg.exe), and create a batch .bat file(for batch video conversion)
Both these will be considered as dangerous behaviours by many AV.

I am an IT technician and I am 100% sure my computer is clean when compiling it. But if you are a paranoid person, download sourcecode+autoit+ffmpeg.exe, right click “compile”, make your own executable :smiley:

And hey @Paul2 , the .au3 is the autoit (google it) source script. This tool looks like a program, but it is actually a simple 60-line script. The script is vb-styled you may find it hard to make it run on macos. Better off rewrite it with similar logic with python or something.
(why it is in autoit? because I had a similar project many years ago. So I copy the code and change few lines and here it is)

@DRM yea right sorry I totally forgot the use cases of “Automatic” resolution. Sadly this tool will not change the project’s resolutions as it can not read human’s mind. You may have to manually pick a project resolution at the beginning of your editing…

@jonray Haha glad to know the proxy is doing its job. Because my workstation is old and I newly bought a 4K camera (you can smell the disaster), this tool saved my own life. It will save me much more time in video editing than the time I spent on making it.

20

I figured that it was that you had to manually set the project’s resolution before starting to edit. I didn’t know though if when swapping the proxy for the original file if the tool would edit the project resolution file based on the original file.

I did test it again by manually setting the project resolution and frame rate as the first step and it worked well. Thanks for sharing this! :slightly_smiling_face:

I suggest adding that detail to your instructions on your webpage for Step 2 since many people will leave the Automatic setting for the video mode. Something like, “Make sure to manually set the video mode for your project’s resolution and frames per second before you start editing.”

Also, it’s been suggested here on the forums that the AC3 audio codec is better for proxy work than AAC. You may want to consider that.

By the way, could there be a button that does the reverse of “Replace Proxy to Original in .mlt” so that it replaces the original files with the proxies instead of creating a backup mlt file?

Thanks again!