Rotate and Scale & Video Quality

Hello, I have another issue with shotcut.

I’d like to do a Timelapse and created the initial video using ffmpeg and image2. I used the full resolution frames (8000x4000 or so…) to create a lossless huffyuv video (24fps) because I figured that high quality input means high quality output. I then applied the Rotate and Scale Filters to do some zooming and panning (also tried Size and Position).

However, the resulting video (1920x1080, 24fps) after export is of awful quality (see screenshot below) compared to the input images which are crystal clear.

I’ve set the Interpolation method to “Hyper/Lanczos” in Advanced Mode, but that didn’t help.

Shotcut v19.09.14. I’ve tried several input sources varying in Size and Quality (Huffyuv, FFV1, H264 CRF0).

Thank you!

I see from your other post you are using Hardware Encoding. Have you tried switching this off and checking the difference in quality?

Thank you for the hint! Unfortunately it does make any difference :slightly_frowning_face:

Only an idea…
Have you tried in a draw program (e.g. Paint.Net, Photoshop)?
To see, if there is also a lost in the quality…
It can be, IMHO, that the downsize is the reason…

2c

Based on your idea I did three things:

  1. Scale the image to 1920x with Gimp
  2. Scale the input video (the lossless huffyuv thing) to 1920x and encode it to h264 with FFMPEG
    (ffmpeg -i input.mkv -vcodec h264 -vf scale=1920:-1 output.mkv)
  3. Scale & Crop the input video (the lossless huffyuv thing) to 1920x1080 and encode it to h264 using shotcut’s Rotate & Scale Filter. I did not introduce any movement (panning).

Below are screenshots. The first two options show perfect quality, the Shotcut option is significantly degraded. I don’t understand why.

You are comparing things that do not make sense to compare:

  1. input is an image of “8000x4000 or so…” resolution
  2. input is a video of unspecified resolution with no filter effect
  3. input is the same video with filter effects

You said you created a huffyuv video from the images. What is the resolution of this video?
I suspect either the video resolution is low or the interpolation option is not reaching the filter for some reason, and it is default to a lower quality default.

Here I opened a photo that is 5152x3864 in Shotcut, added the Size and Position filter to crop and scale it. Then, I exported it using defaults plus Hyper/Lanczos without hardware encoder.

What is your operating system? (could be a build-related problem)

How are you making your screenshot for the Shotcut result? Best not to use screenshot and instead export a frame of video. I used VLC to do that. If you are really making a screenshot you are subject that program’s preview quality. In Shotcut, that depends on Settings > Interpolation as well as Video Mode.

It’s the same resolution as my input frames:

$ ffprobe timelapse_profiled_huffyuv.mkv
ffprobe version 3.4.6-0ubuntu0.18.04.1 Copyright (c) 2007-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
  configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, matroska,webm, from 'timelapse_profiled_huffyuv.mkv':
  Metadata:
    ENCODER         : Lavf57.83.100
  Duration: 00:00:13.04, start: 0.000000, bitrate: 13375515 kb/s
    Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgr0(progressive), 7968x5320, SAR 1:1 DAR 996:665, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      ENCODER         : Lavc57.107.100 huffyuv
      DURATION        : 00:00:13.042000000
$ identify frames/profiled_0000.png
frames/profiled_0000.png PNG 7968x5320 7968x5320+0+0 16-bit sRGB 217MB 0.020u 0:00.029
$ uname -a
Linux monolith 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

However, I think I solved the problem partly:

I used two instances of the Rotate & Scale filter because I didn’t use keyframes properly.

Let me explain. I wanted to zoom in and pan to a starting point at t=00:00:00. For that I used a Rotate & Scale filter with z=175%, x=xxx and y=yyy, but did not define keyframes for that filter.
Then, I instatiated another Rotate & Scale filter because I wanted to zoom out and pan to the “full” picture. For that filter I enabled keyframes. Because of the first filter I had to define values somewhat like z=75%, x=xxx & y=yyy, instead of z=100%, x=0 & y=0. I put the second filter on top of the first, otherwise this would not work.
However, that caused the degradation in quality! Now, I’ve defined only one Rotate & Scale instance but with keyframes. The first keyframe is at t=00:00:00 with z=175%, x=xxx, y=yyy. The second keyframe is at the end of the video with z=100%, x=0, y=0.

From here on I will only talk about Videos which are generated using 1 Rotate & Scale instance
(I’ve also disabled the hardware encoder for now)

This improved the quality drastically. You’ll notice though, that the quality is still not en par with the original image. But I’d attribute that to the video compression.

Here’s the still image taken from VLC which I used in the above screenshot:

However, I still have a problem with “jittering”. It’s best noticable around sharp lines (e.g. on the glacier and the snow fields in my video):

Thank you very much for your patience and help!

I see this no matter what I use for input (full res PNGs in a sequence, or huffyuv video) or what I use for output (H264, huffyuv)

https://trac.ffmpeg.org/ticket/4298

Could this be related?

I’ve implemented the Zoom & Pan thing using ffmpeg. Or at least something very similar to that, because controlling the zoompan filter is a pita.

$ ffmpeg -ss 00:00:00 -t 00:00:07 -i timelapse_profiled_huffyuv.mkv -vcodec h264 -vf "zoompan=s=hd1080:fps=24:d=1:z='if(lte(pzoom,1.0),2.00,max(1.001,pzoom-0.005))':x=px+0.15:y=py+0.15" timelapse_profiled_h264_1920_zoompan_short.mp4
ffmpeg version 3.4.6-0ubuntu0.18.04.1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.3.0-16ubuntu3)
  configuration: --prefix=/usr --extra-version=0ubuntu0.18.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, matroska,webm, from 'timelapse_profiled_huffyuv.mkv':
  Metadata:
    ENCODER         : Lavf57.83.100
  Duration: 00:00:13.04, start: 0.000000, bitrate: 13375515 kb/s
    Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgr0(progressive), 7968x5320, SAR 1:1 DAR 996:665, 24 fps, 24 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      ENCODER         : Lavc57.107.100 huffyuv
      DURATION        : 00:00:13.042000000
Stream mapping:
  Stream #0:0 -> #0:0 (huffyuv (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x56218ab6ef60] using SAR=1/1
[libx264 @ 0x56218ab6ef60] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x56218ab6ef60] profile High 4:4:4 Predictive, level 4.0, 4:4:4 8-bit
[libx264 @ 0x56218ab6ef60] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=4 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'timelapse_profiled_h264_1920_zoompan_short.mp4':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv444p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 24 fps, 12288 tbn, 24 tbc (default)
    Metadata:
      DURATION        : 00:00:13.042000000
      encoder         : Lavc57.107.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
[swscaler @ 0x56218b44e540] Warning: data is not aligned! This can lead to a speed loss
frame=  168 fps=4.2 q=-1.0 Lsize=    3258kB time=00:00:06.87 bitrate=3882.4kbits/s speed=0.171x
video:3255kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.087265%
[libx264 @ 0x56218ab6ef60] frame I:1     Avg QP:22.32  size:141108
[libx264 @ 0x56218ab6ef60] frame P:42    Avg QP:22.55  size: 57276
[libx264 @ 0x56218ab6ef60] frame B:125   Avg QP:25.59  size:  6289
[libx264 @ 0x56218ab6ef60] consecutive B-frames:  0.6%  0.0%  1.8% 97.6%
[libx264 @ 0x56218ab6ef60] mb I  I16..4: 64.9%  0.0% 35.1%
[libx264 @ 0x56218ab6ef60] mb P  I16..4: 10.4%  0.0%  2.7%  P16..4: 40.7% 17.9% 12.5%  0.0%  0.0%    skip:15.8%
[libx264 @ 0x56218ab6ef60] mb B  I16..4:  0.2%  0.0%  0.0%  B16..8: 47.2%  2.8%  0.7%  direct: 0.6%  skip:48.6%  L0:45.0% L1:52.6% BI: 2.4%
[libx264 @ 0x56218ab6ef60] coded y,u,v intra: 19.8% 6.6% 3.6% inter: 6.6% 1.3% 0.9%
[libx264 @ 0x56218ab6ef60] i16 v,h,dc,p: 12% 41% 13% 34%
[libx264 @ 0x56218ab6ef60] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 17% 22%  5%  6%  4% 10%  4%  9%
[libx264 @ 0x56218ab6ef60] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x56218ab6ef60] ref P L0: 66.6% 14.6% 11.9%  6.9%
[libx264 @ 0x56218ab6ef60] ref B L0: 79.2% 17.2%  3.6%
[libx264 @ 0x56218ab6ef60] ref B L1: 88.6% 11.4%
[libx264 @ 0x56218ab6ef60] kb/s:3808.96

As you can see below, the resulting video quality is much better and closer to what I would expect.

In the meantime I tried out OpenShot which is also MLT based. The results are very similar, which suggest a problem with MLT…?

It also helps to know what build you are using, i.e. how did you get Shotcut?

However, I still have a problem with “jittering”.

This is a common problem when animating images with a lot of detail in any software. You need to apply a small amount of Blur: Low Pass Some tools automatically perform something like this in their canned slow pan/zoom effects but not Shotcut.

suggest a problem with MLT

It is using a MLT-native effect, but I hesitate to call it a bug. The first use of Rotate & Scale utilizes the the full resolution of the source. The second use of it must use the output of the first filter, and that will not be good. Libavfilter and zoompan does not offer keyframed animation.

Via the Ubuntu Snap Store. Please have a look at the screenshot below.

I tried that one out, one with a value of 4.0 and one with the default value of 20.0.
With 4.0 I still can see a lot of jitter, while with 20.0 the blur is really strong and there’s still some jitter in the strongest details.

I understand that using two Rotate & Scale instances is a bad idea when you can do everything with one instance. That was clearly an mistake on my side. However, what I don’t understand is, why ffmpeg produces a jitter free, crisp sharp output while MLT does not?

It is not jitter-free to my eyes and has some detail-crawl/moire effect. I have done an analysis of the zoompan code, but here is the code in MLT if you want to try to improve upon it yourself:

Here is a link to related previous discussion:

My suggestion was to use Sharpen, Amount < 50% before the Rotate & Scale or Size and Position.

I am having trouble to reproduce as bad a case as yours with my photos. Mine tend to be lower res (~4000x3000) and denoised a little via JPEG. Do you care to make one of your pics available for me and others to experiment?

I looking into libasvfilter zoom pan. It uses libswscale, which is a simple 2D scaler. MLT uses libswscale for inherent scaling, but Rotate & Scale and Size and Position are using an affine transform. Might be related to that

Hi, sorry for the late reply, I’ve been out and offline. :slight_smile:

Well, then I prefer that over the jitter. It’s less worse in my eyes (subjective of course).

Even if I had the time to dig into the code I still lack domain specific knowledge. :slight_smile:

I tried that out and it improves things a bit, but the problem is still there.

However, after turning off Sharpening in my RAW converter, it improved once more.

Here’s one thing I noticed after playing with Rotate & Scale a bit more: Panning does not cause jitter, but Zooming does. It does not play a role whether I zoom out or in, as soon as I simply zoom (without panning) there’s jitter. If I just pan without zooming there’s no jitter.

https://drive.google.com/drive/folders/1NfJHrssXF-iu8BqATXrOwgPEbr-WFf_Z?usp=sharing

In this folder you’ll find 4 files:

frame_0000_no_sharpen.tif - RAW export without Sharpen
frame_0000_pre_sharpen.tif - RAW export with Sharpen
timelapse_no_sharpen_huffyuv_3s.mkv - 3 seconds of HUFFYUV (lossless) video encoded from frames without Sharpen
timelapse_pre_sharpen_huffyuv_3s.mkv - 3 seconds of HUFFYUV (lossless) video encoded from frames with Sharpen

The TIFs are each ~200MB. The video files ~5GB. Please give me a short notice when you’ve downloaded them, because they take up a substantial bit of my Google Drive allowance. :wink:

I did notice the better quality of the ffmpeg panzoom animation when I played the video fullscreen. I saw the moire effect while playing smaller size embedded in the page. This downscaled playback is yet another example of how high detail/frequency interacts with scaling during animation.

I have downloaded the files you provided. You can remove them now;.

One thing to note is that a sharpen routine does increase the frequency around edges and existing detail, which will contribute to this.