Introducing a new filter: "Deband"

The following zip file has a QML UI for the Deband filter:

deband-1.2.zip (594.9 KB)

To use it, copy the “deband” folder from the archive into the following folder under the Shotcut program root, then restart Shotcut:

<ShotcutRoot>/share/shotcut/qml/filters

The code has been tested against Shotcut 21.03.21 and 21.05.18 and 21.06.15.

The two image files in the archive are for testing the filter. Documentation is provided below.

If the developers are interested, I can create a pull request. I realize the timing is not ideal since a release candidate is already underway.

Comments and suggestions are welcome!


Deband Filter

Table of Contents

What Is It?

Quick How-To

Common Uses

Demonstration

How Does It Work?

Limitations

Alternatives

What Is It?

The Deband filter tries to remove banding from an image by replacing streaks of solid colors (bands) with gradients or dithering.

FilterUI

Quick How-To

Add the Deband filter to a clip. Start with a preset based on the following criteria:

  • For minimal processing, use the “Full range to Limited range” preset. (This is also the default setting.) This preset is designed to gently remove bands that appear when full-range video is dropped onto Shotcut’s limited-range timeline. Minimal processing is good for 2D graphics and PowerPoint-type material where edges should remain crisp.

  • For average processing, use the “FFmpeg default values” preset. This is good for live-action camera footage.

  • If a streaky blue sky is involved, heavier processing from the “Blue gradient in open sky” preset may be needed. For a red sunset, try the “Red gradient in open sky” preset.

These are just starting points. Customize as needed.

Depending on the source video, the results of Deband may seem very subtle. If the output is viewed on a four-inch smartphone screen, this is certainly true. But on a screen that is “medium-sized” or larger, the Deband filter can create profound improvements.

Also, the most distracting thing about bands is their tendency to wiggle like caffeinated worms during video playback. Although Deband results may look subtle on a still image, using Deband to stop the worm-wiggling in a video makes a significant improvement in perceived realism and the viewer’s ability to stay focused.

Common Uses

Banding is a common side effect of the following processes:

  • Full to limited range conversion (including image files)

  • Low-bitrate codecs smearing gradients to reduce file size

  • Compression macroblocking is essentially a series of short bands

  • Posterization or clipping introduced by extreme post-processing filters

“Full range” refers to a video signal (RGB or YCbCr) that is encoded in the 0-255 range. “Full range” is also called JPEG or PC range.

“Limited range” refers to the 16-235 range, which leaves room for sync signals, ringing artefacts, and the Gibbs phenomenon. “Limited range” is also called MPEG, TV, legal, broadcast, or studio range.

Computer screen recordings are often RGB 0-255, and some video cameras save their files as YCbCr 0-255. Since the Shotcut timeline is YCbCr 16-235, a conversion must take place for these sources. Shotcut invokes swscale in a non-dithering mode that often introduces banding (overlap) as colors are squished into the reduced range. The Deband filter tries to detect these bands and convert them back into smooth gradients, typically by adding dithering.

Note that many image file formats (like JPEG, GIF, PNG) are encoded with 0-255 values. They will be converted to 16-235 when placed on the timeline, which makes them prone to banding. The Deband filter therefore has potential to make many still images look better than they would without any processing.

Demonstration

The following image has gradients that go from RGB 0 to RGB 255:

When Shotcut loads this image, banding can be seen in the Video Zoom scope as multiple lines having the same color value. Although the Y (luma) value changes from 16-235 without skipping values, the display conversion back to RGB 0-255 does skip values.

For instance, going from Y 61 to Y 62 correlates to RGB 52 to RGB 54. The skipped RGB 53 causes RGB 54 to look unexpectedly brighter than its neighbors. This discontinuity ruins the illusion of a smooth gradient.

The Video Waveform scope also shows uneven value distribution rather than a solid vertical bar. A solid bar would mean equal representation for every value from 0 to 100 IRE.

Below, we see what the image looks like after applying the Deband filter with the default settings:

Unfortunately, the forum software has shrunk the image so much that the dithering is hard to see. Here is an exaggerated view (enlarged and emphasized with a Curves filter):

The strategy behind the default settings is to assume that bands will be very narrow (at most three pixels wide) as a result of gradients being squished during range conversion. This explains the small default pixel range. We also expect banding to happen only when values are very similar to each other (where overlap happens when squished), therefore the thresholds are small as well. These are the gentlest and least invasive Deband settings. For slightly more processing, use the “FFmpeg default values” preset.

Some scenes, like open blue sky, require more aggressive processing from the Deband filter. @IceFox2421 contributed the following screenshot (used with permission) which features a sky that was splotchy in the game itself, then was captured in 0-255 range, then was brought into Shotcut, which introduced even more banding with the timeline’s full-to-limited conversion. The screenshot also shows what the footage looks like in Adobe Premiere, which benefits from a better full-to-limited conversion process. Lastly, the screenshot goes back to Shotcut using the Deband filter, and shows very competitive results in the final image quality.

On some browsers, this will animate (click the image to enlarge first):

If not, here are the frames for comparison:

For open sky, the strategy behind the settings is to use a higher threshold for blue and a lower threshold for all the other colors. Bear in mind that the Shotcut timeline is YCbCr unless overridden with mlt_image_format=rgb24, meaning blue is manipulated with the Cb of YCbCr rather than the B of RGB. Lastly, we use a large pixel range to scan for slowly-changing gradients, as the sky does not quickly change from one tone to the next.

How Does It Work?

The Shotcut Deband filter uses avfilter.deband (FFmpeg | MLT Framework) for processing. The general idea is to take a pixel and look at its neighbors within Pixel Range. If they’re the same general color (within Threshold range), then assume this area should have been a gradient (or an in-between value) rather than a band. The pixel will be replaced with the average value of its neighbors. If the Randomize option is selected to skew that average, then any bands will in essence become dithered. If dithering introduces too much “snow” or impulse noise into the image, try adding a Sharpen filter with the Amount around or below 30% (which will actually create blur despite the name) and with Size around 12%. Sometimes this can blur small patches of dots out of existence without harming the overall image. In extreme cases, consider a denoise filter to smooth the dot patches out (but not to the point of recreating bands, of course).

Limitations

A more proper way to convert full range video to limited range is to provide a scaler with access to the original 0-255 data, then perform a dithered conversion using error diffusion in linear color space. swscale and zscale can be manually invoked in such a way, but this is not directly available in Shotcut. (See the Alternatives section below for details.) Since Shotcut is currently 8-bit, dithering is not invoked in swscale. Likewise, since Shotcut does not use zscale, conversions are not done in linear space.

Since the Deband filter is seeing data that has already been compressed to the 16-235 range by swscale, Deband does not have the luxury of examining the original 0-255 video data. This is an automatic disadvantage in terms of technical accuracy. But, given proper parameters on a scene-by-scene basis, even a discerning audience would struggle to see the difference.

Alternatives

Video can be manually pre-processed into limited range before importing into Shotcut. This gives swscale or zscale access to the original 0-255 values.

Let’s create a sample 0-255 input video for this demonstration by using the test image provided at the top of this document:

ffmpeg -loop true -i FullRangeGradients.png -t 00:00:04.000 -r 30000/1001 -vsync cfr -pix_fmt gbrp -c:v utvideo -an FullRangeGradients.mkv

Below are two ways to convert the sample input video into a dithered limited-range intermediate.

Convert with swscale passing through 10-bit so dithering can be invoked when returning to 8-bit:
(see this note about dithering on FFmpeg Trac)

ffmpeg -i FullRangeGradients.mkv -filter:v scale=out_color_matrix=bt709:out_range=full:flags=neighbor+accurate_rnd+full_chroma_inp+full_chroma_int:sws_dither=none,format=yuv444p10le,scale=in_color_matrix=bt709:in_range=full:out_color_matrix=bt709:out_range=limited:flags=bicubic+accurate_rnd+full_chroma_inp+full_chroma_int:sws_dither=ed,format=yuv422p -pix_fmt yuv422p -color_range mpeg -colorspace bt709 -color_primaries bt709 -color_trc bt709 -c:v libx264 -qp 0 -g 8 -bf 0 -preset ultrafast -movflags +faststart+write_colr -an -y FullRangeGradients-swscale.mp4

Convert with zscale using dithering:

ffmpeg -i FullRangeGradients.mkv -filter:v zscale=matrix=709:range=limited:filter=bicubic:dither=error_diffusion,format=yuv422p -pix_fmt yuv422p -color_range mpeg -colorspace bt709 -color_primaries bt709 -color_trc bt709 -c:v libx264 -qp 0 -g 8 -bf 0 -preset ultrafast -movflags +faststart+write_colr -an -y FullRangeGradients-zscale.mp4

Stack all three videos on separate tracks in Shotcut and toggle track visibility to notice that the swscale and zscale versions have smoother gradients than the source video.

8 Likes

Wow ! This is great. Thanks a lot @Austin
Will test this filter tomorrow.

This post have an impeccable layout and text formatting by the way :slight_smile:

1 Like

Yes, there was a significant formatting influence from a certain someone’s recent post. :smile:

2 Likes

Can you share a screenshot of the filter UI?

Seems like something we would want to add to Shotcut. Whenever you think you are done tweaking it, go ahead and send a pull request.

Good idea, Brian. I added a filter screenshot to the top of the documentation.

The code has been solid for all of my testing. I figured I would let the forum do weekend testing on it to make sure I didn’t overlook anything, then send a pull request on Sunday evening.

Great addition! A very thoughtful filter. I’m sure I can find good use for this in future videos

2 Likes

Thanks! Another place this filter sometimes helps is for vlog-style footage in high-contrast locations (like outdoor direct sunlight near a shade tree) where the camera is making harsh transitions from highlight to shadow areas. I’ve seen some cases where this filter can make the rolloff a little more graceful (read: make the camera look more expensive). But I have a hunch this is very scene-specific, so I won’t make a general claim about it.

As shown in the documentation, it can definitely do wonders to footage of a sunset though.

3 Likes

I tested the Deband filter on this YouTube video, where you can see some fast moving color banding appearing at around 00:12.

I applied the Deband filter to a portion of the original video and after exporting, I see a distinct improvement when I play the video on my computer. Sadly, after the video was uploaded to YouTube the difference is not so evident anymore.

But here’s a link to download the video: deband_sunset.mp4 - Icedrive
Or you can watch it directly on IceDrive. It’s a 1280x720 clip, but try watch it on full screen. Pause at 00:09 seconds you’ll see a big difference.

And here’s a frame exported from Shotcut. If you zoom the image at 150% or 200%, you can clearly see (I hope) the difference. On the left there is a lot of color pixelization (micro-banding ?), not so much on the right.

1 Like

Fantastic test, @MusicalBox! I’m on a 13" laptop at the moment and the difference on IceDrive is definitely clear. Not quite as clear on YouTube, as expected due to their compression.

The clip you selected shows pretty severe macroblocking (good choice). I would not have instinctively reached for a Deband filter to fix it. FFmpeg has at least six dedicated filters for deblocking (deblock, spp, uspp, fspp, pp7, pp) and up until now I would have attempted to use them first. But your results with Deband are as good if not better than those! I guess technically a macroblock is just a really short band lol. You’ve opened my eyes to a new possibility.

Thanks for testing!

P.S… Shotcut has the fspp deblocking filter already included. It’s called “Reduce Noise: Quantization” if you want to run it head-to-head with Deband out of your own curiosity.

1 Like

Honestly, I chose that video because it’s the first one I found that was useable :slight_smile:
Short, and there was visible artefacts that looked to me like fast moving color banding.

At one point, I’ll find a video with real color banding and do more tests :slight_smile:

1 Like

I pushed the Deband settings a little harder and got these results (using the screenshot from your last post, cloning the left side to the right, click it to see full screen):

Gotta be kidding… the forum software converted my PNG screenshot into JPEG, and the compression puts noise back. The PNG is flawlessly smooth. Here’s a zip file with the PNG:

Sunset.zip (1.4 MB)

Here are the settings I used, Deband followed by a little Sharpen to enhance the ridgeline. Since this sky is red instead of blue, I raised the “Cr” threshold quite a bit. (For anyone wondering why, the “r” in Cr means “Red-Difference”.)

SunsetSettings

SharpenSettings

2 Likes

I find that YouTube also adds additional banding when the final Shotcut output doesn’t. I’d love to know how much I need to overcompensate to avoid what YouTube will later add.

Lower than 1440p, I doubt it’s possible. The banded blocky mushiness we see on YouTube is already the best quality that can be stuffed into the terribly-limited bitrate that YouTube allocates for “low viewership channels”. They provide higher bitrates to some “high viewership channels”, but the definition is very fluid (and probably linked to ad revenue, lol). So even if Shotcut overcompensates, there isn’t enough YouTube bitrate to represent it. The quantizer will strip the details right back down to where they were to fit the bitrate limit.

The story is a little different at 1440p and higher, when the different codecs (VP9/AV1) and sometimes higher bitrates kick in. Then any overcompensation actually sticks, such as details enhanced by the Sharpen filter. It doesn’t take much overcompensation at 1440p+ to land in a good-looking place, at least for the projects I’ve uploaded.

Actually, authoring in 1080p but then changing the Export > Advanced resolution to 1440p or 2160p with Lanczos interpolation will keep an existing workflow at the easier 1080p, still look extremely good even upscaled, and then get the better YouTube codec. Lots of people do this.

1 Like

Yes, I usually upscale my uploads to 1440. Aside from getting a better codec, the quality is definitely better

1 Like

Status update:

I have added a “Red gradient in open sky” preset to the code to accommodate red sunsets, and updated the documentation accordingly. Then I submitted a pull request with the new code. It seems very solid at this point.

I tried this and it seems to have worked flawlessly, this solves my problem, thank you :slight_smile:

1 Like

@austin, thanks so much for this! Apologies if this has been touched on earlier in this thread (I didn’t get time to read it all) but my interest would whether it smooths out gradient colour backgrounds (full-screen). When I get a chance I’ll test it out.

Fantastic job on doing the work to add this filter, @Austin. There are just some things I want to bring up after testing it a little.

Wouldn’t it be better to not double up the values in the preset listings? It’ll be confusing to a user testing out the presets and wondering why the same setting is repeating in two differently named presets.

On top of that, the Minimal Strength preset gives the same exact values as the Full Range To Limited Range preset which is the same as the “defaults” preset. Is it intentional to have three separate preset listings that are actually all the same?

Also, when you say “default setting” do you mean the actual preset listed that says “defaults”? Because when I click on all of the default buttons the values turn into the same ones as the Average Strength preset. Shouldn’t pressing the default buttons revert to the same values as the preset that is listed as “default”?

There’s no preset named “FFmpeg default values”. Is that supposed to be the “defaults” preset?

By the way, since there is a Minimal Strength and an Average Strength, how about adding a preset that is a stronger than those two? Not one that is maxed out obviously but a stronger preset that may be more appropriate for a video with a bit more issues.

1 Like

:thinking:

2021-07-03_17.36.34

@MusicalBox

:point_down: