Rough pixelation in chroma key; color subsampling not smoothened?

Using chroma key-advanded, I get fringed edges in the the resulting mask. Mask has pixelation in half resolution. Even using slope, the chroma key has a resolution of only 960 x 540 pixels instead of Full-HD. Footage comes from usual full-HD camera. See the attached shot. (done by “show mask” and “export frame”, but with preview scaling OFF. Foto then enlarged). Only the slightly visible small pixels are in full resolution; they obviously come from the luma criterion (intensity delta). But the chroma and hue delta make the large pixelation. Of course there is chroma subsampling, but this should not be visible. (For any rendering such images, there must be a smoothener to the croma signal to low-pass it’s spacial frequency, for fulfilling the sampling theorem).

  • Is it possible that this smoothening is missing before the chroma key catches the image?
  • What can I do to get smooth mask? Blurring the mask afterwards seems to be no good option, this should happen to the chroma signal before reaching the chroma key (as long as it is linear).

similar/related: Color on edges of text/masks/etcShotcut chroma mask pixels

Hi @Steve2
I very rarely use the Chroma key filter, but I found this video a few weeks ago, and the guy is getting excellent results. Maybe it can help you with your issues.

1 Like


Sometimes order of (filter) list is important…
Can you show (or name) the used filter(s)?

Best regards

I just tested it, it happens without any other filters before or after.
Just take chroma-key advanced, color-space “hue-chroma-intensity”. Then take “alpha-channel:view”.
(Usually I have hue-delta narrower than intensity-delta. But basically the pixelation happens in all settings once chroma is involved).

I also played the same video in “media player (classic home cinema)” in original size and I took a screenshot of it. This I loaded into shotcut and apllied the same filter. Result: the mask is 100% o.k. as it should be!

Here the properties of the footage:
Shotcut Footage-properties

Thank you! it’s great tutorial. I’ll try it.
He blurrs the chroma key afterwards and then does key-spill,
which surely will help me, even without solving the initial problem.
Maybe his footage comes from better camera.

1 Like

One more idea:
How many tracks do you have under the clip with chroma filter?
If none, use one, and place under the clip with chroma a simple color (can be transparent).

Best regards

Under the track is 1 other track (the background to be punched-in)…But for the alpha channel this is irrelevant. The double size pixels appear already in alpha channel, as visible by “alpha channel: view”.

I think it is easy to reproduce. Use footage directly from camera, not down-sized before. Use the color-space “hue-saturation-intensity”, with narrow hue. Check at a place where brightness of blue-screen is similar to the image.

Best regards

This topic was automatically closed after 90 days. New replies are no longer allowed.