Yuva420p export presets convert RGB to YUV using bt601

pdr · August 17, 2020, 4:04pm

Under the “alpha” export category, the WebM VP8/9 with alpha channel presets use

mlt_image_format=rgb24a
pix_fmt=yuva420p

But the RGB > YUV conversion uses bt601

how to reproduce:
eg. import shotcut logo into a project, export using one of the above presets. Import the exported file and the colors will be shifted. (You can check with other programs, or apply 601>709 conversion to fix it)
https://shotcut.org/media/

I suggest the default should use bt709
Is there an additional switch on the “other” tab where user can control the RGB>YUV conversion ? eg. Through swscale (out_color_matrix=) or zscale (matrix=) ?

https://mltframework.org/plugins/ConsumerAvformat/
https://mltframework.org/plugins/ConsumerAvformat/#colorspace
adding -colorspace does not appear to affect actual conversion, just metatdata flags. (I tried colorspace=1, colorspace=bt709, colorspace=709)

brian · August 17, 2020, 11:50pm

What is the colorspace set to in your video mode?

pdr · August 18, 2020, 2:55am

BT.709

I double checked the .mlt and it said 709 too

The 709/601 mismatch is quite easy to see on HD colorbars

Austin · August 18, 2020, 1:37pm

I may be looking in the wrong place, but the convert_image() function seems to call a helper that always uses BT.601 for RGB[A] conversion rather than select color space according to the video mode. The helper function is shown below:

github.com

mltframework/mlt/blob/7da01504d6844412b6e26c03b7c98214a1730343/src/modules/core/filter_imageconvert.c#L113


		YUV2RGB_601( yy, uu, vv, r, g, b );
		rgb[3] = r;
		rgb[4] = g;
		rgb[5] = b;
		yuv += 4;
		rgb += 6;
	}
	return ret;
}


static int convert_rgb24a_to_yuv422( uint8_t *rgba, uint8_t *yuv, uint8_t *alpha, int width, int height )
{
	int ret = 0;
	int stride = width * 4;
	int y0, y1, u0, u1, v0, v1;
	int r, g, b;
	uint8_t *s, *d = yuv;
	int i, j, n = width / 2 + 1;


	if ( alpha )
	for ( i = 0; i < height; i++ )

shotcut · August 19, 2020, 12:26am

filter_imageconvert.c

That filter is only used when MLT is built without libswscale, and yeah it is limited but does not apply to Shotcut.
I reproduced this and fixed it for the next version 20.09.

brian · August 19, 2020, 1:46am

Impressive investigative work, @Austin

Glad you found a fix.

I always found these lines to be suspicious. But maybe they are unrelated:

github.com

mltframework/mlt/blob/c998f8ebd0d3c04cbb299878517387737a02e15f/src/modules/avformat/filter_avcolour_space.c#L103


      
          	else
          		av_image_fill_arrays(in_data, in_stride, in, in_fmt, in_width, in_height, IMAGE_ALIGN);
          	if ( out_fmt == AV_PIX_FMT_YUV422P16LE )
          		mlt_image_format_planes(mlt_image_yuv422p16, out_width, out_height, out, out_data, out_stride);
          	else
          		av_image_fill_arrays(out_data, out_stride, out, out_fmt, out_width, out_height, IMAGE_ALIGN);
          	struct SwsContext *context = sws_getContext( in_width, in_height, in_fmt,
          		out_width, out_height, out_fmt, flags, NULL, NULL, NULL);
          	if ( context )
          	{
          		// libswscale wants the RGB colorspace to be SWS_CS_DEFAULT, which is = SWS_CS_ITU601.
          		if ( out_fmt == AV_PIX_FMT_RGB24 || out_fmt == AV_PIX_FMT_RGBA )
          			dst_colorspace = 601;
          		error = mlt_set_luma_transfer( context, src_colorspace, dst_colorspace, use_full_range, use_full_range );
          		sws_scale(context, (const uint8_t* const*) in_data, in_stride, 0, in_height,
          			out_data, out_stride);
          		sws_freeContext( context );
          	}
          	return error;
          }

github.com

mltframework/mlt/blob/bd3fa58ceeae20c3c9cddfc572fbf86c0a5d9793/src/modules/avformat/producer_avformat.c#L1426


      
          		sws_freeContext( context );
          	}
          	else if ( *format == mlt_image_rgb24 )
          	{
          		int flags = mlt_get_sws_flags(width, height, src_pix_fmt, width, height, AV_PIX_FMT_RGB24);
          		struct SwsContext *context = sws_getContext( width, height, src_pix_fmt,
          			width, height, AV_PIX_FMT_RGB24, flags, NULL, NULL, NULL);
          		uint8_t *out_data[4];
          		int out_stride[4];
          		av_image_fill_arrays(out_data, out_stride, buffer, AV_PIX_FMT_RGB24, width, height, IMAGE_ALIGN);
          		// libswscale wants the RGB colorspace to be SWS_CS_DEFAULT, which is = SWS_CS_ITU601.
          		mlt_set_luma_transfer( context, self->yuv_colorspace, 601, self->full_luma, 0 );
          		sws_scale( context, (const uint8_t* const*) frame->data, frame->linesize, 0, height,
          			out_data, out_stride);
          		sws_freeContext( context );
          	}
          	else if ( *format == mlt_image_rgb24a || *format == mlt_image_opengl )
          	{
          		int flags = mlt_get_sws_flags(width, height, src_pix_fmt, width, height, AV_PIX_FMT_RGBA);
          		struct SwsContext *context = sws_getContext( width, height, src_pix_fmt,
          			width, height, AV_PIX_FMT_RGBA, flags, NULL, NULL, NULL);

system · November 15, 2020, 4:04pm

This topic was automatically closed after 90 days. New replies are no longer allowed.