Intermediate Files for Editing


#21

Since my previous post I have done some testing on huffyuv and ffv1. I’m seeing some color shifts which I don’t like. For example, given an input of R = 16, G = 180, B = 16, by actual measurement huffvuv and ffv1 are giving me:

R = 18, G = 149, B = 25.

To the naked eye the huffyuv and ffv1 look darker due to the reduced green conent and I do not consider this acceptable. OTOH, I get

R = 17, G = 181, B = 17

Using x.264 with crf 0. Due to rounding error there will always be 2 - 3 points of error so I consider this acceptable.

“Mathematically lossless” is well and fine but if the colors are shifted that drastically it’s a deal breaker in my book. I guess x.264 with crf0 is considered quasi-lossless?

I’ve been using ffmpeg on a command line. Give me a while and I can test it with the Shotcut presets.


New version 18.11: And Hardware Encoding For All
#22

ProRes gives:

R 28 -G 171 - B 28

ShotCut’s H.264 gives:

R 28 -G171 -B26. I get the same results with ShotCut’s HEVC and MPEG-2


#23

What are you using to encode it? I haven’t seen any color shifts on HuffYUV myself but it’s possible you’re shifting color during the RGB<>YUV conversion that huffyuv uses.


#24

The accurate x.264 colors are obtained by using ffmpeg directly from the command prompt with no Shotcut involved. The rest are obtained with Shotcut’s presets.

For clarity, I am starting with a .bmp image and encoding it to huffyuv, ffv1, etc.


#25

It looks like it may be a bmp issue


#26

Sounds like two things in play here. One is RGB vs sRGB to YUV colorspace conversion. The other is MPEG range vs JPEG range video. You’re probably familiar with both issues, but I’ll give a brief explanation for everyone else.

MPEG range is more of a broadcast restriction that says RGB values have to be between 16 and 235. 16 is black and all-235s is white. JPEG range is 0-255. This can be controlled with the ffmpeg -color_range parameter. Converting improperly between them can cause raised or crushed whites/blacks plus color shifts in-between.

The other issue is simple colorspace conversion error. Maybe converting from PNG instead of BMP can be a workaround.

I’ve noticed color shifts exporting from Shotcut that I don’t get using ffmpeg directly. Not sure the issue there either. Will write more if I get a chance to test.


#27

It’s not a bmp issue. The same bmp gives good colors when converted using ffmpeg directly and not going through ShotCut.

I think this is it. I’m not worried about it, though as I can use my direct ffmpeg scripts.


#28

Forgot to follow up on this… In theory, x264 at CRF 0 is true lossless. The rounding error is probably colorspace conversions and there’s not a lot that can be done about that.


#29

There will always be rounding error in calculations involving YUV due to the use of floating-point coefficients. There is YCoCg but it doesn’t subsample well. I don’t think you could do 4:2:2 or 4:2:0 in YCoCg.

I was going to use Huffyuv as my interpositive format until I discovered these color errors, so I’m going with good ol’ x.264 with CRF 0. x.264 is a real workhorse.

I am borrowing an old film term:


#30

Interesting note, in further research I discovered WHY I wasn’t seeing a color shift on my end is that my camera is recording as YUV by default according to mediainfo, interesting discovery to say the least I wonder if it has an option to go between that and rgb internally.


Built in proxy generation
#31

I lost a little in translation… were you seeing color shifts with command-line ffmpeg as well, or only Shotcut presets?

I just did a round of tests using the same color you did using ffmpeg:

ffmpeg -f image2 -s 720x480 -r 29.97 -i "016,180,016.png" -pix_fmt yuv420p -color_range mpeg -c:v ffv1 yuv420p-mpeg.ffv1.avi

I changed -c:v to be HuffYUV, UtVideo, FFV1, and ProRes (and yes some complained that yuv420p was incompatible). I played back their outputted videos with MPC-BE media player on Windows 10, did a screen capture, and eye-dropped the frame to get the RGB. On every codec, the RGB values were within 1. This was true for -color_range of JPEG or MPEG.

I got confused because you ended up with x264 CRF 0. I thought the Shotcut color shifts didn’t bother you because you used scripts, but if you used scripts, then HuffYUV would still be a candidate and you wouldn’t need x264 CRF 0 … so I got confused.

If your ffmpeg is shifting colors, two thoughts come to mind. One is that your ffmpeg is fine and it’s a media player that doesn’t know how to do MPEG/JPEG color range conversion correctly. MPC-BE does. The second option could be an old ffmpeg. I’m using 4.0.2 and get no shifts with the lossless codecs.

Having said that, you’ll probably get better compression with x264 CRF 0 so maybe that’s still a good deal. :slight_smile:


#32

If you checked the stack exchange link I posted part of why he’s seeing shifts and you’re not is the usage of a BMP vs PNG. Unless they’ve updated it(they haven’t as far as i’m aware) the BMP decoder in ffmpeg is an antique (2005) and requires manually declaring what it’s color space is.


#33

I assume the article is old. I did tests with both PNG and BMP, and the results were identical and unshifted. @chris319 also stated that it was not a BMP issue. Colorspace shouldn’t have to be declared because all these formats with the exception of ProRes have native rgb24 support, same as BMP, so it should just work.


#34

It is indeed old although I can’t find any evidence that ffmpeg has been updated to correct it, shouldn’t but it very well may need it.


#35

Agreed. It makes me appreciate x.264 more.

The problem is, I’m still seeing color shifts in ShotCut’s H.264 presets. I’ll test some more to make sure.


#36

Notwithstanding the subsequent discussion into how each intermediate files are working in practise, my takeaway from the thread is:

  1. Try using the original files. If they work in the editor without stuttering etc (because the computer is struggling to decode the frame level information on the fly) then all good. No further action - work with the originals.

  2. If the computer does struggle (stuttering etc), then export to ProRes at 100% quality.

NB: Whilst the intermediate creates a full frame for every frame (in theory improving the editing experience) it does also create a larger file and may therefore cause the editor to suffer similar problems to the original files, albeit for a different reason.

(As an aside… Where does Cineform fit into the discussion? Go Pro’s editor converts the source MP4 files into Cineform to edit. No real choice for the user… Why might Go Pro avoid allowing the user of their software to use the original file?)


#37

Sounds like you nailed it. I might add two very minor additions:

1b) If the original file is variable frame rate like a cell phone, always create an intermediate even if the editor can play the original back. The playback will not be smooth or sync well until it is constant frame rate.

2b) If the computer still struggles even with ProRes, there is the option of going to proxies and using the original file for the final render (no need to keep the ProRes first attempt).

Cineform is designed to be an intermediate format like ProRes and DNxHD. It is quite capable with multi-generation encoding with no loss, and it even threads well. The main reason we haven’t mentioned it is that, to my knowledge, ffmpeg does not have an encoder for it yet. Since Shotcut is built on top of ffmpeg, that means you couldn’t export to it either.

GoPro paid big bucks to buy the Cineform format. They probably convert everything in Studio because users would complain if native H.264 stuttered. GoPro Tech Support doesn’t want to ask people how fast their computers are, so they chose a format that will work on pretty much all computers. Their audience is average users and kids, not video professionals.


#38

Looks like it is in ffmpeg but it’s still a work in progress, gopro open sourced it in 2017
https://gopro.com/news/gopro-open-sources-the-cineform-codec
https://wiki.multimedia.cx/index.php/Cineform_RAW


#39

To make the engine easier to implement. Seeking is as simple as going to the frame in one format and codec, and decoding reference frames is not required. Of course, it is also faster to seek this way. Maybe everything also gets conformed to the same resolution and frame rate in the process, which can also make it simpler.

All this discussion about RGB values is very short on the description of the process and as Austin mentioned color spaces and ranges. Please be aware that by default MLT renders everything to YUV 4:2:2 with MPEG ranges. If you are working with a full range source or targeting a RGB codec, you should add mlt_image_format=rgb24 to the Other tab.


#40

Now I am getting some grievous color errors in Shotcut no matter which codec I use. Same methodology as yesterday but different results.

The mp4 files I coded with ffmpeg on the command line are still good. Weird. I tried different settings in Shotcut with no improvement.

I can’t see the commands Shotcut is sending to ffmpeg so ???

The one change since yesterday is that I ran DisplayCal with a new setting and reloaded the table it generates. I hate to think this is the problem, maybe related to my GPU?