Intermediate Files for Editing


#61

It depends on filters.There are a few filters that operate in YUV only, which would necessitate a conversion to YUV and back to RGB when using mlt_image_format=rgb24


#62

I have the impression that all transitions and track compositions occur in YUV also. So regardless of filters, there will be a conversion to YUV if you use the timeline. But maybe I am mistaken in my understanding.


#63

Still, I would be satisfied if HuffYUV had the same facility for setting pixel format as FFV1.


#64

No, the track blending is using frei0r.cairoblend, which is RGB, and there is an optimization to not blend if not needed. You are correct about all transitions at this time. I made a list of the YUV filters in this post:


#65

The HuffYUV bitstream format can be found here:

https://multimedia.cx/huffyuv.txt

Here is a snippet where the image format is declared in the header:

To find out image format you need to get true bit count: it is stored in 
bpp_override if biSize > sizeof(BITMAPINFOHEADER), else (or if bpp_override 
is 0) it's stored in biBitCount.
Then you can find out image format with this table

+-----+-------+
| bit | image |
+-----+-------+
| 16  | YUY2  |
| 24  | RGB   |
| 32  | RGBA  |
+-----+-------+

In the rest of this document YUV will indicate image of type YUY2, and RGB 
will be used for both RGB and RGBA.

Scanning through the rest of the document, and the way that the data itself is stored, the format by nature cannot store higher than 4:2:2. It only does YUY2 encoding. The table above is consistent with the pixel formats indicated by ffmpeg -h encoder=huffyuv. If 4:4:4 is an absolute requirement, then HuffYUV cannot serve your needs.


#66

Fine, then. Just add 4:2:0 support to HuffYUV and we’re good. Maybe also RGB or RGBA

I’ve seen hundreds of satellite feeds come in over the years. They give the pixel format and I can’t remember ever seeing one come it at 4:4:4. They are all either 4:2:0 or 4:2:2.


#67

The source files are (generally?) compressed (I assume to save space and processing power on the source device (camera, phone, Go Pro, etc). When I create an intermediate file I’m effectively uncompressing the file to work with in the editing process? It opens up editing options and is a better start point when it comes to compressing the final edit into a delivery format/file?


#68

I’m not sure what to do with this. The gray box quotes one of my previous posts, and the follow-up text quotes the OP which started this entire topic. I don’t see a new question in this post. The answers to the OP’s questions were detailed in a four-point bullet list at the end of post #9: Intermediate Files for Editing


#69

@chris319, did you download the newly released 18.11.13 version? I’m wondering how you feel about the fixes that were made to the color issue you were reporting.


#70

If I use my eyedropper program on Shotcut’s color picker tool I see 16-180-16 in the color patch in the color picker window, immediately to the left of the fields for entering R,G and B values, however I see 3-154-9 in Shotcut’s preview window.

If I load my .mp4 file which has 16-180-16 over the entire raster, I see 4-189-4 in Shotcut’s preview window.

If I load this same mp4 file into my other color checking program, which reads YUV pixels directly out of the file rather than using an “eyedropper” to read pixel values under the mouse on the screen, I see 16-179-16. An mp4 file encoded with Shotcut’s lossless H.264 shows 0-84-0. I get the same colors from H.264 whether I export as 420, 422 or 444.

Using Ut video I also see 0-84-0. If I view the Ut video file on SMPlayer and use the eyedropper I see 18-221-9, and 17-221-9 if I play it back using VLC.

I get 4-189-3 using ffv1 and Huffyuv with SMPlayer.

I haven’t found a path to accurate colors in the latest version of Shotcut.


#71

Here is the ffmpeg code I am using to encode my bmp to mp4. I get colors +/- 1:

ffmpeg -y -loop 1 -t 10 -r 59.94 -s 1280x720 -i raster.bmp -vf zscale=matrix=709:range=full,format=yuv420p -c:v libx264 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -crf 0 -an output.mp4


#72

-vf zscale

Shotcut/MLT uses libswscale and not zscale, which can explain differences. Also, libswscale has various interpolation and rounding accuracy options that will make MLT differ slightly from some ffmpeg command line executions. Here are the options that MLT uses for libswscale for pix_fmt conversion:


#73

Still, we’re not getting accurate colors even when ffmpeg is not involved, such as when using Shotcut’s internal color generator. See post #70 in this thread. Note that there is a difference between the preview window and the color patch displayed by Shotcut’s color picker.


#74

There is still a bug in the player/preview if you load a file with BT.601 color into a BT.709 project (a color generator or still image defaults the project to 709), then the colors are not transformed and you get the problem you describe. If you change Settings > Video Mode to SD NTSC. Then, you get something much closer to expectation:

What is fixed in v18.11.13 is if you use the color generator and then export, you get more accurate conversion even if the target is BT.709:

The bmp shown in Paint was generated using ffmpeg 4.0.2 from the MP4 exported from Shotcut using its color generator and lossless H.264. The eyedropper was made from the bmp loaded into Paint. Not great, but not as bad as before and what you are reporting.

I do not know about your MP4 as I do not have it (generated using that ffmpeg command above?), but maybe some mismatch between the source and target colorspace coefficients are causing a problem somewhere that requires further investigation. I do not know much about zscale, but why is “range=full” while the format is not yuvj420p? That looks like a problem to me like it would be using full 8-bit range without signaling it. When I run that command with range=full and range=limited, both give color_range=tv in the ffprobe output.

Here is the result of loading range=limited into Shotcut, and using the eyedropper on its player:

While range=full gives inaccurate color:

Lastly, here is range=full,format=yuvj420p in Shotcut, which is fairly accurate:


#75

Yes, that is the code I used to generate the raster mp4 file:

ffmpeg -y -loop 1 -t 10 -r 59.94 -s 1280x720 -i raster.bmp -vf zscale=matrix=709:range=full,format=yuv420p -c:v libx264 -color_primaries bt709 -color_trc bt709 -colorspace bt709 -crf 0 -an output.mp4

I found that I got inaccurate colors if the range was not explicitly set to “full”. I can try this with yuvj420 pixels, with and without explicitly setting it to “full” and see what the results are. Not only do I have to check to see if the colors are right but I also need to see if the video will play back at all on VLC, SMPlayer and Windows Media Player. I will report my findings here.

I have a couple of programs which I wrote and use to check video color. Some work with RGB and some with YUV. My code has been scrutinized by guys on another forum so I’m pretty confident of it.

All of my work is in the BT.709 color space. IMO BT.601 is passe and has no business in computer video in 2018. BT.601 was only ever intended to be a bridge between analog and digital video.

Only recently has the Chrome browser started decoding video in the BT.709 color space. Microsoft Edge uses BT.709 but Firefox is stuck in BT.601. I and others have been “squeaky wheels” and it is an open Firefox bug but no action for quite some time. The standard for computer, and by extension, web video is sRGB, which uses the same coefficients as BT.709, so you decode the colors as if they were 709. It is logical that Edge uses 709 because Microsoft developed sRGB in conjunction with Hewlett Packard. Standards are wonderful things provided you stick to them.


#76

Just to add. I wish I’d found this thread earlier. And as a sidenote, from all the research I did this thread is of the highest quality I have found on the subject.

Recently after doing tests between Prores vs FFV1 with a view of using as a mezzanine codec on my limited hardware this is what I found. Prores at standard quality which is approx. 100Mb/s for 1080p (is it different for 4k?) and FFv1 whose files are approx %20 larger. I could see a visible difference in quality, I guess because FFv1 is lossless but supposedly Prores at standard quality is perceptably lossless (according to some at this profile). Just mentioning…

However, it quickly became apparent to me that FFv1 cannot be used as an everyday editing/mezzanine/intermediate codec unfortunately. Decoding speed is too slow. Encoding speed was slower and could not peak my cores consistantly.


#77

So we’re to infer that FFV1 looked better to you than ProRes? If FFV1 looked better to you than ProRes then ProRes isn’t exactly “perceptually lossless”, now is it? :slight_smile:

Here’s what I do if delivering to YouTube or something similar: edit in the native format, probably mp4. As you have to upload an intermediate file to YouTube, export to a lossless format such as FFV1 or HuffYUV and upload that to YouTube. It’s going to be a heckuva long upload so you might as well start the upload and go to bed. YouTube is going to resample your video no matter what, so you save a generation of lossy encoding by making your “upload” file a lossless format. To borrow a film term, this would be an interpositive.

I have a 2 TB hard drive and disk space has not become an issue yet, but if you need to recover the disk space just burn your interpositive to DVD.

I tested FFV1 and HuffYUV with Shotcut’s white-noise generator. White noise is random and will give a codec a hard time encoding and decoding. In my casual experimenting, players seemed to have an easier time with HuffYUV. In FFV1 I had audio “hash” but the video was frozen. It didn’t look like the swarm of crawling ants that you would expect. HuffYUV showed me a swarm of crawling ants so that is my preference for interpositives. I’m under the impression that HuffYUV and FFV1 are truly lossless and the rest, including ProRes and lossless H.264 are “quasi-lossless”. Is this about right? One of the best features of Shotcut is the wide array of available export formats. I have looked at commercial NLE programs and none had the lossless export formats of Shotcut. As of version 18.11.18 the lossless formats are exporting without color error, so many thanks to Dan for that.

Here is a short video shot in H.264/mp4, the title added in Shotcut, exported to HuffYUV and the HuffYUV mkv file uploaded to YouTube:


#78

Which was the point I was trying to make, I guess not very well or obviously.

That FFv1 doesn’t work well with white noise seems interesting alright. My tests with Prores and FFv1 were with my own files for an editing job and more importantly was quick and dirty just enough to convince me to re-encode my intermediate files with Prores albeit with a slight loss in quality. :wink:


#79

One wonders if the so-called “perceptably lossless” codecs are really lossless to the naked eye or if they’re “lossless enough”. One could make a whole study of this.

I figured white noise would be a good way to torture test a codec. I liked the results I got with HuffYUV.


#80

Fortunately, Apple already made the whole study for us. :slight_smile: They’ve published a white paper on ProRes, and Page 25 lists the all-important target bitrates needed to be visually lossless:

In @independent’s case, a 1080p video (I will assume at 30p) would require 147 Mbps at vprofile=2 (ProRes 422) and that profile doesn’t claim to be visually lossless. vprofile=3 (ProRes 422 HQ) is the first visually lossless profile, and it requires 220 Mbps. So, if ProRes had only 100 Mbps to work with, then there’s no way it could meet the visually lossless expectations put on it at that bitrate and of course FFV1 would look better. ProRes was getting only half of the bitrate it needed to do its job.

To get the most out of ProRes in Shotcut, make sure the Quality slider is always at 100% so that it uses the full target bitrate specified on Page 25 of the white paper. Then, on the Export > Other tab, add vprofile=2 if you want “standard” mode or add vprofile=3 if you want visually lossless mode. Any setting less than 100% will not give ProRes enough bitrate to do its work.

ProRes, DNxHD, Cineform, and the like are plenty lossless to the human eye even after many generations of re-encoding. But they have to be given the full bitrate they need.