Failure in deinterlacing?

It doesn’t fill in the missing lines with all-new material. It merges lines 1,3,5 with lines 2,4,6 from the next field and does comb filtering removal to fix offsets. This merge is why the rate drops from 50i to 25p. At this point, there are no more source lines left over to get from 25p to 50p. A deinterlacer has to invent new material to create more frames. All “real” lines were already used up by merging two fields into a single frame. This is why it isn’t trivial to get to double rate. This is why there is a big quality difference between the fast field-doubling deinterlacers and the slow interpolation deinterlacers.

Before you say the next frame should simply merge 2,4,6 with 3,5,7… please remember that the previous frame and this new frame would both have 2,4,6 in them. Perceptually, half the frame has not changed, which means the perceived frame rate will not actually double. If averaging is added to the process, there is a little improvement. But it’s not the same as full temporal restoration by having an interpolator invent totally new and reasonable lines to fill in the gaps.

I wasn’t sure exactly how to interpret this, so for the sake of clarity… An interlaced field is half of a frame, which happens at each 1/50th of a second. That means all fields of a frame are not present until 1/25th of a second intervals. This is why a single-rate deinterlacer drops the frame rate from 50 to 25. It is merging odd+even fields into a single frame. The loss of temporal resolution and the addition of comb artefacts is why the merged footage looks less smooth.

If you are happy with the look of real-time VLC playback, that’s great. That level of quality could theoretically be worked into Shotcut and probably maintain real-time preview provided MLT supported frame rate changes. (I am not a developer and do not speak for them; I’m just sharing what I know.) However, this is not the best that deinterlacing can look. Some people set the bar much higher, and it is possible to get there with extra processing time.

To your point, if MLT supported frame rate modification (converting a source to double-rate), then quickly fabricating frames to get from 50i to 50p could be done by algorithms like w3fdif and bwdif which analyze three source fields to help invent the missing lines. It might be good enough for most people in most situations.

I don’t understand this one. If the conversion renders in the original field rate, that means the output is 50p. To use that output, the timeline would need to be 50p, which is what we’ve referred to here as a “higher project frame rate”. How could the conversion running at 50p field rate be dropped onto a 25p timeline without removing half the frames? Doing so would produce the same not-smooth movement we are seeing today.

Oooooh, as I type this, I think I see where the confusion is now. The missing detail is that for all practical purposes, all processing and output of Shotcut is progressive. When export options are set for interlace, that doesn’t mean processing happens at 50 fields per second despite the sound of it. It means processing happens at 25p, hence 50i converted to 25p… then at export time, the progressive frames are encoded and signaled as interlaced in the metadata. However, there is no 50-field temporal resolution present. It is only 25-frame resolution even though the output format is flagged as interlaced. If Shotcut output was analyzed field at a time, and if deinterlacing had been triggered during processing, then there would be no time advancement from fields 1,3,5 to fields 2,4,6 (assuming TFF) as there would be with true interlaced material. So yeah, this little detail will probably disappoint you as it changes the math of everything.

I think I finally understand what you’re saying now. Given the following 50i video…

FRAME.FIELD
1.1
1.2
2.1
2.2 <-- Export process is at this point
3.1
3.2
4.1
4.2

Let’s say we’re exporting and there is a Size, Position, Rotate filter on the clip, which needs a complete progressive frame to feed to the scaler. Bear in mind that Shotcut is internally storing video as frames at 25p, not fields at 50i. So the first change needed would be a way to flag the timeline as interlaced rather than progressive so that the export process steps through time in increments of 1/50th rather than 1/25th. Then, I’m assuming you want to construct a complete frame at position 2.2 by double-rate deinterlacing. This complete frame would be fed to the scaler, then only the even lines of the scaled image would be fed back into the export stream to overwrite the existing even lines. The odd lines from the scaler would be discarded. The end result, whether exported as true 50i or as 25p with interlace metadata, would show time movement with each new field.

This works in theory (mostly). The all-important detail is how good the deinterlace at position 2.2 happens to be. If the deinterlaced frame is a merge with 2.1 and/or 3.1, then the scaler is going to see “time fragments” of past and/or future. The passage of time will not stay segregated between odd and even lines after a scale, because up- or down-sizing the image will merge lines together (and therefore the points in time they represent). If 2.2 doesn’t look like a totally independent reconstruction of the event happening at 2.2 (which is what motion compensation and neural networks try to fabricate), then we’re going to get fragments of 2.1 and/or 3.1 mixed into 2.2, which means those fragments won’t look new when we view 3.1 next. If 3.1 doesn’t look totally new and different from 2.2 (where motion is concerned), then we aren’t going to perceive an increase in frame rate or smoothness.

The other complication is mixing progressive and interlaced videos on the same timeline. In the above example, if a 25p video is dropped on the 50i timeline and the export process steps through time in 1/50th increments, it means double processing for the 25p videos. In theory, Shotcut could know the clip was progressive, calculate filters for all N.1 positions, cache those images, and reuse them for N.2 positions. That would prevent export times from doubling, and prevent temporal shifts that would happen when filters are applied twice per frame but on alternating lines. It gets even more interesting if a 30p video or even an 8p surveillance video is dropped onto a 50i timeline, which Shotcut allows. The same concepts still work, but the code gets complex.

I’m not sure how else to maintain the 50i feel yet keep the timeline at 25p. Maybe you have a better way and I over-complicated it. Unfortunately, interlace is complicated regardless of the method used, which is why I deinterlace externally before editing in progressive, and call it a day. :rofl:

Happy that you got the basic idea!

If you take lines 1,3,5 from each top field and create lines 2,4,6 as a combination from beighbouring bottom fields. (or vice versa), you get a frequency-halving deinterlace (e.g. 25p from 50i), just as is implemented now. Identical quality you would get when taking lines 2,4,6 from each bottm field and inserting lines 1,3,5 from neighbouring fields. There is no quality difference between both versions, and each produces 25 p. Only when performing both versions in alteration, using all top and bottom fields, you will do real deinterlacing job into real 50p, because all lines from all fields come into use, in their proper moment. Keep in mind that the temporal difference is 1/50 seconds (or 1/60) for all neighbouring fields, in any healthy interlaced footage.

Maybe you want to implement it for a try, to see the immense difference in quality. Use a challenging movie with much motion and edges. You will find that deinterlaced 50p is not only much smoother than 25p, but in addition, the quality loss of the substituted lines is reduced, because those lines are then alternating.

Where to implement? I believe I understand the problems you mentioned.
My first suggestion was to generally deinterlace all 50i clips at the beginning, and from then on internally then treat them just as 50p clips. (as it happens when ou do it with external softwae) Everything would work well, but it would have 3 disadvantages, that’s why my second suggestion was different. Here the disadvantages: 1) converting prior to the processes is basically out of the concept used now. 2) unnecessary spoiled resources in case not needed (if exported into 25p or 50i and no geometrical filters or speed changing effects used). 3) The feature “export interlaced” (if anybody uses it at all) would suffer quality, because top and bottm frames could be exchanged any moment in the 50 Hz timeline, then the re-interlacing catches the wrong lines, causing unnecessary motion blur. But there are no other problems. This shortcome exists only in this case of re-interlacing.

Here I repeat and explain my second suggestion:
Use the existing deinterlacer in double rate and use is the same way same as now. Shotcut already activates it in the necessary steps and places, as Brian has explained it.

– For conversion into 25p, deinterlace can be processed in 25 Hz or 50 Hz, and timeline can be 25 Hz or 50 Hz. (From a 50 Hz-deinterlaced stuff, It does not matter whether un-even or even frames are used in the 25 Hz-export or if the sequence becomes swapped)
–.For conversion into 30p or 50p or 60p, deinterlace with 50 Hz before converting into new framerate (important).
– For speed changing effects, deinterlace with 50 Hz (important).
– For geometrical filters, deinterlace before the filter.

  • If converted into 50i and no geometrical filter and no speed change: no need to deinterlace (as now also done), but can not harm either, if the following point is kept in mind.

Timeline: Only to avoid the “problem 3” (export in interlaced format), it could be useful to keep 25 Hz steps in the timeline. How to keep 25 Hz steps in the timeline while transferring 50 Hz contents: In the present version, interlaced format inplements this automatically: The timeline increments in 25 Hz steps, but each frame carries 2 fields in a temporal distance of 1/50 s, therefore the process transfers (hidden) 50 Hz, as can be seen in the healthy 50i output. The same can be acheived using a 50 Hz timeline but marking each frame which was was a top field, These markings will come in 25 Hz steps. They could guide the points of cutting But this should be ignored during speed changing effects, because they require good temporal resolution. After speed effects or after inserted clips of any other frequency, the raster markings could be resumed s soon as de-interlaced stuff comes again. But all this is only for the option of interlaced export in identical interlaced format, I see no other constellation making this necessary.

I’m adding a feature for the next release to add a “Deinterlace” option to the “Convert to Edit Friendly” conversion. This option adds a bwdif filter to the FFMpeg conversion command which will result in a doubled frame rate.

1 Like

This seems to be a good solution!
Looking forward to it…
Thank you.

This is now available in the beta:

Testing would be appreciated.

It’s fundamentally working. Congratulations!
Needs refinements and easier operation. All problems I faced and report here below, seem only due to the implementation as pre-conversion. That’s why let me first give my suggestion:

Suggestion:

Instead of improving the present implementation, you could remove it for the coming release, and target the same thing (it works very good) as an improvement of the deinterlacing which already was implemented. If I understand rightly, the former deinterlacing does not create extra files, so it needs no extra encoding/compressing. And it automatically happens whenever needed, which is a) for export and b) before doing any geometry-affecting filters. In these cases, double rate should not disturb, only improve. Implementing it there will simplify the use a lot, compared to now.

(It will need careful testing by the programmer, and observing the implications mentioned here Failure in deinterlacing?, but I guess it is possible. With e.g. 25p-footage, the timeline could be set to 50 Hz and the export process steps through time in 1/50th, and it has to know that the material was interlaced, which already exists because this triggers the deinterlacer and can do it even before geometry filter.

No idea if the deinterlacer used for geometry filter is also defined by the dropdown choice in export > advanced > deinterlacer. Maybe you can add a choice “doubling rate” to this de-interlace dropdown, or provide a new setting in the general menu with the options to either run deinterlacing as used to, or run all in doubling rate. But maybe this option is not even necessary and all deinterlacing can be done in double frequency. All scenarios seem to match well with the double rate. Only for export into identical interlaced format, deinterlacer should not be used, but if I know rightly, it is already done this way.

Then the user only needs to chose the project frequency, all else happens automatically.

Since the first opened file automatically presets the project standard, interlaced files should then set the project to their field rate, instead of their frame rate. This is the only point for which a setting should be created (to make it an option), so that the new version can behave identical as the old one. For the quality improvement of the new interlace (which is: avoiding identical images in neighbouring frames), a disable option seems not necessary.

Problems in present Beta-Version (observe only if my suggestion can’t be used):

  1. File size is 10 x larger than the original footage (straight from the camera), even in smallest quality setting, --> Why nor define the quality in relation to the data rate of the original? As default, it should be double, due to the double frame rate. With middle quality, labelled “good”, the result is not usable. File size is 50x larger than original. Windows media player can not play it. VLC player shows complete mess (motion blur). The player of Shotcut can play it, but when pulling it into the timeline, Shotcut crashed sometimes (no response, “inactive” while 0% processor according to taskmanager in Windows 8.1).

  2. Make it easier by removing the “advanced” button in “convert”, and always display all. Otherwise it is dangerous and tricky, because the user is completely unclear if the (hidden) settings are kept or will disappear, and at which occasion this happens, so we have to have it open anyway.

  3. Finding a setting until the export delivers 50 fps seems very complicated. My first try had 50 fps but with images changing in 25 Hz. I got correct result only when during export, de-interlacing is deactivated, which seems not always possible, because it is fixed on YADIF, and in global settings, video-mode is set to 50p /or 60p. But I did not understand the meaning of the tickbox in properties > convert > advanced > “override frame rate”. Maybe it means: match the frame rate of the setting in Main-Menu > “video mode” to the converted result? In this case, the labels could be named better. And it is unclear which purpose has the numeric input for frame rate, and when to use it

  4. Using it in the present way is not easy (like using an external de-interlacer): We have to create separate files for each clip and use more conversion time for the additional enoding/compression step.

a) It requires much more work and we have many things, many of which are higher priority. Please volunteer.
b) It is often a bad idea due to multiple generations of lossy compression.

With middle quality, labelled “good”, the result is not usable.

We have had this dialog option for 1.5 years, and it works good for me and many others. This dialog says it is for “Edit-friendly conversion” and integrated into Shotcut by replacing project files. It is not intended to be a general purpose transcode tool. How it plays in other players is not our concern. Edit-friendly also implies intra-only compression, which is heavier.

Make it easier by removing the “advanced” button in “convert”, and always display all.

I disagree. This dialog is used for several purposes, and many of our users will change things without understanding what they are doing (a.k.a. “dangerous and tricky”). Simply putting the options under a section with the heading “Advanced” is not as explicit as clicking it.

I did not understand the meaning of the tickbox in properties > convert > advanced > “override frame rate”.

You probably do not want to use that with the deinterlace option. In theory, I think it can be used together, but I do not know if we tested that. If it is not giving expected results, maybe we need to disable one when choosing the other.

Our engine does not currently support deinterlace at double rate, and it is not a simple change.

It is my design intention that any of the options can be used in any combination. If anyone finds a combination that does not provide expected results, please report it to me.

In the case of the deinterlace option, you do not need to override a frame rate - the doubled frame rate will be calculated automatically.

I agree. The scope of work is too great to do in a short time - which is why we decided to offer this two-step solution which required much less work to implement.

I just tested converting a PAL SD DV file to 100p using the Blend option, output to MP4, and it worked fine for me. I also tested same clip to 60p using the Interpolation option, output to DNxHD MOV, and that worked as well. I loaded each in Automatic video mode to adopt their frame rate, and they played fine, and stepping frame-by-frame shows a unique image per frame. I also tested with a NTSC HDV input, deinterlace, and unnecessarily specifying a framerate override to 59.94 fps. Maybe it is possible that some source format+codec causes a problem, but that is largely out of our control. DV and HDV are most important to me, that works well, and so this feature (deinterlace + framerate override) is staying.

People should also offer label and tooltip suggestions if there are ways that we can clarify the purpose of each option in the user interface.

Best by people who know most implications. For “override framerate”, suggestion to add “in [where?]”
(in the generated file? /in global Shotcut setting / in default of export definition?)

SInce I found out the combinations by testing, I suggest these label texts:
“Deinterlace field by field (doubling frames)” instead of “Deinterlace”
“Convert frame rate”: (instead of “Override frame rate”)
“Rate conversion mode” (instead of “Frame rate conversion”).
and this dropdown only active when “Convert framerate” is checked (same like the framerate numerical), because if I understand rightly, it is not relevant if deinterlacing alone is checked, and in this context, the dropdown content “Duplicate” would irritate.

I tested deinterlacing 50i (25fps) and at same time convert to higher framerate (100p by ‘blend’; 60p by ‘motion compensated’) and both worked well. So no need to disable one when choosing the other.
Great feature alltogether! Allowing quality results when mixing footage of different standards.

1 Like

P.S: I may have to supplement my own suggestion. What happens when “deinterlace” is not checked? I suppose, converting the frame rate to the project setting? And if “deinterlace” is checked? I guess, no frame-rate converting except the doubling by deinterlace. Are there separate options to de-interlace with and without rate converion? I think yes and they should be kept. To avoid others guessing and to make it self-explaining, the framerate conversion could be set by a ‘radio button’ of 2 checkboxes of which one or none can be selected, so in total 3 checkboxes:
deinterlace field by field (doubling fps)
convert frame rate to project setting ((if that’s what it does))
convert frame rate to numeric input.
with second option as default; last option un-checking second option and vice versa; first option possible alone or together.

For what it’s worth, I like most of the labels the way they are.

“Deinterlace” by itself is the most correct because “field by field” or “deinterlace to field rate” may not necessarily be the final outcome if someone overrides the frame rate at the same time.

If someone wants to know if deinterlace is frame rate or field rate, they can see the value in the numeric input box for perfect clarity. So it isn’t necessary to specify it in the label and risk being in conflict with a frame rate override.

“Convert frame rate” implies a conversion when a conversion isn’t necessarily always happening, such as the case of deinterlace to field rate. “Override” is precisely what’s happening if specified.

I actually like the “Rate conversion method” label @Steve2 proposed, but the current label is equally clear to me.

The second and third options seem redundant to me. The numeric input shows what the rate would be from deinterlace whether single- or double-rate is used, and what the rate would be if an override is specified. One text box covers all the bases. “Convert to Edit-Friendly” works strictly at the media file level and has no awareness of the project frame rate (it doesn’t even require a project to use it), so there can’t be any project-related settings or labels. Hence, the redundancy with option 2.

The lots of guessowork and un-certainty in my case show the necessity of well explaining labels which serve not only for those who have created it and already know the implications.

No, the source frame rate is kept.

And if “deinterlace” is checked? I guess, no frame-rate converting except the doubling by deinterlace.

Correct

Are there separate options to de-interlace with and without rate converion?

That is not as useful since the editor engine can already deinterlace at frame rate. Again, this is not a general transcode tool. It is intended to provide a) optimized (edit-friendly) media and b) fill gaps in the engine (i.e. variable frame rate in addition to the things in Advanced).

To avoid others guessing and to make it self-explaining, the framerate conversion could be set by a ‘radio button’ of 2 checkboxes of which one or none can be selected, so in total 3 checkboxes:
deinterlace field by field (doubling fps)
convert frame rate to project setting ((if that’s what it does))

The convert to edit-friendly feature is not intended to be only project-specific. So, it does not offer to convert to project setting, which may change at a later point. You can already choose the current project frame rate without the need to clutter the UI with an additional option.

“Deinterlace field by field (doubling frames)” instead of “Deinterlace”

That is perhaps too complicated for a label, but it can go into a tooltip. There is already a tooltip that you can review. What could make this more clear is that the numeric frame rate field (when Override frame rate is off) doubles when Deinterlace is checked (although I think there are some exceptions beyond the obvious of progressive source).

“Rate conversion mode” (instead of “Frame rate conversion”).

I do not think this is better or worse, but the strings have already been submitted to translators. I do not like to change them unless it is really beneficial.

I prefer “Convert” here since we have “Frame rate conversion.”

These are helpful comments. I tried to balance clarity and brevity, but there might be room for improvement. I will leave the labels alone for now because of the translation timing as Dan mentioned. But I will also keep them in mind as the UI evolves over time.

Good catch. This is now fixed.

The default value in the Frames/sec field is set to the project frame rate. So, if you choose “Override frame rate” and do not change anything else, it will convert to the project frame rate.

I don’t understand how this works. If I start Shotcut in Automatic video mode with no project in progress, then drag in one of my 240fps slow-mo videos and say Convert to Edit-Friendly, the default override frame rate is 240fps, which came from the video file, not the project. This makes sense to me. But if no project is loaded and it’s in Automatic mode, wouldn’t the override default be 25fps in that scenario?

No. 240 came from the project video mode which was set automatically by Shotcut when you opened the first video file.

Notice that in the menu “Video Mode” is under the “Project” heading. In Shotcut, you are always working in a project - even if you haven’t bothered to save it or name it yet.

1 Like