Version 16.04 introduces the “Reduce Noise” filter. This filter can be useful for reducing noise or film grain in a source clip. The filter works by creating a Gaussian blur of each frame and then replacing pixels in the original frame with the blurred pixels. Configuration parameters are provided for the radius and strength of the blur. Another configuration parameter defines the threshold to determine if a pixel should be replaced by its blurred version. If the difference between the original pixel and the blurred pixel is below the threshold, then the pixel will be replaced. This feature allows the user to decide how much edge detail should be preserved. A lower threshold value will only blur the flat areas of the image. A larger threshold value will blur then entire image.
I have been learning this filter and experimenting with it yesterday for 2+ hours. I want to share my (hopefully constructive) critique and a suggestion here.
CRITIQUE:
I honestly think this plugin at its current state is useful in a limited context. Depending on the footage treated, it may produce visible improvement in quality on source footage with certain characteristics, but lot of times in my testing it was not producing satisfactory outcome, and I would eventually end up not using it. I do not have image processing expertise in the area of noise reduction, so I can’t tell if the very algorithm used in this plugin has a potential for becoming better with more fine-tuning or not. Currently I will probably not consider using the plugin on most of my footage, except in perhaps special scenarios. I will be looking forward to updates in the future versions of this plugin. But for now I decided to stick to my other workflow for noise reduction with a 3rd party tool. Please see my next paragraph.
SUGGESTION:
I have been working with a 3rd party tool for noise reduction, that produces incredibly good results, fabulously good actually. I would like to share it here, because perhaps your development team (both R&D and the bizdev) could find a way to integrate this. The tool is called Noiseware, and is available both commercially as well as for free as Noiseware Community Edition, which as far as I remember is also open source. I am using the community edition, on Linux via wine. Except for video production, where I started using the commercial edition because of batch processing. I can export video clip as sequence of frames, batch process them with Noiseware, then import back into a video clip. This process is slow, and incorporates extra steps with 3rd party software included, but the outcome is absolutely worth all the hassle - the resulting denoised clips are just plain beautiful. Because of the community edition, which I think is based on open source libraries, I would like to suggest you consider this for bringing a truly spectacular noise reduction capabilities to Shotcut.
Attached please see a side-by-side comparison of a frame of footage before and after noise removal with Noiseware. The source frame comes from your tutorial on video restoration, so that you can compare the outcome to what you can achieve using your current filter:
*Btw, while Noiseware allows for manual fine-tuning of noise reduction profile, the result included came out of a built-in preset. I just use a preset in my workflow, and I have never been let down by the results.
Thanks for your feedback.
I was not aware of Noiseware before you mentioned it. The results you posted to look compelling. Unfortunately, I can’t find any source code releases for Noiseware. It appears to be a “pay only” product and the community edition seems to be very old. If you know where I can get access to the source, please do share.
Regarding the current “Reduce Noise” filter, it is currently using the smartblur filter from ffmpeg.
Documentation:
https://ffmpeg.org/ffmpeg-filters.html#smartblur-1
Source:
The algorithm itself is the same one used in the “Smart Blur” filter in Gimp. There is a video tutorial here that describes the options:
I agree that the filter is not perfect. But I do think that it can be effective in reducing grain and noise in most content. I choose it because, at the time, I found it to be the most effective free and open source filter that is able to run in real time (I found some other filters that were more effective, but could not be run in real time).
I will keep an eye out for any new open source noise reduction filters that come along. If you see any, please let me know.
I do admit that I was not sure about the open source part, just I thought it was. But I will try to find out. Because the Community Edition, despite being old, does great job. The result I uploaded came from the Community Edition.
I guess the key aspect of the critique I gave is down to the point you made: “…that is able to run in real time (I found some other filters that were more effective, but could not be run in real time)” - this point I find very fair. Noiseware definitely is not at the “real time” level, yet it is still pretty fast. Despite my earlier critique, under this argument I do now see where this filter comes from, and I guess I am more appreciative toward it as well.
Now while I am trying to find anything on the source openness (hopefully) of the Noiseware Community Edition, perhaps an intermediate solution - perhaps a plugin could be considered that could delegate the process (after “Analyze” button like in some other filters) to locally installed Noiseware application provided by the user. Depending on a license perhaps the Community Edition could even be shipped with, but if a user had professional edition, the advantage could be taken of the batch mode. Even if this option does seem a bit overkill at first, the results can potentially be soo good, that it might just be worth a shot! After all, I feel it totally pays off to go about the process in a totally manual DIY fashion. It is just so worth seeing the outcome.
Btw, if only you are interested, I could even contribute by trying to write a script for calling Noiseware CE for Windows, Linux and Mac (on Linux and Mac it would depend on wine). You just tell me what calling format you’d like for my helper program, how would you provide frames to me as input. And I can handle the rest.
For Win and Mac plugins are available for some host applications, so perhaps Shotcut could use that (user would provide plugin, SHotcut would provide interface to use it). Linux isn’t covered. I could do what I described.
Thanks for the offer. However, I predict that a plugin to script out to a closed source application in wine is unlikely to be accepted by the project leaders. My suggestion would be to find the best free/open source denoise implementation out there and then see if we can get that integrated somehow. For example, open the Gimp and find out which filters work best in there. Or look at RawTherapee and see what they have available. There are a lot of noise reduction implementations scattered across various open source projects.
Sorry for English, translated by Google translator.
I found for myself a simple super method of cleaning video and pointing the difference(sharpening) through ffmpeg, here is the line for the terminal
ffmpeg -i imput.mp4 -vf edgedetect=low=0.1:high=0.4,hqdn3d=4.0:3.0:6.0:4.5,nlmeans=10:11:1 -vf unsharp=7:7:3:7:7:3,dctdnoiz=4.5,pp=tn/hb/vb -vf unsharp=7:7:3:7:7:3,hqdn3d=4.0:3.0:6.0:4.5,pp=tn/hb/vb/dr -vf dctdnoiz=4.5,unsharp=5:5:1.1:5:5:1.1,pp=tn/hb/vb/ha/va/h1/v1/dr/al output.mp4
This is how the three filters are then three and three and the fourth time three filters in one line.
By coding on my dual-core celeron, the video minute at 720 takes somewhere ~40 minutes. This is better than sharpening and removing noise filters in Shotcut
I have been working with Noise Reduction for restoration of old Super8 films for some months now. Avisynth scripts produce amazing noise reduction while at the same time keeping most of the real detail. The sources of the libarys used should be available. I can’t say that I understand more than the mere basics of those scripts. The free software “Film9” uses some refined scripts that originally came from an avisyth user named videofred and have been modified by quite some other users to a point where it produces awesome quality. Film9 offers a GUI with preview based on Virtualdub2 that will take user entries work them into a script and preview and then process a film. It does not only degraining but also sharpening, motion compensation (1-pass) and some color correction (all optional). On my rather old i7 the is done near realtime depending on the settings of course - a newer processor will probably do that. The question is if those scripts an the dependancies can be packed into a filter for shotcut…
Since you’re restoring old films, maybe you could try using this filter to see how it works. Made a UI for it, just needs some testing. I don’t have old film to use for testing.
Maybe try and see how it works by itself and in conjunction with the reduce noise filter that comes with Shotcut.
hqdn3d.zip (2.2 KB)
Just copy the folder in the zip file to \Shotcut\share\shotcut\qml\filters
If you can dig up where the source code lives, I would be interested to take a look.
hqdn3d was one of the first filters I tried. But it could not perform in real-time on my system.
I don’t have any old film to test the filter. Applied it to regular clips and fiddled with the parameters. The real time performance was good. Maybe all the recent updates to Shotcut makes it perform better?
@sauron
Nice work on the noise filter.
Purposely tried it with the latest version of SC and on the weakest computer I have:
I set the filter then play.
Delay from deselecting to selecting filter “on the fly” is about 2 secs and the time line
does not play in real time but it’s good enough.
If this filter works on this miserable computer, it will work on pretty much anything.
I will give the filter a try and compare it to what I have achieved with Avisynth
The Restoration script that is the basis for the restoration software I have been using is to be found here (original script by videofred and modified by johnmeyer):
the script depends heavily on avisynth and plugins thereof all to be found here:
I am not deep enough into programming as to judge how much effort it would be to facilitate a filter based upon this script but I wager it might not be a simple task.
The script does pretty much everything in one run if the options are set so but the tasks can also be run seperatly (the Film9 Software that uses the script roughly as a basis - heavily modified I am sure - can e.g. only run the degraining which gives me nearly realtime on my 8 year old PC…
Ok I have tried to mimik the settings I have used for an old 1984 Super 8 restoration (link to example above) by adding hqdn3d, sharpen, color grade and stabilize to the clip. I would have taken the whole clip but I couldnt figure out how to auto-white-balance the whole track (the avisynth script offers an auto-white-balance / auto rgb gamma) - so I went for a single snippet just to test (is there such a thing as per track auto-white?). This is the result (Left - processed by Shotcut // Right - processed by Film9):
The whole pipeline (besides stabilize) runs in realtime - that is the good thing. In my opinion the denoising is giving some nice effect - all in all I think it is not on par with the one I have been using for that parcicular task. Spots and scratches mostly stay untouched and increasing temporal filtering did not really do what I expected (do you have some advice on the parameters?). And this was a clean part of the film… Perhaps I was simply holding it wrong!
Denoing /degraining without loosing detail is difficult. I guess I didn’t get the parameters right and therefore sharpning will exagerate the noise deteriorating the result.
I came across Fred’s scripts a few years ago and decided not to use them.
It’s all based on avisynth scripts which are windows only.
These scripts in turn, call pre-compiled filters which are only in .dll (windows) format and for many of these filters, although freeware, no source code is available.
For example this line:
backward = MAnalyse(superfilt, isb = true, blksize=16,overlap=8,search=3,dct=0)
calls the MAnalyse filter with the parameter values for superfilt, isb, blksize, overlap, search and dct and populates the variable “backward”.
The MAnalyse function could be one of several in a dll.
Another limitation I found with quite a few of the avisynth filters is that they don’t all work with all the popular color spaces and conversions before and after calling a filter is common.
I will take your word on that as I have no deep insight into the techs of Avisynth. I knew there is a Linux port (AVXSynth) but not all functions are already working an the plugins probably would need porting too… As I already guessed - probably not trivial and not the main focus of shotcut altough nice degraining and in particular spot/dirt removal might come in pretty handy at times…
More interesting for “everydays life” would be auto-white. I wasn’t able to find that in the filters - white balance seems to work on a single scene/ frame base only?
You could try using the levels filter.
The small example you provided didn’t look too bad. The one processed with Shotcut needs some work to get it to look like the one done with Film9.
For those that use windows, I guess avisynth is a possibility but having to script everything is a bit of a pain.
What some people have achieved using avisynth is pretty impressive.
I did a test using the following filters on SC:
Quite satisfied with the results.
Must add that both noise filters remove exactly what their name says, noise.
They do nothing for scratches, film blemishes and spots.
BTW, have you tried the white balance filter in SC?
EDIT:
A thought, for the odd spot that pops up now and again, thinking perhaps the spot remover filter may do the trick at times.
The scripting is indeed a downside and therefore I went with Film9 that does all that in a GUI. The preview is rendered as you change the settings via a Virtualdub2 window.
I might upload some dirty raw scanned 18 fps footage of that film to tinker with if there is interest.
Why did you add both denoising filters? Is there some guide as to properly use them?
I tried the white balance but that will only work for one scene even if I set the filter for the whole track. As soon as there is a scene change white balance will be off again. Perhaps I have not used it properly? It would be nice if the white would be corrected automatically throughout the whole timeline and over clip borders…
On a sidenote: is there a way to make the color wheels on the color grade tab bigger - even a minicule movement of the mouse changes the color quite a bit - hard to get it on spot.
That would be great, thank you.
Included them both as I suspect they use different algorithms to reduce noise, then it was easy to select/deselect them on the fly and see the changes.
Correct, that is the downside.
If there aren’t too many scene changes, then cutting and creating different clips will allow for independent WB for each.
Painful but do-able in the absence of auto WB.
Completely agree.