Piano video edit, how to achieve this in Shotcut?

I have this source video and need to do 2 things to get desired result.

  1. Make everything dark on the bottom of the screen and only show hands and the keyboard.
  2. Use the lights template background to lighten up the part of the keyboard when keys are pressed.
    keylights.JPG - Google Drive

Any help appreciated.

The result image should look like this

Hi @Viktor_CZ

Is the original video available somewhere so we can test with it instead of using a single image?

Making the bottom part dark (except for the hands) should be doable.
Highlighting each note as the pianist plays is also possible, but that would be a LOT of work.

1 Like

To manually create such an effect in the shotcut program, it will take you a lot of time and I will try to explain why.

The lighting effect can be achieved using color layers and the GPU filter “Glow”, but there is one problem - the pianist’s fingers. If you simply apply a layer of backlight, the keys will glow through the pianist’s fingers. You need to create a finger mask, shotcut does not yet know how to make auto-dynamic masks, you will have to do this manually using glaxnimate, but you need to understand that each frame can take you at least 10 minutes, provided that you already know how it is created in the glaxnimate program. The second task is practically impossible to accomplish in shotcut - an attempt to create illumination on the pianist’s palm from the light of the key. In your example, you can see how the light from the key shines on the pianist’s palm, but in the original video there is no light, without illumination of the palm, the key illumination will look very artificial.

1 Like

Yes sure, I have 2 videos that I want to blend together.

  1. Piano keyboard with hands
    piano_play.mp4 - Google Drive
  2. MIDI generated visual effects video
    _1080p60_High_2024-03-24_22-58.mp4 - Google Drive

In the end result the top part of video2 (falling notes) will fill up blank top of video1 and the lights keys in video2 should blend together with keyboard part of video1.

Thank you for your explanation, I exactly know what you mean by the glow trough the fingers. That is the main reason I am seeking advice here actually.

And I almost resolved this problem 18months ago when I was dealing with exactly the same situation for the first time. I was new to Shotcut and the video I produced must have been just a newbie lucky shot, because it took me only couple of hours and I was very happy with the result. Obviously it is still glowing through the hand, but at least it looks quite real and not so artificial. But then I did terrible mistake, I was not still completely happy with the light effect so I tried to tweak it further and I totally screwed up the whole project and was never able to go back to the same result. I spent several days in frustration that I did not backed up the project trying to replicate my first effort, but I was not able to properly remember what exact steps I took to create this effect so well.
Now after long time I still do not trust myself that I will reproduce this result so I wanted to discuss with someone on how to best approach it.
The good thing is that I still have the original project in “broken” state, and I believe I did use the glow effect together with simple shape mask and opacity, but I am afraid it will not help as I am sure it will be showing the hand rather transparent.

Here is the whole video if you want to see it. I was not happy with the bottom area, even though darkened, it looks like a dark blanket over everything. And the lights are too “squared” but I am sure it can be improved to almost perfection if tweaked properly further.

1 Like

I can offer you a quick, but not very accurate solution to your question. You can use the “motion tracker” filter to track the movement of each hand and attach it to “mask: simple shape”, you need to select the size of the mask and apply it to the backlit layer, adjust the mask so that the backlight does not pass through the mask but only behind it outside. It will look better than now.

Hi @Viktor_CZ - that is a very impressive video you made. Yes, it could be improved with some work, but it works really well as it is. Impressive work. I agree, the hands do vanish at the bottom though.

Nice performance of Erik Satie’s Gnossienne No. 1, by the way!

1 Like

Hi @Viktor_CZ ,
The hands are better - but not perfect…

What I did:
I applied Saturation and Brightness filters to the original video, then expoted as an Mp4 and used this video as “Mask from File”.

Timeline:


Black colour clip on V1
Piano video on V2 with “Mask from file” applied. Invert, and play with threshold values!

It produced some artifacts on the black portion so I crudely used black colour clips to hide them (V3 and V4). This would have to be worked on to avoid the parts of the video where the hands go over to the left or right:

Maybe you could use the spot filter as an alternative?

2 Likes

WOW, that is a clever idea!
Will definitely try this soon.

1 Like

hi @jonray !
Thank you for you feedback!
Unfortunately as I said above, I do not have knowledge on how I did that effect anymore as I destroyed that project with further edits and the video is only thing that is left from that effort so my task is to find out how to do it from scratch again.

As for the performance, its my daughter when she was 8. Now she is 10 and has improved a lot so I’ve just created a new YT channel where I will be posting her work.
My hope is that even my video editing skills will improve so I can make nice videos for her.

And that work you’ve done for me…WOW! I am so impressed!
I was initially bit afraid to post this problem of mine expecting that people will send me to study documentation and not to bother them, but this?
This is the best forum I have ever seen! Not only you get advised but people actually fix the problem for you. I am truly gobsmacked. :slight_smile:

I will look at your steps tonight, try to reproduce them and let you know how it went.
Thank you so much!!!

On a side note, I think I have created unnecessary challenge thinking its gonna be easy to mask the area under the piano. And I thought I will learn something new on the top which looks like I have already. But for future videos I will put some black sheet covering the floor so this edit won’t be required hopefully.
Nice trick with the “Mask from file” though! :wink:

1 Like

My pleasure - glad to help!

Wow - she is certainly talented - 10 years old - that is amazing. I’m a piano/music teacher so I appreciate her talent. She must keep up her practice!! *

Thank you! There are a number of kind folks on this forum all who try to help others out - and often when I help people I learn something myself. All good!!
Yes, having a black floor would certainly help next time. Sometimes a non-tech solution is the best…

PS * This video may be useful for your daughter with her scales…Made with Shotcut, of course! :laughing:

1 Like

Hi @jonray !

After couple of sleepless night I am done with the video! Thank you so much for your help, it was a life saver for me :wink:

I was lucky again to find the way I did it back then, so the key lights part was not actually that hard as I thought and did not take me so long and I I think I managed to improve it to the point where I am happy with it. The way to go here was to use a Dodge blend mode together with Blur and Chroma Key on the lights track and Opacity on the keys track.

The hardest was the bottom part and masking the bottom right corner was very hard, at least I have learnt how to use the Keyframes for the first time (you can see in at 1:00 into my video, not great but not bad)

Let me know what you think?

I still see lot of room for improvement, but at least I have a “know how” now and I made sure each step is saved to a separate project so I don’t loose the work I have done so far and can use it as a template for future videos.

Thank you for the scales video, this method looks very good! It will be useful for me as well as I have started to learn piano too :slight_smile:

4 Likes

Hi @Viktor_CZ - wow, that is superb. Impressed! Thank too for explaining some of your methods, too. I love the way you highlight the keys being pressed. Very effective.
I’d be interested how you were able to generate the falling “blobs”. Is there a program you use for this, and is it free? :laughing:

Well done. Good luck with learning piano too. :+1:

1 Like

Very nice indeed :+1:
Good thing you had this lighted keys file. That saved you a ton of work.

Hi @jonray, thank you for your feedback!
I think its good for now, but I still have some todo list, eg. find the way how to lit up black keys, make different kinds of key press light effects or use the dark bottom area under the hands for some additional effects. I will look into those later.

This video I’ve just made I will leave as a concept, I will re-record everything from scratch and make a new video that will go public to my daughters YT channel. Once I am done I will post it here so you can see if I managed to improve some more.

As for the falling notes, there is a program I use called Keysight. It’s not free, it is around 31$. But I think its a good value for money as it offers a lot of visual effects with MIDI, it’s even possible to use with OBS for live performance.
You can find it on Steam, time to time its on sale so you can get it for better price if you’re happy to wait.

not sure what do you mean by lighted keys file. The old project or video?

It actually did not help me too much, I did not even open the old project and rather started from scratch.
It is very easy to loose yourself in a maze of options in Shotcut and so many ways how to do things, last time I spent days looking for something that takes 10 minutes and gave up.
This time I did edits step by step and once I discovered how to best blend the key lights then I immediately saved and backed up the project so I don’t repeat the same mistake again :slight_smile:
Hopefully have good idea now on how to do it and I hope there is only way to improve and not the other way around :smiley:

I did try to use the motion tracker, but it failed for me, I got lost a bit in the process.
But I found another way that is acceptable for my result, but I definitely want to learn how to use the motion tracker as I have some other things to try with it in my mind.

Thank you for your advice though!

I meant this file that you shared above.
WbKg8HOz9z

chrome_fdaDabwxQB

Didn’t you use it to highlight the piano keys?

Ahh that is what you meant, sorry sometimes I am very slow and don’t understand simple instructions from people :smiley:
And yes I used that video track of course, it is the part of my project.
Pretty much the only way how to do it unless you have a silent grand piano with LED lights fitted which some people use to make videos like that.