Adding song subtitles (Karaoke-style) using Overlay HTML filter/Webvfx

jonray · April 23, 2018, 2:56pm

So - I set myself a challenge - how easy is it to add subtitles to a song (Karaoke-style) using the Overlay HTML filter together with Elusien’s Webvfx framework?

Well, I did it - I took the hilarious “Alien Song” video and added a “simple” fade - in/ fade-out" effect just by applying a 50-second HTML overlay. The result is below (If anyone is not familiar with this video, I strongly suggest you watch it right until the end… it’s funny! And just a minute long):

(https://streamable.com/1u82i)

The main problem was the time it took tweaking the timings of the html/webvfx animation. It took a long time (an hour or so), and took a LOT of trial and error! An added problem was that each time I tweaked an individual line of the song’s beginning and end-points, I had to play the clip from the beginning each time to check it came into view at the right time. Very time-consuming.

An advantage of this method, however, is that once the timings have been worked out, it’s pretty easy to change global things like using different text animations, such as moving the text up and down, each subtitle replacing the other, like THIS:

https://streamable.com/iiah1

Or finally, one more example in which I’ve added a translucent grey box and made the text fly off to the right as it is replaced:

https://streamable.com/5zw0l

All in all I thought my experiment was useful and may be to others so I’m sharing it here.
Regards
Jon

Elusien · April 23, 2018, 7:26pm

Brilliant! Thanks for sharing.

john_solo · April 24, 2018, 4:54pm

Looks great! Are you going to share your code?

Elusien · April 28, 2018, 1:31pm

I’ve been creating an exploding/imploding text option for my WebVfx framework and thought this might be a good test vehicle. So here is my karaoke version - enjoy.

sauron · April 28, 2018, 5:25pm

Very impressive. Don’t think I have anything that could replicate this.

jonray · April 29, 2018, 11:15am

Wow, that’s really impressive, Elusien, and great fun. Looking forward to experiment with it when your new webvfx framework is out. Thanks!

jonray · April 29, 2018, 11:21am

Hi Sauron,

As per your suggestion on your recent thread, I tried to replicate your version of Alien song and have succeeded in coming up with this, done by creating html/css/webvfx code and importing it into the Overlay HTML filter in SC:

https://streamable.com/xfk3j

It was a valuable training exercise for me, and I learned a lot from doing it. Not quite identical but pretty close I think. With more work I could get it closer yo yours…
I’ll post the html code when I get more time, later today or tomorrow.
Cheers!
Jon

PS Hope you don’t mind if I copy the link to your version here so others can compare:

https://streamable.com/ozymy

jonray · April 29, 2018, 11:23am

Hi John, indeed I will post the code here when I find a little spare time. Thanks.

sauron · April 29, 2018, 11:25am

It’s brilliant what can be done with html/css/webvfx.

Don’t mind at all.

jonray · April 29, 2018, 1:18pm

Indeed! By the way, I wasn’t sure which font you used, but I found it from a site called identifont. It’s called Harlow, and it was first published in 1979!!! Pretty old…

Elusien · April 29, 2018, 2:01pm

Here is the code for the subtitles I created for the movie (ignore the “f=???”, that is the ‘framestamp’ and only used as an aide-memoire) :

<!DOCTYPE html>
<html lang="en">
<head>
	<meta charset="utf-8">
	<title>Subtitles</title>
	<style>
		html, body	{margin:0; padding:0; width: 100vw; height: 100vh; background-color: transparent; overflow: hidden; border: solid 1px #000;}
		#band		{position: absolute;  width: 100vw; height:   6vh; left: 0; top: 94vh; padding: 0 0; background-color: #000;
					 font-size: 4vh; font-family: Orbitron, 'Bauhaus 93', cursive; font-weight: bold;}
		div 		{position: absolute;  width: 100vw; height:   6vh; left: 0; top: 0; text-align: center; background-color: transparent; color: #ddd;}
	</style>
</head>
<body data-control='60:30'>
<div id='band' class='webvfx' data-animate='{start: 0.0, end: 1.0, 0%: {backgroundColor: #a00;}, 50%: {backgroundColor: #0a0;}, 100%: {backgroundColor: #00a;}}'>
	<div f='09;11' class='webvfx' data-explosion='{start: 0.15675 , end: 0.21845, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		At first, I was afraid, I was petrified
	</div>
	<div f='13:02' class='webvfx' data-explosion='{start: 0.21845 , end: 0.27849, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		Kept thinking I could never live without you by my side
	</div>
	<div f='16:20' class='webvfx' data-explosion='{start: 0.27849 , end: 0.35019, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		But then I spent so many nights thinking how you did me wrong
	</div>
	<div f='20:29' class='webvfx' data-explosion='{start: 0.35019 , end: 0.41690, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		And I grew strong and I learned how to get along
	</div>
	<div f='24:29' class='webvfx' data-explosion='{start: 0.41690 , end: 0.47749, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		And so you're back, from outer space
	</div>
	<div f='28:18' class='webvfx' data-explosion='{start: 0.47749 , end: 0.54475, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		I just walked in to find you here with that sad look upon your face
	</div>
	<div f='32:19' class='webvfx' data-explosion='{start: 0.54475 , end: 0.56365, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		Huh!!!
	</div>
	<div f='33:23' class='webvfx' data-explosion='{start: 0.56365 , end: 0.61534, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		Weren't you're the one who tried to hurt me with goodbye?
	</div>
	<div f='36:26' class='webvfx' data-explosion='{start: 0.61534 , end: 0.64647, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		You think I'd crumble?
	</div>
	<div f='38:22' class='webvfx' data-explosion='{start: 0.64647 , end: 0.67927, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		You think I'd lay down and die?
	</div>
	<div f='40:21' class='webvfx' data-explosion='{start: 0.67927 , end: 0.70762, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 90%: explode}'>
		Oh, no, not I
	</div>
	<div f='42:12' class='webvfx' data-explosion='{start: 0.70762 , end: 0.80000, begin: hidden, ease: easeInOutSine, 0%: implode, 20%: wait, 20%: explode}'>
		I WILL SURVIVE
	</div>
</div>
</body>
<script src='./webvfx.js'></script>
</html>

jonray · April 29, 2018, 6:58pm

Thanks, Elusien. Will use this to get to understand how it works.

Here is the full HTML/CSS/Webvfx code of the top video subtitles (simple fade in/out):

    <!DOCTYPE html>
<html lang="en"><head>
<meta charset="utf-8">
<title>I Will Survive subtitles</title><style>
	body {
	background-color:powderblue:
	}
	#container {
    height: 1080px; 
    width: 1920px; 
    padding: 0;
    top: 50%; 
    left:50%; 
    position: absolute;
	margin: -540px 0 0 -960px;
	background-color:transparent;
    overflow:hidden;
    opacity:1;
}
#global_fade {
    height: 1080px; 
    width: 1920px; 
    padding: 0;
	top: 50%; 
    left:50%; 
    position: absolute;
	margin: -540px 0 0 -960px;
	background-color:transparent;
    overflow:hidden;
    opacity:1;
}
#text_container {
position:absolute;
top:850px;
height: 90px;
width:100%;
overflow: hidden;
background-color:transparent;
}
#subtitle01 {
position:absolute;
height:90px;
width:1800px;
left:50%;
margin-left:-900px;
text-align:center;
font-family:arial;
color:white;
font-size:60px;
text-shadow: 2px 2px grey;
background-color:transparent;
opacity:0;
}
	</style></head>
	<body data-control="50:30 ">
	<div id="container">
	<div id="global_fade" class="webvfx" data-animate="{start: 0.6, end: 0.9, ease: &quot;easeInOutSine&quot;, 
  0% : {opacity:1;  }, 
  100% : {opacity:0;} 
  }">
  
    <div id="text_container">

  	  <div id="subtitle01" class="webvfx" data-animate="{start: 0.000, end: 0.075, ease: &quot;easeInOutSine&quot;, 
  0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
  }">At first I was afraid, I was petrified,</div> 

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.075, end: 0.160, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">Kept thinking I could never live without you by my side,</div>

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.160, end: 0.260, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">But then I spent so many nights thinking how you did me wrong - </div>

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.260, end: 0.300, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">And I grew strong, </div>

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.300, end: 0.340, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">And I learned how to get along - </div>

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.340, end: 0.380, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">And so you're back - </div>

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.380, end: 0.420, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">From outer space ... </div>

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.420, end: 0.520, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">I just walked in to find you here with that sad look upon your face ...</div>

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.520, end: 0.530, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">Huh ...</div>
   
  <div id="subtitle01" class="webvfx" data-animate="{start: 0.530, end: 0.600, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">Weren't you the one who tried to hurt me with goodbye? </div>

  <div id="subtitle01" class="webvfx" data-animate="{start: 0.600, end: 0.640, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">Do you think I'd crumble? </div>

<div id="subtitle01" class="webvfx" data-animate="{start: 0.640, end: 0.700, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}"> Did you think I'd lay down and die? </div>

<div id="subtitle01" class="webvfx" data-animate="{start: 0.700, end: 0.740, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}">Oh no, Not I .... </div>

 <div id="subtitle01" class="webvfx" data-animate="{start: 0.740, end: 0.780, ease: &quot;easeInOutSine&quot;, 
 0%: {opacity:0;},
10%: {opacity:0.8;},
20%: {opacity:1;},
90%: { opacity:1;},
100%: { opacity:0;} 
}"> ... I will survive .... </div>
  </div>
    </div>
	  </div>

john_solo · April 30, 2018, 1:10pm

Thanks, @jonray! Do you have a trick for determining the start and end times, or do you calculate it with timecodes, or do you try numbers until they work?

Elusien · April 30, 2018, 1:31pm

John,

Do you have a trick for determining the start and end times, or do you calculate it with timecodes, or do you try numbers until they work?

I created a webpage called “Frames_to_Time” to convert “Framestamps” (HH:MM:SS;FF) to Normalised Times (0.0 to 1.0).

Frames_to_Time Utility
When creating some animations it helps to know what “normalised time” corresponds to a certain frame in the clip. A webpage has been created to enable users to input the frames in the format “HH:MM:SS;FF” (where “FF” is the number of the frame). This “framestamp” can be observed between the timeline and the preview screen.

It is particularly useful for example if you are creating a “subtitle” Overlay HTML filter, so you can synchronise the subtitles precisely with the video. This utlity can be found on my website (Shotcut - Elusien's Contributions) at Frames_to_Time. There is a description there of the workflow to use it.

jonray · April 30, 2018, 2:18pm

Yes, thanks Elusien, I was just going to mention your Frames to Time calculator!! - Not tried it out yet, but it’s a genius idea.

John, there was a LOT of trial and error going on before the timings of each subtitle were finalised !! Fortunately in song subtitles each one should be approximately the same length (ie one line of music, usually two or four bars), except for the short ones in “I Will Survive” - eg.“And I Grew Strong” which is half the length of the others (in theory).

So after working out via trial and error that the correct length for the first subtitle was from 0.00 to 0.075 (my data-control is set to 50:30 (50 seconds at 30 fps)), that gave me a starting point for the timings of theother subtitles.

But yes, it took a long time because I had to preview the html from the beginning each time. Elusien’s utility will make it much easier.

Kim_Tae_Hyung · May 2, 2018, 12:59pm

hahhahahaha it’s funny…

but it’s too short