The Speech To Text tool can analyze the audio for a project and generate text in the Subtitles Panel
Speech To Text was added in version 24.10.29.
Using The Tool
- Place your video in the Timeline.
- In the Subtitle panel, click on the Detect speech… button
- Wait for the 2 jobs to complete (the Speech to Text job might take a while to complete depending on the length of your video).
About Speech To Text
Shotcut’s Speech To Text feature uses AI based on OpenAI’s Whisper, courtesy of the whisper.cpp project.
Our builds include a basic model that has decent speed and accuracy but not a big size. (You can think of the model as the brain.) You can download a bigger and better better brain (model) in ggml format and configure it in the Speech to Text dialog, but it will be slower.
The dialog creates two jobs that appear in the Jobs panel: one to export audio and another to convert to text. The results are added to the Subtitles panel as a new top-level Subtitle Track.
Currently, the only GPU our build supports is Apple Silicon. Otherwise, it is heavily multi-threaded on the CPU.
Known Quirks:
- Subtitle items sometimes start earlier than expected. Timing is provided by the model and tool, and we lack the skills and resources to improve this.
- Expect there to be occasional errors. Like humans and non-ideal conditions, it is not perfect. We will not take action on bug reports about some piece of audio not converting to the expected text.
OpenAI has made some warnings about the usage of their Whisper models:
In particular, we caution against using Whisper models to transcribe recordings of individuals taken without their consent… We recommend against use in high-risk domains like decision-making contexts, where flaws in accuracy can lead to pronounced flaws in outcomes.