Skip to content

TranscriptFor developers, content & research

Clean up YouTube auto captions: punctuation, names, errors

Fix YouTube auto captions without retranscribing: punctuation, names, misheard words and fillers, with every word kept at its time. Tested on 15 videos in four languages.

3 min readBy CleanScriptView as Markdown

On this page
  1. Before and after
  2. Misheard words
  3. Fillers and false starts
  4. What we measured
  5. How to tell what you got
  6. When to rely on it
  7. Next steps

YouTube's automatic captions are made for the screen. As text they are hard to read: no capitals, no punctuation, lines that break mid-sentence, and names heard as the nearest common words. YouTube's own help page says they "might misrepresent the spoken content due to mispronunciations, accents, dialects, or background noise".

CleanScript corrects the caption text itself. It doesn't retranscribe the audio, and every word keeps the moment YouTube's captions say it was spoken, so each line's times are its own words'.

Before and after #

YouTube's automatic captions for Steve Jobs' 2005 Stanford Commencement Address, from 1:00:

Text
I dropped out of Reed College after the
first 6 months but then stayed around as
a drop in for another 18 months or so
before I really
quit so why' I drop
out it started before I was
born my biological mother was a young
unwed graduate student and she decided
to put me up for
adoption she felt very strongly that I

The same passage after correction:

I dropped out of Reed College after the first 6 months, but then stayed around as a drop-in for another 18 months or so before I really quit. So why did I drop out? It started before I was born. My biological mother was a young unwed graduate student, and she decided to put me up for adoption. She felt very strongly that I should be adopted by college graduates.

Punctuation and capitals are back, and "why' I drop" is "why did I drop". The timed lines follow the sentences, each timed by its own words:

JSON
[
  { "start": 64.92, "end": 69.2, "text": "drop-in for another 18 months or so before I really quit." },
  { "start": 69.2, "end": 72.16, "text": "So why did I drop out?" },
  { "start": 72.16, "end": 75.12, "text": "It started before I was born." }
]

This video also has Stanford's own captions, so a normal request returns those, as written. We used its automatic track to measure the correction.

Misheard words #

Some fixes go beyond punctuation. Real ones from the videos we measured:

Automatic caption Corrected Video
I write the blog weight but why Wait But Why Tim Urban, TED
gold, frankincense, and mayo myrrh Sir Ken Robinson, TED
optimistic neoism nihilism Kurzgesagt – In a Nutshell
Please welcome Ted Harrington Kit Harington The Late Show with Stephen Colbert

Fillers and false starts #

The text is clean verbatim: fillers ("um", "uh"), stutters and false starts are removed, and every other word stays, in the speaker's order. Numbers are never changed, and nothing is added beyond a word or two the captions missed. A speaker change (>> in YouTube's captions) starts a new line and a new paragraph.

What we measured #

We took 15 public videos in English, French, Spanish and German that have both automatic captions and captions their creator wrote. We compared the automatic captions word by word with the creator's, before and after correction, ignoring capitals and punctuation:

  • 409 misheard words fixed: wrong in the automatic captions, right after correction.
  • 117 words changed that the creator kept: mostly false starts and fillers, which correction removes on purpose and some creators keep in their captions.

Some errors remain. Creator captions are never corrected.

How to tell what you got #

The source field says it:

source Meaning
creator The creator's own captions, as written
auto YouTube's automatic captions, corrected
transcribed No usable captions, so we transcribed the audio, then corrected it
translated Our translation, with a warning naming the original language
Python
# pip install cleanscript-ai
from cleanscript_ai import CleanScript, transcript_text

client = CleanScript()
result = client.transcript("https://youtu.be/jwnez8HdN7E")
print(result.source)  # "auto" for this video
print(transcript_text(result)[:200])

A passage that can't be corrected within these limits, even on a careful second try, comes back as captioned.

When to rely on it #

Corrected text is right for reading, summaries, search and notes. It is not a verbatim record. Before you publish a quote, open the link to its moment and listen.

Next steps #

Sources

  1. Use automatic captioning - YouTube Help

Spot something wrong? Tell us.