Languages

Seven languages are available, with or without a text. English is measured against human raters; the six others have been checked on native speakers only.

$ openpronounce bonjour.wav "Bonjour, je suis développeur" --lang fr

What a native speaker scores

220 recordings of native speakers per language, reading sentences of their own language, then the same recordings with the text hidden. A native reading a long sentence gets about 75 in English; the other languages should land in the same place.

LanguageCodeNatives, readingNatives, no text
Englishen73.770.7
Frenchfr76.073.0
Spanishes76.672.3
Germande76.372.5
Italianit72.570.4
Portuguesept74.672.1
Dutchnl75.770.8

About one word in a hundred read by a native is reported, and a word replaced by another is found three to four times in five.

What is not known yet

No corpus of non-native speech rated by experts exists for these six languages under a licence that allows it. So nobody can say yet how closely the scores follow a French or a German rater. On the few groups of native and non-native speakers that could be compared, natives come out ahead in short answers and read sentences, but not on an Italian text read aloud, where natives reading fast scored under careful non-native speakers.

If you have rated recordings in one of these languages, measure it: your report would be the first.

What they need

Every language other than English needs the espeak-ng program installed, to turn text into sounds. It runs as a separate program; English uses a pronouncing dictionary instead.

Adding a language

A language needs a speech model for its transcription, an espeak-ng voice, and a native reference: a small JSON file computed from about 120 native recordings, placed in a folder that OPENPRONOUNCE_MODELS_DIR points to. The scripts that build it are in the repository, under benchmarks/.