Jargonautica
Society and Impact

Language Coverage

Models are far better in English than in most languages, and worst in those with least written material online. That gap decides who gets a competent assistant and who gets an unreliable one, and it maps closely onto existing economic inequality.

Technical view

Performance correlates with representation in training data, and tokenisation efficiency compounds it: under-represented scripts consume several times more tokens for the same content, raising cost and consuming context. Approaches include deliberate multilingual data curation, continued pre-training for specific languages, tokeniser adaptation, and community-led dataset construction. Evaluation is itself a gap, since benchmarks in most languages are translated rather than native and miss culturally specific competence.


Part of AI Terminology Index, section: Society and Impact.

How it connects

Language Coverage links to 9 other terms. The faint lines are links between those terms. See it in the full graph.

Training CorpusTraining CorpusData MixtureData MixtureCultural BiasCultural BiasContinued Pre-trainingContinued Pre-train…Content ModerationContent ModerationMachine TranslationMachine TranslationSovereign AISovereign AIDigital DivideDigital DivideByte Pair EncodingByte Pair EncodingLanguage Coverage