Building a 227-Million-Word Uzbek Corpus

Ask any ML engineer what the hardest part of building a low-resource language model is and they will say the same thing: data. Not model architecture, not compute — data. For Uzbek the problem is acute. The language switched scripts twice in a century (Arabic to Cyrillic to Latin), much of its literature exists only on paper, and the web text that does exist is heavily contaminated with machine translation and spam.

What 227 million words actually means

Our pre-training corpus is 227 million words of curated Uzbek text. 'Curated' is the operative word — the raw material we processed was several times larger. The pipeline that got us from raw to curated:

• Script normalization — unifying Cyrillic and Latin Uzbek into a consistent representation, including the messy in-between conventions real people use online.
• Machine-translation detection — filtering out text that was auto-translated into Uzbek, which teaches a model exactly the broken patterns we're trying to fix.
• Deduplication — near-duplicate detection across sources, because news syndication makes Uzbek web text extremely repetitive.
• Quality scoring — native speakers labeled samples, and we trained lightweight classifiers to scale their judgment across the whole corpus.

The next frontier: 16,000 books

Web text has a ceiling. The deepest, richest Uzbek — literary prose, academic writing, technical vocabulary — lives in books that have never been digitized. We have a pipeline of 16,000 digitized books queued for the next training run. Book text is qualitatively different from web text: longer coherence, richer vocabulary, and the kind of formal register that enterprise and government use cases demand.

On top of the pre-training corpus sits the layer we consider our real asset: 75,000+ lines of human-written supervised fine-tuning data. Pre-training data teaches a model what Uzbek looks like; SFT data teaches it what to do. Every line was written by a native speaker — instructions, reasoning chains, refusals, translations, cultural explanations.

You can rent GPUs. You cannot rent a decade of Uzbek books or a team of native speakers who know why a sentence sounds wrong.

The corpus grows weekly, and each Tomaris conversation generates signal about where the model is still weak. Data is not a phase of this project that ends — it is the project.