Learning From What Didn’t Work

The challenges encountered during VoiceLK model training, including speech intelligibility, pronunciation, vocabulary handling, and the improvements that followed.

T.H. Mihiran Thilakarathna

2026-10-21 Visualizing the challenges encountered during VoiceLK model training and how analysis, debugging, and iterative improvements helped build a more reliable text-to-speech system.

Visualizing the challenges encountered during VoiceLK model training and how analysis, debugging, and iterative improvements helped build a more reliable text-to-speech system.

Learning From What Didn’t Work

Understanding the challenges behind training a more reliable Sinhala–English text-to-speech system.

Training our first AI voices was an important milestone, but the early experiments also revealed several challenges. Generating speech was only part of the task; the output also needed to be intelligible, pronounce technical terms correctly, and handle Sinhala–English code-switching consistently.

One challenge was speech intelligibility. Our early model outputs showed that longer and more effective training was needed to improve how clearly the generated speech could be understood. Through continued experiments, we observed a reduction of approximately 43% in the intelligibility gap after longer training.

Pronunciation was another important area. Sinhala and English follow different pronunciation patterns, and technical terms can be difficult to process consistently when both languages appear in one sentence. This reinforced the importance of our custom grapheme-to-phoneme pipeline, pronunciation lexicon, and language-routing mechanisms.

We also encountered a vocabulary issue during development. Some symbols were not handled correctly by the model’s vocabulary configuration. After fixing the vocabulary, we achieved zero out-of-vocabulary symbols in the tested configuration.

These experiments taught us that improving a text-to-speech system requires more than selecting a neural architecture. Data preparation, pronunciation handling, vocabulary coverage, and training duration all influence the final result.

Although the improvements were encouraging, some limitations remained. Further evaluation was still needed to assess intelligibility across different inputs, and formal human evaluation had not yet been completed.

Every unsuccessful experiment helped us identify what needed attention next. By analysing these limitations, we could make more informed decisions and move towards a more reliable VoiceLK system.