Training Our First AI Voices

Exploring how VoiceLK uses the VITS neural architecture to learn from prepared speech data and generate Sinhala–English speech.

T.H. Mihiran Thilakarathna

2026-10-21 Visualizing VoiceLK’s first AI voice-training experiments using the VITS neural architecture, with prepared speech data transformed into synthesised speech.

Visualizing VoiceLK’s first AI voice-training experiments using the VITS neural architecture, with prepared speech data transformed into synthesised speech.

Training Our First AI Voices

Turning prepared speech data into a neural text-to-speech model.

After preparing our speech dataset and developing the language-processing pipeline, the next step was to train our first AI voices.

For VoiceLK, we used the VITS (Variational Inference with adversarial learning for end-to-end Text-to-Speech) architecture to generate speech from phoneme sequences. This approach combines text encoding, duration prediction, normalising flows, and neural audio generation within a single model.

Our model uses a Transformer-based text encoder to process phoneme sequences and a stochastic duration predictor to model the timing of speech. Normalising flows and a HiFi-GAN-based component support the generation of audio waveforms.

The VoiceLK model contains approximately 36.3 million parameters. Training involved preparing the text and audio inputs in a consistent format so that the model could learn the relationship between pronunciation sequences and recorded speech.

Building the first models was an important milestone, but training alone was not enough to guarantee clear and natural speech. We also needed to examine the model's outputs, identify pronunciation problems, and evaluate how well it handled Sinhala–English text.

These early experiments helped us understand the challenges of neural speech synthesis and guided the improvements we made in later stages.

Our first AI voices were the beginning of an iterative process: train, listen, evaluate, and improve.