Skip to content

Latest commit

Β 

History

17 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Tha (ថអ)

Khmer text normalization: turn written text into spoken words for TTS, and spoken words back into written text for ASR.

Paper: arXiv:2609.30984

Install

pip install tha

Tha needs pynini. On macOS, install it from conda-forge first:

conda install -c conda-forge pynini

Written to spoken

import tha

tha.normalize_text("αž‘αž·αž‰αž“αŸ… 02/01/2024 αžαž˜αŸ’αž›αŸƒ $1.05")
# αž‘αž·αž‰αž“αŸ… αžαŸ’αž„αŸƒαž‘αžΈβ–αž–αžΈαžšβ–αžαŸ‚β–αž˜αž€αžšαžΆβ–αž†αŸ’αž“αžΆαŸ†β–αž–αžΈαžšαž–αžΆαž“αŸ‹β–αž˜αŸ’αž—αŸƒαž”αž½αž“ αžαž˜αŸ’αž›αŸƒ αž˜αž½αž™β–αžŠαž»αž›αŸ’αž›αžΆαžšβ–αž”αŸ’αžšαžΆαŸ†β–αžŸαŸαž“
Input Output
1,234 αž˜αž½αž™αž–αžΆαž“αŸ‹β–αž–αžΈαžšαžšαž™β–αžŸαžΆαž˜αžŸαž·αž”αž”αž½αž“
12.5% αžŠαž”αŸ‹αž–αžΈαžšβ–αž…αž»αž…β–αž”αŸ’αžšαžΆαŸ†β–αž—αžΆαž‚αžšαž™
5km αž”αŸ’αžšαžΆαŸ†β–αž‚αžΈαž‘αžΌαž˜αŸ‰αŸ‚αžαŸ’αžš
10:23 αž˜αŸ‰αŸ„αž„β–αžŠαž”αŸ‹β–αž˜αŸ’αž—αŸƒαž”αžΈβ–αž“αžΆαž‘αžΈ
012 345 678 αžŸαžΌαž“αŸ’αž™β–αžŠαž”αŸ‹αž–αžΈαžšβ–αžŸαžΆαž˜αžŸαž·αž”αž”αž½αž“β–αž αžΆαžŸαž·αž”αž”αŸ’αžšαžΆαŸ†αž˜αž½αž™β–αž…αž·αžαžŸαž·αž”αž”αŸ’αžšαžΆαŸ†αž”αžΈ
-5 αžŠαž€β–αž”αŸ’αžšαžΆαŸ†

It also handles ordinals, fractions, phone numbers, e-mails, license plates, ranges, scores and more. Words of one number are joined with ▁, which you can change:

normalizer = tha.Normalizer(separator=" ")  # reuse it, loading takes ~2s
normalizer.normalize("$1.05")  # αž˜αž½αž™ αžŠαž»αž›αŸ’αž›αžΆαžš αž”αŸ’αžšαžΆαŸ† αžŸαŸαž“

Spoken to written

import tha

tha.inverse_normalize_text("αž‘αž·αž‰αž“αŸ…αžαŸ’αž„αŸƒαž‘αžΈαž–αžΈαžšαžαŸ‚αž˜αž€αžšαžΆαž†αŸ’αž“αžΆαŸ†αž–αžΈαžšαž–αžΆαž“αŸ‹αž˜αŸ’αž—αŸƒαž”αž½αž“αžαž˜αŸ’αž›αŸƒαž˜αž½αž™αžŠαž»αž›αŸ’αž›αžΆαžšαž”αŸ’αžšαžΆαŸ†αžŸαŸαž“")
# αž‘αž·αž‰αž“αŸ…02/01/2024αžαž˜αŸ’αž›αŸƒ$1.05
Input Output
αžŠαž”αŸ‹αž–αžΆαž“αŸ‹ 10,000
αž˜αŸ‰αŸ„αž„αž”αŸ’αžšαžΆαŸ†αž”αžΈαž€αž“αŸ’αž›αŸ‡ 8:30
αžŠαž€αžŠαž”αŸ‹αž’αž„αŸ’αžŸαžΆαžŸαŸ -10Β°C
αž˜αŸ’αž—αŸƒαžŠαž€αžŠαž”αŸ‹ 20 - 10
αž˜αž½αž™αž…αŸ†αž“αž½αž“ αž˜αž½αž™αž…αŸ†αž“αž½αž“
αž–αžΈαžšαŸ„αŸ‡ αž–αžΈαžšαŸ„αŸ‡

A single number word on its own stays a word, and number words inside other words are left alone. For Khmer digits, use tha.InverseNormalizer(khmer_digits=True).

Text cleanup

import tha.normalize
import tha.hashtags
import tha.parenthesis

tha.normalize.processor("αž˜αž·αž“β€‹αž²αŸ’αž™")                  # αž˜αž·αž“αž±αŸ’αž™
tha.hashtags.processor("Hello world #αž›αž»αž” hello")           # Hello world  hello
tha.parenthesis.processor("Hello (ignored) world")        # Hello world

Development

uv sync
make test

On macOS, pynini is built from source, so install OpenFst first:

brew install openfst
CPPFLAGS="-I$(brew --prefix)/include" LDFLAGS="-L$(brew --prefix)/lib" uv sync

Citation

@misc{yath2026tha,
  author        = {Yath, Seanghay},
  title         = {{THA}: Weighted Finite-State Text Normalization and Inverse Text Normalization for Khmer},
  year          = {2026},
  eprint        = {2609.30984},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  doi           = {10.48550/arXiv.2609.30984},
  url           = {https://arxiv.org/abs/2609.30984}
}

About

πŸ“’ Tha (ថអ) - A Khmer Text Normalization and Verbalization Toolkit

Topics

Resources

Stars

11 stars

Watchers

1 watching

Forks

Releases

Used by

Contributors

Languages