cmuclmtk

CMU language model toolkit for building and manipulating language models

brewmacoslinux

Try with needOr install directly

About

Language model tools (from CMU Sphinx)

idlm_mixerngram-countngram-mergengram-prunengramtext2idngramidngram2lmevallminterpolate

Count n-grams from a text file$ ngram-count -text corpus.txt -order 3 -write counts.txt

Build a language model from n-gram counts$ idngram2lm -idngram counts.txt -vocab vocab.txt -arpa model.arpa

Evaluate perplexity of a language model on test data$ evallm -lm model.arpa -text test.txt