unigram

Telegram for Windows

Build unigram and bigram language models, implement Laplace smoothing and use the models to compute the perplexity of test corpora.

Get n-grams from text

Fast tokenizer for language models, compatible with SentencePiece, Tokenizers, Tiktoken and more. Supports BPE, Unigram and WordPiece tokenization in JavaScript, Python and Rust.

Next Word Prediction using n-gram Probabilistic Model with various Smoothing Techniques

The Unigram website build with React and Bootstrap.

NLP tutorials and guidelines to learn efficiently

Telegram bot that generates random messages using a Markov chain, now for Node.js

High-precision, zero-dependency Byte-Fallback Unigram tokenizer with dual-offset tracking, arithmetic isolation, and multilingual Unicode protection.

Unigram Tokenization realization from scratch

High-Performance Azerbaijani Tokenizers (30% fewer tokens, 40% faster than multilingual alternatives)

All AUT's principles and applications of artificial intelligence course projects.

NLP extra project at AUT Artificial Intelligence course (Fall 2020)

Word2Vec using Hierarchy Softmax and Negative Sampling with Unigram & Subsampling

Python Web Crawler implementing Iterative Deepening Depth Search

We have implemened an NLP Project to recognize the correct poets. In the project, we have used the bigram, unigram and backoff model for smoothing.

UNB Fall-2018 NLP Assignments 💬

Performance evaluation of sentiment classification on movie reviews

COMP6721: Introduction to Artificial Intelligence Assignments (GOFAI, Machine Learning, NLP)