Transformers
Transformer-based AI models for natural language processing, computer vision, and other machine learning applications, plus real costs and trade-offs.
posts
Tiny BPE Trainer – A Fast and Lightweight BPE Trainer in C++
Tiny BPE Trainer is a fast BPE trainer in C++17. Train your own tokenizer with this lightweight BPE trainer. HuggingFace-compatible output.
Continue Readingposts
A Fast, UTF-8 Aware C++ Tokenizer for NLP & ML
A blazing-fast, header-only C++ tokenizer with UTF-8 support, vocabulary encoding, and transformer-ready sequences, ideal for NLP and machine learning.
Continue Reading