A minimalist implementation of a transformer model in pure Rust, built from scratch without external dependencies.
- 🧠 Transformer architecture implementation
- 🔤 Custom tokenizer
- 📊 Matrix operations
- 🔢 Embedding layer
- 🎭 Multi-head attention mechanism
- 🔀 Feed-forward networks
- 📏 Layer normalization
- 🌡️ Temperature-controlled text generation
- Clone the repository
- Place your training text in the project root
- Run the project:
cargo run --release
Rustformer includes:
- Basic matrix operations
- Tokenization of input text
- Embedding layer
- Positional encoding
- Multi-head attention mechanism
- Feed-forward networks
- Layer normalization
- Training loop
- Text generation with temperature sampling