I built a small language model from scratch. Here's how it works.

2026-10-17来源 dev.toEnglish

I've been building Orin AI (AE-Assistant), a small language-model project designed to run locally on ordinary laptop hardware without a dedicated GPU.

The project explores a decoder-only Transformer, custom BPE tokenization, and experiments with recurrent-depth architectures and retrieval-based grounding.

I made a video walking through the project and showing the model running.

YouTube breakdown:

https://www.youtube.com/watch?v=zraZKknc7LQ

GitHub repository:

https://github.com/Harper-16/Orin_AI

I'm still developing it, and I'd appreciate constructive feedback from people interested in small language models and training models from scratch.

What would you improve first?

原文：https://www.a8q9.cn/en/article/i-built-a-small-language-model-from-scratch-here-s-how-it-works-f080d6.html

** I built a small language model from scratch. Here's how it works.**

相关文章

I built a small language model from scratch. Here's how it works.