Show HN: A layer on top llama.cpp enabling big MoE streaming on phone and laptop

2026-10-20来源 Hacker News ShowEnglish

Adaptive Streaming Optimizer: Run any Big MoE model on any device, any OS, any HW, local OpenAI-compatible server. Android, Windows, Mac, IOS, Linux. CPU, GPU, NPU (35B on phone 12gb RAM 10 tok/s decode 100-300+ prefill, Windows, Mac even more: 25+ tok/s 16 gb RAM)

Comments URL: https://news.ycombinator.com/item?id=50168132

Points: 1

# Comments: 0

原文：https://www.a8q9.cn/en/article/show-hn-a-layer-on-top-llama-cpp-enabling-big-moe-streaming-on-phone-and-laptop-5a15a5.html

相关文章