Neuro: a compiled language for AI, with a compiler written in Rust
I started Neuro about a year ago. You write the model, the training loop and the GPU code in one language, and the compiler gives you a native binary that runs without Python. The compiler is written in Rust, and the language borrows a lot from Rust too.
This is from a program in the repo that trains on the CPU and predicts on the GPU:
@grad func loss(w: &mut Tensor<f32, [2, 1]>) -> Tensor<f32, []> { val x: Tensor<f32, [4, 2]> = [[1.0, 2.0], [2.0, 0.0], [0.0, 1.0], [3.0, 1.0]] val y: Tensor<f32, [4, 1]> = [[5.0], [2.0], [2.0], [5.0]] val miss = x @ w - y return Tensor::scalar((&miss * &miss).mean()) } @gpu func predict(x: &Tensor<f32, [1, 2]>, w: &Tensor<f32, [2, 1]>) -> Tensor<f32, [1, 1]> { x @ w } Shapes are part of the type, so swapping the product in loss to w @ x fails at compile time:
error: the operands of `@` do not multiply: &mut Tensor<f32, [2, 1]> against Tensor<f32, [4, 2]> Ownership will look familiar. Values move by default, and there are borrows with exclusivity checks, lifetimes and deterministic Drop, with no garbage collector. The rough edge is that a few ownership cases leak a buffer, because the compiler won't free what it can't prove it owns. I'd rather leak than free twice, but I want those cases gone.
The compiler is a Cargo workspace with a crate per stage: logos for the lexer, a Pratt parser, a typed HIR shared by both backends, inkwell for LLVM 23 and melior for MLIR 23. inkwell and melior have to load the same libLLVM, so the build points all of them at one LLVM and MLIR install. melior doesn't wrap mlirTranslateModuleToLLVMIR, so that call goes straight through mlir-sys.
On the CPU, mandelbrot takes 166 ms in Neuro, rustc -C opt-level=3 and clang++ -O3. Rust does win some. format_floats is 38 ms in Rust against 115 ms in Neuro, because Neuro calls the C library conversion and Rust has its own formatter. int_divide is 89 ms against 95 ms. The GPU side is behind: timed warm, PyTorch beats Neuro on all four GPU benchmarks, from about even on matmul to roughly 10x on reductions. The benchmark script refuses to report timings if the implementations disagree on output, and the full tables are in the README.
This is alpha, and breaking changes are expected. There's no neural network library yet, so you write layers and optimizers by hand. That comes after the current phase. Multithreading, a language server and a package manager are also missing. You have to build from source against LLVM and MLIR 23, and GPU code runs on Linux only.
The license is shared source, not open source. You can use and modify the compiler, use it inside a company, and sell whatever you build with it. Shipping the compiler itself in a commercial product needs a commercial license or a renamed, source-available fork.
If you've used inkwell and melior together, or built ownership into a language other than Rust, I'd like to hear what you'd do differently.
Source: GitHub - PanzerPeter/Neuro: An AOT compiled programming language using LLVM · GitHub
Docs: https://neuro-lang.netlify.app
1 post - 1 participant