rai

Classevelabs/rai

★ 0 stars Rust AI/LLM Updated 1d ago

CPU-only LLM inference engine in pure Rust — 4-bit quantized models, hand-written AVX2 kernels, speculative decoding, and a local HTTP/MCP server. No GPU, no Python runtime.

View on GitHub → 🔍 Audit Wallet Slippage →

Quick Install

Copy the config for your editor. Some servers may need additional setup — check the README.

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "rai": {
      "command": "cargo",
      "args": [
        "run",
        "--",
        "rai"
      ]
    }
  }
}

README Excerpt

**A CPU-only LLM inference engine written in Rust.** RAI runs 4-bit quantized language models with hand-written AVX2 kernels — no GPU, no CUDA, no Python runtime, no PyTorch, no GGML, no BLAS. Load a `.raimodel` file and generate text on any supported x86-64 machine. Built by [ClassEve](https://classeve.com). Licensed under Apache-2.0.

Tools (8)

OPENAI_API_KEYRAI_API_TOKENRAI_CAPACITYRAI_DATA_PATHRAI_EMBEDDING_PROVIDERRAI_HOSTRAI_MCP_MUTATIONS_ENABLEDRAI_PORT

Topics

avx2classevecpu-inferencellm-inferencemcp-serverquantizationrust

rai

Quick Install

README Excerpt

Tools (8)

Topics

Related AI/LLM Servers