Top Rust LLM repositories on GitHub

CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies

• agentic-coding
• ai-coding
• anthropic
• claude-code
• cli
• command-line-tool

Open-source, community-driven agent harness

• cli
• deepseek
• llm
• rust
• terminal
• tui

Hundreds of models & providers. One command to find what runs on your hardware.

• llm
• skill
• localai
• gguf
• mlx
• unsloth

YC (S26) | AI that knows what you've seen, said, or heard. Records everything you do, say, hear 24/7, local, private, secure. Connect to OpenClaw, Hermes agent and 100+ apps

• ai
• computer-vision
• llm
• machine-learning
• multimodal
• agents

Open-source Agent Operating System

• agent-framework
• ai-agents
• llm
• mcp
• open-source
• openclaw

Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.

• ai
• context
• embedded
• faiss
• knowledge-base
• knowledge-graph

Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local processing. no cloud required. Meetily (Meetly Ai - https://meetily.ai) is the #1 Self-hosted, Open-source Ai meeting note taker for macOS & Windows.

• meeting-minutes
• meeting-notes
• llm
• mac
• windows
• rust

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

• ai
• artificial-intelligence
• deep-learning
• gpt
• llm
• llmops

Incremental engine for long horizon agents 🌟 Star if you like it!

• ai
• change-data-capture
• data-indexing
• etl
• indexing
• python

All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.

• claude
• gemini
• ollama
• openai
• function-calling
• cli

The AI framework that adds the engineering to prompt engineering (Python/TS/Ruby/Java/C#/Rust/Go compatible)

• baml
• llm
• boundaryml
• guardrails
• llm-playground
• playground

Run frontier LLMs and VLMs with day-0 model support across GPU, NPU, and CPU, with comprehensive runtime coverage for PC (Python/C++), mobile (Android & iOS), and Linux/IoT (Arm64 & x86 Docker). Supporting OpenAI GPT-OSS, IBM Granite-4, Qwen-3-VL, Gemma-3n, Ministral-3, and more.

• llm
• on-device-ai
• sdk
• stable-diffusion
• vlm
• go

⚙️🦀 Build modular and scalable LLM Applications in Rust

• ai
• llm
• agent
• artificial-intelligence
• automation
• large-language-model

Coding Agent Harness

• ai
• claude
• cli
• coding-agent
• llm
• mcp

A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.

• ai
• chatgpt
• claude
• cli
• command-line
• command-line-tool

AI enabled pair programmer for Claude, GPT, O Series, Grok, Deepseek, Gemini and 300+ models

• ai-workflows
• artifical-intelligense
• claude-3-7-sonnet
• command-line
• multi-agent-reinforcement-learning
• shell

Fast, flexible LLM inference

• llm
• rust
• uqff

Postgres with GPUs for ML/AI apps.

• ml
• machine-learning
• ai
• ann
• artificial-intelligence
• classification

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

• ai
• ggml
• llm
• ml
• rust

A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.

• ai
• anthropic
• anthropic-claude
• api
• chain-of-thought
• claude

A blazing fast inference solution for text embeddings models

• ai
• embeddings
• huggingface
• llm
• ml

LSP-AI is an open-source language server that serves as a backend for AI-powered functionality, designed to assist and empower software engineers, not replace them.

• ai
• auto-completion
• developer-tools
• ide
• language-client
• llama

A tiny, fast Rust CLI that drives a real browser over the Chrome DevTools Protocol — built for coding agents.

• rust
• agent
• browser-automation
• llm

Control what your AI can see. LeanCTX (Lean Context) is the context intelligence layer for AI agents — one local Rust binary that decides what they read, remembers what they learn, guards what they touch, and proves what they save. 60–90% fewer tokens as the receipt. 76 MCP tools, 30+ agents, local-first.

• ai
• cursor
• llm
• mcp
• rust
• token-optimization

A secure persistent personal agent server in Rust. One binary, sandboxed execution, multi-provider LLMs, voice, memory, Telegram, WhatsApp, Discord, Teams, and MCP tools. Secure by design, runs on your hardware.

• rust
• ai-agent
• ai-assistant
• llm
• mcp
• sandbox