Top Rust LLM repositories on GitHub

CLI proxy that reduces LLM token consumption by 60-90% on common dev commands. Single Rust binary, zero dependencies

• agentic-coding
• ai-coding
• anthropic
• claude-code
• cli
• command-line-tool

Hundreds of models & providers. One command to find what runs on your hardware.

• llm
• skill
• localai
• gguf
• mlx
• unsloth

Run agents that work based on what you do. 24/7 local screen & mic recording for the superintelligence era

• ai
• computer-vision
• llm
• machine-learning
• ml
• multimodal

Open-source Agent Operating System

• agent-framework
• ai-agents
• llm
• mcp
• open-source
• openclaw

Coding agent for DeepSeek models that runs in your terminal

• cli
• deepseek
• llm
• rust
• terminal
• tui

Memory layer for AI Agents. Replace complex RAG pipelines with a serverless, single-file memory layer. Give your agents instant retrieval and long-term memory.

• ai
• context
• embedded
• faiss
• knowledge-base
• knowledge-graph

The open context layer that gives AI agents grounded, governed SQL across 20+ data sources.

• bigquery
• duckdb
• llm
• openai
• postgresql
• rag

Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local processing. no cloud required. Meetily (Meetly Ai - https://meetily.ai) is the #1 Self-hosted, Open-source Ai meeting note taker for macOS & Windows.

• meeting-minutes
• meeting-notes
• llm
• mac
• windows
• rust

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

• ai
• artificial-intelligence
• deep-learning
• gpt
• llm
• llmops

All-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more.

• claude
• gemini
• ollama
• openai
• function-calling
• cli

The AI framework that adds the engineering to prompt engineering (Python/TS/Ruby/Java/C#/Rust/Go compatible)

• baml
• llm
• boundaryml
• guardrails
• llm-playground
• playground

A CLI tool to convert your codebase into a single LLM prompt with source tree, prompt templating, and token counting.

• ai
• chatgpt
• claude
• cli
• command-line
• command-line-tool

AI enabled pair programmer for Claude, GPT, O Series, Grok, Deepseek, Gemini and 300+ models

• ai-workflows
• artifical-intelligense
• claude-3-7-sonnet
• command-line
• multi-agent-reinforcement-learning
• shell

⚙️🦀 Build modular and scalable LLM Applications in Rust

• ai
• llm
• agent
• artificial-intelligence
• automation
• large-language-model

Fast, flexible LLM inference

• llm
• rust
• uqff

Postgres with GPUs for ML/AI apps.

• ml
• machine-learning
• ai
• ann
• artificial-intelligence
• classification

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

• ai
• ggml
• llm
• ml
• rust

A high-performance LLM inference API and Chat UI that integrates DeepSeek R1's CoT reasoning traces with Anthropic Claude models.

• ai
• anthropic
• anthropic-claude
• api
• chain-of-thought
• claude

A blazing fast inference solution for text embeddings models

• ai
• embeddings
• huggingface
• llm
• ml

Coding Agent Harness

• ai
• claude
• cli
• coding-agent
• llm
• mcp

LSP-AI is an open-source language server that serves as a backend for AI-powered functionality, designed to assist and empower software engineers, not replace them.

• ai
• auto-completion
• developer-tools
• ide
• language-client
• llama

A portable open-source operating system for agents. ~6 ms coldstarts, 32x cheaper than sandboxes. Powered by WebAssembly and V8 isolates.

• agent
• ai
• javascript
• llm
• sandbox
• v8

A secure persistent personal agent server in Rust. One binary, sandboxed execution, multi-provider LLMs, voice, memory, Telegram, WhatsApp, Discord, Teams, and MCP tools. Secure by design, runs on your hardware.

• rust
• ai-agent
• ai-assistant
• llm
• mcp
• sandbox

All-in-one platform for search, recommendations, RAG, and analytics offered via API

• actix-web
• diesel
• postgresql
• qdrant
• qdrant-vector-database
• rust

A fast Rust based tool to serialize text-based files in a repository or directory for LLM consumption

• ai
• cli
• large-language-models
• llm
• rust-lang
• serialization