本地LLM RAG服务

Ollama MCP Server

A Model Context Protocol (MCP) server that gives AI agents access to local LLMs via Ollama — including a full RAG pipeline for persistent, zero-cost codebase search.

Why This Project

AI coding agents like Claude Code search codebases by reading files — each file's contents enters the context window, consuming API tokens. Ask a question about a 50-file project and the agent might read 10-15 files before finding the answer. That's thousands of lines of code flowing through the API, most of it irrelevant.

This MCP server addresses that in two ways:

1. RAG Tools — Reduce Context Window Usage

The standout feature. Instead of reading files one by one, the agent calls rag_query with a natural language question. The search, embedding, and retrieval all happen locally — only the relevant chunks (typically 5 snippets, ~200 lines total) enter the context window. The rest of the codebase never touches the API.

• rag_index a project once — all file reading, chunking, and embedding runs locally
• rag_query returns only the relevant code chunks, not entire files
• Persistent across sessions — index once, query repeatedly without re-reading files
• Incremental re-indexing — only changed files get reprocessed

This is where the real token savings happen. Targeted retrieval instead of brute-force file reading.

2. Local LLM Tools — Offload Inference Costs

The other 11 tools (ollama_chat, ollama_code_review, ollama_summarize, etc.) offload inference work to your local GPU. These don't reduce context window tokens — if Claude passes text to a local model, that text is already in the context. But they save API inference costs: the local model does the thinking for free instead of the API model spending output tokens on it.

Be honest about what saves what:

What Makes This Different

• RAG as composable MCP tools — Not a standalone chatbot. The retrieval pipeline sits alongside code review, summarization, and embedding tools in one toolbox. The agent adapts its strategy per question rather than following a hardcoded pipeline.
• Persistent project memory — Each project gets its own .rag/ directory with ChromaDB vector storage. Index once, query across sessions. Re-indexing only processes changed files (content-hash tracking).
• Multi-language code-aware chunking — AST-based extraction for Python, regex + brace-balancing for C/C++, PDF page-group extraction via PyMuPDF. Not naive line splitting.
• VRAM-aware model orchestration — Automatic model swapping on constrained GPUs (e.g. 8 GB). Embeddings use mistral, answer generation uses qwen2.5-coder, with clean VRAM handoff.
• Zero cost, zero data leakage — Everything runs on your machine. No API keys for the local models, no cloud vector DBs.
• Any MCP client gets it — Not locked to one editor or agent. Anything that speaks MCP can use these tools.

Tools

Core LLM Tools (11)

RAG Tools (4)

Architecture

Your question
  -> mistral embeds it (local)
  -> ChromaDB finds top-k similar chunks
  -> VRAM swap (~5s on 8GB GPU)
  -> qwen2.5-coder reads chunks + question
  -> generates grounded answer

Chunking Strategy

• Python: AST-based extraction of functions and classes
• C/C++: Regex + brace-balancing for functions, structs, enums
• PDF: Page-grouped text extraction via PyMuPDF (3 pages/chunk, 1-page overlap)
• Everything else: Fixed 50-line chunks with 10-line overlap

Storage

Each indexed project gets a .rag/ directory:

project/
  .rag/
    chroma/          <- ChromaDB persistent vector storage
    index_meta.json  <- stats: files, chunks, timestamp, model info

Model Routing

The server automatically selects the best model for each task:

VRAM is managed automatically — only one model loaded at a time on constrained GPUs.

Prerequisites

1. Ollama installed and running: https://ollama.com
2. Python 3.11+
3. GPU recommended (works on CPU but slower)

Required Models

Pull these models before first use:

ollama pull mistral
ollama pull qwen2.5-coder:7b
ollama pull qwen3:8b
ollama pull gemma3:4b

Optional (for extended features):

ollama pull qwen3-nothink    # fast mode
ollama pull mistral-16k      # long context summarization

Installation

git clone <this-repo>
cd ollama-mcp-server
pip install -r requirements.txt

Configure Claude Code

Add to your Claude Code MCP settings (~/.claude/settings.json or project .claude/settings.local.json):

{
  "mcpServers": {
    "ollama-agent": {
      "command": "python",
      "args": ["C:/path/to/ollama-mcp-server/server.py"]
    }
  }
}

Then restart Claude Code or run /mcp to connect.

Usage Examples

Index a project

> Use rag_index to index my project at D:/myproject

The agent will call rag_index("D:/myproject") and report:

Indexed 92 files -> 274 new chunks.
Chunk breakdown: python: 150, markdown: 80, json: 44
Storage: D:/myproject/.rag/

Query with natural language

> How does the authentication middleware work?

The agent calls rag_query which:

1. Embeds your question locally
2. Retrieves the most relevant code chunks from ChromaDB
3. Feeds them to qwen2.5-coder for a grounded answer
4. Returns the answer with source file references

Check index status

> What's the RAG status for D:/myproject?

Returns chunk counts, file counts, disk usage, and language breakdown.

Clean up

> Forget the RAG index for D:/myproject

Deletes the .rag/ directory and frees disk space.

Project Structure

ollama-mcp-server/
  server.py          <- Entry point: imports modules, runs MCP
  app.py             <- FastMCP instance (shared across modules)
  config.py          <- Constants: model routes, file extensions, limits
  helpers.py         <- Model resolution, VRAM management, formatting
  chunking.py        <- Code/doc chunking (Python AST, C/C++, PDF, generic)
  rag.py             <- ChromaDB client management, embedding, metadata
  tools.py           <- 11 core LLM tools
  tools_rag.py       <- 4 RAG tools
  requirements.txt   <- Python dependencies

Supported File Types

The RAG indexer handles 45+ file extensions including:

Code: .py, .c, .cpp, .h, .js, .ts, .java, .go, .rs, .rb, .swift, .kt, .cs, .php, .zig, .asm

Documents: .md, .txt, .pdf, .html, .xml

Config: .json, .yaml, .toml, .ini, .cfg

Files over 512 KB (text) or 50 MB (PDF) are skipped. Directories like .git, node_modules, __pycache__, build, .venv are automatically excluded.

License

MIT