量化金融RAG服务器

FellowQuant RAG Server

A local RAG (Retrieval-Augmented Generation) server built for quantitative finance research documents. Ingests PDFs, EPUB books, LaTeX source files, and Jupyter notebooks with structure preservation — financial tables stay as tables, mathematical formulas stay as LaTeX, code blocks stay intact. Exposes a REST API and MCP server.

Installation

pip install rag-server

After installation, run rag-server setup (or any rag-server command) to configure the MCP server in Claude Code.

Requirements

• Python 3.12+
• Docker (for Qdrant vector database)
• A running LLM endpoint (vLLM, llama.cpp, or AWS Bedrock credentials)
• GPU with ~2.5 GB VRAM (BGE-M3 ~1 GB + Qwen3-Reranker ~1.2 GB)

CLI Commands

MCP Setup

rag-server setup prompts for:

• Global scope (--scope user): MCP available in all Claude Code projects
• Local scope: writes .mcp.json in the current directory (commit to share with your team)

If the claude CLI is not installed, setup writes .mcp.json and prints instructions.

What it does

• Ingests PDFs (via Docling), EPUB books, .tex files, and .ipynb notebooks with layout-aware chunking
• Retrieves using three-mode hybrid search: BM25 keyword + BGE-M3 dense + BGE-M3 sparse, fused via RRF and reranked by Qwen3-Reranker-0.6B
• Answers questions with a local LLM (vLLM, llama.cpp, or AWS Bedrock) and inline citations
• Integrates with Claude Code/Codex via HTTP MCP — 5 tools: retrieve, ask, list_documents, get_document, delete_document

Setup (development / clone-and-run)

For most users: install via pip install rag-server (see Installation above). The steps below are for contributors cloning the repo directly.

1. Install dependencies

uv sync

2. Start Qdrant

docker compose up -d qdrant

3. Configure LLM

Copy and edit the LLM config:

cp llm.yaml.example llm.yaml
# Edit llm.yaml: set provider (vllm/llamacpp/bedrock), model, base_url

4. Run database migrations

uv run alembic upgrade head

5. Start the server

./scripts/start.sh

This sets PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (prevents CUDA memory fragmentation stalls), binds to loopback-only 127.0.0.1:8001, and tees logs to .logs/rag_server.log. Browser-origin requests are rejected because this local service does not expose an authentication layer. The Compose app binds 0.0.0.0 only inside its container while publishing RAG and Qdrant ports exclusively on host loopback.

REST API

All endpoints are under /api/v1. A health check is at /health.

Documents

Query

Retrieve request

{
  "query": "Sharpe ratio under non-normal returns",
  "top_k": 10,
  "document_ids": ["uuid-1", "uuid-2"],
  "min_score": 0.3
}

document_ids and min_score are optional. Omit document_ids to search across all documents.

Ask request

{
  "query": "What is the Kelly criterion for fractional betting?",
  "top_k": 10
}

Returns SSE token stream by default. Add ?streaming=false for a single JSON response.

Errors

All errors use RFC 7807 Problem Details format:

{
  "type": "about:blank",
  "title": "Not Found",
  "status": 404,
  "detail": "Document abc123 not found"
}

Upload size limit: 100 MB (returns 413).

MCP Server (Claude Code)

The .mcp.json at the project root points clients at the shared local MCP endpoint:

http://127.0.0.1:8001/mcp

Tools

Notes

• File upload is done via the REST API (POST /api/v1/documents), not through MCP
• MCP now proxies to the shared FastAPI runtime, so multiple clients do not each load their own BGE-M3/reranker stack

Manual MCP test

# Browse tools interactively (development only)
uv run mcp dev src/rag_server/mcp_server.py

# Re-run setup to (re-)register with Claude Code
rag-server setup

Technology stack

Verification scripts

# Verify the installed wheel (entry points, assets, subcommands)
bash scripts/smoke_test_cli.sh

# REST API — requires running server at localhost:8001
python scripts/verify_api.py

# Earlier phases (run in order against a running server)
python scripts/verify_storage.py
python scripts/verify_ingestion.py
python scripts/verify_retrieval.py
python scripts/verify_llm.py

Data layout

data/
├── uploads/      # Uploaded files, stored as {sha256}{ext}
├── qdrant/       # Qdrant persistent storage (mounted into Docker)
└── bm25.pkl      # BM25 index (rebuilt after each document is indexed)

DATA_DIR environment variable controls the base path (default: ./data).

Configuration

Environment variables (.env)

DATA_DIR=./data   # optional override; default is repo-local rag-server/data
QDRANT_HOST=localhost      # or container name inside Docker
QDRANT_PORT=6330
MAX_UPLOAD_SIZE=104857600  # 100 MB in bytes
RAG_ASK_ENABLED=true       # set false for retrieval-only mode without an LLM endpoint

Retrieval-only mode

Use retrieval-only mode when the workstation GPU needs to run other models and you do not want a resident vLLM/llama.cpp process for /ask synthesis:

rag-server start --no-ask
# or
RAG_ASK_ENABLED=false rag-server start

In this mode, /api/v1/retrieve and MCP retrieve keep working. /api/v1/ask and MCP ask return HTTP 503 with a clear message. Restart with rag-server start --ask or set RAG_ASK_ENABLED=true to re-enable synthesized answers. The RAG server itself only uses an OpenAI-compatible client for llama.cpp/vLLM; freeing VRAM also requires stopping the separate llama.cpp/vLLM server process if it is running.

LLM (llm.yaml)

llm:
  provider: vllm            # vllm | llamacpp | bedrock
  model: Qwen/Qwen2.5-7B-Instruct
  base_url: http://localhost:8000/v1
  context_chunks: 5
  max_context_tokens: 8000

See llm.yaml.example for all options including Bedrock and the system prompt.