MCP记忆服务器

MCP Memory Server

A self-hosted semantic memory layer for Claude Code and other MCP clients. Store memories as text, search them by meaning, and capture thoughts from your phone — all on your own infrastructure.

Built with FastMCP, Postgres + pgvector, and OpenAI or local (Ollama) embeddings.

Architecture

Claude Code ──HTTPS──▶ Reverse proxy ──▶ FastMCP server ──▶ Postgres + pgvector
                            ▲                  │
Phone/browser ──HTTPS───────┘                  ▼
  (capture form)                         OpenAI API
                                   (embeddings + metadata)

• Database: Postgres 17 with pgvector — stores text alongside vector embeddings (1536 dims for OpenAI, 768 for nomic-embed-text) and the model that produced each one
• Server: Python 3.13 + FastMCP — Streamable HTTP transport with Bearer token auth
• Embeddings: OpenAI text-embedding-3-small by default, configurable to any OpenAI-compatible API or to Ollama's native API (see Local embeddings). The server refuses to start if the configured model doesn't match the stored vectors
• Search: Hybrid — a vector candidate list fused by reciprocal rank fusion (RRF_K, default 60) with a keyword list that admits only strong matches: memories containing the query's identifier-like tokens verbatim (hostnames, config keys, hashes, error codes; weighted 2×), otherwise memories matching every query term, provided they are also in the vector top 10 (so long memories can't get in on word count alone). Each result's keyword_mode says which applied
• Metadata: the calling client can pass a memory's action_items and memory_type. Otherwise, if OPENAI_API_KEY is set, GPT-4o-mini extracts them (plus tags and entities); it's best-effort and never blocks storage
• Duplicate detection: Cosine similarity check before insert (default threshold 0.95, configurable via DUPLICATE_THRESHOLD; the right value depends on the embedding model)
• Indexing: HNSW (not IVFFlat) — works on empty tables

Tools

Prompts

Setup

Prerequisites

• Docker and Docker Compose
• An OpenAI API key, unless you use local embeddings (Ollama) and client-supplied metadata
• A reverse proxy that handles TLS (e.g. Caddy, Nginx, Traefik)

1. Configure environment

cp .env.example .env

Edit .env:

POSTGRES_USER=memory
POSTGRES_PASSWORD=<generate-a-strong-password>
POSTGRES_DB=memory
DATABASE_URL=postgresql://memory:<password>@mcp-memory-db:5432/memory
OPENAI_API_KEY=sk-...
MCP_API_KEY=<generate-with-openssl-rand-hex-32>

# Optional — scoped API keys that force a source name. Define these in a
# gitignored docker-compose.override.yml rather than here (see "Scoped API keys"):
# MCP_API_KEY_<NAME>=<token>      # Memories stored with this key get source="<name>"

# Optional — embedding provider (defaults to OpenAI):
# EMBEDDING_PROVIDER=openai        # or "ollama" (native /api/embed)
# EMBEDDING_API_URL=https://api.openai.com/v1/embeddings
# EMBEDDING_API_KEY=sk-...        # Defaults to OPENAI_API_KEY; never sent to Ollama
# EMBEDDING_MODEL=text-embedding-3-small
# EMBEDDING_DOC_PREFIX=           # task prefixes for asymmetric models (nomic, E5, BGE)
# EMBEDDING_QUERY_PREFIX=

# Optional — similarity thresholds. These are model-specific (see Local embeddings):
# DUPLICATE_THRESHOLD=0.95         # store rejects a near-duplicate at or above this
# FIND_RELATED_THRESHOLD=0.85      # find_related default
# FIND_RELATED_NEIGHBORS=10        # nearest neighbours find_related checks per memory (1-100)

# Optional — reciprocal rank fusion constant for hybrid search (default 60):
# RRF_K=60

The DATABASE_URL must use the container name (mcp-memory-db), not the service name (db), to avoid DNS collisions if the server container is on a shared Docker network.

2. Start the stack

docker compose up -d

This starts two containers:

• mcp-memory-db — Postgres with pgvector, initialised by init.sql
• mcp-memory-server — FastMCP server on port 8000

3. Configure your reverse proxy

Point your domain at the mcp-memory-server container on port 8000. The server exposes:

• /mcp — MCP endpoint (Bearer token auth handled by the server)
• /webhook/capture — REST endpoint for external capture (Bearer token auth handled by the server)

4. Connect Claude Code

claude mcp add memory-server \
    --transport http \
    --scope user \
    --header "Authorization: Bearer <your-MCP_API_KEY>" \
    -- https://your-domain.example.com/mcp

Restart Claude Code. The eleven tools and two prompts will be available in every project.

Webhook / Capture Form

The /webhook/capture endpoint accepts POST requests for storing memories from external sources:

curl -X POST https://your-domain.example.com/webhook/capture \
  -H "Authorization: Bearer <your-MCP_API_KEY>" \
  -H "Content-Type: application/json" \
  -d '{"content": "Remember this", "source": "curl", "tags": ["test"]}'

You can build a mobile-friendly web form that POSTs to this endpoint. If you use a reverse proxy like Caddy, you can handle authentication at the proxy layer and inject the Bearer token server-side so the API key never reaches the browser.

Backups

The included backup.sh script dumps the database with 14-day retention:

# Add to cron (e.g. daily at 3am). Invoke via bash so a missing execute bit
# on a fresh checkout can't silently break backups.
0 3 * * * /bin/bash /path/to/mcp-memory-server/backup.sh >> /path/to/backups/cron.log 2>&1

Failure alerting (dead-man's switch). Because the most common silent failure is "the cron never ran," backup.sh sends a success heartbeat to an Uptime Kuma Push monitor instead of only alerting from inside the script. Create a Push monitor (heartbeat interval slightly longer than your backup cadence, e.g. 25h), then drop its push URL into a gitignored .backup-env next to the script:

# .backup-env (gitignored — never committed)
KUMA_PUSH_URL="https://<your-kuma>/api/push/<token>?status=up&msg=OK"

If the backup ever fails to run or complete, no heartbeat arrives and Kuma alerts via its own notification channels. If .backup-env is absent, no heartbeat is sent and backups still work.

Local embeddings (Ollama)

Embeddings can run on your own hardware with Ollama, so memory text never leaves your network for embedding. (Leave OPENAI_API_KEY unset as well and nothing goes to OpenAI: clients supply action items themselves, see Metadata.)

Settings for nomic-embed-text (768 dims), with its task prefixes and thresholds calibrated on a real store of about 1,700 memories:

EMBEDDING_PROVIDER=ollama
EMBEDDING_API_URL=http://ollama:11434/api/embed   # the default for ollama
EMBEDDING_MODEL=nomic-embed-text
EMBEDDING_DOC_PREFIX="search_document: "
EMBEDDING_QUERY_PREFIX="search_query: "
EMBEDDING_NUM_THREAD=2          # match to your CPU; omit to let Ollama choose
EMBEDDING_MAX_TOKENS=2048       # nomic's context; longer inputs are truncated
DUPLICATE_THRESHOLD=0.975
FIND_RELATED_THRESHOLD=0.90

Why these thresholds: nomic's cosine scale runs higher than OpenAI's (unrelated memories score about 0.63, against 0.34). Light edits of a memory (typo, added line, dropped sentence) all scored at least 0.981, while distinct memories about closely related facts reached 0.962. 0.975 sits between the two.

Memories longer than the model's context are embedded from their start only, and are flagged metadata.embedding.truncated. Keyword search still indexes their full text.

The Ollama container must share a Docker network with the server. Pin the model: re-pulling a tag can change its weights, and with them every vector.

Switching an existing store to another model

Vectors from different models can't be compared, so switching means re-embedding every memory. migrations/ has the steps for OpenAI → nomic:

1. 001-embedding-model.sql adds the embedding_model column. Run it before deploying a server version that writes it.
2. 002-nomic-768.sql phase A adds staging columns while the server keeps running. backfill_embeddings.py fills them; it can reuse vectors from an earlier run where the content is unchanged.
3. Stop the server, rerun the backfill to catch late writes, then run phase B. It swaps the columns in one transaction and aborts, changing nothing, if any row lacks a current vector. Set the Ollama env above and start the server.

The old vectors stay in embedding_openai. To roll back: stop the server, rename the columns back, re-embed any memories created since the switch with the old model, and restore the old env. Once you're sure, drop the column: ALTER TABLE memories DROP COLUMN embedding_openai; VACUUM FULL memories;

Metadata

Metadata lives under metadata.ai, separate from user-supplied metadata. resolve_action_items, unresolve_action_items and weekly_review read it.

Client-supplied (recommended). The calling model passes action_items (concrete follow-ups the user still has to do; most memories have none) and optionally memory_type (observation, task, decision, idea, reference, person_note, meeting) to store_memory, store_memories or update_memory. The convention reaches clients through the tool descriptions and server instructions, so no per-client setup is needed. These are stored with extracted_by: "client". The caller has the conversation's context, so it can tell a real to-do from a plan it's only describing, and the text goes to no one new.

Server-side extraction (fallback). If the client passes neither field and OPENAI_API_KEY is set, GPT-4o-mini extracts the type, 1–3 topic tags (merged into tags), entities and action items. It's skipped for near-duplicates and never blocks storage. Leave OPENAI_API_KEY unset to turn it off.

Replacing a memory's action_items through update_memory resets which of them are resolved.

Authentication

Authentication uses a Starlette BaseHTTPMiddleware that validates the Bearer token at the HTTP layer, before FastMCP processes the request. This works around a known FastMCP bug where get_http_headers() returns stale or missing headers during tool execution with the Streamable HTTP transport. If FastMCP's own middleware sees your headers correctly in a future release, you could switch back — but the Starlette approach is arguably more correct anyway since auth belongs at the transport layer.

Scoped API keys

You can create additional API keys that force a source value on all memories stored with that key. This is useful for external integrations where you want to identify the origin without trusting the caller to set it.

Keep these keys out of version control: define them in a gitignored docker-compose.override.yml, which Compose merges automatically over the base file:

# docker-compose.override.yml (gitignored — do not commit)
services:
  server:
    environment:
      MCP_API_KEY_<NAME>: ${MCP_API_KEY_<NAME>}   # forces source="<name>"

The suffix after MCP_API_KEY_ becomes the forced source name (lowercased). The token values themselves live only in your untracked .env.

MCP client compatibility

Some MCP clients (including Claude Code) double-serialise structured parameters — sending '["a","b"]' (a JSON string) instead of ["a","b"] (a native array). The server handles this transparently using Pydantic BeforeValidator on all list and dict parameters, so tags and metadata work regardless of client behaviour.

Docker networking note

The server container needs to be on your reverse proxy's Docker network so the proxy can reach it. The database container should stay on the default (internal) network only. If both containers share a network with other stacks, use explicit container_name values and reference those in DATABASE_URL to avoid DNS collisions with common service names like db.

Credit

Inspired by Nate B Jones and his Open Brain guide. His approach uses Supabase and OpenRouter — managed services that are faster to set up. This project takes the same concept and makes it fully self-hosted.

License

MIT