语义搜索服务

Semantic Search

Semantic search over markdown files. Find related notes by meaning, not just keywords. Detect duplicates before creating new notes.

Supports two server transports:

• stdio MCP — For Claude Code integration (one process per session)
• HTTP — Combined MCP-over-HTTP + REST on one port; one warm process shared by all clients

Features

• Semantic search using sentence-transformers
• Duplicate/similar note detection
• Auto-updating index with file watcher
• Multi-directory support
• Inline tag extraction (#tag-name)

Install

CPU-only install — recommended for macOS (any Mac, Apple Silicon or Intel) and Linux/Windows without an NVIDIA GPU. Saves ~5GB of CUDA binaries. On macOS, Apple GPU (MPS) is still auto-detected and used via PyTorch's built-in MPS backend — the "CPU" label refers only to the absence of CUDA, not to the compute device at runtime.

uv tool install --index https://download.pytorch.org/whl/cpu \
  git+https://github.com/bborbe/semantic-search

CUDA install — only for Linux/Windows with a dedicated NVIDIA GPU. Not applicable to macOS (NVIDIA CUDA is not supported on Mac).

uv tool install git+https://github.com/bborbe/semantic-search

Upgrade

uv tool upgrade semantic-search

Server Modes

stdio MCP (per-session Claude Code)

Spawns one process per Claude Code session. Simple, but each session loads its own ~400 MB–1 GB model copy.

claude mcp add -s project semantic-search \
  --env CONTENT_PATH=/path/to/vault \
  -- \
  uvx --from git+https://github.com/bborbe/semantic-search semantic-search-mcp serve

Tools available:

• search_related(query, top_k=5) — Find semantically related notes
• get_content(path, snippet, query, context_lines) — Retrieve file content from indexed vaults
• check_duplicates(file_path) — Detect duplicate/similar notes

HTTP (shared across all clients)

Single long-running process serves MCP-over-HTTP at /mcp plus REST at /search, /duplicates, /content, /health, /reindex. All Claude Code sessions and REST clients share one warm indexer, and each request names the scope it wants — so one process gives every client its own view without a second index.

SEMANTIC_SCOPE_MAP=scopes.yaml semantic-search-http --host 127.0.0.1 --port 8321

The variable may be omitted, in which case the scope map is read from ~/.config/semantic-search/config.yaml.

Scopes are declared in scopes.yaml — see design/per-vault-scoping.md for the contract and the fail-closed default. Point Claude Code at it via MCP config; clients differ only by the scope in their URL:

{
  "mcpServers": {
    "semantic-search": {
      "type": "http",
      "url": "http://127.0.0.1:8321/mcp?scope=personal"
    }
  }
}

REST endpoints:

scope is required on /search, /duplicates, and /content. Omitting it returns HTTP 400 MISSING_SCOPE; an unknown name returns 400 UNKNOWN_SCOPE. Repeating the parameter resolves to the union of the named scopes' roots — /search?q=...&scope=personal&scope=work searches both scopes in one request, and the order the scopes are named in does not change the result set. There is deliberately no union-wide fallback: naming no scope, or any undeclared name, fails the whole request closed.

Example queries:

# Search
curl 'http://127.0.0.1:8321/search?q=kubernetes+deployment'

# Find duplicates
curl 'http://127.0.0.1:8321/duplicates?file=notes/my-note.md'

# Health check
curl 'http://127.0.0.1:8321/health'

Two-Step Flow

Search for related notes, then fetch the full content of any result:

# Step 1: Search for related notes
curl 'http://127.0.0.1:8321/search?q=kubernetes+deployment'
# Returns: [{"path": "notes/k8s-guide.md", "score": 0.92}, ...]

# Step 2: Fetch the content of a result
curl 'http://127.0.0.1:8321/content?path=notes/k8s-guide.md'
# Returns: {"path": "/full/resolved/path.md", "content": "# Kubernetes Guide\n...", "mode": "full"}

Snippet Mode

Retrieve a focused snippet around a specific query term within a file:

# Get a focused snippet around "service mesh" in the file
curl 'http://127.0.0.1:8321/content?path=notes/k8s-guide.md&snippet=true&query=service+mesh&context_lines=10'
# Returns: {"path": "...", "content": "...\n## Service Mesh\n...", "mode": "snippet"}

Remote Deployment

get_content and GET /content enable remote deployment of semantic-search clients. Callers no longer need filesystem access to the vault directory — all content retrieval happens over HTTP/MCP from any network location. The server enforces path validation: files outside indexed roots are never served.

Claude Code Plugin

This repo also ships as a Claude Code marketplace plugin with commands for setup, search, and research.

Install

claude plugin marketplace add bborbe/semantic-search
claude plugin install semantic-search

Update

claude plugin marketplace update semantic-search
claude plugin update semantic-search@semantic-search

Quick Start

# One-shot interactive setup: installs the binary, writes the launchd/systemd
# unit, registers the MCP server in your Claude config.
/semantic-search:configure

# Search indexed markdown
/semantic-search:search kubernetes deployment

# Multi-step research across results
/semantic-search:research kafka backup strategy

Commands

Run in Background

For production-style usage, run semantic-search-http as a background service so every Claude Code session (and any REST client) shares one warm process.

Quick example (macOS):

launchctl load ~/Library/LaunchAgents/com.github.bborbe.semantic-search-http.plist

Quick example (Linux):

systemctl --user enable --now semantic-search-http.service

CLI Commands

One-shot commands without running a server:

# Search
CONTENT_PATH=/path/to/vault semantic-search search "kubernetes deployment"

# Find duplicates
CONTENT_PATH=/path/to/vault semantic-search duplicates path/to/note.md

Binaries

Configuration

Environment Variables

Multiple Directories

Index multiple directories by separating paths with commas:

CONTENT_PATH=/path/to/vault1,/path/to/vault2,/path/to/docs

All directories are indexed together and searched as one unified index.

Excluding Files with .semanticignore

Place a .semanticignore file at the root of any vault to exclude paths from indexing. Each vault uses its own rules independently; a missing .semanticignore means "index everything" (no change for existing vaults).

Syntax — gitignore-style patterns powered by the pathspec library's gitwildmatch dialect. The full gitignore rule set applies:

• Patterns are matched relative to the vault root.
• Trailing / matches directories (and everything inside them): archive/
• ** matches across directory boundaries: **/draft.md
• Leading / anchors a pattern to the vault root: /scratch.md
• ! negates a pattern (re-includes a previously excluded path): !archive/keep.md

Behaviour — matching paths are excluded from:

• Full index rebuilds (rebuild_index)
• Single-file add/update events (add_file_to_index)
• File-watcher created/modified/moved events

The .semanticignore file itself is never indexed.

Runtime reload — editing .semanticignore while the watcher is running triggers an atomic reload of that vault's rules. Subsequent file events immediately use the new patterns; no restart is required.

Example .semanticignore:

# Exclude an entire directory
archive/

# Exclude all draft files in any subdirectory
**/draft.md

# Re-include one specific file from the excluded directory
!archive/keep.md

# Anchor a pattern to the vault root only
/scratch.md

How It Works

First run downloads a small embedding model (~90MB) and indexes your markdown files (<1s for typical vaults). The index auto-updates when files change via filesystem watcher.

Indexed Content

Each markdown file is indexed with weighted components:

Development

# Clone
git clone https://github.com/bborbe/semantic-search
cd semantic-search

# Install dev dependencies
make install

# Run checks
make check

# Run tests
make test

License

BSD 2-Clause License — see LICENSE.