多智能体消息路由系统

Agent Switchboard

Route every message to the right AI agent.

Agent Switchboard is a multi-agent chat system. Each message passes through a small pipeline: a moderator screens it, an LLM router connects it to the best-suited agent, the agent answers using only the tools it is allowed (MCP services, RAG, a calculator, and more), and an evaluator checks the answer and hands the question to another agent if it falls short. Every step streams live to a web UI or a terminal CLI.

Agents are plain YAML, so adding a new specialist takes a config entry, not code. Models from Ollama (local), Gemini, Claude, and ChatGPT can be switched per conversation.

Contents

• Features
• How It Works
• Quick Start
• Configuration
• API
• Project Structure
• Acknowledgements
• License
• Disclaimer

Features

• Agents
  • YAML-driven — define roles, system prompts, and allowed tools in config
  • Multi-provider LLM — Ollama (local), Gemini, Claude, ChatGPT behind a unified Chat protocol
  • Tool calling — every capability is a tool (calculator, time, user context, RAG, MCP)
  • MCP — finance, weather, and Tavily web search exposed as tools via FastMCP
  • RAG — topic-aware local vector search with embeddings, exposed as a tool
• Orchestration
  • Moderator — pattern-based safety filter
  • Router — LLM intent classification picks the best agent, or asks a clarifying question when the query is too vague
  • Evaluator — judges response quality and forwards to another agent when needed
• Web UI
  • Real-time thinking steps with streaming responses (SSE)
  • Multi-turn chat with full conversation history
  • Edit any earlier message and restart from there
  • Token usage and estimated cost per request
• CLI
  • Interactive terminal chat with markdown rendering, syntax highlighting, and a thinking spinner

Built-in Agents

How It Works

• Frontend — React + TypeScript + Tailwind, streams orchestration events via SSE
• CLI — React + Ink terminal UI, same SSE stream as the web frontend
• Backend — FastAPI, stateless (the client sends the full message history with each request)
• Orchestrator — an async generator that runs the pipeline:
  1. Moderator — blocks unsafe queries (pattern matching)
  2. Router — LLM call that picks the best agent from each agent's role, or returns a clarifying question
  3. Agent — LLM call with filtered tools (each agent sees only its allowed tools)
  4. Evaluator — LLM call that judges the answer; if insufficient, forwards to another agent (up to max_forwards times)
• Tools — calculator, time, user context, RAG search, and MCP services (finance, weather, Tavily web search)

Thinking UI & Streaming

Routing decisions, tool calls, agent outputs, and evaluation results stream to the UI as they happen. The thinking section collapses automatically once the final answer arrives.

Multi-turn Chat & Edit

Conversations keep their full history. Edit any earlier user message and the conversation restarts from that point with the new text.

Tool Calling

Agents use native LLM tool calling — tools are plain functions passed directly to the model. Each tool call shows up as a thinking step.

Model Notes

• Gemini / Claude / ChatGPT follow tool-calling and formatting instructions more reliably, but are subject to API rate limits.
• Ollama (local) models such as qwen2.5:7b may ignore formatting instructions (e.g. LaTeX delimiters) or route less accurately. Larger local models generally do better.

Quick Start

Prerequisites

• Python 3.12+
• Node.js 18+
• uv (Python package manager)
• Ollama (for local models and RAG embeddings)

1. Clone and pull models

git clone https://github.com/Bambuuai/agent-switchboard.git
cd agent-switchboard

ollama pull qwen2.5:7b
ollama pull nomic-embed-text

If you skip the pulls, the backend downloads the Ollama models on first use (which can take a while).

2. Backend

cd backend
uv sync

# Optional: add API keys for Gemini / Claude / OpenAI / Tavily
cp .env.example .env

cd src
uv run uvicorn main:app --reload

API docs at http://localhost:8000/docs

3. MCP Services (optional)

Local finance (port 8030) and weather (port 8031) servers for the analyst and weather_expert agents:

cd services
uv sync
uv run python -m mcp_services.finance.server &
uv run python -m mcp_services.weather.server &

Or run each one in its own terminal without the trailing &. Services started after the backend are connected automatically on the next request.

Tavily web search works out of the box once TAVILY_API_KEY is set in backend/.env.

4. Web UI

cd frontend
npm install
npm run dev

Open http://localhost:5173

5. CLI (alternative to the web UI)

cd cli
npm install
npm run build

# Interactive chat (prompts for model selection)
npx chat-cli

# With a specific model
npx chat-cli --model gemini-2.5-flash

# Custom backend URL
npx chat-cli --server http://remote:8000

# List available models
npx chat-cli models

In-session commands: /model (switch model), /clear (reset history), /quit (exit), /help

Configuration

All configuration lives in backend/config/config.yaml. Restart the backend after editing it.

Agents

Each agent has a role (read by the router), a system_prompt, and a list of allowed tools:

agents:
  assistant:
    role: "General chat, greetings, simple questions, time, math"
    system_prompt: |
      You are a helpful, friendly assistant.
      Be concise but informative. If you don't know something, say so rather than making things up.
    tools: [get_current_time, get_user_context, calculate]

  analyst:
    role: "Finance, stocks, market data, and financial calculations"
    system_prompt: |
      You are a senior financial analyst with deep expertise in equity research.
      ...
    tools: [get_stock_price, get_income_statement, get_balance_sheet, get_cash_flow, get_historical_prices, get_company_info, get_key_ratios, compare_stocks, calculate]
    disclaimer: "This is for informational purposes only and not financial advice. ..."

See config.yaml for all five agents.

Adding your own agent

1. Add an entry under agents: with a role, system_prompt, and tools.
2. Write the role as a clear description of what the agent handles — it is all the router and evaluator see when choosing an agent.
3. List tools by name. Built-in tools are calculate, get_current_time, get_user_context, and search_knowledge_base; every tool exposed by a configured MCP service is available under its own name (e.g. get_weather, tavily_search). Use all or omit tools to give the agent every tool.
4. Optionally add a disclaimer, which is shown under any answer the agent contributed to.

The first agent in the list is the fallback when the router's output can't be parsed.

LLMs

Entries under llm become selectable models in the UI dropdown and the CLI:

llm:
  - class: backend.agent.llm.ollama.OllamaChat
    model: qwen2.5:7b
    params:
      temperature: 0.3
  - class: backend.agent.llm.gemini.GeminiChat
    model: gemini-2.5-flash
    api_key_env: GEMINI_API_KEY
    params:
      temperature: 0.3
  - class: backend.agent.llm.anthropic.AnthropicChat
    model: claude-haiku-4-5-20251001
    api_key_env: ANTHROPIC_API_KEY
    params:
      temperature: 0.3
  - class: backend.agent.llm.openai.OpenAIChat
    model: gpt-4.1-nano
    api_key_env: OPENAI_API_KEY
    params:
      temperature: 0.3

Models whose API key is missing are hidden from the model list.

Workflow

workflow:
  max_forwards: 2  # Max agent-to-agent forwards per query

MCP Services

Each service is an MCP endpoint; all of its tools are discovered at startup. format tells agents how to present that service's results:

mcp:
  services:
    weather:
      url: http://127.0.0.1:8031/mcp
      format: "Bullet list with bold labels: **City**, **Temperature**, **Condition**, **Humidity**."
    tavily:
      url: https://mcp.tavily.com/mcp/?tavilyApiKey=${TAVILY_API_KEY}
      timeout: 30

RAG

Markdown and text files in backend/knowledge/<topic>/ are embedded with nomic-embed-text and searched by the search_knowledge_base tool. Add a topic by creating a folder and listing it under rag.params.topics with a name, description, and directory. top_k sets how many documents are returned per search.

Pricing

The pricing section sets per-model token costs ($ per 1M tokens) used for the cost estimate in the UI. These may differ from actual provider pricing. Local models cost zero.

API

The backend is stateless: send the whole conversation each time.

curl -N http://localhost:8000/chat \
  -H "Content-Type: application/json" \
  -d '{
        "model": "qwen2.5:7b",
        "messages": [{"role": "user", "content": "What is 15% of 240?"}],
        "user_context": {"city": "Tokyo", "timezone": "Asia/Tokyo"}
      }'

model and user_context are optional. The stream emits thinking, answer, done (with token usage and cost), and error events, followed by data: [DONE].

Project Structure

backend/
  config/config.yaml           # Agents, LLMs, MCP endpoints, RAG, pricing
  knowledge/                   # Documents for RAG indexing (by topic)
    space/                     #   Solar system, space exploration
    history/                   #   Ancient civilizations, modern history
    geography/                 #   Countries, landmarks, natural wonders
  src/main.py                  # FastAPI entrypoint
  src/backend/
    agent/                     # Agent framework
      agent.py                 #   Agent class (name, system_prompt, act)
      tools.py                 #   Tool definitions, build_tools, filter_tools
      types.py                 #   Chat/Agent protocols, Message, Reply, UserContext
      llm/                     #   Provider implementations (Ollama, Gemini, Anthropic, OpenAI)
        registry.py            #   Model registry, resolve by name
        pricer.py              #   Token cost calculator
      mcp/                     #   MCP client and tool handler
      rag/                     #   RAG client (numpy vector search, topic-aware)
    orchestrator/              # Orchestration pipeline
      orchestrator.py          #   Lifecycle (startup/shutdown), stream generator
      router.py                #   LLM-based intent classification
      evaluator.py             #   Response sufficiency check, agent forwarding
      moderator.py             #   Safety filter
    config.py                  # YAML config loader
    api.py                     # REST/SSE endpoints
frontend/                      # Web UI (React + TypeScript + Tailwind)
cli/                           # Terminal UI (React + Ink)
services/                      # MCP servers (finance, weather)

Acknowledgements

Originally based on llm-rag-chat-demo by Naoki Shibuya.

License

MIT

Disclaimer

The financial analysis features in this project are for educational and demonstration purposes only. Nothing produced by the analyst agent constitutes financial advice, investment recommendations, or endorsements. Financial data may be delayed, incomplete, or inaccurate. Always consult a qualified financial advisor before making investment decisions.