AI导哈

本地语音代理

一个完全本地的智能语音代理，使用Whisper进行离线语音转文本，通过本地LLM理解意图，并从结构化SQL数据库或DuckDuckGo网络搜索中获取答案。

  • by Muaf1998
  • Python
  • 本地部署
  • 更新于
在 GitHub 查看 Muaf1998/MCP-Voice-Agent-with-SQL-and-Web-Fallback

使用说明

MCP Powered Voice Agent

A fully local, intelligent voice agent built using the Model Context Protocol (MCP). It listens to your voice, transcribes it offline using Whisper, understands your intent via a local LLM (Ollama), and fetches answers from a structured SQL database or falls back to a DuckDuckGo web search.

It comes with both a sleek PyQt5 Desktop GUI and a Command Line Interface.

Features

  • 🎙️ 100% Local Voice Transcription: Uses OpenAI's Whisper model (base) to convert speech into text privately and offline.
  • 🧠 Local LLM Orchestration: Powered by llama3.2 running via Ollama and orchestrated with LangChain's Tool Calling Agent.
  • 🗄️ Database MCP Tool: A built-in SQLite database tool that the agent can query for structured information (e.g., querying staff schedules).
  • 🌐 Web Search Fallback: If the local database doesn't have the answer, the agent smartly falls back to DuckDuckGo to answer general knowledge questions.
  • 🖥️ Desktop GUI: A responsive PyQt5 interface with a "Push to Talk" chat window.

Prerequisites

  1. Python 3.12 is highly recommended (Python 3.14 is currently incompatible with some LangChain/Pydantic binary wheels).
  2. Ollama installed and running with the llama3.2 model.
    ollama run llama3.2
  3. FFmpeg installed on your system (required by Whisper for audio processing).
    • Mac: brew install ffmpeg
    • Ubuntu/Debian: sudo apt update && sudo apt install ffmpeg
  4. PortAudio installed on your system (required by PyAudio).
    • Mac: brew install portaudio
    • Ubuntu/Debian: sudo apt install portaudio19-dev

Installation

  1. Clone the repository:

    git clone <your-github-repo-url>
cd mcp-voice-agent

  2. Create and activate a virtual environment:

    python3.12 -m venv .venv
source .venv/bin/activate  # On Windows: .venv\\Scripts\\activate

  3. Install the required dependencies:

    pip install -U pip
pip install -r requirements.txt

Usage

Ensure your virtual environment is activated and Ollama is running.

Run the Desktop GUI (Default):

python main.py

Click the "Push to Talk" button, ask your question, and wait for the agent to think and respond!

Run the Terminal / CLI Version:

python main.py --cli

Testing the Tools

  • Database Tool Trigger: Ask something like "What is Alice Smith's schedule?" -> The agent will recognize it needs to check the staff database.
  • Web Fallback Trigger: Ask something like "Who won the Super Bowl in 2024?" -> The agent will realize the database doesn't know this and will fall back to DuckDuckGo.

Architecture & MCP Details

The Model Context Protocol (MCP) integrates the tools. Although currently bound directly into the LangChain executor for speed and simplicity, the tools in the tools/ directory are built using FastMCP.

This means you can easily detach them and run python tools/database.py to start a standalone MCP server that any standard MCP Client (like Claude Desktop) can connect to!

相关 MCP