AI导哈

MCP结构化数据分析工具

Study MCP 是一款利用模型上下文协议（MCP）将结构化数据转化为可查询接口的服务，支持通过调用明确工具对事件、文章和博客等数据进行聚合、筛选和精确统计，适用于弥补 RAG 在结构化数据挖掘和定量查询方面的不足。

  • by beatanemeth
  • Jupyter Notebook
  • 本地部署
  • 1 星标
  • 更新于
在 GitHub 查看 beatanemeth/ai-engeneering-study-mcp

使用说明

Study MCP

This repository explores how Model Context Protocol (MCP) can be used to overcome a common limitation of RAG-based systems when working with structured, real-world data such as events, articles, and usage metrics.

This project and the engineering experience behind it are discussed in the Medium article:
Making AI Queries Work: From RAG’s Limits to MCP in Practice


Table of Contents

  1. Background & Motivation
  2. How MCP Improves InsightHubAI
  3. Data Preparation
  4. MCP Tools Development
  5. MCP Server Setup
  6. Technical Details
  7. Summary

Background & Motivation

In the mentioned article, it was identified that InsightHubAI struggled with data mining–style questions. This highlights a classic “RAG vs. Structured Data” problem:

  • Semantic search (RAG) is excellent at answering questions like “What is the policy on X?”
  • But it performs poorly for questions like “How many times did X happen?”

The reason is simple: RAG retrieves text chunks, not data rows. Counting, aggregating, ranking, and filtering require structured access to data — not semantic similarity.


How MCP Improves InsightHubAI

MCP allows structured data to be treated as a queryable system, rather than a passive pile of text.

Instead of the AI:

  • guessing answers based on retrieved snippets, or
  • attempting to infer numbers from prose,

it can:

  • invoke explicit tools
  • apply deterministic logic
  • and return exact, verifiable results

In short, MCP turns your data into a functional database interface for the LLM.


Data Preparation

Download Data

The following datasets were downloaded as JSON files:

  • Events
  • Articles
  • Blog posts

The original datasets are stored locally in the /data folder, which is not committed to GitHub.

To preserve structure without exposing real data:

  • a /data_dummy folder is included
  • it contains representative files with dummy values matching the original schemas

Note: For simplicity, this is a one-time download rather than a continuous ingestion pipeline.

Clean Data

After downloading, the data was cleaned and normalized to ensure:

  • Consistent date formats
  • Structured categories and tags
  • Reliable numeric fields (attendance, views, likes, waitlist counts)

Data preparation was performed using Jupyter Notebook and Pandas.

📂 See: /data_prepare This folder documents the full data preparation process, including cleaning, normalization, and schema validation.

The cleaned datasets were saved locally in /data_prepared, which is excluded from GitHub. Instead, a /data_prepared_dummy folder is included with cleaned dummy data reflecting the final structure.


MCP Tools Development

The design and development of MCP tools is documented step by step.

📂 See: /mcp_tools_development In particular, review the README.md in that directory for:

  • question formulation
  • tool taxonomy design
  • reasoning behind Aggregator, Analyst, and Thematic Finder tools

This section focuses on how to think about MCP tools, not just how to implement them.


MCP Server Setup

📂 See: /mcp_server

  • server.py — MCP server implementation
  • client.py — simple client located in the project root for testing and interaction

This setup demonstrates how the developed MCP tools can be exposed and invoked programmatically.


Technical Details

This project is built using a local-first approach, prioritizing open-source tools and data privacy.

Prerequisites

  • Python 3.10 or higher
  • LLM Runtime: Ollama (running qwen2.5:7b)

Operating System Used

  • Linux Mint 21.2 (development environment used)

MCP Stack Overview

  • Protocol Framework: FastMCP (Python-based)
  • Intelligence Layer: Ollama running the Qwen 2.5 (7B) model.
  • Data Processing: Pandas & Jupyter.

Python Virtual Environment

It is best practice to use a virtual environment to isolate project dependencies.

1. Initialize the Environment

Create the virtual environment in the project root:

python3 -m venv .venv

2. Activate the Environment

# macOS/Linux:
source .venv/bin/activate

# Windows (Command Prompt):
.venv\Scripts\activate.bat

# Windows (PowerShell):
.venv\Scripts\Activate.ps1

Your command prompt will now show the environment name, like (.venv) user@host:~/project$, indicating that it is active.

3. Install Project Dependencies

This installs all dependencies required to explore the full project end-to-end.

# Update pip
python -m pip install --upgrade pip

# Install dependencies
pip install -r requirements.txt

IMPORTANT: Whether you have installed the entire repository or only want to use a part of it, you must consult the dedicated README for each service to understand how to set it up and run it correctly:

  • MCP Server: /mcp_server/README.md
  • JWT Microservice: /data_get/jwt_microservice/README.md
  • Jupyter Notebooks: /data_prepare/README.md

4. Shutting Down

When you are finished working, simply run:

deactivate

Your command prompt will return to its default state, and the environment name (.venv) will disappear.

Jupyter Cleanup & Git Hygiene

This project uses nbstripout to keep notebook outputs out of version control and a pre-commit hook to ensure consistent formatting for all files.

Configuration details can be found in .pre-commit-config.yaml.

To check all files and automatically clean notebook outputs before committing, run:

pre-commit run --all-files

Summary

This repository documents a practical exploration of:

  • why RAG alone is insufficient for structured analytics
  • how MCP enables deterministic reasoning over real-world data
  • and how to design MCP tools that align with clear analytical intent

It is intended as both:

  • a learning project, and
  • a reference for designing MCP-based AI systems that go beyond semantic search.

相关 MCP