AI导哈

arXiv论文RAG检索服务

该MCP服务用于发现arXiv论文、下载PDF、提取结构化内容并建立向量索引，支持通过Redis和LLM对已索引论文执行语义检索与RAG问答，适用于研究助手、文献综述和论文问答场景。

  • by Muhammadyousafrana
  • Python
  • 本地部署
  • 1 星标
  • 更新于
在 GitHub 查看 Muhammadyousafrana/research-extract-pdf-papers

使用说明

arxiv-paper-rag-mcp

A Model Context Protocol (MCP) server for discovering research papers on arXiv, downloading their PDFs, extracting structured content, and indexing that content for semantic search and RAG workflows.

This project is designed for AI assistants, research tooling, and local paper workflows that need:

  • arXiv paper discovery by topic
  • PDF download and extraction
  • structured content parsing into readable Markdown
  • chunking for embeddings
  • Redis-backed vector retrieval
  • LLM-based question answering over indexed papers

Why this name?

arxiv-paper-rag-mcp is more precise and scalable than research-extract-pdf-papers because it clearly communicates the three core ideas of the repo:

  • arXiv-based paper discovery
  • RAG/vector retrieval workflow
  • MCP server integration

It is also easier to search and remember in GitHub and package ecosystems.

Features

  • Search arXiv by topic
  • Download paper PDFs
  • Extract text from PDFs and normalize it into Markdown
  • Split content into meaningful chunks
  • Embed chunks with Gemini, Voyage, or OpenAI models
  • Store embeddings in Redis
  • Retrieve metadata for saved papers
  • Ask natural-language questions about an indexed paper
  • Run as an MCP server for external AI clients

Repository structure

.
├── research_server.py   # MCP server and tool implementations
├── web_app.py           # Optional web interface for browsing and querying papers
├── test_server.py       # Unit tests for the server logic
├── pyproject.toml       # Python project configuration
├── uv.lock              # Locked dependencies
├── Dockerfile           # Docker image definition
├── Makefile             # Common dev automation commands
├── README.md            # Project documentation
├── LICENSE              # MIT license
├── papers/              # Runtime metadata and downloaded paper info
├── static/              # Frontend/static assets for the web app
└── .env.example         # Example environment configuration

Getting started

Prerequisites

  • Python 3.13+
  • Redis
  • uv package manager
  • An embedding provider API key (Gemini, Voyage, or OpenAI)

Install

git clone https://github.com/Muhammadyousafrana/arxiv-paper-rag-mcp.git
cd arxiv-paper-rag-mcp
uv sync

Configuration

Copy the example environment file and fill in your keys:

cp .env.example .env

Example:

GEMINI_API_KEY=your_key_here
REDIS_URL=redis://localhost:6379/0
OPENAI_API_KEY=your_openai_key_if_used
VOYAGE_API_KEY=your_voyage_key_if_used

Running the server

python research_server.py

Once running, the server exposes MCP tools to connected clients.

Core MCP tools

  • search_papers(topic, max_results)
  • extract_info(paper_id)
  • index_paper(paper_id)
  • query_paper(question, paper_id)
  • admin_list_papers()
  • admin_delete_paper(paper_id)
  • admin_redis_stats()
  • admin_redis_clear()

Example usage

search_papers(topic="retrieval augmented generation", max_results=5)
extract_info(paper_id="2301.07041")
index_paper(paper_id="2301.07041")
query_paper(question="What is the main contribution?", paper_id="2301.07041")

Development

make test
make lint
make format

License

MIT License. See the LICENSE file for details.

Notes

This project is intentionally built around a research workflow: discover papers, ingest them, and make their content available to LLM-backed agents via structured retrieval. It is especially useful for building research assistants, literature review tooling, and paper-grounded question answering systems.

相关 MCP