多源研究协调服务器

# Clone the repository
git clone https://github.com/yourusername/research-mcp.git
cd research-mcp

# Install dependencies
npm install
# or
bun install

# Build TypeScript
npm run build

{
  "mcpServers": {
    "research": {
      "command": "node",
      "args": ["/path/to/deep-research-mcp/dist/index.js"],
      "env": {
        "PERPLEXITY_API_KEY": "your-key",
        "GEMINI_API_KEY": "your-key",
        "OPENAI_API_KEY": "your-key",
        "ARXIV_STORAGE_PATH": "/path/to/storage/",
        "CONTEXT7_API_KEY": "your-key"
      }
    }
  }
}

I need to research transformer architectures.
Can you use the start_research tool to give me a comprehensive overview?

{
  "query": "How do transformer architectures work?",
  "depth_level": 2
}

Use the read_report tool to verify what the persona cited:
{
  "citation": "[R-135216:5-19]"
}

I'm building an AI memory companion that extracts entities and deduplicates memories.
I need to create 600+ test examples for evaluation, but my current template-based
approach creates unrealistic data. Research the best approaches for creating
high-quality evaluation datasets.

Context:
- Solo developer with 20 hour budget
- Already reviewed papers on Excel formula repair and DAHL biomedical benchmark
- Found that synthetic data is 40% simpler than real data
- Random template filling doesn't work

Specific questions:
1. What makes evaluation data representative?
2. How to generate hard negatives?

Tech stack: Python, Neo4j, LangSmith

{
  "query": "How to create high-quality evaluation datasets for LLM testing?",
  "project_description": "AI memory companion with semantic extraction/dedup",
  "current_state": "85 test examples, need 600+",
  "problem_statement": "Template-based generation creates unrealistic data",
  "constraints": ["Solo developer", "20 hours"],
  "domain": "LLM evaluation datasets",
  "depth_level": 4,
  "papers_read": ["Excel formula repair", "DAHL biomedical benchmark"],
  "key_findings": ["Synthetic data 40% simpler than real data"],
  "rejected_approaches": ["Random template filling"],
  "sub_questions": [
    "What makes evaluation data representative?",
    "How to generate hard negatives?"
  ],
  "tech_stack": ["Python", "Neo4j", "LangSmith"],
  "output_format": "actionable_steps",
  "include_code_examples": true
}

{
  "query": "How do transformer architectures work?",
  "depth_level": 3,
  "report": true,
  "report_path": "/Users/name/Documents/research/"  // optional
}

graph TD
    A[Query + Context] --> B[Root Planning: 2-5 LLM Consensus]
    B --> C[Main Query Strategy]
    B --> D[Identify Shared Docs]
    D --> E[Fetch Base Context7 Docs]
    A --> F[Plan Each Sub-Q Independently]
    
    C --> G[Execute Main Query]
    E --> G
    F --> H[Execute Sub-Qs in Parallel]
    E --> H
    
    G --> GM[Extract Global Manifest]
    GM --> I[Phase 1: Main Synthesis]
    I --> J[Extract Key Findings ~500 tokens]
    J --> K[Phase 2: Sub-Q Syntheses Parallel]
    H --> K
    
    K --> PVR{PVR Check: Score >= 0.85?}
    PVR -->|No| RR[Re-roll Contradicting Sections]
    RR --> K
    PVR -->|Yes| L[Code Validation vs Context7]
    L --> M[Critical Challenge]
    M --> N{Depth >= 4?}
    N -->|Yes| O[Multi-LLM Consensus]
    N -->|No| P[Sufficiency Vote]
    O --> P
    P --> Q{Sufficient?}
    Q -->|Yes| R[Report with Inline Citations]
    Q -->|No| S[Re-synthesize]
    S --> I

LangSmith provides dataset management [context7:langsmith] which supports 
version control [perplexity:langsmith-docs] as validated in recent research 
[arxiv:2024.12345].

read_report(citation="R-182602:q1")          # Read sub-question 1
read_report(citation="R-182602:q1:20-50")    # Lines 20-50 of sub-Q 1
read_report(citation="R-182602", full=true)  # Full report (last resort)

# Verify Context7 MCP is accessible (if installed separately)
# Verify arXiv MCP server is installed:
uv tool run arxiv-mcp-server --help

{
  "mcpServers": {
    "research": {
      "command": "node",
      "args": ["/path/to/deep-research-mcp/dist/index.js"],
      "env": {
        "UV_BINARY_PATH": "/opt/homebrew/bin/uv",  // Apple Silicon
        // or "/usr/local/bin/uv" for Intel Mac / Linux
        "PERPLEXITY_API_KEY": "your-key",
        "GEMINI_API_KEY": "your-key"
      }
    }
  }
}

curl -LsSf https://astral.sh/uv/install.sh | sh

{
  "mcpServers": {
    "research": {
      "command": "node",
      "args": ["/absolute/path/to/research-mcp/dist/index.js"],
      "env": {
        "PERPLEXITY_API_KEY": "your-key",
        "GEMINI_API_KEY": "your-key",
        "OPENAI_API_KEY": "your-key"
      }
    }
  }
}

{
  "command": "npx",
  "args": [
    "tsx",
    "/Users/name/Desktop/Personal and learning/quick-mcp/research/src/index.ts"
  ]
}

# Research Results: [Your Query]

## Complexity Assessment

**Level**: 4/5
**Reasoning**: Complex research requiring academic papers and library documentation

## Research Action Plan

**Estimated Time**: ~45s

**Steps Executed**:
1. **perplexity**: Search for recent approaches and best practices _(parallel)_
2. **deep_analysis**: Analyze web findings for technical insights
3. **context7**: Fetch React and TypeScript documentation _(parallel, shared + specific)_
4. **arxiv**: Search academic papers on evaluation datasets
5. **consensus**: Validate findings across multiple models

## Synthesis with Inline Citations

### Overview

LangSmith provides comprehensive dataset management [context7:langsmith] which enables
evaluation workflow automation [perplexity:langsmith-docs]. Recent research shows that
synthetic data generation requires careful attention to distribution matching [arxiv:2024.12345].

\`\`\`typescript
// Code validated against Context7
import { Dataset } from "langsmith";

const dataset = new Dataset("my-eval-set");
\`\`\`

### Sub-Question 1: What makes evaluation data representative?

Representative data must match real-world distributions [deep_analysis] and include
edge cases from production logs [context7:langsmith]. Studies indicate that 600+ examples
provide sufficient statistical power for small effect detection [arxiv:2024.67890].

## Code Validation Summary

✅ 3 code blocks validated
✅ 1 syntax correction applied (outdated API method)

## Multi-Model Consensus
[3 LLMs validated findings—shows agreement/disagreement]

## Critical Challenge
[Critical validation—alternative perspectives and gaps identified]

## Quality Validation

**Vote Result**: 2 sufficient, 1 insufficient
**Status**: ✅ Response is sufficient

**Model Feedback**:
- ✅ **gemini-3-flash-preview**: Response comprehensively addresses the query with actionable steps
- ✅ **gpt-5-mini-2025-08-07**: Good coverage of edge cases and validation methods
- ❌ **claude-3.5-haiku**: Could benefit from more code examples

---

**Report ID**: R-182602

**Usage Examples**:
\`\`\`
read_report(citation="R-182602:overview")      # Read overview section
read_report(citation="R-182602:q1")            # Read sub-question 1
read_report(citation="R-182602:q1:20-50")      # Lines 20-50 of sub-Q 1
read_report(citation="R-182602", full=true)    # Full report (last resort)
\`\`\`

# Generate responses for benchmark dataset
npm run benchmark:generate              # Generate only missing responses
npm run benchmark:generate:new-set      # Clear all responses and start fresh

# Regenerate only a single category (useful for iterating on a category). Example:
npm run benchmark:generate -- --category synthesis --force

# Run full benchmark comparison (generates new results)
npm run benchmark:compare

# View latest benchmark results
npm run benchmark:view

# Compare multiple benchmark runs over time
npm run benchmark:compare-results

================================================================================
DECISION MATRIX: MCP vs Perplexity
Generated: 2025-12-21T02:17:33.512Z
Total Comparisons: 69
================================================================================

RECOMMENDATIONS
--------------------------------------------------------------------------------

Use Perplexity for:
  - multi_hop_reasoning (win rate: 80%, p<0.05)
  - synthesis (win rate: 90%, p<0.05)
  - latency-critical tasks (<2s vs ~5min)

No clear winner (prefer lower cost):
  - single_hop_factual (60% vs 40%, p=0.63)

DETAILED BREAKDOWN
--------------------------------------------------------------------------------
Category              | MCP Win  | P(Sup)  | 95% CI          | Recommendation
--------------------------------------------------------------------------------
single_hop_factual    | 60%      | 0.63    | [-0.80, 1.30]   | TIE
multi_hop_reasoning   | 20%      | 0.01    | [-2.20, -0.35]  | USE_PERPLEXITY

SWITCHING POINTS:
  - Simple factual lookup: USE PERPLEXITY (faster)

====================================================================================================
BENCHMARK COMPARISON
2 benchmark runs found
====================================================================================================

Category              | Run 1         | Run 2         
----------------------|---------------|---------------
                      | Dec 20, 09:30 | Dec 21, 02:17 
----------------------|---------------|---------------
single_hop_factual    | 70%           | 60% ↓         
multi_hop_reasoning   | 30%           | 20% ↓         
synthesis             | 10% →         | 10% →         

RECOMMENDATION CHANGES
----------------------------------------------------------------------------------------------------
single_hop_factual: Run 1: TIE → Run 2: TIE
multi_hop_reasoning: Run 1: TIE → Run 2: USE_PERPLEXITY

Legend: ↑ = improved, ↓ = declined, → = stable