天气AI代理服务

Weather AI Agent Service

Production-grade AI agent for weather forecast intelligence: Built with LangChain 1.0, LangGraph 1.0, FastAPI and OpenAI. Features 15-agent multi-agent orchestration, auto-routing (intent-based query classification), 7-layer memory architecture, advanced reasoning (Tree/Graph-of-Thought), emotional intelligence, two-tier semantic caching (40-60% cost reduction), multi-layer caching (L1+L2+L3+Semantic), Prometheus metrics and observability, real-time weather data via dual MCP servers, RAG-enhanced knowledge base, Chain-of-Thought reasoning, Human-in-the-Loop (HITL) approval workflows, self-evolving AI with constitutional guardrails, comprehensive golden dataset evaluation (185 test cases, ZERO safety violations) and Two-Tier Semantic Caching -LLM resposne and Tool caching.

From zero to production: Progressive implementation showcasing enterprise AI patterns including multi-agent orchestration (3→8→15 agents), auto-routing architecture, 7-layer memory systems (99.7% storage reduction), context window optimization (50-60% token reduction for <4K tokens/query), semantic caching (65-85% hit rate with similarity matching), full observability stack (Prometheus, Grafana, OpenTelemetry), adversarial testing (RAGAS/DeepEval), constitutional AI (12-layer safety guardrails), LangGraph-bigtool Dynamic Tool Registry & Semantic Discovery, Context Window Optimization, Two-Tier Semantic Caching and [planned] self-evolving agentic dual-loop architecture that continuously learns, adapts, and improves based on real-world performance.

How It Works: This project follows a progressive complexity model—each module builds on the previous one, making it accessible to beginners while scaling toward advanced concepts.

Current Stage: ✅ Level 9 Complete — Two-Tier Semantic Caching: Query/Tool/LLM Response Caching with Similarity Matching, Life-Safety Bypass, Prometheus Metrics (100% LangChain v1.x/LangGraph v1.x Compliant)

Read the Medium Blog Post Series

Table of Contents

• Weather AI Agent Service
  • Table of Contents
  • Technology Stack
  • Testing Tools & Interfaces
  • Quick Start
    • System Requirements
    • Prerequisites
    • MCP Servers Required
    • Setup (15 minutes)
      • Option 1: Quick Start with Makefile (Recommended)
    • Required API Keys
    • MCP Server Configuration
    • RAG & Vector Store Configuration (Level 2)
    • Agent Feature Flags (Level 2)
    • LangSmith Observability (Optional)
    • Optional/Advanced Configuration
      • Option 2: Manual Setup (Without Makefile)
    • Common Makefile Commands
    • Project Structure
    • Progressive Learning Path
  • API Endpoints Reference
  • Testing (Progressive Build Approach)
  • Contributing
  • License
  • Support

Technology Stack

Core Framework:

AI/ML & LLM:

Databases & Storage:

Observability:

Data Sources (via MCP Servers):

• NOAA/NHC - Hurricane tracking data
• NWS API - Weather alerts & forecasts

Testing Tools & Interfaces

Quick reference for testing and debugging the Weather AI Agent:

Service Access URLs

Observability Stack (Level 8 - Signal Correlation)

Signal Correlation (Level 8): Click exemplars on metrics → Jump to traces → View related logs

Test Guides by Level:

Quick Start

System Requirements

Install these first (one-time setup):

1. Python 3.13.5+

# Using pyenv (recommended)
curl https://pyenv.run | bash
pyenv install 3.13.5
pyenv global 3.13.5

2. uv (ultra-fast package manager)

# macOS/Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

# Verify installation
uv --version

3. Docker Desktop

Download from https://www.docker.com/products/docker-desktop

Prerequisites

Note: The following are required BEFORE running make install-dev:

• Python 3.13+ (installed above)
• uv (installed above)
• Docker Desktop (installed above)
• API Keys: OpenAI, LangSmith

MCP Servers Required

This project uses 2 MCP servers (Docker + HTTP Streamable transport) for weather data:

MCP Servers Setup Guide - Step-by-step instructions to clone, build, and run both servers as Docker containers.

Development Tools:

• LangSmith Studio Setup Guide - Official LangChain visual debugger for testing agents locally with real-time visualization of prompts, tool calls, and execution flow. Perfect for debugging all 3 MCP weather tools.
• Docker Usage Guide - Complete guide for managing Docker containers with production and development modes, hot reload setup, and troubleshooting.

Setup (15 minutes)

Option 1: Quick Start with Makefile (Recommended)

1. Clone repository

git clone https://github.com/kumaran-is/weather-agent.git
cd weather-agent

2. Configure environment

cp .env.template .env

Edit .env with your configuration. Below are all available settings organized by category:

Required API Keys

MCP Server Configuration

For Local Development (default):

For Docker Deployment: URLs auto-configured via Docker Compose (use http://weather-mcp:8080 and http://hurricane-mcp:8081 - no manual .env changes needed)

Setup Guide: See MCP Servers Setup for complete installation instructions

RAG & Vector Store Configuration (Level 2)

Agent Feature Flags (Level 2)

Example: /weather/query?enable_rag=true&enable_cot=true (full Level 2 capabilities)

LangSmith Observability (Optional)

Access LangSmith Studio: http://localhost:8123 (after running make docker-up-dev)

Optional/Advanced Configuration

Configuration Priority: Runtime params > Environment variables (.env) > Code defaults

3. Install all dependencies

This creates .venv/ and installs 121 packages (including dev tools):

make install-dev

Or alternatively:

make sync

4. Run verification

Run 9 automated checks:

make verify

Expected output: ALL 9 CHECKS PASSED!

5. (Optional) Activate virtual environment manually

Note: uv run and make commands work without activation.

source .venv/bin/activate

6. Check installed versions

make version

Option 2: Manual Setup (Without Makefile)

1. Clone repository

git clone https://github.com/kumaran-is/weather-agent.git
cd weather-agent

2. Configure environment

cp .env.template .env

Edit .env and add your API keys and MCP server paths (see Option 1 for details).

3. Install all dependencies

Creates .venv/ automatically:

uv sync

4. Run verification

Run 8 automated checks:

uv run python verify_setup.py

Expected output: ALL 9 CHECKS PASSED!

5. (Optional) Activate venv manually

source .venv/bin/activate

6. Check versions

uv pip list | grep -E "langchain|langgraph|fastapi"

Note:

• uv automatically creates and manages .venv/ virtual environment
• Commands like uv run and make work without manual venv activation
• Manual activation (source .venv/bin/activate) is only needed for direct Python commands

Common Makefile Commands

Development & Testing (Progressive Build Approach)

Progressive Build Philosophy: make test-dev automatically tests all implemented features for the current level without requiring Makefile updates as you progress through levels.

Docker Commands (4 Services: weather-mcp:8080, hurricane-mcp:8081, qdrant:6333, weather-ai-api:8000)

Production Mode (recommended for testing production builds):

Development Mode (recommended for coding with hot reload):

Production vs Development:

Typical Workflows:

Development Workflow (Recommended - Progressive Build):

# 1. Start in dev mode with hot reload
make docker-up-dev

# 2. Load RAG knowledge base (one-time setup)
make rag-load

# 3. Run unified test suite (tests ALL current features: Level 1+2+3)
make test-dev

# 4. Edit code in ./backend (changes reflect immediately)

# 5. View logs
make docker-logs-dev

# 6. Stop when done
make docker-down-dev

Production Workflow:

# Start all services
make docker-up

# Check status
make docker-ps
make docker-health

# Debug issues
make docker-logs

# Stop all services
make docker-down

RAG Commands (Level 2: Qdrant Vector Database)

Datasets Guide: See RAG Datasets Guide for complete details on mock data sources, Kaggle datasets (2.9M+ records), download instructions, and dataset statistics.

Typical RAG Workflow:

# 1. Validate datasets
make rag-validate

# 2. Load knowledge base (one-time setup)
make rag-load

# 3. Test retrieval (verify 603 documents loaded)
make rag-test

⚠️ Important: Qdrant Data Persistence

Your embedded documents (603 chunks) are stored in a Docker named volume and persist across container restarts:

Data is PRESERVED when:

• ✅ make docker-restart / make docker-restart-dev - Safe to restart
• ✅ make docker-down / make docker-down-dev - Safe to stop containers
• ✅ make docker-up / make docker-up-dev - Data reloads automatically

Data is DELETED when:

• ❌ make docker-clean / make docker-clean-dev - Removes volumes (5-second warning)
• ❌ docker-compose down -v - The -v flag deletes volumes

Verify data persists after restart:

# Check document count
make rag-test

# Restart Qdrant
make docker-restart

# Check again - same 603 documents!
make rag-test

When to run make rag-load again:

1. After make docker-clean (volumes deleted)
2. When adding new documents to backend/data/raw/
3. When switching between production and dev (different volumes)

Note: Production and development use separate volumes (weather-ai-qdrant-data vs weather-ai-qdrant-data-dev), so you may need to load data separately for each environment.

Docker Troubleshooting

Common Docker Build Errors:

# Error: "parent snapshot does not exist: not found"
# Cause: Docker BuildKit cache corruption (common after Docker restarts/updates)
# Fix:
docker builder prune -f
make docker-rebuild-dev

# If still failing, full cleanup:
docker system prune -a --volumes -f
make docker-rebuild-dev

Memory Commands (Level 3a+: Redis + Neo4j)

Typical Memory Workflow:

# 1. Start development containers
make docker-up-dev

# 2. Check all services are running
make docker-ps-dev

# 3. Verify health
make docker-health

# 4. Test memory connectivity
make memory-test

# 5. Explore Redis data (interactive CLI)
make memory-redis-cli

# 6. Open Neo4j Browser (graph visualization)
make memory-neo4j-browser

# 7. Stop when done
make docker-down-dev

Observability Commands (Level 8: Prometheus + Grafana + Loki + Tempo)

Typical Observability Workflow:

# 1. Start dev containers (includes observability stack + Tempo)
make docker-up-dev

# 2. Check observability status (includes Tempo)
make observability-status

# 3. Verify signal correlation (Level 8)
make verify-signal-correlation

# 4. Open Grafana dashboards (Signal Correlation dashboard)
make grafana-open
# Login: admin / weatherai2025

# 5. View Prometheus metrics (with exemplars)
make prometheus-open

# 6. Query logs via Loki (in Grafana) - click trace_id to jump to Tempo
make loki-logs

Evaluation Commands (Level 5b + Level 6: Golden Dataset Testing & Quality Gates)

All 8 evaluation commands validated and working correctly:

Quality Gates (5 gates, all must pass for deployment):

Typical Evaluation Workflow:

# Quick smoke test (recommended first)
make eval-quick

# Test specific category
make eval-category CATEGORY=hurricane

# Full evaluation pipeline (all 185 cases)
make eval-full

# View results in LangSmith
# https://smith.langchain.com/datasets

Quality Gates Guide: See docs/knowledgebase/quality-gates-guide.md for comprehensive details on what quality gates are, how they work, and what to do when they fail.

Project Structure

weather-agent/
├── backend/                     # Backend application (Level 8 Complete)
│   ├── config/                  # Configuration (SINGLE LOCATION)
│   │   ├── settings.py          # Centralized app settings (50+ env vars)
│   │   ├── llm_config.py        # LLM use case configuration
│   │   ├── memory_config.py     # Memory system configuration
│   │   ├── cache_config.py      # Cache system configuration
│   │   └── tool_registry_config.py  # Tool Registry configuration (Level 7)
│   ├── data/raw/                # RAG knowledge base (603 documents)
│   ├── migrations/              # Database migration scripts
│   ├── src/                     # Source code
│   │   ├── agents/              # 15 specialized agents (weather, triage, hurricane, etc.)
│   │   ├── api/main.py          # FastAPI app (13 endpoints + Prometheus metrics)
│   │   ├── cache/               # Multi-layer + Semantic Caching (Level 9)
│   │   │   ├── l1_memory_cache.py        # L1 in-process LRU (<1ms, life-safety bypass)
│   │   │   ├── l2_redis_cache.py         # L2 Redis distributed (<10ms, life-safety bypass)
│   │   │   ├── l3_anthropic_cache.py     # L3 Anthropic prompt cache (transparent)
│   │   │   ├── query_normalizer.py       # Query normalization (aliases, case, whitespace)
│   │   │   ├── semantic_matcher.py       # Semantic similarity matching (Qdrant)
│   │   │   ├── two_tier_cache.py         # Two-tier architecture (exact + semantic)
│   │   │   ├── tool_cache.py             # Tool result caching with bypass
│   │   │   └── llm_cache.py              # LLM response caching
│   │   ├── context/             # Context Window Optimization (Level 8)
│   │   │   ├── context_optimizer.py      # 5-phase optimization pipeline (50-60% reduction)
│   │   │   ├── query_type_detector.py    # Auto-classification (EMERGENCY/COMPLEX/SIMPLE)
│   │   │   ├── semantic_chunker.py       # Semantic text chunking
│   │   │   ├── relevance_filter.py       # Relevance-based filtering
│   │   │   ├── dynamic_assembler.py      # Dynamic context assembly
│   │   │   └── hierarchical_loader.py    # Hierarchical context loading
│   │   ├── evaluation/          # Evaluation framework
│   │   ├── guardrails/          # 12-layer safety guardrails
│   │   ├── hitl/                # Human-in-the-Loop approval nodes
│   │   ├── mcp/                 # MCP client integration
│   │   ├── memory/              # 7-layer memory system
│   │   ├── models/              # Pydantic models (weather, hurricane, health, tool_registry)
│   │   ├── observability/       # Full Observability Stack (Level 8 + Level 9)
│   │   │   ├── callbacks.py             # LangSmith tracing callbacks
│   │   │   ├── logging.py               # Structured JSON logging with trace context
│   │   │   ├── metrics.py               # Prometheus metrics (context optimization + LLM)
│   │   │   ├── cache_metrics.py         # Cache metrics (Level 9: hit/miss/latency/similarity/cost)
│   │   │   └── tracing.py               # OpenTelemetry tracing integration
│   │   ├── orchestration/       # Multi-agent workflow orchestration
│   │   ├── rag/                 # RAG pipeline (embeddings, hybrid search)
│   │   ├── reasoning/           # ToT/GoT advanced reasoning
│   │   ├── registry/            # Tool Registry & Semantic Discovery (Level 7)
│   │   │   ├── tool_registry.py         # Thread-safe singleton registry
│   │   │   └── semantic_discovery.py    # AI-powered tool discovery
│   │   ├── routing/             # Auto-routing system
│   │   ├── services/            # Shared business logic services
│   │   ├── tools/               # LangChain tools (MCP + RAG + Hurricane)
│   │   │   ├── weather_tools.py         # Weather MCP tool wrappers
│   │   │   ├── rag_tools.py             # RAG retrieval tools
│   │   │   └── hurricane_tools.py       # Hurricane MCP tool wrappers (Level 7)
│   │   ├── utils/               # Utility functions and helpers
│   │   └── workflows/           # LangGraph workflows
│   └── tests/                   # Backend-specific tests
├── docs/                        # Documentation
│   ├── setup/                   # Setup and configuration guides
│   ├── test-guide/              # Test guides by level (9 guides)
├── observability/               # Observability stack configuration (Level 8)
│   ├── grafana/                 # Grafana dashboards and datasources (signal correlation)
│   ├── loki/                    # Loki log aggregation config
│   ├── prometheus/              # Prometheus metrics config (exemplar storage)
│   └── tempo/                   # Tempo distributed tracing config (Level 8)
├── scripts/                     # Evaluation and automation scripts
├── tests/                       # Test suite (27 test files)
│   ├── unit/                    # Unit tests
│   └── integration/             # Integration tests
├── .env.template                # Environment variables template
├── CHANGELOG.md                 # Version history and release notes
├── Dockerfile                   # Multi-stage production container
├── docker-compose.yml           # Production orchestration (9 services)
├── docker-compose.dev.yml       # Development orchestration (hot reload)
├── langgraph.json               # LangGraph Studio (4 graphs)
├── LICENSE                      # MIT License
├── Makefile                     # Development and deployment commands
├── pyproject.toml               # Project config (v1.7.0)
└── README.md                    # This file

Key Directories:

• backend/src/agents/: 15 specialized agents for weather intelligence (supervisor, triage, hurricane, emergency, forecaster, climate, historical, research, personalization, meta-prompt, critique, debate, reflection, self-healing, alert manager)
• backend/src/cache/: L1 (in-process), L2 (Redis), L3 (Anthropic) caching
• backend/src/context/: Context Window Optimization (Level 8) - 5-phase pipeline achieving 50-60% token reduction
• backend/src/memory/: 7-layer memory (conversation, session, episodic, semantic, procedural, emotional, reflective)
• backend/src/observability/: Full Observability Stack (Level 8) - Tracing, Structured JSON Logging, Metrics, Alerts
• backend/src/registry/: Tool Registry & Semantic Discovery (Level 7) - Thread-safe singleton, AI-powered search
• backend/src/orchestration/: Multi-agent workflows (supervisor, parallel execution, auto-routing)
• backend/src/evaluation/: 4-pillar evaluation framework (effectiveness, efficiency, robustness, safety)
• backend/src/tools/: LangChain tool wrappers (weather MCP, hurricane MCP, RAG retrieval)
• docs/test-guide/: 9 test guides (L0-L8, 18 scenarios for Level 8)
• observability/: Prometheus, Grafana, Loki, Tempo configuration (Level 8 signal correlation)
• scripts/: Evaluation automation (quality gates, batch evaluation, dataset upload)
• tests/: 27 test files (unit + integration, including 2 new Level 8 test files)

Progressive Learning Path

• Level 0 (v0.1.0): Setup ✅ COMPLETE
• Level 1 (v0.2.0-v0.2.1): ReAct + HITL + LangSmith Studio ✅ COMPLETE
• Level 2 (v0.3.0): ReAct + CoT + RAG + Hybrid Search (Semantic + Keyword) ✅ COMPLETE
• Level 3a (v0.4.0): 2-Layer Memory (Conversation + Session) ✅ COMPLETE
• Level 3b (v0.5.0): Advanced Reasoning (Tree-of-Thoughts + Graph-of-Thoughts) ✅ COMPLETE
• Level 3c (v0.6.0): Full 7-Layer Memory + Emotional Intelligence + Personalization ✅ COMPLETE
• Level 4 (v0.7.0): Multi-Agent Orchestration + Auto-Routing ✅ COMPLETE
  • Auto-Routing v0.6.0: Intent-based query classification (<1ms) ✅ COMPLETE
  • Level 4a: 3-Agent Foundation (Triage + Specialist + Alert) ✅ COMPLETE
  • Level 4b: 8-Agent Orchestration (+ Supervisor + parallel) ✅ COMPLETE
  • Level 4c: 15-Agent Production (+ Meta-Prompt + Debate + Self-Healing + Circuit Breakers) ✅ COMPLETE
• Level 5a (v0.8.0): Multi-Layer Caching (L1+L2+L3) ✅ COMPLETE
• Level 5b (v0.9.0): Evaluation Framework + 12-Layer Guardrails ✅ COMPLETE
• Level 5c (v0.10.0): Full Production + Observability Stack ✅ COMPLETE
• Level 6 (v1.3.0): Self-Evolving AI Platform ✅ COMPLETE
  • Level 6a: Context Optimization + Ragas/DeepEval Evaluation ✅ COMPLETE
  • Level 6b: TruLens + AgentBench + Auto-Prompt Engineering ✅ COMPLETE
  • Level 6c: Constitutional AI + Production Testing Infrastructure ✅ COMPLETE
• Level 7 (v1.5.0): LangGraph-bigtool Tool Registry & Semantic Discovery ✅ COMPLETE
• Level 8 (v1.6.0): Context Window Optimization & Full Observability Stack ✅ COMPLETE
  • Level 8a: Query Type Detection + 5-Phase Optimization Pipeline ✅ COMPLETE
  • Level 8b: OpenTelemetry Tracing + Structured JSON Logging ✅ COMPLETE
  • Level 8c: Prometheus Metrics + Grafana Dashboards + Alert Rules ✅ COMPLETE
  • Test Coverage: 114/114 tests passing (18 scenarios) ✅ COMPLETE
• Level 9 (v1.7.0): Two-Tier Semantic Caching Architecture ✅ COMPLETE
  • Level 9a: TwoTierCache Base Class + QueryNormalizer + SemanticMatcher ✅ COMPLETE
  • Level 9b: Tool Result Cache with Life-Safety Bypass ✅ COMPLETE
  • Level 9c: LLM Response Cache + Prometheus Metrics ✅ COMPLETE
• Level 10: Self Evolving Agentic Architecture: Self-evolving agentic systems that continuously learns, adapts, and improves based on real-world performance. 🔜 NEXT
• Level 11: Advanced HITL 🔜 FUTURE

Current Status: Level 9 (v1.7.0) - Two-Tier Semantic Caching Architecture (100% LangChain v1.x/LangGraph v1.x Compliant) | Test with make test-dev

API Endpoints Reference

All REST endpoints (13 total):

Access Points:

• Swagger UI: http://localhost:8000/docs (interactive testing with 37 example scenarios)
• ReDoc: http://localhost:8000/redoc (comprehensive API documentation)
• Prometheus Metrics: http://localhost:8000/metrics (plain text format)

Example Usage:

# Simple weather query
curl -X POST "http://localhost:8000/weather/query" \
  -H "Content-Type: application/json" \
  -d '{"query": "What is the weather in Miami?", "user_id": "test_001"}'

# Hurricane query (triggers multi-agent routing)
curl -X POST "http://localhost:8000/weather/query" \
  -H "Content-Type: application/json" \
  -d '{"query": "What is the status of Hurricane Milton?", "user_id": "test_002"}'

# Emergency evacuation query (triggers l4b orchestration)
curl -X POST "http://localhost:8000/weather/query" \
  -H "Content-Type: application/json" \
  -d '{"query": "Should I evacuate from Tampa Beach?", "user_id": "test_003"}'

# Create Category 4 hurricane alert (requires HITL approval)
curl -X POST "http://localhost:8000/weather/hurricane/alert" \
  -H "Content-Type: application/json" \
  -d '{"category": 4, "message": "Cat 4 Hurricane approaching Tampa", "thread_id": "alert-001"}'

# Approve pending alert
curl -X POST "http://localhost:8000/weather/hurricane/approve/alert-001" \
  -H "Content-Type: application/json" \
  -d '{"approved": true, "reviewer_notes": "Verified with NHC"}'

# Health check
curl http://localhost:8000/health

# Context optimization health metrics (Level 8)
curl http://localhost:8000/health/context

# Tool Registry health check (Level 7)
curl http://localhost:8000/health/tools

# MCP server health check
curl http://localhost:8000/mcp/health

# Cache statistics
curl http://localhost:8000/cache/stats

# List all registered tools (Level 7)
curl http://localhost:8000/tools/list

# Prometheus metrics (includes Level 8 context optimization metrics)
curl http://localhost:8000/metrics

LangGraph Studio Graphs (4 graphs for visual debugging):

• weather_agent: Unified agent with RAG + CoT
• weather_hitl_workflow: Hurricane alert HITL approval workflow
• multi_agent_workflow: Level 4a 3-agent orchestration
• level4b_workflow: Level 4b 8-agent parallel execution

Testing (Progressive Build Approach)

Current Test Coverage: 49 test files + 185 golden dataset test cases + Level 9 semantic caching integration tests covering ALL levels (L1-L9)

Unified Testing Command (Recommended):

# Test ALL implemented features (Level 1-9: 49 unit tests + 185 golden dataset cases + semantic caching tests)
make test-dev

This single command:

1. ✅ Restarts dev environment (10 Docker services, clean state)
2. ✅ Runs complete unit test suite (49 tests across all levels)
   • Level 1 (7 tests): ReAct Agent, HITL, MCP integration
   • Level 2 (4 tests): RAG, CoT, Hybrid Search
   • Level 3 (15 tests): 7-Layer Memory, ToT/GoT reasoning, 3-Layer Caching (L1+L2+L3)
   • Level 4 (20 tests): 15-Agent Multi-Agent Orchestration, Supervisor, Parallel Execution
   • Level 5 (3 tests): Production API, Guardrails, Evaluation Framework
   • Level 9 (5 tests): Semantic Caching (Query Normalization, Semantic Matching, Life-Safety Bypass, Metrics Export)
3. ✅ Verifies LangChain v1.x compliance (100% required, zero deprecated imports)

Golden Dataset Evaluation (Level 6):

# Run comprehensive evaluation with 185 test cases
make eval-full  # ~30-35 minutes

# Or test specific categories
make eval-category CATEGORY=simple      # 70 simple queries
make eval-category CATEGORY=hurricane   # 35 safety-critical queries
make eval-category CATEGORY=complex     # 50 multi-location queries
make eval-category CATEGORY=edge        # 30 edge cases

Progressive: As you build more levels, make test-dev automatically includes new tests without Makefile changes!

Level-Specific Test Guides:

• Level 2 Test Guide - RAG + CoT + Hybrid Search (6 Swagger tests + 6 Studio configs)
• Level 3 Test Guide - Memory + ToT/GoT + Multi-Layer Caching
• Level 4 Test Guide - Multi-Agent Orchestration (15 specialized agents)
• Level 5 Test Guide - Production Deployment + Observability
• Level 6 Test Guide - Advanced Evals + 185 Golden Dataset Cases
• Level 7 Test Guide - LangGraph-bigtool Tool Registry & Semantic Discovery
• Level 8 Test Guide - Context Window Optimization & Full Observability Stack (Tracing, Structured Logging, Metrics and Alerts)
• Level 9 Test Guide - Two-Tier Semantic Caching (Query/Tool/LLM Response Caching, Life-Safety Bypass, Prometheus Metrics)

Prerequisites: Docker services running (10 services in dev mode)

# Start all services (weather-ai-api, MCP servers, databases, observability)
make docker-up-dev

# Load RAG knowledge base (603 documents)
make rag-load

# Run complete unit test suite (49 tests)
make test-dev

# Run comprehensive golden dataset evaluation (185 cases)
make eval-full

Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Commit changes: git commit -m 'Add amazing feature'
4. Push to branch: git push origin feature/amazing-feature
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: See docs/ directory

Built for teams exploring agentic AI systems

Version: 1.7.0 | Last Updated: 2025-12-17