Qontinui MCP 视觉自动化服务

qontinui-mcp

Lightweight MCP server for Qontinui Runner - enables AI-driven visual automation.

Installation

pip install qontinui-mcp

Quick Start

1. Start the Qontinui Runner (desktop application)

2. Configure your AI client (Claude Desktop, Claude Code, Cursor, etc.)

Add to your MCP configuration:

{
  "mcpServers": {
    "qontinui": {
      "command": "qontinui-mcp",
      "args": [],
      "env": {
        "QONTINUI_ROOT": "/path/to/your/qontinui-root"
      }
    }
  }
}

QONTINUI_ROOT should point at the directory holding your qontinui repos. It is where .automation-results and .dev-logs are resolved from. Set it — an MCP server inherits whatever working directory the AI client launched it from, so without it the server falls back to that arbitrary directory (and logs a warning): results and logs would be read from, and written to, wherever the client happened to start.

3. Run workflows via AI

The AI can now:

• Load workflow configuration files
• Run visual automation workflows
• Monitor execution status
• Control which monitor to use

Configuration

Environment variables:

Reaching a Windows runner from WSL

The runner binds 127.0.0.1 only, and its Host gate refuses, among others, a request whose Host is neither a loopback name with the runner's bound port nor a QONTINUI_RUNNER_ALLOWED_HOSTS entry (403 HOST_NOT_LOOPBACK).

• WSL1, or WSL2 in mirrored networking mode (networkingMode=mirrored in .wslconfig) shares the Windows loopback, so the default works. Every process in every WSL distro then reaches the runner as a loopback caller, credential routes included, so use it only where you trust every WSL user and container on the machine.
• WSL2 in NAT mode (the default) cannot reach the runner, because the runner listens on no address WSL can reach. Switch to mirrored mode. Forwarding a Windows port to the runner is not a supported setup: every client that can reach the forwarded address arrives at the runner as 127.0.0.1, with a local caller's access.

Development

The repo path-depends on qontinui-schemas, which must be checked out next to this one (../qontinui-schemas). The checks CI runs are declared in .qontinui/ci.toml; run the same ones locally:

# Setup: schemas must sit beside this checkout
git clone https://github.com/qontinui/qontinui-schemas
git clone https://github.com/qontinui/qontinui-mcp
cd qontinui-mcp

# The CI checks
poetry check --lock                       # poetry.lock matches pyproject.toml
poetry install
poetry run ruff check src tests           # lint
poetry run ruff format --check src tests  # format
poetry run mypy src tests                 # strict typecheck
poetry run pytest -q

# Run the stdio MCP server (it waits for a client; see Quick Start for config)
poetry run qontinui-mcp

With pre-commit installed (it is not in the dev group, e.g. pipx install pre-commit), pre-commit install runs ruff (lint + format) and mypy on every commit.

Features

Area A: SSE Event Streaming

Real-time event streaming via Server-Sent Events (SSE) for monitoring workflow execution.

Endpoint: /sse/events

Client Usage:

from qontinui_mcp.client import QontinuiClient

client = QontinuiClient()

def handle_event(event: dict):
    print(f"Event: {event['event_type']} - {event}")

await client.subscribe_events(callback=handle_event, timeout=60)

Event Types:

• qontinui/execution_started - Workflow begins
• qontinui/execution_progress - Step completion
• qontinui/execution_completed - Workflow ends
• qontinui/test_started - Test begins
• qontinui/test_completed - Test ends
• qontinui/image_recognition - Match found/failed
• qontinui/error - Error occurs
• qontinui/warning - Non-fatal issue

Area B: MCP Prompts

Parameterized prompt templates for common automation tasks. Prompts aggregate context from the runner to provide structured debugging, analysis, and verification workflows.

Area C: Tool Caching

Version-based tool caching to optimize MCP tool list requests.

Endpoint: /tool-version

Response:

{
  "version": "abc123...",
  "tool_count": 35,
  "test_count": 12
}

The MCP server caches tools and invalidates the cache when:

• The runner's tool version changes (config loaded, tests added/removed)
• Cache is older than 5 minutes (fallback)

Area E: Permission System

Fine-grained permission control for tool calls inspired by OpenCode's permission system.

Permission Levels:

Configuration:

from qontinui_mcp.permissions import get_permission_service, PermissionLevel

service = get_permission_service()

# Auto-approve only read operations (default)
service.configure(auto_approve_levels={PermissionLevel.READ_ONLY})

# Auto-approve all operations (trusted context)
service.auto_approve_all()

# Custom permission handler
service.on_request = lambda req: input(f"Allow {req.tool_name}? (y/n)") == "y"

Area F: MCP Resources

Read-only data access via URI scheme for accessing runner data.

URI Scheme: qontinui://{type}/{id}

Resource Types:

Area G: Inline Python Execution

Execute arbitrary Python code with optional dependency isolation via uvx.

Tool: execute_python

Parameters:

• code (required): Python code to execute
• dependencies: List of pip packages to install
• timeout_seconds: Execution timeout (default: 30)
• working_directory: Working directory for execution

Example:

# Simple calculation
result = await client.execute_python(
    code="return {'sum': 1 + 2, 'product': 3 * 4}"
)
# result.data["return_value"] == {"sum": 3, "product": 12}

# With dependencies
result = await client.execute_python(
    code="""
    import requests
    resp = requests.get('https://api.example.com/data')
    return resp.json()
    """,
    dependencies=["requests"],
)

Area H: Agent Spawning

Hierarchical task decomposition by spawning sub-agents with focused tasks.

Tool: spawn_sub_agent

Parameters:

• task (required): Task description for the sub-agent
• tools: List of tool names to restrict the sub-agent to
• max_iterations: Maximum turns/iterations (default: 10)
• context: Additional context to provide

Example:

result = await client.spawn_sub_agent(
    task="Verify that the login form works correctly",
    tools=["run_workflow", "capture_screenshot", "execute_test"],
    max_iterations=5,
    context="The login page is at /login with username and password fields."
)

Available Tools

Core Tools

Task Management Tools

Automation Run Tools

Test Management Tools

Test Types:

• playwright_cdp - Browser DOM assertions using Playwright
• qontinui_vision - Visual verification using image recognition
• python_script - Custom Python verification logic
• repository_test - Run pytest, Jest, or other test frameworks

Log Tools

Log Types:

• general - General executor events
• actions - Workflow action/tree events
• image-recognition - Image recognition results with match details
• playwright - Playwright test execution results

DOM Capture Tools

AWAS (AI Web Action Standard) Tools

Tools for interacting with websites that support the AWAS standard.

Advanced Tools

Example Usage

Basic Workflow Execution

# In an AI conversation:
"Load the config at /path/to/workflow.json and run the 'login_test' workflow on the left monitor"

Test-Driven Verification

# Create a verification test
"Create a Playwright test that verifies the login button is visible and enabled"

# Execute the test
"Run the login_button_visible test and show me the results"

# Debug failures
"Use the debug_test_failure prompt for test abc123 with screenshots"

Automation Analysis

# Analyze a failed automation run
"Analyze the most recent automation run and identify why it failed"

# Debug image recognition
"Debug the template matching for the 'submit_button' template"

Architecture

qontinui-mcp (MCP Server)
    |
    v
QontinuiClient (HTTP Client)
    |
    v
qontinui-runner (Desktop App, port 9876)
    |
    v
Python Subprocess (Qontinui Execution)

The MCP server is a thin wrapper that:

1. Exposes runner capabilities via MCP protocol
2. Provides permission control for tool calls
3. Caches tool definitions for performance
4. Aggregates context for structured prompts
5. Streams events via SSE for real-time monitoring

License

Licensed under the GNU Affero General Public License v3.0 or later (AGPL-3.0-or-later). See LICENSE for full terms.