网页抓取 MCP 服务器

SEC-API Internal MCP Servers

• MCP server to fetch URLs with cloakbrowser
• Returns HTML, html-to-text and PDF files

Tools

The two website tools take:

get-pdf takes pdfUrl and useProxy. It first tries a plain download with a browser agent, and it falls back to the cloak browser, because some hosts, such as sec.gov, answer 403 to that agent. The magic number %PDF- decides whether the answer holds a file: a login page, a soft 404 and the HTML of the built-in PDF viewer all come with status 200.

A GET on http://127.0.0.1:22001/mcp shows all tools as an HTML page. The MCP protocol runs over POST on the same route.

Run in Docker (recommended)

The container holds everything: node, Xvfb and the Chromium of cloakbrowser. Nothing opens a window on your desktop.

docker compose up -d --build
docker compose logs -f

The server then listens on http://127.0.0.1:22001/mcp.

What the container gets:

• 10 Xvfb displays, :99 to :108, one per proxy, started by docker-entrypoint.sh
• the MS core fonts, the Liberation fonts and Segoe UI. Chromium draws a box glyph for each missing font, and that breaks the canvas fingerprint
• ./modules, ./config and .env as read-only mounts, thus a code change needs a restart but no rebuild
• ~/.secdotenv as a read-only mount, because secdotenv needs that key to decrypt .env
• ./output as a writable mount for the page cache

A code change needs docker compose restart. A change of package.json, the Dockerfile or docker-entrypoint.sh needs docker compose up -d --build.

Proxies

useProxy: true sends the request through the SOCKS5 proxies on the Tailscale host. This works from the container, because Docker Desktop routes the TCP connection through the host, and the host holds the Tailscale route.

useProxy: false → your own line
useProxy: true  → the proxy pool

Run on the host

Three entry points, all with the same tools:

# http, for Claude Code
node modules/server-browser.js

# stdio, for an app that spawns the server as a local process
node modules/server-browser-stdio.js

# stdio in, http out: a bridge to the server in docker, for Claude Desktop
node worker/mcp-stdio-to-http.js

On macOS the browser window comes up on your desktop and takes the focus. macOS has no X server, and macOS clamps a window position into the area of the attached displays, thus the window cannot hide. Use Docker instead.

Connect a client

Claude Code

Add .mcp.json to the project root:

{
  "mcpServers": {
    "browser-mcp": {
      "type": "http",
      "url": "http://127.0.0.1:22001/mcp"
    }
  }
}

Or add it from the command line:

claude mcp add --transport http browser-mcp http://127.0.0.1:22001/mcp

Run /mcp to confirm the server. Restart the session if it does not show.

Claude Desktop and Cowork

Do not use a custom connector. Claude dials a connector from the cloud, thus it needs a public HTTPS URL and cannot reach your machine. The dialog rejects http://127.0.0.1.

Edit the config file instead, with the app closed, because the app writes that file on exit:

open -e "$HOME/Library/Application Support/Claude/claude_desktop_config.json"

An HTTP entry does not work either. A type: http url in this file gives no server in the app, thus the config takes a local process only.

Use worker/mcp-stdio-to-http.js. The app spawns that worker, and the worker sends each message to the HTTP server of the container:

"mcpServers": {
  "browser-mcp": {
    "command": "/Users/jan/.nvm/versions/node/v24.12.0/bin/node",
    "args": ["/Users/jan/VSCodeProjects/sec-api/sec-api-internal-mcp-servers/worker/mcp-stdio-to-http.js"]
  }
}

Use the full path of your node binary, because the app does not load your shell profile. MCP_HTTP_URL points the worker at another endpoint, and the default is http://127.0.0.1:22001/mcp.

The container must run for this:

docker compose up -d

The worker keeps the browser pool of the container warm, because every request lands in the same server. A code change then needs docker compose restart only, and no restart of the app.

Two other ways, both without the worker:

// the stdio server inside the container. one browser start per tool call.
"command": "/usr/local/bin/docker",
"args": ["exec", "-i", "browser-mcp", "node", "modules/server-browser-stdio.js"]

// the stdio server on the host. a browser window opens on your desktop.
"command": "/Users/jan/.nvm/versions/node/v24.12.0/bin/node",
"args": ["/Users/jan/VSCodeProjects/sec-api/sec-api-internal-mcp-servers/modules/server-browser-stdio.js"]

Restart the app after a config change. The app spawns the process at start and never reloads it.

Scripts

They run against the server on config.mcp.httpUrl, thus the container must run. MCP_HTTP_URL points them at another server.

# list every tool with its inputs. exits 1 when the server offers none
node scripts/test-mcp-tools-list.js

# call get-pdf and check the header and the end marker of each file.
# without an argument it takes two known URLs, one per download path
node scripts/test-mcp-tool-get-pdf.js [url ...]

# download one pdf to output/tmp.pdf, without the mcp server
node scripts/get-pdf.js [url]

Notes

• The page cache goes to output/cache/<date>/, as .html and as .txt. A PDF goes to the same folder, as .pdf.
• npx cloakbrowser info reports the state of the browser and of the fonts:
  复制

  docker compose exec browser-mcp node node_modules/cloakbrowser/dist/cli.js info
  

  Win fonts reads 3/8. Calibri, Consolas, Marlett, MS UI Gothic and Franklin Gothic are licensed Microsoft fonts, and no package ships them. Copy them from a Windows machine to get the full set.
• The free cloakbrowser license covers one session at a time. The factory opens one browser per proxy, thus useProxy: true with many proxies needs a key.