网页抓取MCP服务

Firecrawl Lite MCP Server

Web scraping for AI agents, minus the infrastructure. One process. Your machine. No scraping cloud in the loop.

Your agent needs to read web pages. Firecrawl is great at that, but self-hosting it means Redis, a Playwright service, API workers, and a weekend. Firecrawl Lite is the other option: a single Node.js process with headless Chrome (stealth-patched) that turns URLs into clean Markdown, and speaks the two protocols agents actually use:

• MCP — for Claude Desktop, Claude Code, Cursor, and anything else that talks Model Context Protocol.
• Firecrawl-compatible REST API — for any agent or harness that already speaks the Firecrawl SDK. Point FIRECRAWL_API_URL at it and it quietly impersonates a self-hosted Firecrawl (scrape only — no crawl or search).

Nothing leaves your box except the page requests themselves. No Firecrawl account. No API keys required at all unless you want the optional LLM-powered extraction tools — and for those, you bring your own model.

60-second start

As an MCP server (Claude Code shown; Claude Desktop and Cursor configs are below):

claude mcp add firecrawl-lite npx -- -y firecrawl-lite-mcp-server

As a Firecrawl-compatible API (for agents, scripts, anything HTTP):

docker run -d -p 3000:3000 ariangibson/firecrawl-lite-mcp-server:latest
curl -X POST localhost:3000/v2/scrape -H 'Content-Type: application/json' -d '{"url":"https://example.com"}'

That's a working scraper. Everything below is optional.

What you get

The Firecrawl-compatible API exposes scrape_page as POST /v2/scrape. See the API section for the exact contract.

Hook it up

MCP clients

{
  "mcpServers": {
    "firecrawl-lite": {
      "command": "npx",
      "args": ["-y", "firecrawl-lite-mcp-server"],
      "env": {
        "LLM_API_KEY": "sk-...",
        "LLM_PROVIDER_BASE_URL": "https://api.openai.com/v1",
        "LLM_MODEL": "gpt-5.5"
      }
    }
  }
}

claude mcp add firecrawl-lite \
  --env LLM_API_KEY=your_key \
  --env LLM_PROVIDER_BASE_URL=https://api.openai.com/v1 \
  --env LLM_MODEL=gpt-5.5 \
  -- npx -y firecrawl-lite-mcp-server

{
  "mcpServers": {
    "firecrawl-lite": {
      "command": "npx",
      "args": ["-y", "firecrawl-lite-mcp-server"],
      "env": {
        "LLM_API_KEY": "sk-...",
        "LLM_PROVIDER_BASE_URL": "https://api.openai.com/v1",
        "LLM_MODEL": "gpt-5.5"
      }
    }
  }
}

Agents that speak Firecrawl

Any framework with a "self-hosted Firecrawl" option works unchanged — set its Firecrawl URL to your Firecrawl Lite instance. The server implements the scrape endpoint the SDKs call; search returns a clear 501 because this is a renderer, not a search engine, so pair it with whatever search provider your agent supports.

# ~/.hermes/.env
FIRECRAWL_API_URL=http://your-server:3000
# FIRECRAWL_API_KEY=...   only if you set one on the server

# ~/.hermes/config.yaml
web:
  search_backend: ddgs        # DuckDuckGo, no key. Or searxng, brave-free, ...
  extract_backend: firecrawl  # → Firecrawl Lite

from firecrawl import Firecrawl

fc = Firecrawl(api_key="unused", api_url="http://your-server:3000")
doc = fc.scrape("https://example.com", formats=["markdown"])
print(doc.markdown)

Configuration

Everything is an environment variable, and everything has a default. Full annotated list in .env.example.

Endpoints

Running via npx with nothing set = MCP over stdio. Enable any of these and it becomes an HTTP server on PORT (default 3000) instead. The Docker image turns on /mcp and the Firecrawl API out of the box.

/health is always there when HTTP is on. Without Docker: ENABLE_FIRECRAWL_API=true npx -y firecrawl-lite-mcp-server.

LLM (only for extract_*)

# OpenAI
LLM_PROVIDER_BASE_URL=https://api.openai.com/v1        LLM_MODEL=gpt-5.5
# Anthropic
LLM_PROVIDER_BASE_URL=https://api.anthropic.com/v1     LLM_MODEL=claude-haiku-4-5
# xAI
LLM_PROVIDER_BASE_URL=https://api.x.ai/v1              LLM_MODEL=grok-4
# OpenRouter
LLM_PROVIDER_BASE_URL=https://openrouter.ai/api/v1     LLM_MODEL=openai/gpt-5.5
# Ollama (local)
LLM_PROVIDER_BASE_URL=http://localhost:11434/v1        LLM_MODEL=llama3.3

Scraping behaviour

Proxy

PROXY_SERVER_URL=http://proxy.example.com:10001-10010   # a port range = automatic rotation
PROXY_SERVER_USERNAME=...
PROXY_SERVER_PASSWORD=...
PROXY_LLM_API=false   # proxies are for target sites; LLM calls go direct unless you say otherwise

Deploying

docker run -d -p 3000:3000 \
  -e LLM_API_KEY=... -e LLM_PROVIDER_BASE_URL=... -e LLM_MODEL=... \
  ariangibson/firecrawl-lite-mcp-server:latest

Or use the bundled docker-compose.yml with a .env file. It uses a wget health check on purpose: the Alpine image has no curl, and a curl check will restart-loop the container (see Troubleshooting).

Images are multi-arch (amd64 / arm64), published to ariangibson/firecrawl-lite-mcp-server on Docker Hub and ghcr.io/ariangibson/firecrawl-lite-mcp-server. Tags: latest moves on every push to main; stable only moves on releases — track stable in production (it pairs well with Watchtower for hands-off updates); version tags (1.5.0, 1.5, 1) pin exactly.

Remote MCP clients: Claude Code → claude mcp add -t http firecrawl-lite http://your-server:3000/mcp. Claude Desktop → Settings → Connectors with an HTTPS URL, or mcp-proxy http://your-server:3000/sse if you don't have a certificate (needs ENABLE_SSE_ENDPOINT=true).

Firecrawl-compatible API

Implements the slice of the Firecrawl v2 API that SDK clients use for extraction.

formats defaults to ["markdown"] and accepts both "markdown" and { "type": "markdown" }; screenshot is accepted but ignored. Two extensions beyond stock Firecrawl: stripLinkUrls (boolean) and maxChars (number) override the SCRAPE_STRIP_LINK_URLS / SCRAPE_MAX_CHARS defaults per request — and when link URLs are stripped, links is always included so nothing is lost. With FIRECRAWL_API_KEY set, both endpoints return 401 before anything else. Crawl, map, batch jobs, and the other async endpoints aren't implemented.

Troubleshooting

Could not find Chrome — the npx postinstall step downloads Chrome the first time; if it's missing: npx puppeteer browsers install chrome. Corrupted? rm -rf ~/.cache/puppeteer and run it again.

Scraping works, extract_* doesn't — it's the LLM call. The server logs the provider's status and response body to stderr. 401 = bad key. 400 = wrong model name or a tuning param the model rejects. 429 = you're being rate limited.

Container restart-loops with SIGTERM right after "listening on port 3000" — a curl health check is failing because the image has no curl. Use wget --spider http://localhost:3000/health (the bundled compose file already does).

Agent says Firecrawl isn't configured / extract fails — curl http://your-server:3000/health should show endpoints.firecrawlApi = "enabled". If you set FIRECRAWL_API_KEY on the server, the agent needs the same value. Search failing is expected unless the agent has a separate search provider.

Under the hood

For the curious or the contributing: config.ts parses every env var once into a typed object; browser.ts owns one stealth browser session (launch flags, proxy, UA, navigation, cleanup, retries); scraper.ts puts rotation, retries, and scroll/settle heuristics behind two methods, scrape and screenshot; htmlToMarkdown.ts does the cleaning and conversion; index.ts is just MCP tool definitions, LLM extraction, and transports. The browser and clock are injected, so the whole scrape pipeline is tested through a stub browser — npm test needs no Chrome and no network.

npm install && npm run build && npm test

Releases: bump version, git tag vX.Y.Z, push the tag. CI publishes to npm (with provenance) and Docker Hub / GHCR.

Credits

Inspired by the Firecrawl team and their official MCP server. This is an independent, deliberately small take on the same idea. If you want the managed, enterprise-grade version with crawling, search, and a team behind it, that's firecrawl.com.

License

MIT — see LICENSE.