AI导哈

数据与研究

crawl4ai

brettdavies/cra…ai-skill

Use when scraping JavaScript-heavy pages or SPAs, crawling multiple URLs concurrently, extracting structured data with reusable CSS/JSON schemas, or building automated web data pipelines. Wraps the Crawl4AI library (`crwl` CLI and Python SDK) with schema-generation patterns for LLM-free extraction. Triggers on crawl4ai, crwl, scrape JS-heavy site, scrape SPA, headless browser scrape, schema-based extraction, batch crawl, sitemap crawl, web data pipeline. SKIP when a static HTML page can be read with `defuddle` / `fetch-web` — those are faster cold-start and don't need a browser.

  • 1113 次安装
  • 49 星标
  • 收录于

安装命令

npx skills add https://github.com/brettdavies/crawl4ai-skill --skill crawl4ai

SKILL.md

Crawl4AI

Verified against crawl4ai VERSION. PEP 723 pins in scripts/*.py and tests/*.py floor at that version.

Overview

Crawl4AI wraps a headless browser (Playwright) plus a markdown-aware content pipeline. Use it when defuddle/curl can't reach the content — JavaScript-rendered pages, login-gated content, infinite scroll, multi-URL concurrency, repeatable schema-based extraction.

This skill exposes both interfaces of the underlying library:

  • CLI (crwl) — quick, scriptable commands: CLI Guide
  • Python SDK — full programmatic control: SDK Guide

Invoked with a URL argument

Crawl4AI

Verified against crawl4ai VERSION. PEP 723 pins in scripts/*.py and tests/*.py floor at that version.

Overview

Crawl4AI wraps a headless browser (Playwright) plus a markdown-aware content pipeline. Use it when defuddle/curl can't reach the content — JavaScript-rendered pages, login-gated content, infinite scroll, multi-URL concurrency, repeatable schema-based extraction.

This skill exposes both interfaces of the underlying library:

  • CLI (crwl) — quick, scriptable commands: CLI Guide
  • Python SDK — full programmatic control: SDK Guide

Invoked with a URL argument