What it does
webclaw turns websites into clean markdown, JSON and context an AI model can read. It can scrape, crawl, extract, summarize, diff and research, and it uses TLS fingerprinting to get past anti-bot checks without running a browser. It comes as a CLI, an MCP server, a REST API and SDKs.
Use cases
- 01Convert a website into markdown for an AI agent
- 02Crawl a site and extract structured fields
- 03Compare two versions of a page to see what changed