AcademyMCP serversData and reporting

mcp-server-webcrawl for Searching Crawl Archives

SEO, web and research teams that keep crawl archives and need to find pages, errors or content in them.

  • Recommendedour rating
  • 46gitHub stars
  • Not statedlicence
  • 13 days agolast update
claude mcp add mcp-server-webcrawl -- uvx mcp-server-webcrawl

What it does

An MCP server that lets an AI client search and retrieve data from web crawls made with tools such as WARC, wget, Katana, SiteOne, HTTrack, ArchiveBox and InterroBot. It offers full-text search with boolean operators and can filter resources by type and HTTP status.

Use cases

  1. 01Find pages with a given HTTP status in a crawl
  2. 02Run full-text search across archived websites
  3. 03Filter a crawl by resource type

Questions about
mcp-server-webcrawl for Searching Crawl Archives.

What is mcp-server-webcrawl for Searching Crawl Archives used for?

SEO, web and research teams that keep crawl archives and need to find pages, errors or content in them. An MCP server that lets an AI client search and retrieve data from web crawls made with tools such as WARC, wget, Katana, SiteOne, HTTrack, ArchiveBox and InterroBot. It offers full-text search with boolean operators and can filter resources by type and HTTP status.

How do I install mcp-server-webcrawl for Searching Crawl Archives?

Run this in your terminal: claude mcp add mcp-server-webcrawl -- uvx mcp-server-webcrawl

Is mcp-server-webcrawl for Searching Crawl Archives open source?

The code is on GitHub (pragmar/mcp-server-webcrawl). Check the licence in the repository before using it at work.

Is mcp-server-webcrawl for Searching Crawl Archives safe to use?

It passed our automatic scan for credential theft, hidden instructions and risky install commands. Third party open source software. Sabemos AI does not maintain it. Check the code and permissions before you connect it to company data.