AcademyMCP serversDevelopment

doc-scraper

Development teams that want their AI agent to work from the exact documentation they chose.

Use it nowWant it running at your company?
  • Recommendedour rating
  • 104gitHub stars
  • Apache-2.0licence
  • todaylast update

Use it now

Open inClaudeChatGPTGeminiPerplexity
git clone https://github.com/Sriram-PR/doc-scraper.git
Open the original ↗

Third party software. By copying, downloading or installing it you accept the Academy legal notice: you are responsible for reviewing its code and permissions before use.

README.md

Loading the file...

What it does

Crawls documentation sites into local Markdown or JSONL files and serves them to your AI agent over MCP. The agent can search the saved docs offline, read pages, check whether they are still fresh and compare crawls.

Use cases

  1. 01Save a framework's docs locally for an agent to search
  2. 02Check whether saved documentation is out of date
  3. 03Compare two crawls to see what changed in the docs

Questions about
doc-scraper.

What is doc-scraper used for?

Development teams that want their AI agent to work from the exact documentation they chose. Crawls documentation sites into local Markdown or JSONL files and serves them to your AI agent over MCP. The agent can search the saved docs offline, read pages, check whether they are still fresh and compare crawls.

How do I install doc-scraper?

Run this in your terminal: git clone https://github.com/Sriram-PR/doc-scraper.git

Is doc-scraper open source?

Yes. The code is on GitHub (Sriram-PR/doc-scraper) under the Apache-2.0 licence.

Is doc-scraper safe to use?

It passed our automatic scan for credential theft, hidden instructions and risky install commands. Third party open source software. Sabemos AI does not maintain it. Check the code and permissions before you connect it to company data.

Related resources

Read the guide: Private AI on your company documents ↗

  1. 001Claude APIDevelopment teams adding Claude features to their own products or internal tools.OfficialSkill
  2. 002MCP BuilderDevelopment teams that want to connect an internal or third-party service to Claude through an MCP server.OfficialSkill
  3. 003Web Artifacts BuilderDevelopment and product teams that want to build interactive prototypes or internal tools as a single shareable HTML file.OfficialSkill
  4. 004server-gitDevelopment teams that want an AI assistant to explore repository history and changes.OfficialMCP server
  5. 005server-everythingDevelopment teams that build or test MCP clients and need a server that covers every protocol feature.OfficialMCP server
  6. 006Context7Development teams that want their AI coding assistant to work from current library documentation.OfficialMCP server · ★ 63k