AcademyWorkflowsData and reporting

Autonomous AI crawler

Data, sales and research teams that need to collect specific details from many websites, such as contact pages or social profiles.

Use it nowWant it running at your company?
  • Recommendedour rating
  • 60,593views on n8n

Use it now

Open inClaudeChatGPTGeminiPerplexity
Open on n8n.io ↗

Paste it straight onto an n8n canvas with Ctrl+V or Cmd+V.

Third party software. By copying, downloading or installing it you accept the Academy legal notice: you are responsible for reviewing its code and permissions before use.

autonomous-ai-crawler.json

Loading the file...

What it does

An n8n workflow with an AI agent that starts from a website address and navigates the site to find a specific type of information, such as social media profile links. It has one tool to read page text and another to extract links, and it stores input and output in Supabase.

Use cases

  1. 01Find social media profiles for a list of sites
  2. 02Locate contact details on company websites
  3. 03Extract a custom field by editing the schema

Connects

HTTP Request · Supabase · HTML · AI Agent · OpenAI Chat Model · Structured Output Parser · Call n8n Workflow Tool

Questions about
Autonomous AI crawler.

What is Autonomous AI crawler used for?

Data, sales and research teams that need to collect specific details from many websites, such as contact pages or social profiles. An n8n workflow with an AI agent that starts from a website address and navigates the site to find a specific type of information, such as social media profile links. It has one tool to read page text and another to extract links, and it stores input and output in Supabase.

How do I import Autonomous AI crawler into n8n?

Copy the workflow JSON from this page and paste it onto the n8n canvas with Ctrl+V or Cmd+V. Then connect your credentials in each node.

Is Autonomous AI crawler safe to use?

It passed our automatic scan for credential theft, hidden instructions and risky install commands. Third party open source software. Sabemos AI does not maintain it. Check the code and permissions before you connect it to company data.

Related resources

Read the guide: AI data analysts: ask your data ↗

  1. 001markitdownOperations and analyst teams that need to give reports, contracts and spreadsheets to an AI assistant as plain text.OfficialMCP server · ★ 189k
  2. 002XLSXFinance and data teams that build or update Excel models and reports and want them to recalculate correctly.OfficialSkill
  3. 003server-fetchMarketing and research teams that want an AI assistant to read and summarise public web pages.OfficialMCP server
  4. 004Scrape and summarize webpages with AIMarketing and research teams that want quick summaries of articles from a website.RecommendedWorkflow · 292k views
  5. 005AI agent that can scrape webpagesOperations and research teams that want a chat agent able to read web pages and answer questions about their content.RecommendedWorkflow · 212k views
  6. 006Pull data from services without an n8n integrationOperations and data teams who need to connect n8n to a service that has no dedicated node.RecommendedWorkflow · 194k views