AcademyWorkflowsData and reporting

Scrape and Store Data from Multiple Website Pages

Data and operations teams that need to collect structured data from a multi-page website into a database.

  • Recommendedour rating
  • 80,004views on n8n
Open on n8n.io ↗

Paste it straight onto an n8n canvas with Ctrl+V or Cmd+V.

scrape-and-store-data-from-multiple-website-pages.json

Loading the file...

What it does

An n8n workflow that walks through a paginated website, country by country, extracts the data on each page and saves it to MongoDB. It caches visited pages on local disk and checks the primary value before inserting, to avoid duplicate records.

Use cases

  1. 01Collect reference data from a paginated website
  2. 02Store scraped records in MongoDB without duplicates
  3. 03Refresh the data on a regular schedule

Connects

HTTP Request · MongoDB · uProc

Questions about
Scrape and Store Data from Multiple Website Pages.

What is Scrape and Store Data from Multiple Website Pages used for?

Data and operations teams that need to collect structured data from a multi-page website into a database. An n8n workflow that walks through a paginated website, country by country, extracts the data on each page and saves it to MongoDB. It caches visited pages on local disk and checks the primary value before inserting, to avoid duplicate records.

How do I import Scrape and Store Data from Multiple Website Pages into n8n?

Copy the workflow JSON from this page and paste it onto the n8n canvas with Ctrl+V or Cmd+V. Then connect your credentials in each node.

Is Scrape and Store Data from Multiple Website Pages safe to use?

It passed our automatic scan for credential theft, hidden instructions and risky install commands. Third party open source software. Sabemos AI does not maintain it. Check the code and permissions before you connect it to company data.