What it does
An n8n workflow that walks through a paginated website, country by country, extracts the data on each page and saves it to MongoDB. It caches visited pages on local disk and checks the primary value before inserting, to avoid duplicate records.
Use cases
- 01Collect reference data from a paginated website
- 02Store scraped records in MongoDB without duplicates
- 03Refresh the data on a regular schedule
Connects
HTTP Request · MongoDB · uProc