Field manual · Dispatches from Unit WR-01
Practical notes from the team behind the WebRobot AI web scraper: how to get data off any website, what the law actually says, and how teams run monitors in production. Every guide is written from real robot runs. Want to try one yourself? The interactive demo extracts live data in about a minute.
ENTRY 01
2026-06-18
9 min read
Three practical ways to get data off any website in 2026, compared honestly - and a step-by-step walkthrough of the fastest no-code route.
how to scrape data from a website
ENTRY 02
2026-06-24
8 min read
Scraping public data is legal in most jurisdictions - with real conditions. What the case law says and the checklist every team should follow.
is web scraping legal
ENTRY 03
2026-06-29
11 min read
Five leading scraping tools compared on the things that actually matter: reliability after site changes, agent actions, pricing model, and setup time.
best web scraping tools
ENTRY 04
2026-07-01
7 min read
A practical setup guide: from picking the pages to watch, to hourly checks with Slack alerts, to feeding the data into your pricing decisions.
how to monitor competitor prices
ENTRY 05
2026-07-16
9 min read
Every way to pay for web scraping in 2026, with real numbers: APIs, no-code tools, managed services, and the in-house build nobody budgets honestly.
how much does web scraping cost
ENTRY 06
2026-07-16
8 min read
Sites block scrapers on five signals: IP reputation, request rate, browser fingerprint, behavior, and session state. What fixes each one, and what does not.
web scraping without getting blocked
ENTRY 07
2026-07-16
8 min read
Scraping behind a login is technically easy and legally different from scraping public pages. The rules to check first, then the setup that works.
how to scrape a website that requires login
ENTRY 08
2026-07-03
8 min read
APIs are cleaner but cover a fraction of the web. A practical framework for choosing between official APIs and scraping - and when to combine both.
web scraping vs api
ENTRY 09
2026-07-16
9 min read
The clean job data does not live on the big aggregators. It lives on company career pages powered by a handful of applicant tracking systems. Where to pull it, and how.
how to scrape job postings
ENTRY 10
2026-07-16
10 min read
Amazon gives sellers an official API, but it omits half of what you can see on a product page. Here is what each route returns, the rules, and the fastest setup.
how to scrape amazon product data
ENTRY 11
2026-07-20
10 min read
Google gives you an official Places API, but it charges per call and limits what you can store. Here is what each route returns, the rules, and the fastest no-code setup.
how to scrape google maps
ENTRY 12
2026-07-20
9 min read
Three ways to pipe live website data into a spreadsheet, from a one-line formula to a scheduled agent, compared honestly on what each can and cannot do.
web scraping to google sheets
ENTRY 13
2026-07-20
9 min read
Pull star ratings, review text, reviewer names, and dates off any Google listing without writing code, and turn them into a reputation dashboard that refreshes itself.
how to scrape google reviews
ENTRY 14
2026-07-20
9 min read
Turn any set of news sites into a structured feed of headlines, article text, authors, and dates for media monitoring and research, without maintaining a scraper.
how to scrape news articles
New entries are filed as the robot learns something worth writing down.
FINAL ASSEMBLY
Describe the data you want. Get a clean spreadsheet back tomorrow morning, and every morning after.