STEP 01
Point it at the store
A category page, a search result, or a list of product URLs. If the store needs a search or a location set first, agent actions fill the form before the robot starts reading.
Ecommerce Data Scraping: Product, Price, and Catalog Data From Any Online Store
Ecommerce data scraping turns any online store into structured rows: product name, price and sale price, variants, SKU, stock status, star rating, and review text. WebRobot does it with an AI agent. Describe the fields you want in plain English, point it at a category page or a list of products, and it paginates through the catalog, opens each product, and delivers clean rows to a spreadsheet or your pricing stack on a schedule.
No selectors, no proxy stack, no per-store scraper to rewrite every time a shop changes its theme. The robot re-finds the fields and the hourly price pull keeps landing.
Last updated July 2026
Robot console · WR-01
Standing by Running · s Complete · rows
1 · Pick a target
3 · Fields to extract
Agent log
Crawl graph
Extracted data · rows
Want this data fresh every morning, without lifting a finger?
01 / FIELDS
FIG. 1 · WHAT THE ROBOT PULLS
What an ecommerce web scraper collects from a product page
A product page hides more than it shows. The list view has a name and a price; the variants, stock,
and reviews live one click deeper. The robot reads both levels into one row.
| Field group | What the robot extracts | Powers |
|---|---|---|
| Pricing | List price, sale price, currency, unit price, discount percentage | Price monitoring and repricing rules |
| Identity | Title, brand, SKU or product ID, GTIN or UPC where shown, category path | Catalog matching across stores |
| Variants | Every size, color, and configuration with its own price and stock | Assortment and availability analysis |
| Availability | In stock, out of stock, low-stock flag, shipping estimate, seller | Stockout alerts and demand signals |
| Reviews | Star rating, review count, review text, date, verified-purchase flag | Product research and complaint mining |
| Rank and placement | Position on category and search pages, badges, sponsored flag | Share-of-shelf and merchandising analysis |
Facts like price and availability are exactly the kind of public data a scraper collects cleanly. Photos and written descriptions are copyrighted, so collect the facts, not the creative work. The same engine drives our data extraction tool.
02 / PROCEDURE
FIG. 2 · SETUP
How to scrape ecommerce data in four steps
The first catalog takes about ten minutes to set up. Every price refresh after that runs itself.
STEP 02
Describe the row
Plain English: "title, price, sale price, variants, stock, rating, review count." The robot opens each product for the variant and review detail the grid hides. No CSS selectors.
STEP 03
Set the schedule
Hourly for live price tracking, daily for catalog and stock. Each run delivers only what changed, so you see price cuts and stockouts as movement, not the full catalog again.
STEP 04
Deliver into your stack
Google Sheets or Excel for analysis, Slack for a price-drop alert, or the REST API into your repricer, PIM, or BI dashboard.
03 / RULES
FIG. 3 · THE LINES
Ecommerce scraping that stays on the right side of the rules
Product data has a clean public core and a few sharp edges around marketplaces and copyrighted content.
Knowing the difference keeps a pipeline durable.
GREEN
Public product facts
Price, availability, SKU, specs, and star ratings are facts a store publishes to sell. Facts are not copyrightable, and collecting public data has repeatedly been held not to be unauthorized access. This is the core of price monitoring and assortment work.
YELLOW
Photos and descriptions
Product images and written copy are copyrighted by whoever made them. Collect the facts for analysis; do not republish someone else's photos and descriptions as your own listing. Review text is fine to analyze, but attribute and do not pass it off as yours.
RED
ToS walls and bot protection
Large marketplaces prohibit automated access and enforce it. Where an official product API exists, prefer it for what it covers. Scrape carefully and at low frequency only where terms allow. WebRobot honors robots.txt and rate limits by default.
The full legal picture is in is web scraping legal, and the API-versus-scraping tradeoff is in web scraping vs API.
04 / USE CASES
FIG. 4 · WHO RUNS THIS
Where scraped product data pays for itself
Price monitoring and repricing
Track competitor prices hourly, catch every drop and stockout, and feed the data into a repricing rule so your prices move with the market. The dedicated workflow is on price monitoring.
Brands and MAP enforcement
Watch every retailer selling your product and flag anyone breaking minimum advertised price. A daily file of who is below MAP, by how much, is the evidence your channel team needs to act.
Catalog and PIM building
Pull structured product data (titles, specs, variants, categories) to seed or enrich a catalog without keying it by hand. Match products across stores on SKU and GTIN to build a unified view.
Assortment and share of shelf
Track which products a competitor stocks, what they add and drop, and where you rank on their category pages. Assortment gaps are product-launch opportunities you can see coming.
Product research and review mining
Pull reviews across a category to find the features buyers praise and the complaints that repeat. It is the fastest way to spec a product or a listing around what customers actually say.
Resellers and arbitrage
Compare prices and stock across stores to spot margin, and export the file to a sheet with scrape website to Excel for a ranked buy list.
Pricing is flat: plans on pricing start at $79 per month with no per-record fee. Stores with search forms or logins use agent actions on Scale; see browser automation. For the category overview, start at the web scraping tool pillar.
05 / FAQ
FIG. 5 · FIELD QUESTIONS
Ecommerce data scraping questions, answered
Ecommerce data scraping is the automated extraction of product information (name, price, variants,
SKU, stock status, ratings, and reviews) from online stores into structured rows. Instead of
checking product pages by hand, an agent browses the catalog, opens each product, and delivers a
clean spreadsheet or feed that refreshes on a schedule so prices and stock stay current.
Scraping publicly posted product facts such as price, title, and availability is generally lawful
in the US as public data. The limits are contractual and creative: many marketplaces prohibit
automated access in their terms, product photos and written descriptions are copyrighted, and some
run bot protection. The durable approach is collecting facts (price, stock, specs) at a polite rate
from stores whose terms allow it, or using an official product API where one exists.
You can, and many sellers do for price and rank tracking, but Amazon prohibits scraping in its
terms and runs strong bot protection. Amazon also offers the Product Advertising API to associates,
which returns price, images, and features cleanly but omits things like full review text and some
Buy Box detail. The honest answer: use the API for what it covers and scrape carefully, at low
frequency, for what it does not.
Title, brand, price and any sale price, currency, SKU or product ID, every variant (size, color,
configuration), stock status, shipping estimate, star rating, review count, and review text. From
category pages it also captures rank and position. Combined, that is enough to power price
monitoring, assortment analysis, and catalog building.
Point the robot at a competitor category page or specific products, describe the fields (title,
price, sale price, stock, rating), and schedule it hourly or daily. Each run delivers only what
changed, so you see price cuts and stockouts as they happen and can feed them into a repricing rule
or a dashboard instead of checking pages by hand.
Yes. The robot opens each product, paginates through the review pages, and pulls rating, date,
verified-purchase flag, and review text into rows. Review mining surfaces the features customers
praise and the complaints that repeat, which is product research and competitive intelligence a
star average alone cannot give you.
WebRobot is flat-priced: Launch at $79 per month ($63 yearly) covers 5 robots and 10,000 records,
enough to track a competitor set. Scale at $249 ($199 yearly) adds hourly schedules for live price
tracking, agent actions for stores with search and login, and 100,000 records for large catalogs.
There is no per-record fee, so a growing product set does not grow the bill.
FINAL ASSEMBLY
Every competitor's prices and catalog, refreshing on schedule
Point the robot at the store, describe the row, and get live product and price data in a spreadsheet or your pricing stack.