Freight and 3PL status checks
Pull tracking events and delivery appointments from carrier portals that do not send EDI, then flag late loads.
Web Screen Scraping Tool and Website Screen Scraper for Portals, Logins and Old Web Apps
A web screen scraping tool reads data off a screen that has no export button and no API, then writes it into rows. WebRobot opens the portal in a real browser, signs in, fills the search form, pages through the results and copies the table into Google Sheets, Excel, CSV or a JSON API, every day or every hour, with nobody at the keyboard.
For ops, finance and logistics teams who still copy statuses out of carrier, supplier and insurance portals by hand. Flat plans from $79 a month, no RPA license, no robot machine.
Robot console
No account needed Running · s Complete · rows
01 · The job it replaces
Forty minutes of copy and paste from a portal, every morning
Somebody on the team logs in to a carrier site, types in a date range, clicks through four pages of shipments and pastes the status column into a spreadsheet. Then does the same on the next portal. The vendor has no export, or the export leaves out the one field you need, or the API costs more than the person doing the copying.
That is the classic screen scraping job, and it is a poor use of a salaried person. A website screen scraper does the same clicks on a schedule and writes a dated row for every record, so the sheet also becomes a history of what changed.
Screen to rows
From a portal screen to clean rows in your sheet
The robot reads what the browser renders, after the scripts have run, so tables that load late or sit
behind a search form come through like any other page.
portal.supplier.example/orders
From 09/01
To 09/30
Search
| PO | Item | Status | ETA |
|---|---|---|---|
| 40118 | Pallet wrap 18in | Shipped | 10/08 |
| 40121 | Case labels 4x6 | Backordered | 10/21 |
| 40127 | Corner boards | Packed | 10/09 |
Page 1 of 4 · Next
orders_daily.xlsx
| run | po | status | change |
|---|---|---|---|
| 10-06 | 40118 | Shipped | was Packed |
| 10-06 | 40121 | Backordered | ETA +12d |
| 10-06 | 40127 | Packed | none |
All 4 pages, one sheet, one row per order per run.
Illustrative portal and rows; you name the columns and the portals.
Gets to the data
Signs in, picks the account or location, types dates into a search form and clicks Search.
Reads the screen
Tables, cards, status badges and detail pages, read from the rendered page, not from brittle selectors.
Follows the pages
Clicks the portal's own Next link, up to 10 pages per run, and opens detail pages when a field lives there.
Writes it down
Google Sheets feed, Excel or CSV download, or JSON from the API, with added and removed records counted.
Fit
Web screen scraping or desktop screen scraping
The phrase covers two different jobs. Pick the tool by where the screen lives, not by the name.
WebRobot handles it
Anything that opens in a browser
- Carrier, supplier, distributor and marketplace seller portals
- Insurance, benefits and payer portals where you have an account
- State license lookups, court and permit search pages
- Old intranet apps and vendor web apps with no export
- Public sites that load their tables with JavaScript
Use an RPA suite instead
Screens that are not web pages
- SAP GUI, Oracle Forms and other installed Windows clients
- Mainframe green screens through a terminal emulator
- Apps streamed through Citrix or remote desktop, which need OCR
- Jobs that must also type into Excel macros on someone's PC
UiPath and Power Automate for desktop are built for these. If the same system also has a web version, point a robot at that instead and skip the robot machine.
04 · How it works
Set up a screen scraping robot in four steps
-
1
Paste the portal address
The login page or the page where the data starts.
-
2
Describe the steps and columns
"Sign in, search the last 30 days, give me PO, item, status and ETA."
-
3
Check the first run
You see the rows before anything is scheduled, and rename or drop columns.
-
4
Schedule and deliver
Daily or hourly into Sheets, Excel, CSV or the API, with Slack or webhook alerts on Scale.
Logins and form filling are agent actions on Scale and above. The full walk-through, including what to do about two-factor prompts, is in the guide to scraping a website that requires a login.
Use cases
Screen scraping jobs teams hand to a robot
Each of these starts as a manual task that one person owns and nobody wants.
Supplier order and backorder lists
Read open POs, backorders and new ETAs from distributor portals so purchasing sees changes before the customer does.
Claims and eligibility statuses
Collect claim statuses from payer or carrier portals your office already logs in to, one row per claim per day.
License and permit lookups
Check contractor, broker or professional licenses on state search pages on a schedule instead of one at a time.
Legacy web app with no export
Move records out of an old vendor app before a migration, or keep a nightly copy while the contract runs out.
Competitor and price screens
Public pages that render prices with JavaScript, covered in depth on price monitoring.
Buyer's table
Best screen scraping tools compared for a team without developers
The question that decides it is who keeps the job running next quarter, when the portal changes its layout.
| Option | How you pay | Where it wins | Where it falls short |
|---|---|---|---|
| Your own Selenium or Playwright script | Developer time, plus a server | Full control, any portal, any logic | Breaks when the screen changes; scheduling, retries and delivery are yours to maintain |
| Chrome extension screen scraper | Free to a few dollars a month | A quick copy of the table you are looking at now | Runs only while your browser is open; no login automation or history |
| RPA suite (UiPath, Power Automate) | UiPath Basic from $25 a month; unattended robots and enterprise tiers by quote | Desktop apps, SAP GUI, green screens, Citrix, and long cross-app processes | Needs a builder who knows the studio and, for unattended runs, a robot machine to keep alive |
| Scraping API | Per request or credit | Developers fetching public pages at volume; see ScraperAPI pricing | Returns HTML; the parsing, login flow and sheet are still your code |
| WebRobot | Flat: $79 or $249 a month | Web portals with logins and forms, described in a sentence, scheduled, delivered to Sheets or an API | Browser only: no desktop clients, no Citrix, no SMS two-factor prompts |
Comparing hosted options in more depth? The browser automation page covers multi-step flows, and cloud web scraper explains why a job that runs on someone's laptop keeps missing days.
07 · Screen scraping vs API
When to screen scrape and when to wait for the API
An official API is the better source whenever it exists, returns the fields you need and costs less than the time it saves. Screen scraping earns its place in the gap: no API, an API that leaves out half the screen, or API access that only comes with an enterprise contract.
Plenty of teams run a screen scraper as a bridge for a few months, then switch the same sheet to the API once the vendor opens it. The columns stay the same, so nothing downstream changes. The longer comparison is in web scraping vs API.
API
The vendor publishes one, it covers your fields, and the price fits. Use it.
Scrape
No API, or the screen shows status notes, ETAs or documents the API omits.
Scrape
API access is quoted as an enterprise add-on worth more than the job.
Ask first
The portal's terms forbid automated access or its robots.txt blocks it. WebRobot honors robots.txt, so ask the vendor for a service account or export.
Sizing
How much screen scraping each plan covers
One row read once is one record. A 200-row status table read every weekday is about 4,400 records a month.
about 6,600
records a month
Three public lookup pages of 100 rows each, every weekday.
Launch, $79 a month
about 22,000
records a month
Five carrier portals behind logins, 200 loads each, every weekday.
Scale, $249 a month
about 80,000
records a month
300 open orders checked every hour across a 12-hour shift, weekdays.
Scale, $249 a month
150,000+
records a month
A shared services team running dozens of portals for several business units.
Autopilot, $699 a month
Yearly billing drops Launch to $39 and Scale to $124 a month; every limit is on the pricing page. For a plain daily file, see scrape website to Excel. If you would rather hand the whole job to people, web scraping services compares agency quotes with a subscription, and how credentials are stored is on the security page.
FAQ
Screen scraping tool questions, answered
Screen scraping is used to pull data out of systems that show it on screen but offer no export or API. Typical jobs are copying shipment statuses from carrier portals, order and invoice lines from supplier portals, claim statuses from insurance portals, and records from an old internal web app. The scraper reads what a person would read and writes it into a sheet or database.
It depends on where the data sits. For desktop software such as SAP GUI, Citrix or a mainframe green screen, an RPA suite like UiPath or Power Automate is the right tool. For anything that opens in a web browser, a hosted web screen scraper is cheaper and faster to set up. WebRobot covers the browser side, including logins, forms and pagination.
Reading your own data, or public pages, is generally lawful in the US, and hiQ v. LinkedIn narrowed how the Computer Fraud and Abuse Act applies to public data. Behind a login the portal's terms of use are a contract you accepted, so check them first. Use your own account, never bypass access controls, and ask counsel when the stakes are high.
No, but they overlap. Screen scraping is reading data off a screen. RPA (robotic process automation) is the wider job of clicking, typing and moving data between applications, and screen scraping is one of its building blocks. A web screen scraper like WebRobot does the reading part plus the clicks needed to reach the data, without an RPA license or a Windows robot machine.
Use the API when the vendor offers one that returns the fields you need at a price you accept: it is more stable and usually sanctioned. Use screen scraping when there is no API, the API leaves out fields the screen shows, or the vendor charges an enterprise fee for access. Many teams screen scrape as a bridge while they negotiate API access.
Yes. On WebRobot Scale and above, a robot signs in with credentials you store for an account you own, keeps the session, fills search forms and reads the pages behind the login. Credentials are encrypted at rest and used only for that robot. Authenticator app codes can be handled when you register the secret; SMS and push approvals cannot.
Not with WebRobot. It works on anything that opens in a web browser, including old intranet apps and vendor portals. Native Windows programs, SAP GUI, terminal emulators and apps streamed through Citrix need an RPA tool with desktop drivers or OCR, such as UiPath or Power Automate for desktop. If the same system also has a web version, point the robot there.
Developer libraries cost nothing but engineering time. RPA platforms start around $25 a month for UiPath Basic and climb to custom enterprise quotes once you need unattended robots. Scraping APIs bill per request. WebRobot is flat: Launch is $79 a month for 10,000 records, and Scale is $249 a month for 100,000 records with logins, form filling and hourly runs.
Stop copying portal screens into spreadsheets by hand
Paste the portal, describe the steps and columns, and get the rows in Sheets, Excel or an API every day or every hour.