Tools / Web search and scraping / API
POST a URL to web.scrape and the page is fetched, rendered and returned as markdown, with its title, meta description, language and HTTP status code. Rendering, retries and anti bot handling sit with the providers behind the call, and the next provider takes over when one fails. One key, one call per URL.
Ask your agent
Scrape the pricing pages of these five competitors and tell me what each plan includes.
Connect RouterGrowth once, then ask in plain words. New workspaces start with $1 of credit. Get a key or read the setup guide.
A real run
One call to web.scrape (website scrape), run through RouterGrowth on October 26, 2026. These are the first rows it returned; fields that name a person are masked.
| url | title | language | status_code |
|---|---|---|---|
| https://stripe.com/payments | Stripe Payments | Global Payment Processing Platform | en-US | 200 |
One scrape of the public Stripe Payments page: status 200, its title and language, and 23,856 characters of markdown with 59 links, back in under a second.
Step by step
Create a key in the dashboard and send it as a Bearer token. The same key covers every capability.
Send capability web.scrape with input url to /v1/run. The full URL, with https, is the only field.
The page text is in result.markdown, next to title, description, language and status_code.
Use the markdown only when status_code is 200. A 404 or a login page still returns the text it showed.
Price
List prices on this call, read from the catalog when this page was built. The exact figure is quoted before every run, and a monthly plan lowers it.
In practice
You describe the job; the agent picks the calls, shows the price and runs them.
Scrape the pricing pages of these five competitors and tell me what each plan includes.
Search the web for headless CMS comparison articles, then scrape the top three and pull their feature lists.
Read the homepage of each company in this list and add its industry and headcount next to a one line summary.
Connect
One MCP server for Claude, ChatGPT, Claude Code, Codex and Cursor, or the same call over the CLI and REST with the input above.
claude mcp add --transport http routergrowth https://api.routergrowth.com/mcp
npm install -g routergrowth && routergrowth setup
routergrowth run -c web.scrape -i '{"url": "https://stripe.com/payments"}'curl -X POST https://api.routergrowth.com/v1/run \
-H "Authorization: Bearer $ROUTERGROWTH_API_KEY" \
-d '{"capability": "web.scrape", "input": {"url": "https://stripe.com/payments"}}'Worth knowing
web.scrape reads the page you pass. It does not follow links, crawl a site or page through results.
Menus, cookie text and footer links come through with the body. Trim them before you hand the markdown to a model.
Nothing behind a login is fetched, no form is submitted and no button is clicked.
FAQ
The providers behind the call render the page in a browser before converting it, so text that scripts load for a public visitor is included.
Not in one call. List the URLs first, from the sitemap or a web.search with a site: query, then scrape each one.
The call moves to the next live provider in the waterfall, and the run says which one answered.
No. Fetching, rendering and retries happen on the provider side; your code sends one HTTPS request.
More on Web search and scraping
Reading this as an agent? This page as markdown: /tools/web-scraping/api.md
Connect your agent once. The first calls run on the $1 that comes with signup.