Skip to content

Automation Proxies: Scraping, Monitoring, Testing, AI Agents

Household IPs for sites that block servers, fixed datacenter IPs for tests, monitors and allowlists — and a setup page for each job.

Configure automation proxies
Pricing

Why automated jobs need proxies

Sites ration requests per IP. Past the limit they answer HTTP 429 Too Many Requests, often with a Retry-After header saying how long to wait, and Cloudflare lets site owners block or challenge traffic by IP address, by network (ASN) or by country. Cloudflare also reports that more than half of Internet traffic is now non-human and that AI training crawlers are blocked by default on new Cloudflare domains — so a job running from one server IP meets these rules first.

Results depend on where a request comes from. Retailers can set prices and availability by region, Google says Googlebot’s default IP addresses appear to be in the USA, and Google Search tailors results to location and language. A test or price check from one server shows one country’s version.

Some jobs need the opposite: one address that never moves. Stripe lets you restrict an API key to specific IPv4 addresses — the approach it suggests when your servers have fixed IPs — and lets you block residential proxies. GitHub’s REST API allows 60 unauthenticated requests per hour per originating IP and advises against allowlisting its hosted runners’ IP ranges. A static datacenter IP gives a job an address you allowlist once and a rate limit of its own.

Which proxy for which automation job

Scroll horizontally with the arrow keys to read all content.

Which proxy for which automation job
CriterionUseWhy
Scraping public pages and cataloguesRotating residentialHousehold IPs on consumer networks for sites that block or rate-limit servers
Price checks, rank tracking and geo testingRotating residential, a sticky session per marketEach market keeps its own country or city for the whole run
Playwright and Selenium test runs, CIStatic datacenterFast fixed IPs; move a site to residential if it blocks servers
Uptime checks, stock polling, API jobs, AI agents’ tool callsStatic datacenterOne fixed address to allowlist and a rate limit of its own

6 of the 10 automation pages below start from static datacenter in the console and 4 from rotating residential. Every setting stays editable before you pay.

Automation proxy setups for 10 jobs

Each page has the setup and code for that job — Python, Node.js, Playwright or Selenium — and how to tell a real result from a block page. “View options and pricing” on a card opens the console with that job’s starting setup.

SERP proxies

SERP proxies for rank tracking: check Google results from the searcher's country and city on household IPs, validate every page and record captchas as unknown.

View options and pricing

Selenium proxies

Selenium proxy with username and password in Chrome: HTTP proxy in ChromeOptions plus Selenium's BiDi auth handler, no extensions. Static datacenter IPs for test runs.

View options and pricing

AI agent proxies

AI agent proxy setup: give Browser Use, Playwright MCP and n8n agents their own exit IP — fixed datacenter IPs for tools and APIs, household IPs for browsing.

View options and pricing

Set up automation proxies in three steps

  1. Test the target with one request

    Send one request through the proxy you plan to use and check that the answer is the real page — not a 429, a challenge page or another country’s version.

  2. Open your job’s page

    Each page shows where the proxy goes in that tool and opens the console on the right product.

  3. Keep a steady pace

    Honour Retry-After, follow robots.txt and each site’s terms, and don’t fetch the same page twice when once will do.

    Expected result Fewer 429s, and results you can compare run after run.

Automation proxy FAQ

Residential or datacenter proxies for scraping?

Datacenter when the site accepts server IPs — it’s faster and cheaper per IP. Rotating residential when the site blocks hosting networks or rate-limits server IPs.

Should API calls go through residential proxies?

Usually not. APIs that allowlist callers need a fixed address, and Stripe, for one, lets account owners block residential proxy traffic. Use a static datacenter IP for API jobs.

Does a proxy let me ignore a site’s rate limits?

No. A proxy changes where requests leave from, not the site’s rules. Honour 429 and Retry-After, follow robots.txt and the terms, and keep a steady pace.

How many proxies does an automation job need?

One per worker that needs its own country, session or rate limit, and one fixed IP for each job or environment you allowlist.

Do AI agents need their own proxy?

Give each agent its own exit so one noisy agent can’t get the others rate-limited: a fixed datacenter IP for tool calls and APIs, a rotating residential IP when a browsing task must see what a local visitor sees.

Sources and verification

10
  1. MDN: 429 Too Many Requests

    Official documentation · Checked:

  2. Cloudflare docs: IP Access rules (by IP, ASN or country)

    Official documentation · Checked:

  3. Cloudflare blog: Content Independence Day, one year on

    Third-party source · Checked:

  4. Google Merchant Center Help: About regional availability and pricing

    Official documentation · Checked:

  5. Google Search Central: How Google crawls locale-adaptive pages

    Official documentation · Checked:

  6. Google Search Help: Why your search results might differ from other people’s

    Official documentation · Checked:

  7. Stripe docs: API keys — access policies

    Official documentation · Checked:

  8. GitHub Docs: Rate limits for the REST API

    Official documentation · Checked:

  9. GitHub Docs: GitHub-hosted runners — IP addresses

    Official documentation · Checked:

  10. Proxy302 console: starting setup of each automation page

    Product observation · Checked:

    Read from the site and console source on 18 October 2026: 6 of the 10 automation pages (Playwright, Selenium, restock and uptime monitoring, scheduled API sync and AI agents) start from static datacenter; scraping, price monitoring, rank tracking and geo testing start from rotating residential. Presets are editable starting points, not access guarantees.

Ready to connect?

Create an account, top up once and pick rotating residential, static ISP or datacenter. Rotating residential exits cover 195+ countries and regions.

Get startedPricing →