Skip to content

Updated

Crawlee, Crawl4AI and Apify Proxy Configuration

  1. Crawlee: new ProxyConfiguration({ proxyUrls: [...] }) passed as the crawler's proxyConfiguration.
  2. Crawl4AI: CrawlerRunConfig(proxy_config=ProxyConfig(server, username, password)).
  3. Apify: paste the proxy URLs into the Actor's Input tab under Proxy and browser configuration, or call Actor.createProxyConfiguration({ proxyUrls }). With no session, Crawlee and Apify use the URLs in turn.

Check it right after buyingExample output

curl -x http://USER:PASS@HOST:PORT https://ipinfo.io/json

ip
203.0.113.24
country
US
Matches the country you picked
Day 15
203.0.113.24
Same address for the whole term
Example addresses (203.0.113.0/24), not real exits
01 / 13One proxy URL, three places to put it

Where the proxy goes in three crawling tools, how each one rotates, and how to keep a session on one IP.

One proxy URL, three places to put it

All three tools take the same proxy in the form http://LOGIN:PASSWORD@HOST:PORT, or host, port, username and password as separate fields in Crawl4AI. Crawlee and Apify also accept a list, and use its entries in turn when no session ID is given. Use the HTTP port of the proxy: Crawl4AI and the Playwright and Puppeteer crawlers in Crawlee drive a Chromium-based browser, and Chromium-based browsers generally do not accept SOCKS5 with a username and password.

Each example requests https://ipinfo.io/json, so the output shows the exit country and network.

Crawlee: ProxyConfiguration and sessions

import { HttpCrawler, ProxyConfiguration } from 'crawlee';

const proxyConfiguration = new ProxyConfiguration({
    proxyUrls: ['http://LOGIN:PASSWORD@HOST:2222'],
});

const crawler = new HttpCrawler({
    proxyConfiguration,
    useSessionPool: true,
    persistCookiesPerSession: true,
    async requestHandler({ body }) {
        console.log(body.toString()); // ipinfo.io JSON: check country and org
    },
});

await crawler.run(['https://ipinfo.io/json']);

HttpCrawler works for any content type; CheerioCrawler expects HTML. useSessionPool and persistCookiesPerSession keep a session's cookies together, and Crawlee starts a new session when it detects blocking.

Crawlee: sessions and tiers

Per the Crawlee guide, without a session ID the proxy URLs are used round-robin, and with a session ID the same proxy is reused for that session. tieredProxyUrls lets Crawlee move to a higher tier of proxies when blocking is detected, and it only works when the configuration is used through a crawler.

A Crawlee session maps to one proxy URL, not to one exit IP. If your proxy URL is a rotating gateway, a new IP can still appear whenever a new connection opens, so decide on the gateway side (a session or a fixed IP) when the IP itself must stay the same.

Crawl4AI: proxy_config on CrawlerRunConfig

import asyncio
from crawl4ai import AsyncWebCrawler, BrowserConfig, CrawlerRunConfig, ProxyConfig

run_config = CrawlerRunConfig(
    proxy_config=ProxyConfig(
        server="http://HOST:2222",
        username="LOGIN",
        password="PASSWORD",
    )
)

async def main():
    async with AsyncWebCrawler(config=BrowserConfig()) as crawler:
        result = await crawler.arun(url="https://ipinfo.io/json", config=run_config)
        print(result.html)

asyncio.run(main())

The Crawl4AI documentation recommends setting the proxy per run on CrawlerRunConfig; the proxy argument of BrowserConfig is deprecated.

Crawl4AI: rotation

For several proxies Crawl4AI documents a RoundRobinProxyStrategy from crawl4ai.proxy_strategy, attached with proxy_rotation_strategy on CrawlerRunConfig, and ProxyConfig.from_env, which reads a PROXIES variable in the form ip:port:user:pass separated by commas.

With a rotating gateway you need only one proxy: the gateway hands out the new IP on each new connection. Add a rotation strategy when you hold several separate IPs.

Apify Actors: two ways to add your own proxy

In the Console, open the Actor's Input tab, scroll to Proxy and browser configuration and enter your proxy URLs. Apify's page says this appears only for Actors whose input schema defines it, so an Actor without that field cannot take your proxy from the Input tab.

In your own Actor code, build a proxy configuration from a list of proxyUrls and pass it to the crawler. Apify's documentation says proxyConfiguration.newUrl(sessionId) returns the next URL, and that without a session ID custom URLs rotate round-robin.

Apify SDK: Actor.createProxyConfiguration

import { Actor } from 'apify';
import { HttpCrawler } from 'crawlee';

await Actor.init();

const proxyConfiguration = await Actor.createProxyConfiguration({
    proxyUrls: ['http://LOGIN:PASSWORD@HOST:2222'],
});

const crawler = new HttpCrawler({
    proxyConfiguration,
    async requestHandler({ body }) {
        await Actor.pushData({ response: body.toString() });
    },
});

await crawler.run(['https://ipinfo.io/json']);
await Actor.exit();

Run it inside an Actor. Replace the placeholders and check the stored response for the proxy country.

Where to enter the proxy

Where to enter the proxy
ItemSettingRotation without a session
CrawleeproxyConfiguration option of the crawler, ProxyConfiguration({ proxyUrls })URLs used in turn; a session ID pins one URL
Crawl4AIproxy_config on CrawlerRunConfigNo rotation by default; add RoundRobinProxyStrategy for a list
Apify ActorInput tab, Proxy and browser configuration, or Actor.createProxyConfigurationCustom URLs used in turn; newUrl(sessionId) pins one

From the Crawlee, Crawl4AI and Apify documentation pages named in the sources, read on 20 November 2026. Tool versions change; check the page for your version.

How to do it with Proxy302

Each snippet needs four values from a Proxy302 proxy: host, port, login and password. For scraping, rotating residential traffic is billed per GB (Budget line US$1.50 per GB, Premium line US$5 per GB); HTTP is on port 2222 and SOCKS5 on port 3333 of proxy.proxy302.com, and the same login works for both. Without a session, each new connection gets a new IP, and a connection you keep open keeps its IP.

When the other side needs one fixed address, a datacenter IP lease costs US$0.25 for 1 day or US$3.99 for 30 days, and you choose HTTP or SOCKS5 when you buy it. There is no free trial; the smallest purchase is 1 GB of rotating residential traffic or one IP for one day.

How to do it with Proxy302: steps

  1. Step 1

    Open the rotating residential page, choose the line, then click Review cost and generate. For a fixed IP, buy a datacenter IP on the datacenter pay-per-IP page instead.

  2. Step 2

    Open My assets and copy HOST, PORT, USERNAME and PASSWORD exactly as they are shown there.

  3. Step 3

    Paste them into the code in this guide, using the port that matches the protocol: 2222 for HTTP, 3333 for SOCKS5.

  4. Step 4

    Run the code against https://ipinfo.io/json and check that the country and org fields describe the proxy, not your own connection.

Troubleshooting

407 or an authentication error from the proxy

Check LOGIN and PASSWORD in the URL exactly as in My assets; special characters in a password are percent-encoded in a URL.

Fix Re-copy the values, or use the separate username and password fields where the tool has them (Crawl4AI).

Stop when If curl with the same login also fails, contact Support with the order number.

Timeouts or a browser error such as ERR_PROXY_CONNECTION_FAILED

Check Port matches the protocol: 2222 for HTTP; a SOCKS5 URL with a login is used in a Chromium-based browser.

Fix Switch to http://HOST:2222 and run again; see the connection guide for the other checks.

Stop when If every port times out from your network, test from another network before changing code.

The Actor input has no proxy field

Check The Actor's input schema does not define proxy configuration.

Fix Run your own Actor with createProxyConfiguration, or ask the Actor's author to add the field.

Stop when Do not try to patch an Actor you do not own; use your own code instead.

FAQ

How do I add my own proxy to Crawlee?

Create new ProxyConfiguration({ proxyUrls: [...] }) and pass it as the proxyConfiguration option of the crawler. Crawlee uses the URLs in turn, or one URL per session when you give a session ID.

How do I set a proxy with login in Crawl4AI?

Use CrawlerRunConfig(proxy_config=ProxyConfig(server=..., username=..., password=...)). The BrowserConfig proxy argument is deprecated.

Can I use my own proxy in an Apify Actor?

Yes. In the Console, enter proxy URLs under Proxy and browser configuration on the Input tab when the Actor defines that field, or call Actor.createProxyConfiguration with proxyUrls in your own code.

How do I keep the same IP for a whole crawl session?

A Crawlee or Apify session id keeps the same proxy URL. If that URL is a rotating gateway, also fix the IP on the gateway side by opening a session there or using a fixed IP.

Further reading

Setup guides Proxy302 publishes on GitHub on the same topic:

Reference links

5
  1. Crawlee: Proxy management

    Official documentation · Confirmed on:

    Read on 20 November 2026: ProxyConfiguration with proxyUrls, round-robin rotation without a session ID, newUrl(sessionId), tieredProxyUrls, useSessionPool and persistCookiesPerSession, and the crawler proxyConfiguration option.

  2. Crawl4AI: Proxy and security

    Official documentation · Confirmed on:

    Read on 20 November 2026: CrawlerRunConfig(proxy_config=ProxyConfig(server, username, password)), ProxyConfig.from_string and from_env, the deprecated BrowserConfig proxy argument, and RoundRobinProxyStrategy with proxy_rotation_strategy.

  3. Apify docs: Using your own proxies

    Official documentation · Confirmed on:

    Read on 20 November 2026: custom proxy URLs in the Actor Input tab under Proxy and browser configuration (only for Actors whose input schema defines it), and proxyConfiguration.newUrl(sessionId) in the SDK.

  4. Apify SDK: Proxy management

    Official documentation · Confirmed on:

    Read on 20 November 2026 (search result summary of the page): creating a proxy configuration from a list of proxyUrls, and round-robin rotation when no session ID is given.

  5. Proxy302 product facts and price list

    Product observation · Confirmed on:

    Product facts file of the site (values checked against the console, the backend and the price table): products, prices in US$ (1 PTC = US$1), lease terms, protocols and ports, quantity limits, no free trial.

Ready to try it?

Create an account, top up once, then pick rotating residential, static ISP or datacenter IPs and have one running in your browser within minutes.