Skip to content

Updated

Proxy for Scraping eBay, Etsy and Walmart:
Rates and Geo

  1. For public product pages on eBay, Etsy and Walmart, a rotating residential proxy in the country of the storefront you read is the usual fit: US$1.50 per GB on the Budget line, US$5 per GB on Premium, and the traffic does not expire.
  2. What matters more than the IP is the pace and the path: all three robots.txt files close search pages or search URLs, so read individual product pages slowly and use a platform API where one covers your use.

Check it right after buyingExample output

curl -x http://USER:PASS@HOST:PORT https://ipinfo.io/json

ip
203.0.113.24
country
US
Matches the country you picked
Day 15
203.0.113.24
Same address for the whole term
Example addresses (203.0.113.0/24), not real exits
01 / 08What the exit country changes on a marketplace

Match the exit country to the storefront, keep one connection per job and pace the requests.

What the exit country changes on a marketplace

eBay, Etsy and Walmart show price, currency, shipping options and sometimes the listing itself according to the visitor country and the country site. A request from another country can return a different price or a different delivery estimate, and the number you store is then wrong for your market.

So decide which market you measure, set the proxy country to it and check the exit before the first run. Rotating residential covers 195+ countries; for several markets, keep one connection per country instead of switching inside one job.

What each robots.txt said when we opened it

What each robots.txt said when we opened it
ItemWhat we readWhat it means for a script
eBay (ebay.com)All crawlers (*): /sch/ search result pages disallowed (with a few Allow exceptions), plus URL patterns such as /itm/addToCart and /itm/watch/ and /usr/*/followersDo not crawl search result pages; read only the individual listing pages you need, slowly
Etsy (etsy.com)All crawlers (*): /api/, /cart, /your/ and search URLs with a q= query disallowed, plus many filter parameters; no crawl-delay line; listing pages are not blocked as a wholeStay off search and account paths; listing pages are the public part, so pace them
Walmart (walmart.com)All crawlers (*): /search, /api/, /account/ and /orders disallowed, plus store-finder and checkout-related paths; /ip/ product pages are not on the disallow list; Crawl-delay 5 is set only for SlurpSearch is off limits by this file; product pages are not listed, but keep a polite pace

Sources: robots.txt of ebay.com, etsy.com and walmart.com, opened 20 November 2026. The files change often; read the current one before each project. This is not legal advice, and site terms and platform APIs may say more.

Rates, sessions and rules that keep a job healthy

Without a session, each new connection gets a new IP and a reused connection keeps its IP, so you do not need a fresh IP per request. The Budget line also offers sessions of 1 to 120 minutes if one run should keep one IP. Concurrent connections are not limited, but the target site is what limits you: start with one request every few seconds per site and raise it only while pages keep returning 200.

Collect public pages only. Do not sign in with accounts that are not yours, do not collect personal data, and stop when you see 429 or repeated check pages; adding IPs to push through is not the fix. If you sell on Etsy, its Open API is the supported source for your own shop data.

Check robots.txt, check the exit country, then fetch one page slowly

import time
import urllib.robotparser
import requests

# One connection copied from My assets (HTTP port 2222; the same login also works over SOCKS5 on 3333).
PROXY = "http://LOGIN:PASSWORD@proxy.proxy302.com:2222"
PROXIES = {"http": PROXY, "https": PROXY}
UA = "my-price-research/1.0 (contact: you@example.com)"
SITE = "https://www.walmart.com"
PAGES = ["/ip/EXAMPLE-ITEM-ID"]  # replace with the public product pages you need

rp = urllib.robotparser.RobotFileParser(SITE + "/robots.txt")
rp.read()

print("exit country:", requests.get("https://ipinfo.io/json", proxies=PROXIES, timeout=(5, 20)).json().get("country"))

for path in PAGES:
    if not rp.can_fetch(UA, SITE + path):
        print("robots.txt says no:", path)
        continue
    r = requests.get(SITE + path, proxies=PROXIES, headers={"User-Agent": UA}, timeout=(5, 20))
    print(path, r.status_code, len(r.text))
    time.sleep(3)  # slow and steady; stop if you see 429

Illustrative example: replace LOGIN, PASSWORD, the contact address and the page path. For SOCKS5 install requests[socks] and use socks5h:// with port 3333.

Set it up in 4 steps

  1. Step 1: Pick the line

    Open Rotating Residential Proxies. Budget is US$1.50 per GB (country level), Premium is US$5 per GB and lets you pick a state or city. Packages of 50 to 500 GB lower the per-GB price on Budget.

  2. Step 2: One connection per market

    Set the Region to the storefront country and generate a connection. Copy host, port, username and password from My assets.

  3. Step 3: Test the exit

    Run the script above or open ipinfo.io through the proxy. Expected: the country you set.

  4. Step 4: Run slowly and log

    Fetch only public product pages that robots.txt allows, keep a pause, and store the date and storefront with every price.

Quick fixes

Prices come back in the wrong currency

Check The exit country may not match the storefront.

Fix Check the exit on ipinfo.io and use the domain of the same country.

HTTP 429 or a check page

Check Too many requests too fast.

Fix Slow down, cut parallel requests, and open a Support ticket with the order number if the exit itself looks wrong.

Pages load for you but not in the script

Check The script sends no browser-like headers or no timeout, or the page needs scripts to render.

Fix Set a User-Agent and a timeout; if the data is rendered by scripts, use a browser tool such as Playwright with the same proxy.

Marketplace data collection FAQ

Which proxy for scraping eBay?

Rotating residential in the country of the eBay site you read, from US$1.50 per GB. eBay's robots.txt closes /sch/ search pages, so read single listing pages and pace them.

Can I scrape Etsy or Walmart product data?

Their robots.txt files leave product pages out of the disallow lists but close search paths. Check the terms of use as well, and use an official API where it covers your case.

Do I need a new IP for every request?

No. A new IP is given on each new connection, and a reused connection keeps its IP. Pace requests rather than chasing IPs.

How many GB do I need?

It depends on page size and volume; the traffic guide shows how to estimate it.

Further reading

Setup guides Proxy302 publishes on GitHub on the same topic:

Reference links

6
  1. eBay (ebay.com) robots.txt

    Official documentation · Confirmed on:

    Opened on 20 November 2026: All crawlers (*): /sch/ search result pages disallowed (with a few Allow exceptions), plus URL patterns such as /itm/addToCart and /itm/watch/ and /usr/*/followers

    robots.txt is a crawler-etiquette file and can change; it is not a legal opinion and the site terms may say more.

  2. Etsy (etsy.com) robots.txt

    Official documentation · Confirmed on:

    Opened on 20 November 2026: All crawlers (*): /api/, /cart, /your/ and search URLs with a q= query disallowed, plus many filter parameters; no crawl-delay line; listing pages are not blocked as a whole

    robots.txt is a crawler-etiquette file and can change; it is not a legal opinion and the site terms may say more.

  3. Walmart (walmart.com) robots.txt

    Official documentation · Confirmed on:

    Opened on 20 November 2026: All crawlers (*): /search, /api/, /account/ and /orders disallowed, plus store-finder and checkout-related paths; /ip/ product pages are not on the disallow list; Crawl-delay 5 is set only for Slurp

    robots.txt is a crawler-etiquette file and can change; it is not a legal opinion and the site terms may say more.

  4. Etsy Open API v3 documentation

    Official documentation · Confirmed on:

    Opened on 20 November 2026: the page describes Open API v3 as a REST API for inventory, sales orders and shop management, with guides on authentication and rate limits.

    The documentation is about shop and app integrations; check its terms for any data use beyond that.

  5. Requests documentation: Advanced usage, Proxies

    Official documentation · Confirmed on:

    Opened on 17 November 2026 and re-read 20 November: proxies are passed as a dict to the proxies argument; user:password goes in the URL; SOCKS needs requests[socks]; socks5h resolves DNS on the proxy; requests have no timeout unless one is set.

  6. Proxy302 product facts and price list

    Product observation · Confirmed on:

    Product facts file of the site (dated 10 November 2026), read on 20 November 2026: per-GB prices of both rotating residential lines, traffic does not expire, countries and city targeting, protocols and ports, no free trial.

Ready to try it?

Create an account, top up once, then pick rotating residential, static ISP or datacenter IPs and have one running in your browser within minutes.