Skip to content

Updated

Proxy for Scraping Zillow, Redfin and Idealista Listings

  1. For public listing pages on Zillow and Redfin (US) and Idealista (Spain, Italy, Portugal), a rotating residential proxy in the same country is the usual fit: US$1.50 per GB on Budget, US$5 per GB on Premium with state or city choice.
  2. The robots.txt files matter more than the IP: Zillow opens only listed index pages, Idealista closes sort and paged URLs, and Redfin offers market data as downloads.
  3. Read the file for every URL pattern you plan to use, and go slowly.

Check it right after buyingExample output

curl -x http://USER:PASS@HOST:PORT https://ipinfo.io/json

ip
203.0.113.24
country
US
Matches the country you picked
Day 15
203.0.113.24
Same address for the whole term
Example addresses (203.0.113.0/24), not real exits
01 / 08Why the exit country matters for listings

Property sites are country-bound and strict about paths: choose the exit country, read the robots.txt and pace slowly.

Why the exit country matters for listings

Property portals are local: Zillow and Redfin cover the United States, and Idealista runs separate Spanish, Italian and Portuguese sites. Results, currency and language follow the country site and the visitor location, and a request from another country may be redirected or shown differently.

Set the proxy country to the market you collect and check the exit before running. For neighbourhood-level work inside the US, the Premium line takes a state or city; see the US state and city guide.

What each robots.txt said when we opened it

What each robots.txt said when we opened it
ItemWhat we readWhat it means for a script
Zillow (zillow.com)All crawlers (*): /api/ and /homes/ disallowed, with a list of Allow lines for specific listing-index pages such as /homes/for_sale/ and /homes/for_rent/ and their paged versions; map-rectangle URLs (_rect) disallowedOnly the listed index pages are open; map and API paths are closed, so check each URL against the file
Redfin (redfin.com)All crawlers (*): /stingray/ (with an Allow for newest_listings.rss), /myredfin/, /tools and /api/v1/buildings/properties disallowed; no listing-page path is blocked as a whole. Redfin also publishes downloadable housing market data in its Data CenterListing pages are not listed as closed, but the internal API paths are; for market-level numbers use the downloads first
Idealista (idealista.com)All crawlers (*): /ajax/*, listing photo and virtual-tour paths (/inmueble*/foto, /inmueble*/visita-virtual), many ?ordenado-por= sort URLs, /pagina/ and /*/pagina-*.htm paged URLs disallowedSorting and paging URLs are closed, so a crawl that walks result pages page by page is outside the file; plan around it

Sources: robots.txt of zillow.com, redfin.com and idealista.com, and the Redfin Data Center page, opened 20 November 2026. The files change often and match the longest rule first; read the current one for every URL pattern you use. This is not legal advice, and site terms may say more.

Rates, sessions and data rules

A new IP is given on each new connection and a reused connection keeps its IP, so the number of IPs is not the lever; the pace is. Concurrent connections are not limited by the proxy, but start with one request every several seconds per site and stop on 429 or a check page.

Collect public listing data only, keep the source URL and date with every record, and do not collect owner or agent personal data. If you need market-level statistics rather than individual listings, Redfin's downloads are the cleaner source and cost no proxy traffic.

Check robots.txt for the exact path, check the exit, fetch slowly

import time
import urllib.robotparser
import requests

# One connection per country, copied from My assets (HTTP port 2222; the same login also works over SOCKS5 on 3333).
PROXY = "http://LOGIN:PASSWORD@proxy.proxy302.com:2222"
PROXIES = {"http": PROXY, "https": PROXY}
UA = "my-housing-research/1.0 (contact: you@example.com)"
CHECKS = [
    ("https://www.zillow.com", "/homes/for_sale/"),
    ("https://www.redfin.com", "/city/30772/TX/Austin"),
]  # replace with the public pages you are allowed to read

print("exit country:", requests.get("https://ipinfo.io/json", proxies=PROXIES, timeout=(5, 20)).json().get("country"))

for site, path in CHECKS:
    rp = urllib.robotparser.RobotFileParser(site + "/robots.txt")
    rp.read()
    if not rp.can_fetch(UA, site + path):
        print("robots.txt says no:", site + path)
        continue
    r = requests.get(site + path, proxies=PROXIES, headers={"User-Agent": UA}, timeout=(5, 20))
    print(site + path, r.status_code, len(r.text))
    time.sleep(5)  # keep the pace slow; stop on 429 or a check page

Illustrative example: replace LOGIN, PASSWORD, the contact address and the paths. For SOCKS5 install requests[socks] and use socks5h:// with port 3333.

Set it up in 4 steps

  1. Step 1: Choose the line

    Open Rotating Residential Proxies. Budget is US$1.50 per GB (country level); Premium is US$5 per GB and lets you pick a state or city.

  2. Step 2: One connection per country

    Set the Region to the United States, Spain or the country you read, and generate a connection. Copy host, port, username and password from My assets.

  3. Step 3: Test the exit

    Run the script above or open ipinfo.io through the proxy. Expected: the country you set.

  4. Step 4: Run slowly and log

    Fetch only paths the file leaves open, pause between requests, and save the listing URL, the country site and the date.

Quick fixes

A check page or HTTP 403 appears

Check Requests are too fast, or the path is one the site closes.

Fix Slow down, reduce parallel requests and compare the path with robots.txt; adding IPs is not the fix.

The site redirects to another country page

Check The exit country does not match the site you read.

Fix Check the exit on ipinfo.io and set the country of that site.

Python reports a SOCKS proxy error

Check requests[socks] is not installed.

Fix Run python -m pip install "requests[socks]" or use the HTTP line on port 2222.

Real-estate data FAQ

Which proxy for scraping Zillow?

Rotating residential in the United States, from US$1.50 per GB. Zillow's robots.txt closes /homes/ except listed index pages, so check each URL against it first.

Can I scrape Idealista?

Its robots.txt closes sort and paged result URLs and some media paths. Read the file and the site terms; a proxy does not change what is allowed.

Is there a no-scraping way to get Redfin market data?

Yes: Redfin's Data Center offers downloadable housing market data.

Do I need city-level IPs?

Only for neighbourhood-level work. Premium takes a state or city; country-level collection works on Budget.

Further reading

Setup guides Proxy302 publishes on GitHub on the same topic:

Reference links

6
  1. Zillow (zillow.com) robots.txt

    Official documentation · Confirmed on:

    Opened on 20 November 2026: All crawlers (*): /api/ and /homes/ disallowed, with a list of Allow lines for specific listing-index pages such as /homes/for_sale/ and /homes/for_rent/ and their paged versions; map-rectangle URLs (_rect) disallowed

    robots.txt is a crawler-etiquette file and can change; it is not a legal opinion and the site terms may say more.

  2. Redfin (redfin.com) robots.txt

    Official documentation · Confirmed on:

    Opened on 20 November 2026: All crawlers (*): /stingray/ (with an Allow for newest_listings.rss), /myredfin/, /tools and /api/v1/buildings/properties disallowed; no listing-page path is blocked as a whole. Redfin also publishes downloadable housing market data in its Data Center

    robots.txt is a crawler-etiquette file and can change; it is not a legal opinion and the site terms may say more.

  3. Idealista (idealista.com) robots.txt

    Official documentation · Confirmed on:

    Opened on 20 November 2026: All crawlers (*): /ajax/*, listing photo and virtual-tour paths (/inmueble*/foto, /inmueble*/visita-virtual), many ?ordenado-por= sort URLs, /pagina/ and /*/pagina-*.htm paged URLs disallowed

    robots.txt is a crawler-etiquette file and can change; it is not a legal opinion and the site terms may say more.

  4. Redfin Data Center

    Official documentation · Confirmed on:

    Opened on 20 November 2026: the page offers downloadable housing market data (a weekly and monthly Housing Market Tracker, a Balance of Power report and other trackers) under a Downloads section.

  5. Requests documentation: Advanced usage, Proxies

    Official documentation · Confirmed on:

    Opened on 17 November 2026 and re-read 20 November: proxies are passed as a dict to the proxies argument; user:password goes in the URL; SOCKS needs requests[socks]; socks5h resolves DNS on the proxy; requests have no timeout unless one is set.

  6. Proxy302 product facts and price list

    Product observation · Confirmed on:

    Product facts file of the site (dated 10 November 2026), read on 20 November 2026: per-GB prices of both rotating residential lines, traffic does not expire, countries and city targeting, protocols and ports, no free trial.

Ready to try it?

Create an account, top up once, then pick rotating residential, static ISP or datacenter IPs and have one running in your browser within minutes.