Beyond the Exit IP: How WebRTC and DNS Undo Your Residential Proxy (and How to Verify It)
A residential proxy only controls the path your HTTP requests take. It does not automatically control every way a browser can reveal where it is. That gap is where a surprising number of "clean IP but still blocked" case
A residential proxy only controls the path your HTTP requests take. It does not automatically control every way a browser can reveal where it is. That gap is where a surprising number of "clean IP but still blocked" cases come from, and WebRTC is the loudest version of it.
Here is the failure I see most often in scraping setups that already do IP hygiene right. You route traffic through a residential exit in Berlin. The HTTP layer looks fine: your fetch goes out a Berlin ASN, the site logs a Berlin address, geolocation APIs agree. Then the same page opens an RTCPeerConnection, fires a couple of STUN binding requests, and the reflexive candidate it gathers reports your real outbound address 鈥?the datacenter box the headless browser actually runs on, or a home broadband IP that is nowhere near Berlin. One request says Berlin, the other says "actually, Frankfurt, AS-NOT-A-RESIDENTIAL." Modern anti-bot does not need to block you on that. It just needs to stop trusting you and quietly downgrade you to a challenge or a thin-data page.
What the proxy does not govern
When you hand Chromium an HTTP or SOCKS5 proxy, it proxies the connections you make through the normal network stack. WebRTC's ICE gathering is a different animal: the browser tries to discover the addresses the peer can be reached at, and part of that discovery is direct UDP STUN. On many default configurations that UDP path does not respect the proxy. So the IP you want the page to see and the IP WebRTC thinks it has diverge.
There is a wrinkle that makes this less obvious than it used to be. Recent Chromium builds redact local and private ICE candidates behind mDNS 鈥?instead of a plain 192.168.x.x in the candidate list you now get something like a1b2c3.local. That killed the old-school "print the local IP from WebRTC" trick and made a lot of people think WebRTC was safe now. It is not, for our purpose: the redaction covers local addresses, but the server-reflexive candidate 鈥?the public address a STUN server saw you come from 鈥?still gets collected unless you stop it. And that public reflexive address is exactly the geo/ASN contradiction that matters.
The blunt fix: force WebRTC through the proxy
The supported way to stop non-proxied UDP is the webrtc-ip-handling-policy switch. disable_non_proxied_udp tells the browser not to send UDP that is not going through the proxy, which collapses the leak because the direct STUN path is gone.
from playwright.sync_api import sync_playwright
PROXY = {
"server": "http://gateway.your-residential-provider.net:8000",
"username": "cust-session-42", # session/user encodes the target geo
"password": "***",
}
def launch(p):
return p.chromium.launch(
headless=True,
args=[
"--webrtc-ip-handling-policy=disable_non_proxied_udp",
"--disable-features=AsyncDns",
],
proxy=PROXY,
)
with sync_playwright() as p:
browser = launch(p)
ctx = browser.new_context(locale="de-DE", timezone_id="Europe/Berlin")
page = ctx.new_page()
page.goto("https://example-target.com/listing", wait_until="domcontentloaded")
# ... your extraction ...
browser.close()
Two things to note in that snippet, because they are where people get bitten. First, disable_non_proxied_udp may break WebRTC entirely for pages that genuinely need it 鈥?that is fine when you are scraping a listing page, but it is a real behavior change, not free. Second, the flag is not version-proof. Chromium has moved around how command-line WebRTC policies are honored across releases, and enterprise-policy variants (WebRtcIPHandling) can take precedence. The flag is necessary-but-verify, not set-and-forget. Which is the whole point of the next section.
Do not trust the fix 鈥?measure it
The honest version of proxy work is that you do not know you closed the leak until you have looked at what the browser actually gathers and compared it to what an HTTP request shows. That test is small and worth putting in CI.
Run one gather of ICE candidates inside the browser, and one plain HTTP echo of your public IP through the same proxy, and assert they tell a consistent story.
JS_GATHER_CANDIDATES = """
async () => {
const out = [];
const pc = new RTCPeerConnection({ iceServers: [{ urls: 'stun:stun.l.google.com:19302' }] });
pc.createDataChannel('');
pc.onicecandidate = e => { if (e.candidate) out.push(e.candidate.candidate); };
const done = new Promise(res => { pc.onicegatheringstatechange =
() => pc.iceGatheringState === 'complete' && res(); });
await pc.createOffer().then(o => pc.setLocalDescription(o));
// ICE gathering can hang forever behind some proxies; bound it.
await Promise.race([done, new Promise(r => setTimeout(r, 4000))]);
pc.close();
return out;
}
"""
import json, urllib.request
def http_public_ip(proxy_url):
req = urllib.request.Request("https://api.ipify.org?format=json")
with urllib.request.urlopen(req, timeout=8) as r:
return json.load(r)["ip"]
# Inside a page context after launch():
candidates = page.evaluate(JS_GATHER_CANDIDATES)
http_ip = http_public_ip("http://gateway.your-residential-provider.net:8000")
print("HTTP sees:", http_ip)
for c in candidates:
print("candidate:", c)
What you are looking for: any candidate line containing host or srflx with a public IPv4 that is not the HTTP sees: address. .local mDNS entries are the browser redacting a private address and are acceptable; a raw public srflx that disagrees with the proxy is the leak. If your provider exposes a geo lookup, resolve both IPs and confirm the country (and ideally ASN type) matches. If they diverge, the policy flag did not take on that build, and you are shipping a contradiction to every anti-bot that checks it.
The same "the proxy is not a magic cloak" logic extends past WebRTC. --disable-features=AsyncDns matters because Chromium's built-in asynchronous resolver can resolve names directly and leak the DNS query path to your host's resolver, which geolocates to the wrong place even when HTTP is proxied. And a proxy does nothing at all about Intl.DateTimeFormat().resolvedOptions().timeZone, the WebGL renderer string, or navigator.hardwareConcurrency 鈥?so pairing a Berlin exit IP with a America/New_York timezone and a Google cloud GPU is its own tell. IP is one signal; the job is to keep the signals consistent, not to swap one number and call it done.
Operationally, the cheapest thing you can do is treat the echo-vs-candidate check as a log line instead of a one-off experiment. On every Nth navigation, record the HTTP echo IP, its resolved country and ASN type, and any public srflx candidate you gathered, then alert when they disagree. A mismatch that shows up once is a flaky STUN path; one that shows up on every request from a given exit is the proxy configuration or the browser build, and you want to catch it before the target's trust score does. The failure is rarely loud 鈥?you get valid-looking HTML with prices hidden, results truncated, or a CAPTCHA only on your best-selling queries 鈥?so the signal has to come from your own instrumentation, not from a non-200.
Why the override trick is the tempting wrong answer
The internet full of "stealth" snippets shows you overriding window.RTCPeerConnection with an init script that rewrites or empties the candidate list. It is quick, and it is worse than the flag for a specific reason: a hand-rolled RTCPeerConnection is trivially detectable. A target can call Function.prototype.toString.call(window.RTCPeerConnection) and see that the "native" constructor now prints JavaScript instead of [native code], or compare the methods it expects against what you left behind. An empty candidate array is also a signature: real browsers on real networks gather something. You replaced a geo contradiction with a bot-ness contradiction, which is often the more expensive one.
So the practical ranking, from most to least defensible: honor the policy at the engine level and verify it; if you cannot get the engine policy to hold on your build, run WebRTC through a proxy that actually carries the UDP path; and treat a raw JS monkey-patch as a last resort that itself needs to be made consistent with the rest of the fingerprint. When the whole point is a believable real browser and the target leans hard on these signals, that is also the honest case for using a managed scraping browser that owns the browser-fingerprint-and-IP consistency problem for you, instead of rebuilding it flag by flag and version by version.
The part I would tell you in person
I write this while partnered with Thordata, so treat the product mention as advertising and the reasoning as the part worth keeping. The reasoning generalizes: your provider's residential line (residential proxies) buys you a plausible exit IP; it does not buy you a plausible browser. Thordata's scraping browser exists for the case where you have decided the fingerprint-consistency tax is bigger than the maintenance tax of doing it yourself 鈥?which, on the targets that actually matter, is usually the right call.
The takeaway that survives any provider: after you fix the IP, run the candidate-vs-echo test above on your own build. A closed leak you have measured is worth more than a leak you assumed the flag handled. If the two numbers agree, you stopped contradicting yourself across protocols. If they do not, you did not, and the cleanest residential IP in your pool will not save you from it.
Originally published by Dev.to WebDev. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.