Using proxies with Python requests, minus the usual pain

The smallest thing that works

requests takes a proxies dict keyed by scheme, with credentials inline in the URL.

Note that the https entry also starts with http:// — that means "use an HTTP proxy to reach HTTPS sites", and it is not a typo. Writing https:// there demands the proxy itself terminate TLS, which usually just fails to connect.

• proxies = {"http": "http://user:pass@gateway:8000", "https": "http://user:pass@gateway:8000"}
• requests.get("https://api.ipify.org", proxies=proxies, timeout=15)
• Confirm the returned IP is the proxy before wiring in any real logic

Use a Session instead of a fresh connection each time

Every bare requests.get is a new TCP handshake plus a new TLS handshake. Through a proxy that overhead is larger, and at volume it becomes the bottleneck.

Switch to requests.Session() and attach proxies to the session so the connection pool gets reused. It also keeps cookies, which sticky-session flows require anyway.

Timeouts and retries are not optional

requests has no default timeout. Without one, a single stalled request hangs your job forever, and stalls are more likely through a proxy because the path is longer.

Pass a tuple to separate connect and read timeouts, such as (5, 30). Mount urllib3’s Retry on an HTTPAdapter and retry only timeouts and 5xx. Do not retry 4xx — that is your request being wrong, and retrying just burns traffic.

SOCKS5 needs an extra dependency

requests ships without SOCKS support, so install requests[socks] first. Then point the proxies values at socks5://user:pass@gateway:port.

If you also want DNS resolved through the proxy so you do not leak target hostnames to your local resolver, use socks5h:// rather than socks5://. That trailing h is easy to miss.

Debug order for the usual errors

• 407 Proxy Authentication Required: wrong credentials, or your egress IP is not on the allowlist
• ProxyError or connection refused: wrong host or port — verify once with curl --proxy in isolation
• SSLError: usually a local CA bundle problem. Update certifi rather than disabling verify
• The response shows your own IP: the proxies keys are wrong, or NO_PROXY in your environment matched