curl_cffi: The MIT Library That Beats Cloudflare Without a Browser

TL;DR — You use requests/httpx and sometimes Playwright for scraping. The gap between them is curl_cffi (6,400★, MIT): it impersonates a real Chrome TLS/JA3/HTTP2 fingerprint from plain Python, no browser. When a site blocks requests but serves real browsers, curl_cffi often just works.

The "because Z"

1. TLS fingerprinting is the silent blocker. Most anti-bot (including Cloudflare's lighter layers) doesn't check your IP or headers first — it checks the TLS handshake. Python's requests has a recognizable, non-browser fingerprint. curl_cffi copies Chrome's exactly, at the TLS/JA3/HTTP2 level.
2. It's one pip install. No browser binary, no WebDriver, no headless Chrome memory. For API-shaped targets (JSON endpoints behind light bot protection) it replaces Playwright entirely.
3. MIT, from a Chinese maintainer (lexiforest). It's the standard tool of the Chinese scraping ecosystem for a reason.

from curl_cffi import requests

r = requests.get("https://target.com/api/data", impersonate="chrome124")
print(r.json())  # works where requests got 403

Verdict

• Simple JSON endpoint behind bot protection → curl_cffi (impersonate="chrome").
• Complex UI, login flows, or JS rendering → Playwright stays the right tool.
• Respect robots.txt and ToS — this removes a technical barrier, not a legal one.

Data verified 2026-09-03 · Source: GitHub (lexiforest/curl_cffi, MIT license field).