headless browsers such as Playwright vs curl_cffi: the 6,493 star alternative worth checking

TL;DR — You use requests/httpx and sometimes Playwright for scraping. The gap between them is curl_cffi (6,493★, MIT): it impersonates a real Chrome TLS/JA3/HTTP2 fingerprint from plain Python, no browser. When a site blocks requests but serves real browsers, curl_cffi often just works. The catch is that impersonation is not a one-time setting — it is a version you have to keep current.

Why it matters

Most scraping stacks in the West are bimodal. On one side sits requests or httpx: a few hundred kilobytes of pure Python, instant to start, trivial to deploy — and instantly identifiable as not-a-browser the moment a site looks at the TLS handshake. On the other side sits Playwright or Selenium: a real browser engine, a real rendering pipeline, a real memory footprint, and the ability to pass almost any check because it is the thing the check is looking for.

The problem is that a large class of targets does not need a browser. It needs a browser's fingerprint. A JSON endpoint behind Cloudflare's lighter layers, an internal API guarded by a JA3 check, a price feed that 403s requests but serves Chrome — none of these render JavaScript. They just want to see a Chrome-shaped handshake before they answer. Reaching for Playwright to satisfy that check means paying the full cost of a rendering engine to solve a problem that has nothing to do with rendering.

curl_cffi is the tool that fills that gap. It is a Python HTTP client that reproduces Chrome's TLS/JA3/HTTP2 fingerprint from inside the interpreter. You get the fingerprint of a browser without the browser. That is the whole pitch, and it is why the library became the default in the Chinese scraping ecosystem before it became visible elsewhere: it removes an entire tier of infrastructure from the problem.

The "because Z"

1. No browser to launch: tens of megabytes instead of several hundred, which changes everything on a modest machine.

A headless Chrome install is not a library you import. It is a browser binary, a driver, a process tree, and a rendering engine that allocates memory whether or not the page has anything to render. On a laptop, a small VPS, or a CI runner with a tight memory ceiling, that cost is the difference between running ten workers and running two. curl_cffi is a Python package with a compiled extension. There is no binary to download, no WebDriver to keep in sync, no process to spawn and tear down per request. The exact resident-memory figures for a headless Chrome session versus a curl_cffi process are not published in the data I verified, so I will not put a number on the comparison — but the architectural difference is not subtle, and it is the first reason people reach for this library. If your target is an API-shaped endpoint, you are paying browser overhead for nothing.

2. Chrome's TLS fingerprint is imitated directly from Python, with no rendering engine.

This is the mechanism, and it is worth being precise about. Anti-bot systems — Cloudflare's lighter layers among them — frequently inspect the TLS handshake before they look at your IP or your headers. The order, the extensions, the cipher suites, the elliptic curves, the ALPN negotiation: together these form a fingerprint that identifies the client. requests has a recognizable, non-browser fingerprint. curl_cffi copies Chrome's at the TLS/JA3/HTTP2 level, so the server sees what it expects to see from a real browser. The library exposes this through named profiles:

from curl_cffi import requests

r = requests.get("https://target.com/api/data", impersonate="chrome124")
print(r.json())  # works where requests got 403

No JavaScript engine runs. No DOM is built. No page is rendered. The impersonation happens in the transport layer, which is exactly where the check happens. That is why the library can replace Playwright entirely for JSON endpoints behind light bot protection: it satisfies the check at the layer the check operates on, and skips everything above it.

3. One network call instead of a full session: response time is a different order of magnitude.

A Playwright request is not one request. It is a browser launch, a context creation, a page navigation, a wait for the network to settle, a wait for the renderer, and then the extraction — and if you are scraping many URLs, you either pay that launch cost once and manage a long-lived browser, or you pay it repeatedly. curl_cffi makes one HTTP call. There is no session to establish, no renderer to wait on, no lifecycle to manage. The library's authors describe it as substantially faster than browser automation for the cases it covers; that is a claim published by the authors, not an independent measurement, and I have no verified benchmark figures to give you. But the shape of the difference is structural rather than incremental: you are removing entire stages from the pipeline, not optimizing one of them.

The data

Field	Value
Repository	lexiforest/curl_cffi
Stars	6,493
Licence	MIT
Last commit	2026-09-04 (10 days before verification)
Archived	No
Verified on	2026-09-14
Source	GitHub API

The licence is MIT, a standard permissive open-source licence. For anyone shipping a product, that means you can use curl_cffi in commercial and closed-source software, modify it, and redistribute it, provided you keep the copyright notice and licence text. There is no copyleft obligation, no requirement to publish your own source, and no commercial-licence tier to negotiate. That matters here because the alternative in this space is often a paid anti-bot API or a commercial scraping service; curl_cffi is a dependency you can vendor into a proprietary product without a legal conversation.

The commit date is worth noting for a different reason. Ten days between the last commit and verification is an active project, not an abandoned one — which is directly relevant to the next section.

The trap to avoid

The trap this column exists to expose is this: the imitated fingerprint has to track Chrome's versions. curl_cffi publishes named profiles — chrome124, and others — and a stale profile gets flagged exactly the same way a hand-written fingerprint does. Impersonation is not a setting you configure once and forget. It is a maintenance obligation.

The reasoning is straightforward once you see it. Anti-bot vendors do not just check whether a client looks like Chrome. They check whether it looks like a plausible Chrome — a version that real users are actually running, with the extension set, cipher ordering, and ALPN behaviour that version ships. If you pin impersonate="chrome110" in 2026 and never touch it, you are presenting a fingerprint that no real browser population matches anymore. You have not become invisible; you have become distinctive in a new way. The failure mode is identical to the one you were trying to escape.

The practical consequence is that upgrading curl_cffi is not optional housekeeping. When the library adds a new Chrome profile, that profile exists because the old ones started getting detected. You should treat the version pin in your requirements.txt the way you treat a security dependency: something to review on a schedule, not something to set and forget. The good news is that the project is actively maintained — the last commit is ten days old as of verification — so current profiles are available. The bad news is that this is a treadmill, and it does not stop. If you build a scraping pipeline on curl_cffi and then stop paying attention to it, it will quietly degrade from "works where requests got 403" to "gets 403 like everything else," and you will not get an error message telling you why.

The verdict

• Simple JSON endpoint behind bot protection → curl_cffi (impersonate="chrome"), and keep the library current.
• Complex UI, login flows, or JS rendering → Playwright stays the right tool. curl_cffi does not render, and it will not help you with a page that needs JavaScript to produce its data.
• Pin a profile, but review it. A stale impersonation profile is as detectable as no impersonation at all.
• Respect robots.txt and ToS — this removes a technical barrier, not a legal one.

Data verified 2026-09-14 · Source: GitHub API (lexiforest/curl_cffi, MIT licence field, 6,493 stars, last commit 2026-09-04).

References

1. curl_cffi — MIT, 6493 stars, last commit 2026-09-04, primary language Python.
2. curl_cffi documentation.
3. curl_cffi on PyPI.

All figures were read from the GitHub API (api.github.com) on 2026-09-14. They move: re-check before you ship.