Which headless browser signals still separate bots from real shoppers?
Retire the signals that stopped working
The webdriver flag, a headless string in the user agent, and missing plugins were useful when automation frameworks did not bother hiding. Stealth plugins and patched browsers remove all three before the first request. A ruleset still blocking on them catches only the laziest scripts, the ones your rate limits would have caught anyway, and reports a comforting block count for doing it.
Keep them in the log, because a client tripping the old tells tells you something about its operator. Just stop treating any of them as a verdict. The fleets worth worrying about, the ones running residential proxies and rotating fingerprints, pass every legacy check by design and announce themselves somewhere else entirely.
Rendering is hard to fake at scale
Canvas and WebGL rendering depend on the actual graphics stack: GPU, drivers, operating system. A headless browser on a server renders with software rasterisers and a short, recognisable list of configurations, while real shoppers arrive on thousands of genuine device and driver combinations. A fingerprint that claims a high end phone while rendering like a Linux server is not a smoking gun. It is an inconsistency, and inconsistency is the currency here.
Audio processing, font enumeration, and media codec support work the same way. Each is a claim about the machine. Automation gets caught when the claims disagree with each other: touchscreen support with a desktop rendering pipeline, a timezone that fights the IP location, a language list no human configured. Score the disagreements, not any single value, because real devices are weird too and a single odd value proves nothing.
Behaviour across the session beats the first request
A bot session has a job. It goes to the pages with the data, in an efficient order, at an even pace, and it does not do the wasteful things humans do: no backtracking, no half typed searches, no tab left open during a phone call, no mouse wandering while reading. Individually each is nothing. As a session pattern, efficiency itself is the tell, which is uncomfortable because your best customers are not the most human looking visitors either.
That is why behaviour should move a score, not trigger a block. Feed session signals into the same risk decision as fingerprint consistency and request economics, and reserve blocks for the combined picture. The cost of a false positive is a real customer challenged or turned away. The cost of a challenge step, a CAPTCHA or a verification, is smaller, which is why scoring systems step up friction instead of slamming doors.
Every publicised detection technique teaches the automation frameworks what to patch, and the interval between publication and patch has shrunk to weeks. A detection programme built on a fixed list of tricks decays on a schedule its owners do not control. Budget for the decay: review which signals still separate your traffic every quarter, retire the flat ones, and weight the new ones by what they do on your data, not on a vendors benchmark.
Assume the signal has a half life
The durable layer is economics, not fingerprints. A scraper that must run a full browser, solve challenges, rotate residential proxies, and behave like a distracted human is paying close to human cost per page. Detection wins when it makes the automated route expensive enough that the data is cheaper to license, buy, or live without.
Keep reading: How do residential proxy networks make bot traffic look like neighbourhood traffic?, How do bots exploit mobile app APIs that the website never exposes?.
Frequently asked questions
Is the user agent still useful for bot detection?
As a log field, yes. As a block rule, no. Modern automation sends ordinary user agents, and the fleets that matter rotate them. Claims in the user agent are still worth checking against rendering behaviour, because a mismatch between the two is a useful inconsistency signal.
Is the user agent still useful for bot detection?
No single signal should block on its own. Real devices produce odd values legitimately. Detection works on inconsistency across many signals: rendering, fonts, timezone, device claims, and session behaviour that contradict each other.
Is the user agent still useful for bot detection?
Because headless browsers also run legitimate tools: testing frameworks, monitoring, accessibility aids, and some in app browsers look similar. Blocking the tool rather than the behaviour breaks real users and still misses patched automation.
Is the user agent still useful for bot detection?
Quarterly at minimum, using your own traffic and overturn data. Publicised techniques get patched by automation frameworks within weeks, so a static ruleset loses separating power on a predictable schedule.