Seatext library / BotRefund evidence
How to Avoid False Positives When Detecting Playwright Automation
False positives happen when a single browser anomaly — like a patched API or unusual timing — flags a real person as a bot. The reliable way to avoid this is to treat every...
✓ Built for advertisers who need clear, refund-ready traffic evidence.
False positives in Playwright detection occur when legitimate users trigger automation signals because of privacy tools, corporate networks, unusual devices, or browser configurations that happen to look like automation. The fix is not a stricter rule — it is a broader evidence base. Treat each anomaly as one data point, then require independent confirmation from browser fingerprinting, network reputation, device consistency, and behavioral patterns before you label a session as automated.
Why False Positives Happen in Playwright Detection
Playwright and similar automation frameworks run real browser engines. They can load pages, execute JavaScript, and render pixels just like a human visitor. Detection tools that rely on a single tell — such as the presence of navigator.webdriver, a missing Chrome runtime, or a patched eval function — will flag any browser that happens to show that trait.
Privacy extensions, enterprise security policies, anti-fingerprinting browsers, and even some VPNs modify the same APIs that automation tools touch. A corporate laptop with a hardened browser profile can look more "automated" than a well-configured Playwright script running in stealth mode. If your detection logic stops at the first anomaly, you will block paying customers.
How BotRefund's Multi-Signal Approach Reduces False Positives
BotRefund runs 106 independent checks per visit. The Playwright Init Scripts check is one of them. It looks for a mismatch between the browser's primary execution context and a clean iframe context — a pattern that automation tools often create when they patch APIs before the page loads. But that signal alone never produces a verdict.
Instead, each check contributes one objective fact. The system then cross-checks whether other signals tell the same story. Browser fingerprint consistency, network reputation, hardware concurrency, canvas rendering, mouse movement patterns, scroll behavior, and session timing all feed into an AI prediction model. The model weighs the complete pattern instead of trusting any raw rule. This corroboration-first design is how BotRefund reaches 99% accuracy across 2,500+ brand audits.
Step-by-Step Process to Minimize False Positives
- Collect independent signal types. Do not rely on browser APIs alone. Capture network attributes (IP reputation, ASN, proxy indicators), device signals (hardware concurrency, battery API, screen properties), and behavioral data (mouse tremor, scroll velocity, click timing, form interaction patterns).
- Keep each signal as evidence, not a decision. Store every check result with its raw value and confidence. A single failed check should never auto-block.
- Cross-check context. When one signal suggests automation, ask: do the other 20+ signals agree? A patched
navigator.webdriverplus humanlike mouse tremor, consistent device fingerprint, and residential IP is likely a privacy tool, not a bot. - Use a weighted model, not a rule list. Train or configure a model that learns which signal combinations actually predict automation in your traffic. Rules rot; models adapt.
- Set a decision threshold with a review queue. Sessions above the automation threshold get blocked or challenged. Sessions in a gray zone go to human review or a silent challenge (e.g., a proof-of-work CAPTCHA) that does not disrupt real users.
- Log and audit false positives. Every blocked session that complains or converts later is a training sample. Feed it back to the model weekly.
Key Signals That Distinguish Bots from Humans
No single signal is decisive, but some combinations are highly predictive. The table below summarizes the signal categories BotRefund uses and why each resists false positives when combined with others.
| Signal Category | What It Measures | Why It Resists False Positives |
|---|---|---|
| Browser API consistency | Checks for patched or missing APIs across contexts (e.g., Playwright Init Scripts check) | Privacy tools rarely patch every context identically; automation often does |
| Fingerprint integrity | Canvas, WebGL, audio, font, and hardware fingerprints | Real devices produce stable, self-consistent fingerprints; spoofed ones often conflict |
| Behavioral biometrics | Mouse tremor, scroll physics, click intervals, form typing rhythm | Humans have micro-variance; scripts are either too perfect or use simple randomization |
| Network reputation | IP type (residential, data center, VPN, proxy), ASN, geolocation consistency | Corporate VPNs are identifiable; residential proxies are rare for bots at scale |
| Device sensors | Battery status, accelerometer, gyroscope, touch support | Headless environments often lack sensors or return static values |
| Session logic | Navigation sequence, referrer chain, cookie persistence, storage behavior | Bots often skip steps or show impossible transitions |
Common Mistakes That Increase False Positives
- Blocking on
navigator.webdriveralone. This flag is set by any automation framework and also by some testing tools and accessibility software. - Treating headless Chrome as a bot signature. Many legitimate users run headless for PDF generation, screenshots, or CI pipelines on their own sites.
- Ignoring device context. A Linux desktop with no battery API and a generic fingerprint could be a server — or a developer's workstation.
- Using static blocklists. Data center IP lists catch corporate proxies, cloud CI runners, and legitimate monitoring services.
- No feedback loop. Without logging and reviewing false positives, your rules drift further from reality every month.
Verification: How to Test Your Detection Accuracy
Run a controlled experiment before you trust any detection system in production.
- Sample 10,000 recent sessions with known outcomes (converted, bounced, complained, chargeback).
- Run your detection logic offline. Label each session as bot, human, or uncertain.
- Measure precision (of sessions labeled bot, how many were truly automated?) and recall (of known bots, how many did you catch?).
- Focus on the false positive rate among converters and high-value users. A 1% false positive rate on checkout sessions is catastrophic; 5% on bounce traffic may be acceptable.
- Adjust thresholds until the cost of false positives (lost revenue, support tickets) balances the cost of false negatives (wasted ad spend, skewed analytics).
Key Facts
| Fact | Detail | Source |
|---|---|---|
| Independent checks per visit | 106 (Playwright Init Scripts is one) | S1 |
| Signal handling principle | Each signal is evidence, not a verdict; cross-checked against browser, network, device, and behavior data | S1 |
| Decision method | AI prediction model weighs complete pattern across all signals | S1 |
| Reported accuracy | 99% bot-or-human classification accuracy | S1, S2 |
| Total signals used | 110+ behavioral, browser, hardware, network, and attribution signals | S2 |
| Client audit volume | 2,500+ brands audited | S2 |
| Refund recovery rate | 83% of clients recover funds from Google and Meta | S2 |
| Report format | Refund-ready with click IDs, campaign details, timestamps, session recordings, signal-by-signal reasoning | S2 |
Limitations and When This Advice Does Not Apply
The multi-signal, AI-weighted approach requires enough traffic volume to train and validate the model. Sites with fewer than ~10,000 sessions per month may not have sufficient data for a custom model; they should start with a managed service that pools anonymized patterns across customers.
This guidance assumes you control the detection stack or can choose a vendor that exposes signal-level evidence. If you are locked into a WAF or CDN that only offers binary allow/block rules, you cannot implement cross-checking yourself — you must migrate the evidence layer to a specialized tool.
Advanced adversarial bots that invest in real device farms, residential proxy networks, and behavioral simulation can still evade detection. The goal is to raise the attacker's cost per successful visit, not to achieve perfect detection.
FAQ
Can I just block all headless browsers?
No. Legitimate users run headless Chrome for PDF generation, automated testing of their own sites, accessibility tooling, and server-side rendering previews. Blocking headless outright loses real conversions.
Does the Playwright Init Scripts check detect all Playwright bots?
No single check detects all Playwright automation. The Init Scripts check catches one evasion pattern — API patching before page load — but sophisticated scripts can avoid that specific mismatch. It works because it is one of 106 checks that together cover many evasion angles.
How often should I retrain the detection model?
Weekly retraining is a good baseline for high-volume sites. Lower-volume sites can retrain monthly if they feed false-positive and false-negative samples from review queues.
What if I don't have an ML team?
Use a vendor that provides the model as a service. BotRefund's prediction AI is included in the platform; you do not build or maintain it. You only review the evidence and decide whether to challenge or block.
Will this stop competitor click fraud on Google and Meta ads?
It provides the evidence layer. BotRefund's reports are formatted for Google and Meta invalid-traffic claims. Across 2,500+ audits, 83% of clients recovered funds. Detection alone does not guarantee refunds — you still need to file the claim with platform-ready evidence.
How do I know if my current detection has a false positive problem?
Check your support tickets for "I couldn't access your site" or "Your CAPTCHA is broken" from paying customers. Compare conversion rates before and after enabling strict bot rules. A drop in conversions without a drop in traffic often signals false positives.
What is the cost of a false positive vs. a false negative?
A false positive loses a customer and their lifetime value. A false negative wastes ad spend and poisons analytics. For most e-commerce sites, one lost high-value customer costs more than 100 bot clicks. Set your threshold accordingly.
Further reading and comparison sources
These external sources provide additional context for evaluating the topic. Their inclusion is not an endorsement.
How BotRefund can help
BotRefund runs 106 independent checks — including the Playwright Init Scripts check — on every visit. Each check produces one piece of evidence. Our AI model then weighs the full pattern across browser, network, device, and behavioral signals to classify the session as bot or human with 99% accuracy. You get refund-ready reports with click IDs, timestamps, session recordings, and signal-by-signal reasoning formatted for Google and Meta invalid-traffic claims. Across 2,500+ audits, 83% of clients recovered ad spend.
Limitation: the model needs sufficient traffic volume to maintain accuracy. Very low-traffic sites (under ~10,000 sessions/month) benefit more from the pooled intelligence of the managed service than from a custom threshold. You also need to place our JavaScript on your landing pages; server-only logs cannot provide the client-side behavioral signals that drive the 99% result.