Skip to content
Open
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 43 additions & 8 deletions browsers/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,48 @@ If you're experiencing slower-than-expected browser creation times, review your
- Browsers persist independently of CDP. Depending on your timeout configuration, it will continue running even if the CDP connection closes. You can reconnect to the same `cdp_ws_url` if you're unexpectedly disconnected.
- We recommend implementing reconnect logic, as network interruptions or lifecycle events can cause CDP sessions to close. Detect disconnects and automatically re-establish a CDP connection when this occurs.

## Unsupported Websites
## Site difficulty index

There are some websites that are not supported by Kernel browsers due to their restrictions around automation and associated bot detection. These include:
Block rates for unauthenticated homepage visits from a stealth Kernel browser through a US residential proxy. Sites are sorted by observed difficulty. See [methodology](#methodology) for the test protocol and important caveats — in particular, these numbers reflect a single landing-page request, not login flows or at-scale scraping.

- LinkedIn
- Facebook
- Instagram
- X (Twitter)
- Amazon
- Reddit
This list is incomplete and will grow as we run more tests. Last measured 2026-05-11.

### Hard — significant friction observed

| Site | Block rate | Detection vendor |
|------|-----------:|------------------|
| Yelp | 100% (5/5 blocked) | DataDome |
| Glassdoor | 100% (5/5 challenged) | Cloudflare |
| Indeed | 40% (2/5 challenged) | Cloudflare + Imperva |
| TripAdvisor | 40% (2/5 blocked) | DataDome |

### Light — partial friction observed

| Site | Block rate | Detection vendor |
|------|-----------:|------------------|
| Yellow Pages | 20% (1/5 blocked) | Cloudflare |
| Zillow | 20% (1/5 challenged) | PerimeterX |

### Clear — no blocks observed at this layer

All five sessions returned a usable page. These sites still deploy bot detection — login flows, deep navigation, and high-volume scraping behave very differently — but the public landing page renders cleanly.

Airbnb, Amazon, Booking.com, Cars.com, Crunchbase, eBay, Etsy, Facebook, Facebook Marketplace, GitHub, Google Maps, Google Search, IMDb, Instagram, LinkedIn, Medium, Pinterest, Reddit, Shopify storefronts (Gymshark), Target, TikTok, Walmart, Wikipedia, X (Twitter), Yahoo Finance, YouTube.

### Methodology

For each site, we open 5 concurrent stealth Kernel browser sessions through a US residential proxy and navigate to the public landing URL (e.g. `https://www.linkedin.com`). Each session uses a different exit IP. We then classify the response:

- **Success** — the expected page rendered, no detection signals tripped.
- **Challenged** — a visible CAPTCHA or "checking your browser" interstitial that requires action to proceed (e.g. Cloudflare Turnstile, hCaptcha, DataDome captcha).
- **Blocked** — a hard block page, 403/429 status, or vendor-branded "Access Denied" response.

Block rate combines blocked + challenged. Vendor labels reflect the bot-detection product whose signatures we matched.

<Info>
These results are a floor, not a ceiling. They tell you what the *easiest* automation case — one anonymous homepage visit — looks like. A site that scores 0% here can still be very hard once you add login, repeated requests from the same IP, deep navigation, or large concurrency. We plan to publish login-flow and at-scale benchmarks separately.
</Info>

<Info>
Hitting friction on a site that scored clean here? Check your [proxy type](/browsers/bot-detection/overview#choosing-a-proxy-type) and confirm you're not running headless — those are the two most common causes of unexpected detection.
</Info>
Loading