Lightpanda Browser Torture Test Results
March 22, 2026
Lightpanda Browser Torture Test Results
2026-03-17
22 sites tested across financial data, news, tech docs, forums, SPAs, and search engines.
Results Summary
| # | Site | Category | Result | Lines | Notes |
|---|---|---|---|---|---|
| 1 | Reuters /markets/ | Financial news | FAIL | 0 | Empty response |
| 2 | Bloomberg /markets | Financial news | BLOCKED | 19 | Bot detection ("unusual activity") |
| 3 | CNBC /economy/ | Financial news | PASS | 365 | Full content including breaking news banner |
| 4 | MarketWatch | Financial news | FAIL | 0 | Empty response |
| 5 | Wikipedia | Reference | PASS | 2614 | Full article content |
| 6 | Hacker News | Forum/news | PASS | 168 | All stories with titles, points, comments |
| 7 | Reddit (old.reddit) | Forum | PASS | 615 | Full subreddit listing |
| 8 | GitHub README | Code hosting | PASS | 614 | Full rendered README |
| 9 | StackOverflow | Q&A | PASS | 1817 | Full page including questions |
| 10 | ArXiv | Academic | PASS | 154 | Paper metadata and abstract |
| 11 | NPM | Package registry | BLOCKED | 21 | Cloudflare challenge |
| 12 | CME FedWatch | Financial data | PARTIAL | 185 | Navigation + header, but tool data likely JS-rendered |
| 13 | Yahoo Finance /quote/ | Financial data | PARTIAL | 1640 | Page loads but redirects to lookup (may need cookies) |
| 14 | Substack | Newsletter | PASS | 99 | Newsletter listing with titles |
| 15 | Google Search | Search engine | BLOCKED | 16 | CAPTCHA ("unusual traffic") |
| 16 | SEC EDGAR | Government | BLOCKED | 30 | Requires user-agent declaration |
| 17 | TradingView | Financial SPA | PASS | 318 | Navigation + stock categories rendered |
| 18 | Medium | Blog platform | BLOCKED | 21 | Cloudflare challenge |
| 19 | FRED (St. Louis Fed) | Economic data | FAIL | 4 | HTTP/2 stream error |
| 20 | Finviz | Stock screener | PASS | 101 | Map page with nav (visual content can't render) |
| 21 | Zig lang site | Tech docs | PASS | 129 | Full homepage content |
| 22 | Unusual Whales | Options flow | PASS | 220 | Navigation + mentions API/MCP access |
Score
- PASS (usable content): 13/22 (59%)
- PARTIAL (some content): 2/22 (9%)
- BLOCKED (bot detection): 5/22 (23%)
- FAIL (empty/error): 3/22 (14%)
What Works Well
- DuckDuckGo search (primary search tool)
- News/blog sites without aggressive bot protection (CNBC, E8 Markets, dev.to, Substack)
- Code platforms (GitHub, StackOverflow)
- Forums (Hacker News, old.reddit.com)
- Reference sites (Wikipedia, ArXiv, ziglang.org)
- Financial aggregators (TradingView navigation, Finviz navigation)
What Doesn't Work
- Cloudflare-protected sites: Medium, NPM, Bloomberg (bot challenge pages)
- Google Search: CAPTCHA wall
- SEC EDGAR: Requires custom user-agent header
- Reuters, MarketWatch: Empty responses (likely aggressive bot detection)
- FRED: HTTP/2 protocol issue
Recommendations
- Primary search: DuckDuckGo via Lightpanda (works reliably)
- Financial news: CNBC works; for Reuters/Bloomberg, fall back to web_fetch
- SEC EDGAR: Could work with custom user-agent flag (--user_agent_suffix)
- For Cloudflare-blocked sites: web_fetch (readability extractor) often succeeds where Lightpanda fails, and vice versa. Use both as complementary tools.
- Google: Don't bother; DuckDuckGo is the reliable search path