nicobailon/pi-web-access v0.46.0

on GitHub

pi-web-access 0.46.0 gets more pages past bot blockers and adds three new places to search. Page fetches can now connect the way Chrome does, and pages that still refuse can be read through Exa or Tabstack. You can search through Cloudflare, Google via SearchApi, or Tabstack, which answers with citations instead of a list of links; none of them run or cost anything unless you pick them. Over MCP, the tools now take about half as much of every request to the model, and PDFs, proxies and search errors behave the way you'd expect.

Highlights

• Read more pages that block bots. In a test of 68 mostly bot-protected pages, readable pages went from 18 to about 31 when fetches connect like Chrome. Blocked pages can also be handed to Exa or Tabstack.
• Three new search options, each used only when you pick it: Cloudflare, Google through SearchApi, and Tabstack, which returns one cited answer instead of a list of links.
• Over MCP, the web tools take half as much of every request to the model (about 1.75k → 0.9k tokens).
• Reading a PDF now gives you its text instead of a note saying where the file was saved.
• A proxy you set keeps working after Pi reloads, and a rate-limited free search now says so instead of claiming no search provider is set up.

Full changelog

Highlights

• Read more pages that block bots. Page fetches can connect the way Chrome does: in a test of 68 mostly bot-protected pages, readable pages went from 18 to about 31. Blocked pages can also be handed to Exa or Tabstack.
• Three new search options, each used only when you pick it: Cloudflare, Google through SearchApi, and Tabstack, which returns one cited answer instead of a list of links.
• Over MCP, the web tools take half as much of every request to the model (about 1.75k → 0.9k tokens).
• Reading a PDF now gives you its text instead of a note saying where the file was saved.
• A proxy you set keeps working after Pi reloads, and a rate-limited free search now says so instead of claiming no search provider is set up.

Added

• Search through the Cloudflare Web Search API with provider: "cloudflare". It runs on Ceramic, Exa, or Linkup (cloudflareSearchProvider, default Ceramic) through your AI Gateway, billed to AI Gateway credits or a provider key stored on the gateway. It reads the CLOUDFLARE_API_KEY and CLOUDFLARE_ACCOUNT_ID environment variables, or cloudflareApiKey and cloudflareAccountId in web-search.json; a Cloudflare key saved with Pi's /login is not used. It runs only when you name it, so auto and all never spend on it. Thanks to @kestermcullough for PR #639.
• fetchContent.curl in web-search.json sends the extension's own page fetch through a curl command instead of Node's fetch. Point it at curl-impersonate, which connects the way Chrome does, and many sites that answer Node's fetch with HTTP 403 send the page instead. In a test of 68 pages, most of them behind bot protection, the number the extension could read went from 18 to about 31. Through curl, the extension leaves out its default User-Agent so the command's browser User-Agent is used. Without the setting, Node's fetch is used as before. Thanks to @kestermcullough for PR #640.
• fetch_content can read a page through Exa when a site refuses the direct fetch, using the Exa key you already set for search (exaApiKey or EXA_API_KEY). Add "exa" to fetchRouting.providers and set fetchRouting.allowRemoteHostedProviders to true; the page address is sent to Exa, and Exa bills each page it reads. A page Exa can't read comes back as an Exa error, not an empty page. Thanks to @ccc592 for issue #633.
• Search Google through SearchApi with provider: "searchapi", configured with searchapiApiKey or SEARCHAPI_API_KEY (and an optional searchapiBaseUrl gateway). It runs only when you name it or list it in searchRouting; auto and provider: "all" never choose it. Domain filters become site: clauses and recency maps to time_period. Each search returns at most 10 results, one Google results page. Thanks to @gabsferreira for PR #645.
• Search with Tabstack Research, which returns a cited answer with its sources, using provider: "tabstack" and tabstackApiKey (or TABSTACK_API_KEY). It runs only when you name it; auto and all never choose it. Research runs in fast mode unless tabstackResearchMode is "balanced". Tabstack Extract can also fetch blocked or JavaScript-heavy pages once you add "tabstack" to fetchRouting.providers and turn on fetchRouting.allowRemoteHostedProviders. Thanks to @tessak22 for PR #641.

Changed

• The MCP server's tool definitions shrink from about 7,000 to about 3,500 characters (about 1,750 to 900 tokens) on every request to the model. The provider parameter no longer spells out every provider name; an unknown name is still rejected before any request, and the error lists the valid names. Thanks to @mikeyobrien for PR #644.

Fixed

• fetch_content recognizes mixed-case Content-Type headers, so HTML still goes through readable extraction and Cloudflare challenge detection, and PDF and binary types are handled like lowercase types. Thanks to @Nyarlathoteppppp for PR #632.
• Reading a PDF with fetch_content now returns the PDF's text, with the path of the saved Markdown file above it. Before, the result was only a "saved to" notice, and get_search_content returned that same notice, so a model that cannot open local files had no way to read the PDF. Thanks to @ccc592 for issue #634.
• Searching without an Exa key no longer says that no search provider is set up when Exa's free search answered but couldn't be used. When the free Exa search is rate-limited, the error now says so and tells you to add an Exa API key; when Exa sends back something without search results, the error shows what it sent. Search routing treats these like any other rate limit or unreadable reply, so fallbackOn can move on to the next provider. A search that finds nothing is now an empty result, not an error. Thanks to @vishwajeet7381 for issue #638.
• When no summary model can be used, the reason now lists every model that was tried and why each one failed, instead of naming only the last one. Before, a session with none of the default summary models set up showed one model as "not enabled", which pointed at the wrong fix. Thanks to @vishwajeet7381 for issue #642.
• A proxy you set could be skipped after Pi reloaded its extensions, so web requests went out directly instead of through it. Requests now go through the proxy you chose. Thanks to @xiangsheng for issue #643.
• source_check marks OpenAI and xAI citation snippets as [provider summary, not page text]. Those snippets are cut from the search provider's own answer, not from the cited page, and were shown as if they were quotes from it. Thanks to @panchogn for PR #629.
• fetch_content answer mode asks models that can't turn thinking off, such as Claude Opus 5.5 and Sonnet 5.5, for their lowest thinking level instead of leaving it to the provider. Models that can turn thinking off still run without it. Thanks to @panchogn for PR #629.
• Requests sent through a proxy no longer follow redirects inside curl when ~/.curlrc turns on location, so the extension checks each redirect again. A curl download that fails partway is now an error instead of a short response. A URL with [ ] or { } in it, such as ?ids[]=1, is now sent as written instead of failing or being turned into several requests.