nicobailon/pi-web-access v0.39.0

on GitHub

You can now point PDF fetches at a document parser you run yourself, on your own machine or LAN, with no API key or per-page cost. A failed request falls back to the other engines. Search summaries are more useful: the summary model now reads each source's snippet, so providers that only return links still get a real summary. Answer mode with several URLs now shows every page's answer instead of a list of character counts. This release also makes search less fragile when a provider stalls, comes back empty, or hits a bot check.

Highlights

• Convert PDFs with a document parser you run yourself, with no API key and no per-page cost.
• Search summaries now read each source's snippet, so providers that return only links still get a real summary.
• Answer mode with several URLs shows every page's answer, and stored pages are easier to find again.
• Search is less fragile: stalled Perplexity requests time out, empty results move auto on to the next provider, and DuckDuckGo bot checks are reported as such.
• GitHub pull request and issue fetches no longer repeat long text, and Brave results no longer show leftover HTML.

Full changelog

Highlights

• Convert PDFs with a document parser you run yourself, with no API key and no per-page cost.
• Search summaries now read each source's snippet, so providers that return only links still get a real summary.
• Answer mode with several URLs shows every page's answer, and stored pages are easier to find again.
• Search is less fragile: stalled Perplexity requests time out, empty results move auto on to the next provider, and DuckDuckGo bot checks are reported as such.
• GitHub pull request and issue fetches no longer repeat long text, and Brave results no longer show leftover HTML.

Added

• PDFs can be converted by a document parser you run yourself. Set pdf.provider: "endpoint" and pdf.endpointUrl, and fetch_content sends the PDF to that service and uses the Markdown it returns, with no API key or per-page cost. In auto, a configured endpoint is tried before Datalab, Gemini and the built-in parser, and a failed request falls back to them. The URL passes the same SSRF checks as other fetches, so a LAN host needs its range in ssrf.allowRanges. Thanks to @goodman-b for PR #553.

Changed

• Search summaries now give the summary model each source's snippet along with its title and URL, so providers that return ranked results without an answer still get a summary of what the pages say. Snippets are capped per source and per query (600 and 12,000 characters, or 200 and 3,000 when the provider already answered) to stay within the summary deadline. Thanks to @axelbaumlisto for PR #547.

Fixed

• Single-URL fetches and finished search summaries now end with the stored responseId and a get_search_content(…) call the model can run, so stored content stays reachable when the host hides tool details. The call uses the configured retrieval tool name, or says retrieval is not registered when it is turned off. Thanks to @mothieras for issue #548 and PR #551.
• Truncated fetch_content answers now say the stored content is the original source, not the rest of the answer, and offer to read it from offset zero instead of an answer offset. Thanks to @mothieras for issue #550 and PR #552.
• When a model sends queries, provider or domainFilter as a string starting with [ that isn't valid JSON, web_search now errors that the field must be an array of strings instead of searching the raw string. Thanks to @tschnorpfeil for issue #542 and @MatrixNeoKozak for PR #543.
• auto search now moves on when a provider answers successfully with no results and no answer, and returns the first empty response only when every provider is empty. Thanks to @axelbaumlisto for PR #546.
• Brave search results no longer show raw HTML such as <strong> tags or — entities in titles and snippets. Thanks to @axelbaumlisto for PR #545.
• A Perplexity request that stalls now gives up after 30 seconds instead of holding the search for the whole turn. The timeout is reported as a provider error, so auto search and configured routing move on to the next provider. Thanks to @axelbaumlisto for PR #544.
• DuckDuckGo search now tells a bot check apart from an empty search. A bot check (HTTP 202 or a challenge page) fails with an error saying DuckDuckGo blocked the request, and configured searchRouting treats it as quota. A search that found nothing returns no results instead of failing with "no parseable results". Thanks to @tschnorpfeil for issue #541.
• fetch_content with mode: "raw" on a PDF now points at the default Markdown mode or mode: "answer" with a prompt, instead of only reporting an unsupported content type. Thanks to @goodman-b for PR #555, split out of PR #553.
• fetch_content on a GitHub pull request or issue now stores each fetched body, comment, review comment, file, commit, check and review once: the inline view shows the start and the end of the document holds the rest, so the retrieval hint finds what it points at. Data GitHub was never asked for points at gh instead, and hints use the retrieval tool's configured name, or only gh when retrieval is off. Thanks to @mothieras for issue #549.
• fetch_content answer mode with several URLs now shows each URL's answer, instead of only a list of URLs with character counts. The answers share the output limit of a single-URL answer and say when one is cut short, and the retrieval hint points at the original pages, as for a single URL. See issue #556.