nicobailon/pi-web-access v0.40.0

on GitHub

You can now keep page fetches on the site you asked for. Turn on the new same-origin redirect policy, for every fetch or just one call, and any redirect to a different site is refused before that site is ever contacted. This makes it practical to give a restricted research agent public web access without letting a redirect take it somewhere else. This release also stops Gemini gateway users' video uploads from going around their gateway, makes answer mode work again on videos that come with a thumbnail, and keeps emoji intact in excerpts and paged content.

Highlights

• Keep page fetches on the site you asked for: an opt-in policy refuses any redirect to another origin before that site is contacted.
• With a Gemini gateway configured, video uploads now go through the gateway instead of straight to Google with your gateway key.
• Asking a question about a YouTube or local video works again when a thumbnail comes back with the transcript.
• Emoji and other special characters no longer get cut in half at the edge of an excerpt or a page.

Changelog

Highlights

• Keep page fetches on the site you asked for: an opt-in policy refuses any redirect to another origin before that site is contacted.
• With a Gemini gateway configured, video uploads now go through the gateway instead of straight to Google with your gateway key.
• Asking a question about a YouTube or local video works again when a thumbnail comes back with the transcript.
• Emoji and other special characters no longer get cut in half at the edge of an excerpt or a page.

Added

• An opt-in same-origin redirect policy for page fetches. Turn it on for every fetch with fetchContent.redirects: "same-origin" in config, or for one call by passing redirects: "same-origin" to fetch_content. Every redirect must then stay on the requested URL's scheme, hostname and port. A redirect that leaves is refused before the new site is contacted, with no page content and no fallback to another provider. While the policy is on, extraction paths that fetch pages themselves (Jina Reader, hosted providers, Gemini, and the GitHub and YouTube integrations) are turned off. Thanks to @jordan-droverinsights for issue #560.

Fixed

• With geminiBaseUrl pointing at a gateway, video uploads to the Gemini Files API now go through the gateway. Before, the file was sent to generativelanguage.googleapis.com directly, along with the gateway key. Thanks to @goodman-b for PR #561.
• fetch_content answer mode now answers from a YouTube or local-video transcript that comes with a thumbnail, instead of failing with "Page answer requires textual fetched content". Requests for several video frames are refused in answer mode instead of being answered from a placeholder line. Thanks to @goodman-b for PR #562.
• get_search_content findText excerpts no longer end in half an emoji. A character split at the excerpt edge shows as �, so the text stays valid. Thanks to @yazanabuashour for PR #563.
• Reading stored content page by page with fetch_content and get_search_content no longer splits an emoji across two pages. The whole character starts the next page, and the next-page offset points to it (#564).