OpenViking v0.5.1

中文

本版本包含 v0.5.0 之后的 151 个提交，重点是 Gateway 部署与接入、隐私字段管理、批量写入、多模态检索，以及 Session 和后台任务的可靠性。升级前请阅读下方兼容性与迁移：本次涉及 API 删除、写入语义和存储格式变化。

主要更新

Gateway 可以直接运行在 OpenViking Server 内。 开启 gateway.enabled 后，模型请求和管理接口统一位于 /gateway；独立部署仍受支持。Studio 新增快速配置、从 Server 导入模型设置和真实模型生成测试；上游支持精确 endpoint、自定义认证头及 URL 中的 key 占位符，覆盖 Azure 等接入方式。管理密钥与用户身份分开，配置了管理密钥后，修改上游和 context profile 必须提供它。请求准备、响应转发和历史响应查找也减少了 CPU 开销（#5960、#6100、#6101、#6107、#6118、#6026）。
隐私配置支持字段级引用和原子更新。 ov privacy reference 默认返回绑定 owner 的引用，resolve 才显式读取值；新增字段元数据、字段/目标删除、原子 PATCH，以及对已存内容的扫描和修复。扫描默认 dry-run，返回引用而不返回命中原值；多个字段登记了同一个值时报告歧义。修复单文件 no-split 导入的隐私处理（#6161、#6260）。
批量写入支持更多处理选项。 batch_write 新增整批 processing_mode，每个文件可带 tags 和 ACL；增加内容版本冲突检查及保留冲突文件的选项。变化文件统一维护语义与向量，vectors_only 不刷新父目录语义。正文已写入后，后台刷新入队失败会如实报错，正文不会回滚（#5897、#5750）。
检索支持文字与多图组合。 find、search 的 image_url 可传有序图片数组，Python SDK 支持对应输入；需要兼容的多模态 embedding。CLI 新增 ov search --mode context，可直接输出组装好的上下文。修复召回过的目录阻挡其下新文件的问题，缩短并限制 query planner 的处理，Hermes 排序移除字面词重叠加分（#6128、#5951、#5982、#6039、#6007、#5712）。
Session 和任务状态更准确。 统一 task 与 queue work 跟踪，保留任务 owner；SessionCommit 第二阶段失败会进入终态错误，归档完成标记在任务结果保存后写入；注册任务前释放归档锁。减少消息写入、Session 接口和删除时的重复读取，移除文件任务存储的统一 I/O 限流；资源 reason 触发的记忆抽取入队后即可返回受理状态（#6043、#6122、#6136、#5772、#6153、#6175、#6270、#6165、#6265、#6077、#6157）。
多实例部署能力增加。 Session idle auto-commit 可选 Redis 分布式调度；Account Store 新增 MySQL 和 Redis+MySQL Provider，默认仍为 file。顶层 service_name 统一部署标识及共享缓存命名；多写存储升级为 V2 分区日志、checkpoint 和后台追平模型（#6046、#5933、#5955、#6011、#6014、#5741）。
Compile 和 Studio。 VikingBot Compile 使用可规划的 Map/Shuffle/Reduce 流程，增加临时路由 embedding 接口，并限制失败重试和停滞的 review 循环。Studio 改进 workspace 设置、连接恢复、模型配置及私有地址连接测试；ROOT 可保存启动配置并重启受支持的单 worker 服务，启动失败时尝试恢复之前实际运行的配置（#5750、#5989、#6133、#6066、#6148、#6177、#5730）。
Agent 集成。 新增 Microsoft Agent Framework context provider；LangChain/LangGraph 修复 mapping、文本块、异步初始化和健康状态识别，框架包移到 integrations/frameworks/。共享插件新增 ov-resources skill、透传额外请求头和召回排除项；DSH 在压缩/启动时提交待处理消息；OpenCode 修复 session 归属和过期状态清理；Hermes Quick Local 支持 OpenViking 0.5（#5578、#6017、#5980、#6070、#5908、#5878、#6092、#5988、#5616、#5254、#6243、#5935、#5936、#6114）。
可靠性、安全与观测。 统一模型错误分类和重试边界，识别 Ark 超时，保留输入错误、鉴权失败、超时和网络错误的区别；修复长 embedding 输入截断、PDFium 并发渲染、SVG 依赖、加密快照恢复、原生向量存储错误透传及租户过滤。新增 Connector 导入计数和按来源区分的 ingestion token 指标（#6047、#5950、#6102、#6045、#5771、#5952、#6049、#6052、#5803、#5945、#6245）。
MCP 下载原文件。 新增 download(uri) 工具，返回一次性下载链接，默认约 10 分钟有效；Gateway 可代理文件链接。链接不需要 OpenViking API key，但部署在额外鉴权代理后时仍需该代理要求的请求头（#6119）。
提交与锁释放修复。 Session 在入队前创建 Task 并绑定 owner，修复 commit 返回 500；写入协调消息在生产方交锁及所属工作结束后释放锁，修复写入完成后移动、覆盖和删除仍返回 409。同时修复跨进程交接锁的本地状态清理，以及 HTTP 向量存储读取返回字段的问题（#6258、#6300、#6281、#6193）。
任务后端、解析与接入。 Task/Work 可选 MySQL 持久化后端（默认仍为 file）；parser 新增 parsers.download_temp_dir，允许底层文件系统支持的仅大小写不同文件名。Gateway 避免 DSH 和无工具辅助请求产生重复召回或新会话，LangChain 保留会话角色和工具历史；记忆更新重新读取失败时不再用旧快照覆盖（#6258、#6298、#6295、#6275、#6075）。Studio 完善界面本地化，Gateway 文档增加独立接入章节（#6292、#6287）。

用法示例

# 主包和 Python SDK
pip install --upgrade 'openviking==0.5.1' 'openviking-sdk==0.1.16'

# 独立 CLI
npm install -g @openviking/cli@0.5.1

# 组装上下文；完整结构使用 -o json
ov search '最近的设计决策' --mode context --events-time-decay-protection 7d

# 只获取隐私字段引用，不读取明文值
ov privacy reference credential github-prod api_key

# Python SDK 0.1.16；服务端需配置支持多模态输入的 embedding
result = client.find(
    query="比较这两张图的共同主题",
    image=["./first.png", "./second.png"],
    limit=5,
)

兼容性与迁移

Gateway 地址、管理权限和存储迁移（#5960、#6100）。 先停止旧进程并备份完整 storage_path。客户端地址从 http://host:1935/v1 改为 http://host:1933/gateway/v1；Anthropic 使用 http://host:1933/gateway。独立网关默认仍监听 1935，但同样使用 /gateway 前缀。管理 API 从 /api/v1/admin/gateway/* 改为 /gateway/admin/*。嵌入部署移除 sidecar 和旧端口配置，public_url 使用 Server 对外地址且不带 /gateway。独立命令的连接参数迁入 gateway.conf 或环境变量。
- 首次启动会用旧的 OPENVIKING_GATEWAY_ENCRYPTION_KEY 或 storage_path/encryption.key 解密旧数据，再按当前 OpenViking 加密配置写入。若希望继续加密，必须在首次启动前启用 OpenViking 加密；否则转换后的数据是明文。 缺少旧密钥时拒绝启动。
- OPENVIKING_GATEWAY_ADMIN_TOKEN 不会自动变为管理密钥。需要限制配置修改时显式设置 gateway.management_key 或 OPENVIKING_GATEWAY_MANAGEMENT_KEY；没有管理密钥时，同账号普通用户也可修改共享上游和 context profile。root/admin 身份不能绕过已设置的管理密钥。
- 旧上游 Base URL 会做一次迁移以保持原请求地址；新建/修改按新规则处理。更换上游 base_url 会清除已存的 API key 和请求头值，需要同次提交新地址对应的凭据。详见迁移指南。
移除独立 Session extract（#6131）。 POST /api/v1/sessions/{session_id}/extract 不再提供。改为 POST /api/v1/sessions/{session_id}/commit，再按返回的 task_id 查询后台结果。commit 会按策略归档消息，与旧的直接抽取不同；受理不代表完成。
移除等待专用 timeout（#6048）。 资源、Skill、内容写入、删除和 system wait 的服务端/SDK/MCP 等待参数，以及 CLI 对应命令的 --timeout 已移除。保留 wait；请求时限由客户端 HTTP timeout 配置控制。升级 SDK 后删除方法调用中的 timeout=；不要把 HTTP 超时当成后台任务取消。
write(mode="create") 不再保护已有文件（#5897）。 它现在是 replace 的兼容别名，会覆盖已有正文；batch write 的 create/replace/upsert 同样写成请求的最终正文。依赖旧 409 Conflict 的调用方必须调整，不能继续用 create 作为防覆盖保障。batch write 可按实际需求使用 expected_sha256 和 skip_conflicts；它们不等同于旧的仅创建语义。正文提交后遇到派生处理失败，重试 replace 类写入时使用最终正文，勿直接重放已成功的 append。
trusted 模式要求 root key（#5947、#5946）。 所有部署（包括 localhost）必须配置非空 server.root_api_key；数据请求带匹配的 API key 和 account/user 身份头。trusted 模式新建身份不再生成个人 key，重置个人 key 返回 FAILED_PRECONDITION。已有 key 不会自动删除，也不参与 trusted 认证。
共享缓存、Account Store 与多写存储（#6011、#6014、#5933、#5955、#5741）。 将缓存命名配置迁到顶层 service_name，同一部署各实例保持一致，共享 Redis 的不同部署使用不同值。切换 Account Store Provider 不会自动迁移身份数据；MySQL 表需按仓库 SQL 事先建立，修改 service_name 会选择另一组逻辑分区。
- V2 多写只直接读取 primary，primary 成功后异步持久化事件并同步 backup。旧 sync/ack/retry/operations/excludes/redirects 配置会警告后被忽略；旧持久化协议不自动转换。首次启用会导入 primary 现有文件，不删除 backup 额外旧文件。升级前完整备份，回滚时先恢复兼容备份，不能仅换回旧二进制。primary 写入与事件落盘之间崩溃可能遗漏复制事件，需从 primary 重建受影响 backup。
Session idle 策略与检索校验（#6046、#5926）。 新传入的 idle_timeout_seconds 必须为 0 或至少 300 秒；存量 1–299 秒值仍可读取。检索 limit 和显式 node_limit 必须为正数。
插件采集替换范围扩大（#5937）。 captureFilters 的 s 替换规则现在也处理 tool input 的字符串值和 tool output；d/k 仍只用文本决定保留或丢弃。检查已有规则是否会改写需保留的工具内容。插件需在各宿主中单独更新，升级 Python 包不会自动更新插件。
非空用户组可以直接删除（#6282）。 删除组会移除其全部成员关系，但保留用户和凭证，不重写已有资源 ACL。依赖旧“先清空组才能删除”行为的管理界面应更新提示。
可选 MySQL Task Store（#6258）。 启用前安装 openviking[mysql] 并执行 openviking/service/sql/task_store_init.sql；配置 storage.task_store.provider="mysql"。不要让 FileStore 和 MySQL 同时作为同一部署的写入端。

已知限制

远端索引中，新导入资源的 glob 结果可能稍后才可见；写入后的即时检索应考虑重试。

感谢本次 12 位首次贡献者，名单见下方。

English

This release contains 151 commits since v0.5.0, covering Gateway deployment, privacy fields, batch writes, multimodal retrieval, and Session/task reliability. Read the migration notes before upgrading: this release changes APIs, write semantics and storage formats.

Highlights

Embedded Gateway. Run Gateway inside OpenViking Server with gateway.enabled; model and management traffic use /gateway. Standalone deployment remains supported. Studio adds quick setup, model-setting import and generation-based upstream tests. Exact endpoints, custom authentication headers and API-key URL placeholders cover providers such as Azure. A separate management key controls upstream/profile edits. Request preparation, streaming relay and response lookup use less CPU (#5960, #6100, #6101, #6107, #6118, #6026).
Privacy fields. Owner-bound references, explicit value resolution, field metadata, atomic PATCH, field/target deletion and stored-content scanning are available through the API and CLI. Scans default to dry-run and do not expose matched plaintext; duplicate registered values are reported as ambiguous. No-split file imports receive the privacy handling fix (#6161, #6260).
Batch writes. Batch-wide processing_mode, per-file tags/ACLs, byte-revision conflict checks and conflict-preserving writes extend the content API. vectors_only maintains file vectors without regenerating parent summaries. If derived-work submission fails after content is written, the API reports the error and keeps the committed content (#5897, #5750).
Retrieval. find and search accept ordered image arrays alongside text with a compatible multimodal embedder. Python SDK supports these inputs; ov search --mode context returns assembled context. Recalling a directory no longer hides new files below it. Planner prompts and work are bounded, and Hermes no longer boosts results for literal query-token overlap (#6128, #5951, #5982, #6039, #6007, #5712).
Sessions and background work. Task and queue-work tracking converge, queue work retains its owner, SessionCommit phase-2 failures become terminal errors, and archive completion follows saved task results. Reduced repeated reads improve message writes, Session APIs and deletion. Resource-reason extraction returns once accepted into the queue (#6043, #6122, #6136, #5772, #6153, #6175, #6270, #6165, #6265, #6077, #6157).
Deployment options. Redis-backed idle auto-commit, MySQL and Redis+MySQL Account Store providers, a shared top-level service_name, and V2 multi-write partition logs/checkpoints extend multi-instance deployments. File Account Store remains the default (#6046, #5933, #5955, #6011, #6014, #5741).
Compile and Studio. VikingBot Compile gains a plannable Map/Shuffle/Reduce flow and transient routing embeddings, with bounded retries and stalled-review detection. Studio improves workspace setup, saved connections and model configuration. ROOT can save startup configuration and restart supported single-worker servers, with an attempt to recover the previously running configuration if startup fails (#5750, #5989, #6133, #6066, #6148, #6177, #5730).
Integrations. A Microsoft Agent Framework context provider joins the framework integrations, now under integrations/frameworks/. LangChain/LangGraph, DSH, OpenCode and Hermes receive capture, initialization and session-state fixes. Shared plugins add the ov-resources skill and consistently forward extra headers and recall exclusions. Hermes Quick Local supports OpenViking 0.5 (#5578, #6017, #5980, #6070, #5908, #5878, #6092, #5988, #5616, #5254, #6243, #5935, #5936, #6114).
Reliability and observability. Model error/retry classification preserves input, authentication, timeout and network distinctions. Fixes cover long embedding inputs, PDFium rendering, SVG dependencies, encrypted snapshot restore, native vector-store errors and tenant filtering. Connector-import counts and ingestion-token metrics identify source types (#6047, #5950, #6102, #6045, #5771, #5952, #6049, #6052, #5803, #5945, #6245).
MCP downloads. download(uri) returns a single-use file URL, normally valid for ten minutes, with optional Gateway proxying. OpenViking needs no API key on that URL; an outer authenticated proxy may still require its own headers (#6119).
Commit and lock fixes. Sessions create their Task and bind its owner before enqueueing, fixing commit HTTP 500 errors. Write coordination waits for producer handoff and owned work before releasing locks, fixing subsequent move, replace and delete operations returning 409 after a completed write. Cross-process handoff cleanup and HTTP vector-store response parsing are also corrected (#6258, #6300, #6281, #6193).
Task storage, parsing and integrations. MySQL is an optional Task/Work backend; file remains the default. parsers.download_temp_dir controls parser temporary storage, and case-only filenames follow the underlying filesystem. Gateway avoids duplicate recall/new sessions for DSH and tool-less auxiliary requests; LangChain preserves roles and tool history. Memory updates no longer overwrite from a stale snapshot when the fresh read fails (#6258, #6298, #6295, #6275, #6075). Studio localization and dedicated Gateway documentation are expanded (#6292, #6287).

Upgrade and migration

pip install --upgrade 'openviking==0.5.1' 'openviking-sdk==0.1.16'
npm install -g @openviking/cli@0.5.1
ov search 'recent design decisions' --mode context --events-time-decay-protection 7d
ov privacy reference credential github-prod api_key

Gateway migration (#5960, #6100). Stop the old process and back up its complete storage_path. Embedded client URLs change from http://host:1935/v1 to http://host:1933/gateway/v1; Anthropic uses http://host:1933/gateway. Standalone still defaults to port 1935 but also requires /gateway. Management moves from /api/v1/admin/gateway/* to /gateway/admin/*. Remove the old sidecar/port configuration, set public_url to the server origin without /gateway, and move standalone command-line connection settings into gateway.conf or environment variables.
- On first startup, the old OPENVIKING_GATEWAY_ENCRYPTION_KEY or storage_path/encryption.key decrypts existing data before it is rewritten under the current OpenViking encryption configuration. Enable OpenViking encryption before this startup to keep data encrypted; otherwise the converted data is plaintext. Missing legacy keys prevent startup.
- The old admin token is not a management key. Set gateway.management_key or OPENVIKING_GATEWAY_MANAGEMENT_KEY to restrict configuration edits. Without one, ordinary users can edit shared upstreams/profiles in their account. Admin/root identity does not bypass a configured management key.
- Stored upstream URLs receive a one-time migration to preserve request destinations. Changing base_url clears stored provider credentials and header values; send credentials for the new destination in the same update. See the migration guide.
Session extract removed (#6131). Replace POST /api/v1/sessions/{session_id}/extract with commit, then inspect its returned task_id. Commit archives messages under the retention policy; acceptance is not extraction completion.
Wait-specific timeout removed (#6048). Remove method-level timeout= and the corresponding CLI --timeout from resource, skill, content, deletion and system-wait operations. wait remains; configure the client's HTTP timeout instead. An HTTP timeout does not cancel background work.
Create now overwrites (#5897). write(mode="create") aliases replace and no longer returns the old existence conflict. Batch create/replace/upsert also write the requested final content. Callers relying on create-only protection must change; byte-revision checks and skip_conflicts are not equivalent to create-only semantics. Retry final-content replacement after derived-work failures; do not blindly replay an append that already succeeded.
Trusted authentication requires a root key (#5947, #5946). Every deployment, including localhost, needs a nonblank server.root_api_key. Data requests carry the matching API key and account/user headers. New trusted identities have no personal key; resetting a personal key returns FAILED_PRECONDITION. Existing keys are not deleted and do not authenticate trusted requests.
Shared cache, Account Store and multi-write (#6011, #6014, #5933, #5955, #5741). Move cache naming to top-level service_name; instances in one deployment share it, while distinct deployments on one Redis use different names. Account Store changes do not migrate identities automatically. Create MySQL tables from the repository SQL before use; changing service_name selects another logical partition.
- V2 multi-write reads only the primary and persists events/replicates backups asynchronously after primary success. Legacy sync/ack/retry/operations/excludes/redirects settings warn and are ignored; old persisted protocols are not converted. First enable imports primary files without deleting extra backup files. Back up before upgrading and restore a compatible backup before rolling back the binary. A crash between primary write and event persistence can lose a replication event; rebuild an affected backup from the primary.
Validation changes (#6046, #5926). New positive idle_timeout_seconds values must be at least 300 seconds (0 disables it); existing smaller values remain readable. Retrieval limit and explicitly supplied node_limit must be positive.
Capture filtering (#5937). Substitution rules now rewrite strings in tool inputs and tool outputs. Drop/keep rules still examine text only. Review existing substitution rules and update plugins separately in each host; upgrading the Python package does not update installed plugins.
Nonempty groups can be deleted (#6282). Deletion removes all memberships while retaining users and credentials; it does not rewrite resource ACLs. Update management UIs that assumed a group must first be emptied.
Optional MySQL Task Store (#6258). Install openviking[mysql], apply openviking/service/sql/task_store_init.sql, and set storage.task_store.provider="mysql" before enabling it. Do not use FileStore and MySQL as concurrent writers for one deployment.

Known limitation

With a remote index, newly imported resources may become visible to glob after the import response. Allow for retries when querying immediately after writes.

Thank you to the 12 first-time contributors below.

First-time Contributors

Full Changelog / 完整变更

volcengine/OpenViking v0.5.1 on GitHub