github PrimeIntellect-ai/verifiers v0.3.0
on GitHub

5 hours ago

Highlights

  • Multi-agent environments. An Agent you run like a function and an Env that writes the control flow between agents as plain Python — one Episode per task, with bundled single-agent, best-of-n, and agentic-judge envs (#1939) and turn-level interleaving between agents (#2049).
  • ACP integration. Supported harnesses (Codex, Claude Code, ...) run through ACP adapters with resumable sessions (#2257), and stateful harnesses keep one live process across turns over runtime processes (#2249).
  • Client-side tasksets. The client owns the taskset and ships each task's data to a stateless env server; resume identity is the task's content, so a changed task re-runs instead of silently matching (#2039).
  • Improved performance. The interception server owns and multiplexes the live clients (#2218, #2279), and the train client's tokenizers come from a process-wide elastic renderer pool (#2218).
  • New harnesses and integrations. browser_use for CDP browser control (#2192), Hermes Agent (#2173), OpenClaw (#2174), NeMo Gym V1 (#2212), pluggable skills in harnesses (#2102), and execution-time network isolation with egress allow/block lists on Docker and Prime (#2024, #2115).

Breaking

Configs

  • Every run axis lives under [env]. --taskset.* / --harness.* become --env.taskset.* and per-agent --env.<agent>.* (--env.agent.harness.id on a plain run); per-run caps and retries move onto the agent (#1939).
  • Runtime is an agent field. [env.agent.harness.runtime] moves to [env.agent.runtime] and HarnessConfig.runtime is gone — where a rollout provisions its box is the seat's decision, not the harness's (#2106).
  • env.max_concurrent removed; serving is its own block. serve.max_concurrent bounds a worker's episodes and env.max_concurrent_agents (default 1) bounds one episode's agents; --pool.* → --serve.pool.*, --address → --serve.address, --id/--args/--extra-env-kwargs → --legacy.*; EnvServerConfig is gone (#2157).
  • ServingConfig renamed to ServeConfig (same fields: pool, address, max_concurrent) (#2237).
  • TrainClientConfig.pool_size replaced by multiplex, with inverted meaning. pool_size was "how many renderers"; multiplex is "concurrent rollouts per renderer" (default 256) — drop the old value to take the default, or set it to concurrency / desired_renderers (#2218).
  • share_runtime removed from the agentic judge. --env.id agentic-judge is isolated judging; same-box grading is the explicit --env.id shared-agentic-judge, and the judge always follows the solver's runtime policy (#2234).

Artifacts (trace, episode)

  • traces.jsonl holds episodes. Each line is one Episode — {id, env, ok, errors, traces} — written whole after the env-rollout finishes; traces gain ok and agent.name/agent.trainable (#1939).
  • Trace API renames. Trace.stamp → record_run, capture_error → record_error, error → last_error; the agent_name/trainable/runtime passthroughs are removed — read trace.agent.name/.trainable/.runtime (#2172).
  • The agent is stamped on the trace. trace.agent is {config, runtime, name, trainable}: agent.model/agent.sampling/agent.harness become agent.config.*, and the top-level trace.runtime field moves to agent.runtime; consumers without the run's packages read records via WireTrace/WireEpisode (#2106).
  • Timing.generation → Timing.agent (GenerationSpan → AgentSpan, Trace.split_generation → split_agent_time) (#2187).
  • Episode.env is a typed EnvInfo (persisted as {"id": ...} — read episode.env.id), and Episode.error → Episode.last_error (#2187).
  • vf.StrictBaseModel is gone. The whole record tree parses non-strict — unknown fields are ignored instead of rejected, so additive schema evolution no longer breaks older readers (#2187).

Authoring surface

  • Deprecated group rewards. @group_reward, run_group, and group resume are removed; -r n means n independent episodes, and sibling comparison lives in an env's finalize() (e.g. best-of-n) (#1939).
  • Tool declaration is explicit. The Task.tools/Taskset.tools classvars, server_config, and resolve_server_config are removed — override toolsets(cls, config) (a classmethod) and construct each server with exactly the config it needs (#2199).
  • Agents no longer take a live client. Agent(config, client=...)/make_agent(client=...) are removed — put the endpoint on AgentConfig.client. Harness hooks see a config-valued ctx.client (ClientConfig, not a live Client); the rollout's live client is at RolloutSession.client (#2218). Agent.run(shared_tools=...) is now Agent.run(tools=...) (#2094).
  • Import moves. Config classes live in verifiers.v1.configs.* (#2106), and scoring/push/artifacts/decorators/loaders/retries move under verifiers.v1.utils.* (scoring → utils.score, push → utils.platform) (#2204); RolloutRun is renamed Rollout and takes bundled RolloutTimeouts (#2204, #2205); TaskTimeout.harness → TaskTimeout.agent (#2172). Public vf.* re-exports are unchanged.

CLI

  • The serve CLI is removed. Host an env server programmatically via serve_env, or through a trainer (prime-rl's uv run env) (#2237).

Changelog

New Contributors

Full Changelog: untagged-ea23470cc3f3ddbbca02...v0.3.0

Check out latest releases or
releases around PrimeIntellect-ai/verifiers v0.3.0

Don't miss a new verifiers release

NewReleases is sending notifications on new releases.

Get notifications