Release candidate. Docker :latest and a plain pip install stay on 0.9.9 until 0.10.0 ships, so nothing moves unless you opt in. 0.10.0-rc.4 is 0.10.0-rc.3 plus the three changes below. See the 0.10.0-rc.3 notes for the scope-clone repack, the 0.10.0-rc.2 notes for the security fixes and the 0.10.0-rc.1 notes for the move to pydantic v2 and Starlette 1.x.
All three new settings are off by default. With the defaults, the behaviour changes are a repoint that always notifies (#988) and the log changes (#973).
A repoint always notifies its clients (#988)
This affects opal-server in scopes mode (OPAL_SCOPES=1) only.
- On by default. When
PUT /scopesmoves a scope to another repository or branch, or the previous record can't be read, the sync forces a fetch and publishes an update-all notification to the scope's clients, even when the new head matches the clone's scope ref. Before, moving a scope back to a repository it used before could notify nobody. - A server without this change ignores the new webhook keys, so during a rolling upgrade repoint a scope only once.
- Opt-in: wait for a missing branch or clone.
OPAL_SCOPES_POLICY_BRANCH_REFRESH_INTERVAL(default0, off). When set,GET /scopes/:scope_id/policyon a server whose clone lacks the scope's branch, or has no clone yet, asks every server to sync the scope and holds the request in the existing clone wait, instead of answering 409 (or 503) at once. Each worker asks at most once per interval per scope and branch. It needsOPAL_SCOPES_POLICY_CLONE_WAIT_SECONDS > 0. - Opt-in: prune on fetch.
OPAL_SCOPES_GIT_FETCH_PRUNE(defaultfalse) prunes remote-tracking refs on each scope fetch, so a branch deleted on the remote becomes a miss instead of being served at its last head. It adds seconds per fetch on clones with tens of thousands of branches. - Metrics:
opal_server.scopes.policy_clone_waitgains afortag (cloneorbranch) and the outcomeunrequested. New counteropal_server.scopes.policy_branch_refresh.
Re-clone a clone left hung by a restart (#989, opt-in)
This affects opal-server in scopes mode only, and does nothing unless OPAL_SCOPES_GIT_RECLONE_HUNG_ON_START=true.
- A git fetch or clone that is still running past
OPAL_SCOPES_GIT_FETCH_TIMEOUTcan't be cancelled in place. If the server process ends while one is running, for example on a container restart that keeps the clones' volume, the next start finds a clone that may not sync again. - With the setting on, each such operation leaves a marker file while it runs. At startup, the gunicorn master's preload deletes the clones that still have a marker, so they are cloned again.
- Guard: if the marked clones span at least two of the repositories the current scopes use, and at least half of them, the server keeps every clone instead, since that looks like a git provider outage rather than one bad clone.
- Before turning it on: a deleted clone is cloned again in full, and until that lands the server has no policy for the scopes on it. Make sure a full clone of your largest repository fits inside
OPAL_SCOPES_GIT_FETCH_TIMEOUTand your startup probe's window.
Fewer INFO lines from opal-server, and the master configures logging first (#973)
- Now DEBUG instead of INFO: the per-data-update "Publishing data update to topics" line, and on a scopes policy push "Found new commits", "Publishing to topics", "Refresh scope", "Webhook listener triggered (N)" and "Repo was fetched after refresh request". A push now logs one INFO line, "Triggering policy update for scope". WARNING and ERROR lines are unchanged. To see the demoted lines, set
OPAL_LOG_LEVEL=DEBUG. - The gunicorn master configures logging before it logs anything, with scopes on or off and in the opal-client images too. The master's output now goes through the configured sink and credential scrubbing instead of loguru's default handler.
- Third-party stdlib log records in the master now reach the configured sink at their own level, as they already did in workers.
- An invalid logging setting, such as an unknown
OPAL_LOG_LEVELor an unwritableOPAL_LOG_FILE_PATH, now fails in the master: gunicorn exits with code 1 instead of 3.
- The log file is masked like the console. With
OPAL_LOG_TO_FILE=true, every record written to the file is masked, tracebacks and non-string extras included, and the file followsOPAL_LOG_TRACEBACKandOPAL_LOG_DIAGNOSE.
How to opt in
Docker: every image is published under the 0.10.0-rc.4 tag, plus -alpine:
permitio/opal-server:0.10.0-rc.4
permitio/opal-client:0.10.0-rc.4
permitio/opal-client-standalone:0.10.0-rc.4
permitio/opal-client-cedar:0.10.0-rc.4
permitio/opal-client-eopa:0.10.0-rc.4
pip: pre-releases are skipped unless you ask for them:
pip install --pre opal-client # or: opal-client==0.10.0rc4
pip install --pre opal-server