PgQue v0.3.0-alpha.1
Prerelease — for testing only. This alpha ships the new partition-keys and
producer-idempotency work for early feedback. Install it from
devel/sql/pgque.sql at this tag. The stable install under sql/ remains
0.2.0 and is unchanged; use that for anything real.
This is an SQL-only alpha. The client libraries (gem / PyPI / npm / Go module)
are not being versioned or published in this stamp — client releases will
follow once the partition-keys surface stabilizes.
Highlights
Partition keys — ordered per-key processing (#295)
Consume a queue with strict per-key ordering while still processing different
keys in parallel. Events are distributed across a fixed set of slots; each
slot is consumed by one worker at a time, and ordering is preserved within a
slot. Coordination uses batch-granularity leases rather than advisory
locks, so it is safe behind a transaction pooler (e.g. PgBouncer in
transaction mode) and needs no session-level state.
There are no client-library helpers yet — drive it with the raw SQL described
in blueprints/partition-keys/SPEC.md (see §15). User-facing docs under
docs/ are still pending; the SPEC is the reference for now.
Producer idempotency — pgque.send_idem() (#295)
Deduplicate on the producer side: pgque.send_idem() claims an idempotency
key and appends the event atomically, so a retried producer call does not
enqueue a duplicate event.
New: Ruby client (#234)
A pgque Ruby gem joins the client lineup. (Not published to RubyGems in this
alpha — see the SQL-only note above.)
Fixes rolled in
get_queue_infois reader-callable — consumers can read queue metadata
without writer/admin grants (#267).dlq_replayconcurrent double-replay guarded — two workers can no longer
replay the same DLQ event twice (#287).- Uninstall hardening — scheduler jobs are torn down, and the uninstall
scripts abort loudly instead of silently swallowing errors (#290). transform.shbuild guards — the build fails loud on integrity
regressions instead of emitting a subtly wrong install file (#292).- Go client — NULL type/payload columns scan correctly, and ack failures
back off instead of hot-looping (#285, #284). - TypeScript client — ack/nack now back off on failure; benchmark harness
hardened (#289). - Python client — consumer drains cleanly and survives per-event errors
without aborting the loop (#291). - Experimental SQL — throughput division-by-zero and negative
max_retriesfixed, with CI coverage added (#288).
Continuous integration
PostgreSQL 19 beta 1 is now part of the continuously tested server matrix
(PG 14–19beta1).
Breaking change (Python client)
Event.extra has been removed. It never functioned — data passed via
extra was silently dropped. Passing extra= now raises TypeError at the
call site instead of failing silently. Update callers to stop passing it.
Known limitations — partition-keys alpha
Partition keys are alpha. Read blueprints/partition-keys/SPEC.md before
relying on any of this, and watch pgque.partition_slot_status.
- No sticky-worker takeover (M=N). When a sticky worker stops, its freed
slot is not automatically picked up by another worker. - Stale subscriptions pin rotation silently. A subscribed-but-idle slot
holds back tick rotation with no error. Monitor
pgque.partition_slot_status. i_maxcaps a slot's batch slice. On the partitioned path, events beyond
i_maxin a batch are skipped for that slot and picked up on the next batch.- Every subscribed slot must be polled. If any subscribed slot is never
consumed, rotation pins. Poll all subscribed slots. - Partial
unsubscribe_slotorphans its key-class. Unsubscribing only some
slots of a key-class can leave that class orphaned. - Non-superuser-installer variant is not yet security-tested.
- No client-library helpers yet. Use the raw SQL in the SPEC (§15).
- User-facing docs pending.
docs/coverage is not in this alpha.
Install
begin;
\i devel/sql/pgque.sql
commit;Requires PostgreSQL 14+ and a periodic ticker. See devel/sql/README.md for
ticker setup and role grants.