NikolayS/pgque v0.2.0

PgQue v0.2.0

on GitHub

PgQue 0.2.0 is a major release: first-party client libraries, cooperative consumers, faster batch producers, sub-second ticking, pg_timetable support, pg_tle packaging, and a lot of hardening around security, roles, validation, and upgrades.

Highlights

• First-party clients for Python, Go, and TypeScript with aligned APIs and published GA packages.
• Cooperative consumers / subconsumers let multiple workers share one logical consumer cursor and drain slow downstream workloads in parallel.
• Set-based batch producers make client-side send_batch() dramatically faster than looping over send().
• Sub-second queue ticking lowers producer-to-consumer latency out of the box.
• Scheduler choice: pg_cron and pg_timetable are both supported.
• Optional pg_tle install path for platforms that support trusted-language extensions.
• Security and correctness hardening: fixed role split, pinned SECURITY DEFINER search paths, stricter validation, safer grants, and upgrade tests.

New features and notable changes

Client libraries

• Python client reaches API parity with Go: #82
• TypeScript client reaches API parity with Go: #83
• Client batch producer APIs: #161
• Cross-client parity matrix and docs: #152
• Python cooperative consumer API: #216
• Go cooperative consumer API: #215
• TypeScript cooperative consumer API: #214
• Python client parity wrappers: #247
• Go client parity wrappers: #248
• TypeScript client parity completion: #249
• TypeScript legacy pgq error classification: #251
• TypeScript connection-failure classification: #252

All three clients now cover the same core operations:

• send / send_batch
• receive
• ack
• nack
• force_next_tick / force_tick
• subscribe / unsubscribe
• typed errors
• retry-after handling
• high-level consumer helpers
• cooperative consumer / subconsumer APIs

Cooperative consumers / subconsumers

• SQL cooperative consumers: #208
• Reference docs and role grants for cooperative consumers: #126, #205
• Client support across Python, Go, and TypeScript: #214, #215, #216

Cooperative consumers let one logical consumer split work across multiple subconsumers. This is useful when the queue is not the bottleneck but downstream work is: email APIs, webhooks, SMS, HTTP calls, enrichment jobs, and similar workloads.

Core SQL API:

• pgque.subscribe_subconsumer(...)
• pgque.unsubscribe_subconsumer(...)
• pgque.receive_coop(...)
• pgque.touch_subconsumer(...)

Batch producer performance

• Atomic set-based send_batch(): #161
• Client producer benchmark refresh: #253

send_batch() now avoids client round trips per event and uses PgQue's set-based insert path.

Tick cadence and scheduler support

• Configurable sub-second tick rate, default 10 ticks/sec: #204
• Ticker latency wording and WAL-budget docs: #207, #209, #213
• pg_timetable scheduler support: #219

Tune tick cadence with:

select pgque.set_tick_period_ms(100);

Lower intervals can increase WAL and metadata churn. Benchmark before using very aggressive tick rates.

Packaging and installation

• Optional pg_tle installer: #177
• Client distribution release plumbing and dry-runs: #222
• Python release workflow hardening: #225
• TypeScript npm release workflow: #228
• Install command cleanup after rc.1: #230
• Final 0.2.0 release prep: #254

Published GA packages:

• PyPI: pgque-py==0.2.0
• npm: pgque@0.2.0
• Go: github.com/NikolayS/pgque-go@v0.2.0

Security, roles, validation, and correctness

• Restore producer/consumer role split and cleanup grants: #126, #205
• Pin SECURITY DEFINER search paths and restrict trusted-SQL cursor helper: #176, #205
• Reject invalid receive(max_return) values: #114
• Reject queue names longer than 57 bytes: #122
• Canonical and idempotent DLQ terminal nack() handling: #116
• Empty-batch receive cleanup: #117
• Batch retry hardening: #134
• Receive lock harness and synchronization: #175, #220
• SQL contract and e2e coverage for recent merged changes: #205
• v0.1.0 to HEAD upgrade CI: #226

Important behavior change: pgque_writer and pgque_reader are sibling roles. Apps that both produce and consume must be granted both roles.

Documentation and benchmarks

• Benchmark methodology and tooling: #66, #72, #157, #172
• PgQ heritage and concepts surfaced in docs: #218
• Roadmap and README cleanup: #171, #202, #221
• Upgrade guide polished after GA: #256

Contributors

Thank you to external contributors who landed changes in the 0.2.0 line:

• @hobostay / Qiaochu Hu — multiple bug fixes and dead-code cleanup: #75
• @The-Alchemist / Karl Pietrzak — reference documentation polish: #77
• @alceops / Alce — two-session receive lock harness: #175
• @victoraugustolls / Victor — legacy next_batch_custom main-row locking fix: #235

Install

PgQue is installed in two layers:

1. install the PgQue SQL API inside Postgres;
2. install a client library in your application.

The client libraries do not create the queue schema for you. They expect pgque to already be installed in the target database.

1. Install PgQue in Postgres

From this release tag, run the SQL installer as the schema owner or a superuser:

psql --single-transaction -v ON_ERROR_STOP=1 -d mydb -f sql/pgque.sql

If you are already inside psql, the equivalent is:

\i sql/pgque.sql

For platforms that support pg_tle, the optional wrapper is also available:

\i sql/pgque-tle.sql

The default SQL install does not require CREATE EXTENSION, shared_preload_libraries, or superuser-only background workers.

2. Install an application client

Pick the client for your application runtime:

# Python
pip install pgque-py

# TypeScript / JavaScript
npm install pgque
# or: bun add pgque

# Go
go get github.com/NikolayS/pgque-go@v0.2.0

Benchmarks

Client producer batching

Environment: GitHub Actions ubuntu-latest, PostgreSQL 18 Docker image, one producer/client connection, median of 3 repeats. Full data and methodology: docs/benchmarks.md and benchmark/charts/client_producer_batch_api.csv.

At batch size n=1000:

These are API-overhead benchmarks, not max cluster throughput claims. The point is simple: send_batch() collapses client round trips and uses PgQue's set-based insert path.

Subconsumer scaling demo

Demo setup: same 160-message backlog, fixed sleep(250 ms) per message to simulate downstream work, only the number of subconsumers changes.

Observed demo results:

The animated backlog-drain demo is in the README: docs/images/backlog_race.gif.

Upgrade notes from 0.1.0

The supported upgrade path is to re-run the SQL installer from the 0.2.0 tag:

psql --single-transaction -v ON_ERROR_STOP=1 -d mydb -f sql/pgque.sql

The installer is idempotent and preserves queues, consumers, subscriptions, retry rows, DLQ rows, and existing event tables.

Behavior changes to check:

• grant both pgque_reader and pgque_writer to apps that both consume and produce
• default tick cadence is now 10/s
• pgque.maint() no longer attempts internal VACUUM; vacuum scheduling remains the operator's job
• several public wrapper argument names were normalized, so named-argument calls should use the documented 0.2.0 names

Upgrade docs: docs/upgrading.md.

Verification for this GA

Before publishing GA, the release went through:

• full CI matrix on PostgreSQL 14, 15, 16, 17, and 18
• pg_tle install path
• pg_cron install path
• pg_timetable real-worker path
• v0.1.0 to HEAD upgrade tests on PostgreSQL 14 and 18
• Python client tests
• Go client tests
• TypeScript client tests
• final package dry-runs
• post-publish smoke checks for PyPI, npm, and Go proxy

Published release artifacts verified:

• pgque-py==0.2.0 installs from PyPI and imports as 0.2.0
• pgque@0.2.0 installs from npm, ESM import works, and latest points to 0.2.0
• github.com/NikolayS/pgque-go@v0.2.0 installs and compiles in a fresh Go module

Detailed changelog draft

The longer PR-by-PR draft that led to these release notes is tracked in issue #227:

#227