Celery v5.5.0 Release Candidate 2 is now available for testing.
Please help us test this version and report any issues.
Key Highlights
See What’s new in Celery 5.5 (Immunity) or read the main highlights below.
Using Kombu 5.5.0rc2
The minimum required Kombu version has been bumped to 5.5.0.
Kombu is current at 5.5.0rc2.
Complete Quorum Queues Support
A completely new ETA mechanism was developed to allow full support with RabbitMQ Quorum Queues.
After upgrading to this version, please share your feedback on the quorum queues support.
- New documentation.
- New broker_native_delayed_delivery_queue_type configuration option.
New support for Google Pub/Sub transport
After upgrading to this version, please share your feedback on the Google Pub/Sub transport support.
Relevant Issues: #9351
Python 3.13 Improved Support
Additional dependencies have been migrated successfully to Python 3.13, including Kombu and py-amqp.
Previous Pre-release Highlights
Python 3.13 Initial Support
This release introduces the initial support for Python 3.13 with Celery.
After upgrading to this version, please share your feedback on the Python 3.13 support.
Soft Shutdown
The soft shutdown is a new mechanism in Celery that sits between the warm shutdown and the cold shutdown. It sets a time limited "warm shutdown" period, during which the worker will continue to process tasks that are already running. After the soft shutdown ends, the worker will initiate a graceful cold shutdown, stopping all tasks and exiting.
The soft shutdown is disabled by default, and can be enabled by setting the new configuration option worker_soft_shutdown_timeout. If a worker is not running any task when the soft shutdown initiates, it will skip the warm shutdown period and proceed directly to the cold shutdown unless the new configuration option worker_enable_soft_shutdown_on_idle is set to True. This is useful for workers that are idle, waiting on ETA tasks to be executed that still want to enable the soft shutdown anyways.
The soft shutdown can replace the cold shutdown when using a broker with a visibility timeout mechanism, like Redis or SQS, to enable a more graceful cold shutdown procedure, allowing the worker enough time to re-queue tasks that were not completed (e.g., Restoring 1 unacknowledged message(s)) by resetting the visibility timeout of the unacknowledged messages just before the worker exits completely.
After upgrading to this version, please share your feedback on the new Soft Shutdown mechanism.
Relevant Issues: #9213, #9231, #9238
- New documentation for each shutdown type.
- New worker_soft_shutdown_timeout configuration option.
- New worker_enable_soft_shutdown_on_idle configuration option.
REMAP_SIGTERM
The REMAP_SIGTERM "hidden feature" has been tested, documented and is now officially supported.
This feature allows users to remap the SIGTERM signal to SIGQUIT, to initiate a soft or a cold shutdown using TERM
instead of QUIT.
Pydantic Support
This release introduces support for Pydantic models in Celery tasks.
For more info, see the new pydantic example and PR #9023 by @mathiasertl.
After upgrading to this version, please share your feedback on the new Pydantic support.
Redis Broker Stability Improvements
The root cause of the Redis broker instability issue has been identified and resolved in the v5.4.0 release of Kombu, which should resolve the disconnections bug and offer additional improvements.
After upgrading to this version, please share your feedback on the Redis broker stability.
Relevant Issues: #7276, #8091, #8030, #8384
Quorum Queues Initial Support
This release introduces the initial support for Quorum Queues with Celery.
See new configuration options for more details:
After upgrading to this version, please share your feedback on the Quorum Queues support.
What's Changed
- Fix: Treat dbm.error as a corrupted schedule file by @stumpylog in #9331
- Pin pre-commit to latest version 4.0.1 by @pyup-bot in #9343
- Added Python 3.13 to Dockerfiles by @Nusnus in #9350
- Skip test_pool_restart_import_modules on PyPy due to test issue by @Nusnus in #9352
- Update elastic-transport requirement from <=8.15.0 to <=8.15.1 by @dependabot in #9347
- added dragonfly logo by @auvipy in #9353
- Update README.rst by @auvipy in #9354
- Update README.rst by @auvipy in #9355
- Update mypy to 1.12.0 by @pyup-bot in #9356
- Bump Kombu to v5.5.0rc1 by @Nusnus in #9357
- Fix
celery --loaderoption parsing by @0x2b3bfa0 in #9361 - Add support for Google Pub/Sub transport by @haimjether in #9351
- Add native incr support for GCSBackend by @haimjether in #9302
- fix(perform_pending_operations): prevent task duplication on shutdown… by @moaddib666 in #9348
- Update grpcio to 1.67.0 by @pyup-bot in #9365
- Update google-cloud-firestore to 2.19.0 by @pyup-bot in #9364
- Annotate celery/utils/timer2.py by @hmnfalahi in #9362
- Update cryptography to 43.0.3 by @pyup-bot in #9366
- Update mypy to 1.12.1 by @pyup-bot in #9368
- Bump mypy from 1.12.1 to 1.13.0 by @dependabot in #9373
- Pass timeout and confirm_timeout to producer.publish() by @thedrow in #9374
- Bump Kombu to v5.5.0rc2 by @Nusnus in #9382
- Bump pytest-cov from 5.0.0 to 6.0.0 by @dependabot in #9388
- default strict to False for pydantic tasks by @mathiasertl in #9393
- Only log that global QoS is disabled if using amqp by @thedrow in #9395
- chore: update sponsorship logo by @Niennienzz in #9398
- Allow custom hostname for celery_worker in celery.contrib.pytest / celery.contrib.testing.worker by @SlowMo24 in #9405
- Removed docker-docs from CI (optional job, malfunctioning) by @Nusnus in #9406
- Added a utility to format changelogs from the auto-generated GitHub release notes by @Nusnus in #9408
- Bump codecov/codecov-action from 4 to 5 by @dependabot in #9412
- Update elasticsearch requirement from <=8.15.1 to <=8.16.0 by @dependabot in #9410
- Native Delayed Delivery in RabbitMQ by @thedrow in #9207
- Prepare for (pre) release: v5.5.0rc2 by @Nusnus in #9416
New Contributors
- @0x2b3bfa0 made their first contribution in #9361
- @hmnfalahi made their first contribution in #9362
- @Niennienzz made their first contribution in #9398
- @SlowMo24 made their first contribution in #9405
Full Changelog: v5.5.0rc1...v5.5.0rc2