tastyware/streaq v6.0.0

tastyware/streaq:v6.0.0b1

on GitHub

What's Changed

• Add Trio support (pending anyio support in coredis) by @Graeme22 in #80
  This is a large change, powered by simultaneous changes to coredis, the client used behind the scenes. This release is beta, pending the release of the next major coredis version. Here's a summary of the changes:
  • The library now works with both asyncio and Trio. You can change the backend by passing anyio_backend to workers:

    # asyncio is the default
    worker = Worker(anyio_backend="trio", anyio_kwargs={"restrict_keyboard_interrupt_to_checkpoints": True})

  • streaQ now uses structured concurrency everywhere for more predictability. If you haven't read Nathaniel Smith's article on structured concurrency, go read it now--it's a fantastic explanation of the motivations and an introduction to some of the concepts.
  • As a consequence of SC, async context managers for the worker are back! You'll need to use the context manager for any operation that involves communicating with Redis:

    async with worker:
        task = await sleeper.enqueue()
        print(await task.result(3))

    This means the recommended way of integrating with FastAPI has changed a bit:

    @asynccontextmanager
    async def app_lifespan(app: FastAPI) -> AsyncGenerator[None]:
        async with worker:
            yield
    
    app = FastAPI(lifespan=app_lifespan)

    See here for more details.
• Workers no longer attempt to re-enqueue tasks on shutdown--instead, we assume all shutdowns are unsafe, and allow Worker.idle_timeout to handle retrying. The default idle timeout has been decreased from 5 minutes to 1 minute to reflect this change.
• Redis scripts are now Redis functions in a single Lua library file. This means we now use FCALL instead of EVALSHA, which slightly increases performance in pipelines. This also means Redis versions less than 7.0 are no longer supported.
• Task abortion now handles delayed tasks better, aborting them right away instead of marking them and leaving them untouched
• TaskResult now has slightly better typing. Instead of accessing TaskResult.result which was either the task's output or any raised exception, TaskResult.result now only contains the task's output and errors if you access it when TaskResult.success is False. Exceptions are now contained in TaskResult.exception, and errors if you access it when TaskResult.success is True.

  async with worker:
      result = await task.result(3)
      if result.success:
          print(result.result)  # this is present only if success
      else:
          raise result.exception  # this is present only if not success

• Add date in logs by @nsteinmetz in #91
  Logs now contain date as well as time. Timezone can be customized as before.
• Dependencies bumped, now using dependabot (#81, #86)
• Redis Sentinel now in test coverage everywhere a worker is used
• Tests now use a Docker compose cluster
• Added "Contributing" page to docs
• Benchmarks updated
• Web UI no longer uses alru_cache since it's not compatible with anyio

New Contributors

• @dependabot[bot] made their first contribution in #81

Full Changelog: v5.2.2...v6.0.0