typelevel/cats-effect v3.0.0-M2

on GitHub

Cats Effect 3.0.0-M2 is the second milestone release in the 3.0.0 line. We are expecting to follow this milestone with subsequent ones as necessary to further refine functionality and features, prior to some number of release candidates leading to a final release. The designation "M2" is meant to indicate our relative confidence in the functionality and API of the library, but it by no means indicates completeness or final compatibility. Neither binary compatibility nor source compatibility with anything are assured prior to 3.0.0, though major breakage is unlikely at this point.

What follows are the changes from M1 to M2. For a more complete summary of the changes generally included in Cats Effect 3.0.0, please see the M1 release notes

Major Changes

The most significant changes in this release are undoubtedly the introduction of auto-yielding semantics in fiber evaluation and the final removal of UnsafeRun and related introduction of Dispatcher.

Auto-Yielding

This feature is probably best demonstrated with an example test case:

"reliably cancel infinite IO.unit(s)" in real {
  IO.unit.foreverM.start.flatMap(f => IO.sleep(50.millis) >> f.cancel).as(ok)
}

This test constructs a fiber which loops forever doing absolutely nothing. Specifically, IO.unit.foreverM. This is the functional equivalent of while (true) {}. It then starts this fiber, sleeps for 50 milliseconds, then cancels it.

Unfortunately, this test will hang forever on any single-threaded runtime! The reason for this is the fact that one fiber is hogging the only available thread in such a runtime, and so the sleep is never able to wake up and cancel it. Exemplifying this is the fact that this exact unit test had to be disabled on JavaScript until this change was introduced.

When one fiber is able to hog a thread, preventing other fibers from getting their turn, it is known as a violation of fairness. Most applications written in the Cats Effect ecosystem have experienced this problem in a minor way at some point or another, though it isn't commonly observed in production due to the fact that most Cats Effect usage involves a lot of incidental yielding. A yield, codified by the IO.cede operation, is when the current fiber gives up its carrier thread so that other fibers can take their turn before the current fiber resumes again. However, when fibers, intentionally or otherwise, failed to consistently yield their thread back to the scheduler, they could starve other fibers of resources. The service-level consequence of this problem (when it manifests) is a significantly increased latency jitter metric: responses are still produced very quickly once they get started, but some responses take a significant length of time before they even begin executing.

Auto-yielding resolves this issue entirely. This semantic has long been resisted within Cats Effect's IO due to the fact that auto-yielding fundamentally sacrifices throughput (intuitively, straight-line performance of any single fiber) for fairness (reliable latency). This is often a desirable tradeoff to make, but seldom one which can be automatically determined by the runtime system. A poorly-timed automatic yield can result in significant performance costs for an application which are hard to detect and almost impossible to resolve.

However, the introduction of the work-stealing scheduler in Cats Effect 3 opens up the possibility of automatic yields without any performance penalty! Simply put, when a fiber yields back to the scheduler in Cats Effect 3, it only incurs a cost if some other fiber was already waiting on that specific thread. In other words, in all cases where the automatic yield was unnecessary, the performance penalty of the automatic yield is non-existent: no memory barriers are crossed, and no threads are switched, thus all cache lines are preserved and the fiber continues as if nothing happened. If, however, the automatic yield does find another fiber awaiting access to the thread, then the yield is having its desired result and the other fiber needed to take a turn! Thus, we have all of the benefits of automatic yielding with none of the drawbacks.

Dispatcher

Cats Effect 2 included a pair of typeclasses, ConcurrentEffect and Effect, which were designed to abstract over the notion of running an effect. This was necessary due to the existence of frameworks like Netty, which requires API users to submit tasks which the framework (Netty) itself will run as side-effects at some later point. Other common frameworks in this mould include Play, Akka, Java NIO, and many more.

This scenario is remarkably common, even in higher-level frameworks such as Http4s. Unfortunately, Cats Effect 3 made this considerably more complicated by improving the ergonomics around the async operation. Since the async operation now automatically shifts back to the compute thread pool, that thread pool must be passed to the IO running functions (e.g. unsafeRunSync()) in the form of a runtime parameter. In the case of Cats Effect 3, this is represented by the IORuntime type (and automatically managed for you if you use IOApp), but Monix and ZIO both have their own variation of this idea. Unfortunately (or perhaps, fortunately), this also means that typeclasses like ConcurrentEffect are no longer possible.

In 3.0.0-M1, Cats Effect relied upon the UnsafeRun typeclass to "abstract" over this use-case. This was deeply unpleasant and had a lot of unfortunate consequences on API design, but there didn't seem to be a better approach.

Dispatcher is a better approach. For any situation where you're using Cats Effect code to manage some sort of impure framework which itself needs to run other Cats Effect actions as side-effects, Dispatcher provides a high-performance and fully generic solution which only requires an Async constraint to operate. This allows frameworks such as Fs2, Http4s, streamz, and many others to loosen their constraints dramatically, which opens up the door to significantly more powerful abstraction and higher levels of control for users.

Critically, Dispatcher does not provide a mechanism for running an effect "at the end of the world." For such situations, you still need to use IOApp (or whatever similar functionality is provided for your effect type of choice). What Dispatcher does do is ensure that only one "end of the world" is ever required, which greatly improves the ergonomics, flexibility, and performance of the framework and ecosystem.

Pull Requests

• #1278 Stack safety for Resource (@bplommer)
• #1283 ApplicativeThrow type alias (similar to MonadThrow) (@fthomas)
• #1285, #1296 Removed constraints from most Resource functions (@bplommer)
• #1157, #1295 Special-case IO.race and IO.both for performance (@TimWSpence)
• #1282 Tests for Console.readLine including all sorts of wonky characters (@vasilmkd)
• #1281 Simplified Semaphore API by removing withPermit (@bplommer)
• #1307 Added Resource#onFinalize (@bplommer)
• #1308 Added IO.ref and IO.deferred functions (@bplommer)
• #1303, #1333 Added Dispatcher, a new construct for interacting with code which must run F actions as a side-effect, including common frameworks like Play or Netty (@djspiewak, @RaasAhsan)
• #1303 Removed UnsafeRun, the last vestiges of ConcurrentEffect, since it is no longer necessary to "abstract over the end of the world" (@djspiewak)
• #1326 Ported HotSwap from fs2 (@RaasAhsan)
• #1294 Added a high performance asynchronous blocking queue (@vasilmkd)
• #1346 Fixed Outcome smart constructors (@heka1024)
• #1316 Relaxed Semaphore typeclass constraints by eagerly raising errors on invalid parameters (@bplommer)
• #1340 Implemented fiber auto-yielding to ensure fair-by-default fiber evaluation semantics. Note that this should have almost no performance penalty due to the presence of the work-stealing scheduler (@TimWSpence)
• So… much… scaladoc (@RaasAhsan)

You're all amazing, thank you!