Taaitaaiger/jlrs v0.20.0

V0.20.0

on GitHub

Version 0.20 is a big release that introduces a few new features. This version supports Julia 1.6 up to and including 1.11, the MSRV is 1.77.

Low-level changes

• jl-sys redesign

Historically jlrs exposed many implementation details of the Julia C API and reimplemented others in Rust. Some of these reimplementations existed in jl-sys, others in jlrs. This version changes this completely: jl-sys hides as many implementation details from Rust as possible. For example, most pointers to Julia data are exposed as opaque pointers to Rust; if a specific field must be accessed, a function written in C is used to do so.

• LTO

The major drawback of this previous change is that it introduces a lot of overhead: what used to be a simple pointer dereference is now a function call. On Linux this lost performance can be restored by enabling the lto feature and setting several flags to compile with support for cross-language LTO. See the docs for the prerequisites and relevant flags.

Runtime module changes

Several types and traits have been renamed:

• RuntimeBuilder to Builder
• AsyncRuntimeBuilder to AsyncBuilder
• AsyncRuntime to Executor
• AsyncJulia to AsyncHandle

The sync runtime, Julia, has been deprecated in favor of the local runtime, LocalHandle. Types like LocalHandle and AsyncHandle should be considered handles to the Julia runtime, which shuts down when all handles have been dropped. More on these handles later.

Builder

Added features:

• Multithreaded runtime

When the multi-rt feature is enabled, Julia can be initialized in a way that allows directly calling into Julia from multiple threads. More info about this feature can be found in the section about the multithreaded runtime. Both builder types provide a start_mt and spawn_mt method.

• Thread count

The number of (interactive) threads available to Julia can be set by Builder, not just the AsyncBuilder.

• Starting and spawing runtimes

Some runtimes can either be started or spawned. If it's started, Julia is initialized on the current thread and a new thread is spawned and provided with a handle to the runtime. If it's spawned, Julia is initialized on a separate thread and a handle is provided directly.

Changed features:

• Safety

Initializing Julia is now considered safe. Using a custom system image is unsafe.

Dropped features:

• Worker threads

Support for worker threads has been dropped, Async(Runtime)Builder::n_worker_threads has been removed.

Local runtime

Added features:

• Local runtime

The main difference between the sync and local runtimes is that the local runtime is started without a Stack. As such, the local runtime's LocalHandle can only create statically-sized local scopes, and is unable to create dynamically-sized scopes.

• Using

LocalHandle::using/Julia::using can be called to evaluate a using statement.

Deprecated features:

• Sync runtime

The sync runtime has been deprecated in favor of the local runtime.

Async runtime

Added features

• Using

AsyncHandle::using can be called to evaluate a using statement.

• Closing

An AsyncHandle can be closed by calling AsyncHandle::close, this shuts down the runtime or pool. More on pools later. You can check if an AsyncHandle has been closed by calling AsyncHandle::is_closed.

Changed features:

• Task return channel

Async and persistent tasks always use a tokio oneshot channel to return their result. Methods like AsyncHandle::task no longer take the sending half of a return channel, rather the receiving end of the oneshot channel is returned after the task has been successfully dispatched.

Dropped features:

• async-std

The only implementation of Executor provided by jlrs is Tokio. If you must use async-std, you must implement the Executor trait yourself.

• Worker threads

Support for using the async runtime with worker threads has been dropped.

• Resizing the backing channel

The capacity of the channel used to communicate with the async runtime can no longer be adjusted.

• Posting blocking tasks

Support for posting blocking tasks has been dropped.

Multithreaded runtime

Added features

• Multithreaded runtime

With the multithreaded runtime you can call into Julia from arbitrary threads as long as you have an MtHandle. This handle can be cloned and sent to other threads. The Julia runtime shuts down when all handles have been dropped.

• Thread pools

The multithreaded and async runtimes can be used together to create thread pools that can call into Julia. This pool behaves just like an async runtime, and is initeracted with using an AsyncHandle. Workers can be added or removed with AsyncHandle::try_add_worker and AsyncHandle::try_remove_worker.

Handles

Added features

• Weak handle

A WeakHandle is similar to a LocalHandle and is a safe alternative for the Unrooted target.

• Delegated handle

A DelegatedHandle is also similar to a LocalHandle and is only available in delegated tasks. More on delegated tasks later.

• WithStack

The local and multithreaded runtimes don't allocate a Stack, the WithStack trait can be used to provide these handles with a Stack to enable using dynamically-sized scopes.

Changed featues

• CCall

CCall is now considered a handle instead of existing at top-level.

Memory module changes

Scope

Added features

• Unsized local scopes

Unsized local scopes take their size as an argument at runtime rather than as a const generic.

Changed features

• Scope and LocalScope traits

Methods like GcFrame::scope and GcFrame::local_scope have been moved to the Scope and LocalScope traits respectively. These traits have a type parameter, the type of the data returned by the scope.

• Arbitrary return type

Scopes are no longer required to return a JlrsResult, but can return data of any type. If the compiler is unable to infer the return type, annotate the closure or use the Returning::returning/LocalReturning::local_returning methods.

Data module changes

Layout

Added features

• Complex numbers

When the complex feature is enabled, jlrs maps num's Complex type to Julia's Complex type.

Dropped features

• SSA values

The SSAValue type has been removed.

Managed

Added features

• Static symbols

Symbols can now be cached by defining a static symbol with the define_static_symbol or define_static_binary_symbol macros.

Changed features

• Array redesign: ArrayBase

The array module has been changed quite significantly. There is now a single base type for arrays, ArrayBase. Previously independent types like Array and TypedArray are now aliases for the ArrayBase type.

• Array redesign: ArrayBase parameters

ArrayBase has two parameters, a type parameter T and a const isize N. T is either Unknown to indicate the element type is unknown, or some type that implements ConstructType. In the latter case the type serves as the type constuctor of the element type. Previously T could refer to the layout of the elements in the array, this is no longer the case. The const isize N is the rank of the array if N >= 0, otherwise it is -1 to indicate the rank of the array is unknown.

• Array redesign: Array constructors

Array constructors are implemented for ArrayBase, rather than the old separate types having separate constructor methods. Which methods are available depends on wheter or not the element type is statically known; ArrayBase::new is only available if T: ConstructType. If T is Unknown, ArrayBase::new_for is available which takes the element type as an argument.

• Array redesign: Array accessors

Array accessors have had the opposite treatment compared to arrays: the single accessor type and its many aliases have been disentangled into separate types. There's an accessor for each kind of array layout, distinction between mutable and immutable accessors is also made at the type level.

• Expr promotion

The Expr type has been promoted from an internal to a managed type.

Dropped features

• Internal types

jlrs used to expose many internal types if the internal-types feature was enabled. These types and that feature have been removed. YAGNI.

• Many field accessors

Many accessors for specific fields of managed types have been removed, only accessors for useful fields have been retained. These fields are mostly implementation details of Julia and prone to change, while offering little to no value to external users.

• Array manipulation methods

Several array manipulation methods have been completely removed, e.g. Array::reshape. These functions are no longer available in the C API in Julia 1.11.

Types

Added features

• Fast type and array keys

Types that implement ConstructType and have type parameters have to use a relatively slow caching mechanism, while types that don't have type parameters can use a much faster mechanism. The define_fast_key and define_fast_array_key macros let you define zero-sized types for specific cases that can use the fast caching mechanism.