github shuttle-hq/shuttle v0.12.0
on GitHub

latest releases: v0.49.0, v0.48.3, v0.48.2...
20 months ago

shuttle: v0.12.0 update

We're excited to release shuttle v0.12.0! 🚀

Removing our unsafe FFI

As of this release Shuttle will no longer rely on compiling users projects as dynamic libraries, we have refactored the codegen to embed the required code to configure and add resources to a users project binary. This means we can get rid of the unsafe FFI needed to load the dynamic libraries, which was a big source of memory issues (especially for local runs). As a result of the refactor, we’ll be able to:

  • Get rid of the Sync requirement for services.
  • Get rid of the extra tokio::Runtime that is currently required for shuttle resources to be started on the correct side of the FFI (e.g. shuttle_shared_db, shuttle_secrets).
  • Correctly stop deployments.
  • Remove the version pin on tokio (as of writing it is pinned to 1.22).

Shuttle-next framework

Shuttle-next is a brand new WASM web-framework based on axum and hyper. There are many benefits to using WASM for a backend web-framework, but one of the key features is the isolation it offers. Since WASM modules have no notion of a file system, a shuttle-next project can only interact with the host through resources explicitly declared by us. In this first iteration, the only resource available is an HTTP stream to and from a project.

Shuttle-next is very much a work in progress, and we're releasing this iteration without some of the
features that make shuttle a pleasure to use, for example the ability to effortlessly deploy and connect to a database. It's also not currently possible to use any middleware. We're planning to implement this functionality, and more, in the near future. We're also hoping that the feedback we get from this release will help us see which features we should prioritize, and which areas of the framework's design need the most attention going forward.

Below is a small example of shuttle-next, see the example in our docs for more information.

Note: shuttle-next does not currently work on Windows, it only works on unix systems (i.e mac, Linux and Windows Subsystem for Linux). We aim to resolve this in future releases. 

// lib.rs
shuttle_next::app! {
    #[shuttle_next::endpoint(method = get, route = "/hello")]
    async fn hello() -> &'static str {
        "Hello, World!"
    }
}

Extracting the service integrations

The service implementations for frameworks like Axum, Rocket, Serenity etc., have now been extracted into separate crates. You will now need to depend on that crate, e.g shuttle_axum = "0.12.0", rather than shuttle-service = { version = "0.12.0", features = ["web-axum"] }.

Bug fixes

  • Fixed a bug where windows users had to do a cargo build before they could cargo shuttle run: #721
  • Fixed a bug where the file contents of static assets in the shuttle_static_files directory were deleted in local runs: #717
  • Fixed a bug where passing a path to cargo shuttle init would return an error: #706

Protoc requirement

Due to new dependencies we can’t go without users will now need to install protoc to install and use cargo-shuttle and our other crates. We tried to find a way around this requirement, but it came at the cost of significantly increased build times. We know this extra step required before being able to use shuttle is far from ideal, and we’ll try to find a good way around it in the future.

Mac and Linux

To install protoc from the distributed binary, you can use this script: 

# replace <arch> with your architecture, for example for linux x86_64 replace <arch> with: linux-x86_64
curl -OL https://github.com/protocolbuffers/protobuf/releases/download/v21.9/protoc-21.9-<arch>.zip &&\
        sudo unzip -o protoc-21.9-<arch>.zip -d /usr/local bin/protoc &&\
        sudo unzip -o protoc-21.9-<arch>.zip -d /usr/local 'include/*' &&\
        rm -f protoc-21.9-<arch>.zip

If you'd like to do this yourself without the above script, see instructions here: https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os

On mac you can also install protoc with brew: brew install protobuf.

Note: installing protoc on Linux with apt-get install protobuf-compiler will not be enough, since it will install an old version of protoc (v3.12) but it needs to be at least v3.19. Installing on mac using brew works.

Windows

On Windows the simplest way to install protoc is to first install chocolatey, and then simply run: choco install protoc

Upgrading

Due to the nature of these changes, all projects will be stopped and forced to upgrade. Please see instructions here for how to upgrade, as well as our updated examples and updated docs

First, since your project will now be compiled as a binary and no longer a library, you need to convert it to a binary. That means renaming the lib.rs file where your shuttle_service::main function currently is to main.rs, and you’ll also need to remove the [lib] field from your Cargo.toml.

We have also made some changes to the imports, you’ll now need to import shuttle-runtime and tokio, as well as the service integration for your framework. See the example below. 

# Before for axum apps
axum = "0.6"
shuttle-service = { version = "0.11.0", features = ["web-axum"] }
sync_wrapper = "0.1.1"

# After for axum apps
axum = "0.6"
shuttle-axum = "0.12.0"
shuttle-runtime = "0.12.0"
tokio = "1.26.0"

# Before for serenity bots
serenity = { version = "0.11.5", default-features = false, features = ["client", "gateway", "rustls_backend", "model"] }
shuttle-secrets = "0.11.0"
shuttle-service = { version = "0.11.0", features = ["bot-serenity"] }

# After for serenity bots
serenity = { version = "0.11.5", default-features = false, features = ["client", "gateway", "rustls_backend", "model"] }
shuttle-runtime = "0.12.0"
shuttle-serenity = "0.12.0"
shuttle-secrets = "0.12.0"
tokio = "1.26.0"

Since the service integrations now live in a different crate from where the Service trait is defined, they need to be wrapped in a new-type for us to be able to implement Service for them. This means you need to wrap the return type from your shuttle_runtime::main function in this type, but he have implemented From from for all the integrations, so you can simply return Ok(router.into()). See an example of upgrading an axum app below, and see our updated examples for other integrations: 

// Before in src/lib.rs
use axum::{routing::get, Router};
use sync_wrapper::SyncWrapper;

async fn hello_world() -> &'static str {
    "Hello, world!"
}

#[shuttle_service::main]
async fn axum() -> shuttle_service::ShuttleAxum {
    let router = Router::new().route("/hello", get(hello_world));
    let sync_wrapper = SyncWrapper::new(router);

    Ok(sync_wrapper)
}

// After in src/main.rs
use axum::{routing::get, Router};

async fn hello_world() -> &'static str {
    "Hello, world!"
}

#[shuttle_runtime::main]
async fn axum() -> shuttle_axum::ShuttleAxum {
    let router = Router::new().route("/hello", get(hello_world));

    Ok(router.into())
}

To upgrade your shuttle CLI, run: cargo install cargo-shuttle, or if you’re using cargo-binstall, cargo binstall cargo-shuttle.

If you’d like to upgrade your project container, run the rm and new project commands. This will not delete any databases, and you will keep your project name: 

cargo shuttle project rm 
cargo shuttle project new

Finally, redeploy your shuttle service: 

cargo shuttle deploy

Contributions

  • @kierendavies made their first contribution to shuttle by fixing a nefarious cyclical dependency bug in #700
  • @blixus made some great improvements to our CLI docs in shuttle-hq/shuttle-docs#48
  • @HexPanda made their first contribution to shuttle by fixing the indentation of the poise init code in #687
  • @stavares843 made their first contribution to shuttle by fixing some typos in our repo in #682

Commits for this release

  • feat: shuttle-serenity initial commit poc by @brokad in #429
  • shuttle next wrapper POC by @oddgrd in #431
  • refactor: create runtimes workspace by @chesedo in #432
  • feat: add --provisioner-address arg to both runtimes by @chesedo in #433
  • feat: create a control plane interface (part 1) by @chesedo in #436
  • refactor: combine runtimes into one by @chesedo in #438
  • feat(deployer): add support for building wasm projects by @akrantz01 in #437
  • feat: move factory to runtime by @oddgrd in #444
  • WIP feat: start runtime from deployer by @oddgrd in #450
  • feat: get logs from runtime by @chesedo in #459
  • [WIP] feat(shuttle-next): POC of axum wasm router by @oddgrd in #472
  • Feat(shuttle-next): stop runtime services by @oddgrd in #481
  • feat(next): expand macro into axum routes by @chesedo in #488
  • Feat: parse shuttle::endpoint macro by @oddgrd in #490
  • feat: merge main into shuttle-next by @oddgrd in #543
  • refactor: one store per request by @chesedo in #510
  • refactor: remove the serenity runtime by @chesedo in #549
  • Feat: shuttle next bump deps by @oddgrd in #551
  • feat: return streaming body from wasm router by @oddgrd in #558
  • feat: embed runtime into client and deployer by @chesedo in #559
  • bug: expanded broken merge by @chesedo in #567
  • feat: refactor router and router inner by @oddgrd in #566
  • feat: hook in runtime logs by @chesedo in #568
  • misc: merge main by @chesedo in #572
  • fix: missing feature flag in common dep by @oddgrd in #573
  • feat: change log read to spawn blocking by @oddgrd in #574
  • feat: DX by @chesedo in #577
  • feat: get runtime binary from cargo install by @oddgrd in #578
  • feat: deployer next by @chesedo in #575
  • misc: merge main by @chesedo in #582
  • misc: remove unneeded codegen feature by @chesedo in #584
  • refactor: proto fixups by @chesedo in #585
  • Feat: cleanup fds and tmp by @oddgrd in #586
  • refactor: unwraps and mor by @chesedo in #587
  • feat: build our images with the release profile by @oddgrd in #583
  • Feature/eng 378 axum wasm multiple handlers per endpoint by @oddgrd in #588
  • refactor: find code to wrap legacy runtime by @chesedo in #675
  • feat: remove box self from services, remove syncwrapper from axum service by @oddgrd in #677
  • feat: extract next runtime into separate binary by @oddgrd in #679
  • Feature: remove tokio runtime from all resources by @oddgrd in #680
  • feat: trim the service loader, unpin tokio by @oddgrd in #681
  • Feature/eng 477 make wasm dependencies optional by @oddgrd in #688
  • Feature: eng 465 update all the codegens by @oddgrd in #686
  • Feature: eng 483 trim and fix the tests in shuttle-service by @oddgrd in #693
  • fix: Respect Cargo.lock when building containers by @kierendavies in #700
  • ci: resolve CI errors in shuttle-next by @oddgrd in #580
  • Feature/eng 486 update deployer with runtime changes by @oddgrd in #696
  • feat: extract service integrations into separate crates by @oddgrd in #702
  • [next] tests: CI go green by @chesedo in #704
  • Feature: emit error when shuttle::main function is named main by @oddgrd in #707
  • Feature: eng 484 update init with codegen changes by @oddgrd in #706
  • fix: codegen clippy by @oddgrd in #709
  • misc: v0.12.0-rc1 by @chesedo in #708
  • [next] refactor: update runtime manager by @chesedo in #711
  • Feature: eng 497 update or remove the docs in shuttle by @oddgrd in #710
  • refactor: rename legacy runtime to alpha by @oddgrd in #713
  • refactor: move next to services by @chesedo in #714
  • [next] refactor: remove ids from runtime by @chesedo in #712
  • feat: embed protoc in shuttle-proto by @oddgrd in #715
  • [next] bug: communicating resources by @chesedo in #716
  • refactor: don't install next-runtime from git in local runs by @oddgrd in #718
  • fix: static folder local run clearing file contents, add missing tests in cargo-shuttle by @oddgrd in #717
  • [next] refactor: tracing by @chesedo in #719
  • fix: windows local run path bug by @oddgrd in #721
  • refactor: pull out vendored protoc from shuttle-proto by @oddgrd in #726
  • [next] bug: misc fixes by @chesedo in #725
  • bug: next runtime not sending stop signal by @chesedo in #728
  • chore: v0.12.0 by @oddgrd in #727
  • feat: shuttle-next by @chesedo in #579

Full Changelog: v0.11.3...v0.12.0

Check out latest releases or
releases around shuttle-hq/shuttle v0.12.0

Don't miss a new shuttle release

NewReleases is sending notifications on new releases.

Get notifications