github vercel/turborepo v0.8.5

2 years ago

What's Changed

  • Completely rewritten and revamped internal graph. Fixes long standing bug with construction. You no longer need to specify all possible tasks, turbo will figure out the transitive tasks necessary automatically.
  • Anything passed after a -- to turbo run will be passed through to all build tasks. This is a temporary workaround for parameterized builds. The use case is for something like turbo or Next.js which has OS-specific builds and cache artifacts which would otherwise have an identical hash.
  • turbo CLI output is now color aware, which should make reading build logs on platforms like Vercel and Netlify (and any other non-interactive terminal) easier.

Historical Changelog

v0.8.4

  • turbo is now much more friendly to OSS. In prior versions, a CI run triggered by a pull request by an outside contributor on a turborepo with remote caching would fail because turbo would bail in response to the unauthorized HTTP status code from our API (because the outside contributor doesn't have access to the original repo's secrets and thus TURBO_TOKEN). From now on though, turbo will show this warning message and fallback to local caching in this situation. This is a win for everyone as it allows core contributors with write access to get fastest builds, while still have fairly fast external contributions. In the future, we may explore public remote caches for OSS as well as read-only tokens.

v0.8.3

  • Remove support doublestar globs in workspaces (introduced in v0.8.1) until a faster implementation can be found

v0.8.2

  • The globs you use to specify your package.json workspaces or pnpm-workspaces.yaml now properly handles doublestar globs directives (e.g. packages/**)—turbo will automatically ignore node_modules and bower_components directories for you. This matches up with NPM, PNPM, and Yarn implementations.

v0.8.1

  • Botched release due to GitHub packages bug, do not use. This has been removed from the registry.

v0.8.0

  • turbo can now be run from a workspaces which are in subdirectories (not the git root).
  • Internal changes need for the forthcoming UI, which is flagged off warranted a minor version bump

v0.7.5

  • Fixed issue where including an pipeline package-specific overrides with topological dependsOn created circular task graph dependency

v0.7.4

  • Fix regression with --since flag (it wasn't working)
  • Unknown/misspelled CLI flags will now properly error instead of failing silently

v0.7.3

  • Fixed issue with prune where .gitignore didn't exist. It now safely handles this situation.
  • Fixed issue with prune where turbo's internal representation of the lockfile was being outputted to the root directory instead of to the cache.
  • Fixed infinite loop during turbo link

v0.7.2

  • Improved error messages around auth flow

v0.7.1

  • Invalid flag/config errors now return proper exit code (1)
  • Added ability to specify token, team, and project via TURBO_TOKEN, TURBO_TEAM, and TURBO_PROJECT environment variables. The flags will take precendence over the environement variables when both are present.

v0.7.0

  • Remote caching is finally here! Along with a super super early preview of https://beta.turborepo.com. Remote caching allows everyone on your team and even your CI/CD to recycle previously cached task artifacts. Think of it like Dropbox, but for your builds!

Getting started with v0.7.0

  • Remove all the .npmrc stuff! You no longer need a token to download turbo.
  • Upgrade turbo: yarn add turbo -W
  • Create an account and team on https://beta.turborepo.com/signup
  • Activate your CLI: yarn turbo login
  • Link your monorepo to a new project: yarn turbo link
  • Use turbo! In addition to locally caching artifacts, turbo will now also two-way sync artifacts to your remote cache.
  • Invite your team members so they can use your artifacts too. In your dashboard, go to Users and then blast out some invites. Once they have accepted the invitation, they'll be added to your team and have access to all of your projects. They can then link their local git repos to your shared turborepo.
  • Connect your CI (programmatic non-interactive access): To give your CI/CD access to your turborepo.com team, project, and cache, go to the dashboard and create a personal access token for your account (API Tokens > Create). Copy this token to a safe place (you'll only see it once). Add it to as a secret environment variable in your CI provider. You can then pass it to turbo CLI like so: yarn turbo run build --token=<my_token> --team=<my-team-url> --project=<my-project>

During this limited preview period, storage and bandwidth is free for everyone in the Discord, but please don't abuse it.

Stuff missing/what we're working on

  • Inspect/explore remote cache artifacts in the dashboard
  • Usage UI
  • Cleaner error messages

WARNING/DISCLAIMER: This is rough, and they'll be bugs, but #shippingisperfection.

v0.6.10

  • Added (back) partial support for NPM and PNPM workspaces. When using either of these package managers, turbo will include the entire contents of the relevant lockfile in its hashing algorithm for each package task. This differs from yarn behavior where turbo will include the resolved the versions of dependencies for a given package. The prune command does not yet work with pnpm or npm. These features will be added at a later date.

v0.6.9

  • Remote cache hits now store to other caches (file system).

v0.6.8

  • Improved chrome profiling output (--profile) flag
  • Fixed hashing regression with lockfiles

v0.6.7

  • Removed misleading/useless warning message about reading missing log file during replay.

v0.6.6

  • turbo can now construct hashes without being in a git repository. This is extremely useful for docker deployment and running builds inside of pruned out directories for a PaaS like Vercel or Netlify.

v0.6.5

  • Fix cache regression where the first run was not properly storing artifact output
  • Fix regression with lockfile hash
  • Fix regression with cache worker pool on Linux

v0.6.4

  • Improved error messages

v0.6.3

  • Add hybrid (remote + local) caching
  • Ensure that execution output summary waits until all replay logs have been flushed already.

v0.6.2

  • Internal refactoring related to remote caching

v0.6.1

  • In the presence of an error without --continue, turbo will properly flush/shutdown async cache workers instead of leaving them hanging (resulting in partial artifact outputs).

v0.6.0

  • turbo is now 50-80% faster thanks to refactoring/improving cache behavior. While fetching artifacts from the cache is still synchronous, storing task artifacts is now async (and queued) among a worker pool so that it is non-blocking. The cache internals have been tweaked so that multiple cache implementations (i.e. local and remote) can be multiplexed together in the future.
  • Improved file copying algorithm. turbo will first attempt to use hardlinks to copy files and fallback to a real copy if the hardlink fails. If you experience any regressions, please let us know. If necessary, we can add --no-hardlinks flag and/or "copyBehavior" config key to tune this (either globally or on a per-task basis).
  • Fixed glob bug in pipeline.<taskId>.outputs not respecting ! properly

v0.5.12

  • turbo is around 20% faster thanks to some concurrency/performance improvements to hashing

v0.5.11

  • turbo will now print a warning when .git is not present. It will continue with forced local execution and avoid caching any artifacts (the equivalent of --force and --no-cache flags)

v0.5.10

  • Fixed regression with --includeDependencies and --since flags in turbo run

v0.5.9

  • Pruned lockfile now includes correct file header and line breaks between entries

v0.5.8

  • You can now express package-specific tasks and implicit dependencies between tasks in pipeline like this:

    {
      "turbo": {
        "pipeline": {
          "build": {
            "dependsOn": ["^build"]
          },
          "test": {
            "dependsOn": ["build"]
          },
          "deploy": {
            "dependsOn": ["test"]
          },
          "frontend#deploy": {
            "dependsOn": ["ui#test", "backend#deploy", "backend#health-check"]
          }
        }
      }
    }

    In this example, we illustrate a deploy script of frontend application depends on both the deploy and health-check NPM scripts of backend as well as the test script of a ui package. The syntax is <package>#<task>.

    This seems like it goes against the "test": { "dependsOn": ["build"] } and "deploy": { "dependsOn": ["test"] }, but it does not. Since test and deploy scripts do not have topological dependencies (e.g. ^<task>), they theoretically can get triggered anytime once their own package's build and test scripts have finished!
    Although this feature is extremely powerful, unless they are being used for applications/services for CI/CD orchestration, the general guidance is to get rid of these specific package-task to package-task dependencies in the pipeline (so the turbo can be schedule more optimally).

    Note: Package-tasks do not inherit cache configuration. You must redeclare outputs at the moment.

v0.5.7

  • Fix regression with --scope flag on multi-cluster dependency graphs

v0.5.6

  • Improved lockfile parse performance. turbo now will hash/cache the converted SYML in the cache folder for faster subsequent reads. This hash is not included in task hashes, it's just used for determining if turbo should re-parse the SYML lockfile or not.
  • turbo prune now properly includes lockfile entries from root deps, devDeps, optionalDeps, and peerDeps. You should now safely be able to run yarn --frozen-lockfile inside of the out directory.
  • Performance improvements

v0.5.5

  • Deprecated turbo graph with an error message. This functionality has been moved into turbo run behind the --graph flag.
    If Graphviz is installed, this flag will generate an file visualizing the current task graph.
    The output file format can be controlled with the filename's extension if it is specified. If a filename is not specified, graph-<timestamp>.jpg is outputted. If Graphviz is not installed, this command prints the dot graph to stdout.

    turbo run build --graph
    turbo run build test lint --graph=my-graph.svg
    turbo run build test lint --graph=my-json-graph.json
    turbo run build test lint --graph=my-graph.pdf
    turbo run build test lint --graph=my-graph.png

v0.5.4

  • Fixed regression with --docker flag json directory

v0.5.3

  • Added back --concurrency flag from v0.3.x to limit task execution concurrency. The default is 10.
  • Deprecated --serial flag with a warning message. This will still work for now, but is simply equivalent to --concurrency=1.

v0.5.2

  • Adds back turbo prune behavior from v0.4.x (i.e. out/full, out/json, one sublockfile) via --docker flag. This output
    style is in fact optimal for Docker layer caching. However, the v0.5.x default is also useful when deploying to PaaS providers (e.g. Vercel). So now it can do both!

v0.5.1

  • Fixed regression when copying files and folders recursively

v0.5.0

Breaking changes

  • turbo prune now generates only one folder with all the source code in it (instead of 2). This makes the command more useful to people not using Docker.
  • turbo now includes pipeline.<task>.outputs in its hashing algorithm. Changing it should bust the cache. This is both more correct and also makes it easier to migrate to turbo (i.e. changing outputs yields different hashes meaning the cache is less likely to include unwanted stuff)
  • Added internal optionalDependencies to hash algorithm
  • Fixed regression where turbo's hashing algorithm was not including the OS platform (e.g. darwin/linux/windows) in global hash calculation

Improvements

  • Replaying cached logs no longer blocks graph / task execution resulting in a massive perf improvement
  • Fixed incorrect CLI usage text for turbo prune
  • Improved turbo prune error messages and error message CLI output

v0.4.11

  • Tweak to turbo CLI installation and authentication

v0.4.10

  • Alter install wrapper

v0.4.9

  • Improved error messages

v0.4.9

  • Tweak to turbo installation and authentication

v0.4.7

  • Fix cache outputs in pipelines to respect folders that start with dots

v0.4.6

  • Tweaked turbo installation script

v0.4.5

  • Fixed CLI panic caused by concurrent writes to log colorizer

v0.4.4

  • turbo now properly handles long and shorthand workspaces declarations in package.json

v0.4.3

  • Fixed lockfile parsing when dependencies use string version names (e.g. latest)

v0.4.2

  • Improved error handling and error messages

v0.4.1

  • turbo run will now throw an error if you attempt to run a target that doesn't exist in the pipeline.
  • Added back ">>> FULL TURBO" indicator when 100% of results are cached.

v0.4.0

Breaking changes

  • Logs are now output to .turbo directory instead of turbo directory. This moves the directory out of the middle of your package's source code. Update your .gitignore accordingly.
  • Removed support for NPM and PNPM (although we can add these back later)
  • The --concurrency has been removed. 9/10 times this was used to force serial execution with --concurrency=1.
    Now you can just pass --serial to get this behavior.
  • Turbo no longer includes the hash of the contents of the root lockfile in its hash algorithm.
    Instead, turbo now hashes the set of the resolved versions of a package's dependencies and devDependencies
    based on the root lockfile. The old behavior would explode the cache whenever the root lockfile changed. With this new behavior,
    changing the lockfile will only bust the cache for those package's impacted by the added/changed/removed dependencies. To get this to work, though, you must now specify all
    dependencies and devDependencies in each package.json
    .
  • environmentGlobs have been renamed to globalDependencies

What's New?

  • Completely rewritten from the ground up in Go!! turbo is now more than 10x faster.

  • Added prune command to allow outputting a pruned subset of your monorepo for a given project! Perfect
    for Docker layer caching.

  • Added a graph command that will print a .dot graph to the console and/or output to a file
    which you can view in Graphviz or online

  • We now validate your graph with more useful errors for cycles and multiple roots

  • Added pipeline key to turbo in package.json. This allows users to control the task graph topology
    and per-task options such as outputs. Right now, pipeline exists at the project level,
    but in later releases these will be overridable on per-package basis.

    // <root>/package.json
    {
      "turbo": {
        "pipeline": {
    
          "build": {
            // This `^` tells turbo that this pipeline target relies on a topological target being completed.
            // In english, this reads as: "this package's `build` command depends on its dependencies' or
            // devDependencies' `build` command being completed"
            "dependsOn": ["^build"]
            // You can now also specify outputs on a per-task basis. Logs will also be automatically
            // added to this, so you can omit them.
            "outputs": ["dist/**"]
          },
          "test": {
            // Without a `^`, `dependsOn` can be used to express the relationships between tasks at the package level.
            // In English, this reads as: "this package's `test` command depends on its `build` command first being completed"
            "dependsOn": ["build"]
          },
          "dev": {},
        }
      }
    }

Migration Guide

  • Move your config from turbo.json into package.json under the turbo key.
  • Add an object called pipeline that specifies the relationship of your project's tasks. turbo can now only run those tasks listed in pipeline object.
  • Replace usage of --concurrency=1 flags to just --serial
  • Since turbo now only hashes the resolved version of deps (and not the contents of the root lockfile), you need specify all devDependencies used by a package in the package itself.

v0.3.18

  • Fixed turbo run output showing false error with 0 exit code

v0.3.17

  • Added --continue flag to turbo run (defaults to false). This flag tells turbo whether or not to continue with execution in the presence of an error (i.e. non-zero exit code from a task).
    By default, specifying the --parallel flag will automatically set --continue to true unless explicitly set to false.
    When --continue is true, turbo will exit with the highest exit code value encountered during exection.

v0.3.16

  • Speed improvements

v0.3.15

  • cacheStorageOptions now default to local so they can be omitted in turbo configuration
  • Changed docs to suggest using package.json key turbo for configuration going forward. This will ease future migration to Go implementation.

v0.3.14

  • Fixed caching issue with Lerna due to turbo not properly reading packages list from lerna.json resulting in null hashes.
  • Top-level lint command now has specialized log treatment along with test where turbo will now only cache turbo/run-lint.log.
  • Added documentation around caching ESLint by setting TIMING=1 eslint ...

Full Changelog: v0.8.4...v0.8.5

Don't miss a new turborepo release

NewReleases is sending notifications on new releases.