spack/spack v0.13.0

on GitHub

v0.13.0 is our biggest Spack release yet, with many new major features.
From facility deployment to improved environments, microarchitecture
support, and auto-generated build farms, this release has features for all of
our users.

Spack grew by over 700 packages in the past year, and the project now has
over 450 contributors. Thanks to all of you for making this release possible.

Major new core features

• Chaining: use dependencies from external "upstream" Spack instances
• Environments now behave more like virtualenv/conda
  • Each env has a view: a directory with all packages symlinked in
  • Activating an environment sets PATH, LD_LIBRARY_PATH, CPATH,
    CMAKE_PREFIX_PATH, PKG_CONFIG_PATH, etc. to point to this view.
• Spack detects and builds specifically for your microarchitecture
  • named, understandable targets like skylake, broadwell, power9, zen2
  • Spack knows which compilers can build for which architectures
  • Packages can easily query support for features like avx512 and sse3
  • You can pick a target with, e.g. spack install foo target=icelake
• Spack stacks: combinatorial environments for facility deployment
  • Environments can now build cartesian products of specs (with matrix:)
  • Conditional syntax support to exclude certain builds from the stack
• Projections: ability to build easily navigable symlink trees environments
• Support no-source packages (BundlePackage) to aggregate related packages
• Extensions: users can write custom commands that live outside of Spack repo
• Support ARM and Fujitsu compilers

CI/build farm support

• spack release-jobs can detect package.py changes and generate
  .gitlab-ci.yml to create binaries for an environment or stack
  in parallel (initial support -- will change in future release).
• Results of build pipelines can be uploaded to a CDash server.
• Spack can now upload/fetch from package mirrors in Amazon S3

New commands/options

• spack mirror create --all downloads all package sources/resources/patches
• spack dev-build runs phases of the install pipeline on the working directory
• spack deprecate permanently symlinks an old, unwanted package to a new one
• spack verify checks that packages' files match what was originally installed
• spack find --json prints JSON that is easy to parse with, e.g. jq
• spack find --format FORMAT allows you to flexibly print package metadata
• spack spec --json prints JSON version of spec.yaml

Selected improvements

• Auto-build requested compilers if they do not exist
• Spack automatically adds RPATHs needed to make executables find compiler
  runtime libraries (e.g., path to newer libstdc++ in icpc or g++)
• setup-env.sh is now compatible with Bash, Dash, and Zsh
• Spack now caps build jobs at min(16, ncores) by default
• spack compiler find now also throttles number of spawned processes
• Spack now writes stage directories directly to $TMPDIR instead of
  symlinking stages within $spack/var/spack/cache.
• Improved and more powerful spec format strings
• You can pass a spec.yaml file anywhere in the CLI you can type a spec.
• Many improvements to binary caching
• Gradually supporting new features from Environment Modules v4
• spack edit respects VISUAL environment variable
• Simplified package syntax for specifying build/run environment modifications
• Numerous improvements to support for environments across Spack commands
• Concretization improvements

Documentation

• Multi-lingual documentation (Started a Japanese translation)
• Tutorial now has its own site at spack-tutorial.readthedocs.io
  • This enables us to keep multiple versions of the tutorial around

Deprecations

• Spack no longer supports dotkit (LLNL's homegrown, now deprecated module tool)
• spack build, spack configure, spack diy deprecated in favor of
  spack dev-build and spack install

Important package changes

• 3,563 total packages (718 added since 0.12.1)
• Spack now defaults to Python 3 (previously preferred 2.7 by default)
• Much improved ARM support thanks to Fugaku (RIKEN) and SNL teams
• Support new special versions: master, trunk, and head (in addition to develop)
• Better finding logic for libraries and headers