aws/aws-parallelcluster v2.11.0

AWS ParallelCluster v2.11.0

on GitHub

We're excited to announce the release of AWS ParallelCluster 2.11.0

Upgrade

How to upgrade?

sudo pip install --upgrade aws-parallelcluster

ENHANCEMENTS

• Add support for Ubuntu 20.04.
• Add support for using FSx Lustre in subnet with no internet access.
• Add support for building custom Centos 7 AMIs on ARM.
• Add support for FSx Lustre DataCompressionType feature.
• Add validation to prevent using a cluster_resource_bucket that is in a different region than the cluster.
• Install SSM agent on CentOS 7 and 8.
• Add support for security_group_id in packer custom builders. Customers can export AWS_SECURITY_GROUP_ID environment variable to specify security group for custom builders when building custom AMIs.
• SGE: always use shortname as hostname filter with qstat. This will make nodewatcher more robust when using custom DHCP option, where the full hostname seen by SGE might differ from the hostname returned from EC2 metadata(local-hostname).
• Transition from IMDSv1 to IMDSv2.

CHANGES

• Removed support for Ubuntu 16.04 (ubuntu1604).
• Removed support for Amazon Linux (alinux). Amazon Linux 2 (alinux2) remains fully supported.
  Amazon Linux is no longer supported.
• Make key_name parameter optional to support cluster configurations without a key pair.
• Remove support for Python versions < 3.6.
  • Remove dependency on future package and __future__ module.
• Root volume size increased from 25GB to 35GB on all AMIs. Minimum root volume size is now 35GB.
• Add sanity check to prevent cluster creation in an AWS region not officially supported by ParallelCluster.
• Restrict IAM permissions to only allow cluster IAM instance role to launch instances via run-instances in cluster compute subnet.
• Upgrade EFA installer to version 1.12.2
  • EFA configuration: efa-config-1.8-1 (from efa-config-1.7)
  • EFA profile: efa-profile-1.5-1 (from efa-profile-1.4)
  • EFA kernel module: efa-1.12.3 (from efa-1.10.2)
  • RDMA core: rdma-core-32.1amzn (from rdma-core-31.2amzn)
  • Libfabric: libfabric-1.11.2amzon1.1-1 (from libfabric-1.11.1amzn1.0)
  • Open MPI: openmpi40-aws-4.1.1-2 (from openmpi40-aws-4.1.0)
• Upgrade Slurm to version 20.11.7.
  • Update slurmctld and slurmd systemd unit files according to latest provided by slurm.
  • Add new SlurmctldParameters, power_save_min_interval=30, so power actions will be processed every 30 seconds.
  • Add new SlurmctldParameters, cloud_reg_addrs, which will reset a node's NodeAddr automatically on power_down.
  • Specify instance GPU model as GRES GPU Type in gres.conf, instead of previous hardcoded value Type=tesla for all GPU.
• Upgrade Arm Performance Libraries (APL) to version 21.0.0.
• Upgrade NICE DCV to version 2021.1-10557.
• Upgrade NVIDIA driver to version 460.73.01.
• Upgrade CUDA library to version 11.3.0.
• Upgrade NVIDIA Fabric manager to nvidia-fabricmanager-460.
• Install ParallelCluster AWSBatch CLI in dedicated python3 virtual env.
• Upgrade Python version used in ParallelCluster virtualenvs from version 3.6.13 to version 3.7.10.
• Upgrade Cinc Client to version 16.13.16.
• Upgrade third-party cookbook dependencies:
  • apt-7.4.0 (from apt-7.3.0)
  • iptables-8.0.0 (from iptables-7.1.0)
  • line-4.0.1 (from line-2.9.0)
  • openssh-2.9.1 (from openssh-2.8.1)
  • pyenv-3.4.2 (from pyenv-3.1.1)
  • selinux-3.1.1 (from selinux-2.1.1)
  • ulimit-1.1.1 (from ulimit-1.0.0)
  • yum-6.1.1 (from yum-5.1.0)
  • yum-epel-4.1.2 (from yum-epel-3.3.0)
• Drop lightdm package install from Ubuntu 18.04 DCV installation process.

BUG FIXES

• Use ICP-compliant AL2 repo URLs when building Docker images in China
• Fix a bug that caused clustermgtd to not immediately replace instances with failed status check that are in replacement process.