aws/aws-parallelcluster v3.3.0

AWS ParallelCluster v3.3.0

on GitHub

We're excited to announce the release of AWS ParallelCluster 3.3.0

Upgrade

How to upgrade?

sudo pip install --upgrade aws-parallelcluster

ENHANCEMENTS

• Add possibility to specify multiple EC2 instance types for the same compute resource.
• Add support for adding and removing shared storages at cluster update by updating SharedStorage configuration.
• Add new configuration parameter DeletionPolicy for EFS and FSx for Lustre shared storage to support storage retention.
• Add new configuration section Scheduling/SlurmSettings/Database to enable accounting functionality in Slurm.
• Add support for On-Demand Capacity Reservations and Capacity Reservations Resource Groups.
• Add new configuration parameter in Imds/ImdsSettings to specify the IMDS version to support in a cluster or build image infrastructure.
• Add support for Networking/PlacementGroup in the SlurmQueues/ComputeResources section.
• Add support for instances with multiple network interfaces that allows only one ENI per device.
• Improve validation of networking for external EFS file systems by checking the CIDR block in the attached security group.
• Add validator to check if configured instance types support placement groups.
• Configure NFS threads to be min(256, max(8, num_cores * 4)) to ensure better stability and performance.
• Move NFS installation at build time to reduce configuration time.
• Enable server-side encryption for the EcrImageBuilder SNS topic created when deploying ParallelCluster API and used to notify on docker image build events.

CHANGES

• Change behaviour of SlurmQueues/Networking/PlacementGroup/Enabled: now it creates a different managed placement
  group for each compute resource instead of a single managed placement group for all compute resources.
• Add support for PlacementGroup/Name as the preferred naming method.
• Move head node tags from Launch Template to instance definition to avoid head node replacement on tags updates.
• Disable Multithreading through script executed by cloud-init and not through CpuOptions set into Launch Template.
• Upgrade Python to version 3.9 and NodeJS to version 16 in API infrastructure, API Docker container and cluster Lambda resources.
• Remove support for Python 3.6 in aws-parallelcluster-batch-cli.
• Upgrade Slurm to version 22.05.5.
• Upgrade NVIDIA driver to version 470.141.03.
• Upgrade NVIDIA Fabric Manager to version 470.141.03.
• Upgrade NVIDIA CUDA Toolkit to version 11.7.1.
• Upgrade Python used in ParallelCluster virtualenvs from 3.7.13 to 3.9.15.
• Upgrade Slurm to version 22.05.5.
• Upgrade EFA installer to version 1.18.0.
• Upgrade NICE DCV to version 2022.1-13300.
• Allow for suppressing the SingleSubnetValidator for Queues.

BUG FIXES

• Fix validation of filters parameter in ListClusterLogStreams command to fail when incorrect filters are passed.
• Fix validation of parameter SharedStorage/EfsSettings: now validation fails when FileSystemId is specified
  along with other SharedStorage/EfsSettings parameters, whereas it was previously ignoring them.
• Fix cluster update when changing the order of SharedStorage together with other changes in the configuration.
• Fix UpdateParallelClusterLambdaRole in the ParallelCluster API to upload logs to CloudWatch.
• Fix Cinc not using the local CA certificates bundle when installing packages before any cookbooks are executed.
• Fix a hang in upgrading ubuntu via pcluster build-image when Build:UpdateOsPackages:Enabled:true is set.
• Fix parsing of YAML cluster configuration by failing on duplicate keys.