Azure HDInsight release notes

This article provides information about the most recent Azure HDInsight release updates.

Summary

Azure HDInsight is one of the most popular services among enterprise customers for open-source analytics on Azure.

Release date: December 12, 2022

This release applies to HDInsight 4.0. and 5.0 HDInsight release is made available to all regions over several days.

HDInsight uses safe deployment practices, which involve gradual region deployment. It may take up to 10 business days for a new release or a new version to be available in all regions.

OS versions

• HDInsight 4.0: Ubuntu 18.04.5 LTS Linux Kernel 5.4
• HDInsight 5.0: Ubuntu 18.04.5 LTS Linux Kernel 5.4

For workload specific versions, see here.

What's New?

• Log Analytics - Customers can enable classic monitoring to get the latest OMS version 14.19. To remove old versions, disable and enable classic monitoring.
• Ambari user auto UI logout due to inactivity. For more information, see here
• Spark - A new and optimized version of Spark 3.1 is included in this release.

New Regions

• Qatar Central
• Germany North

What's changed?

• HDInsight has moved away from Azul Zulu Java JDK 8 to Adoptium Temurin JDK 8, which supports high-quality TCK certified runtimes, and associated technology for use across the Java ecosystem.

• HDInsight has migrated to reload4j. The log4j changes are applicable to

  • Apache Hadoop
  • Apache Zookeeper
  • Apache Oozie
  • Apache Ranger
  • Apache Sqoop
  • Apache Pig
  • Apache Ambari
  • Apache Kafka
  • Apache Spark
  • Apache Zeppelin
  • Apache Livy
  • Apache Rubix
  • Apache Hive
  • Apache Tez
  • Apache HBase
  • Apache OMI
  • Apache Pheonix

Updated

HDInsight will implement TLS1.2 going forward, and earlier versions will be updated on the platform. If you're running any applications on top of HDInsight and they use TLS 1.0 and 1.1, upgrade to TLS 1.2 to avoid any disruption in services.

For more information, see How to enable Transport Layer Security (TLS)

End of Support

End of support for Azure HDInsight clusters on Ubuntu 16.04 LTS from 30 November 2022. HDInsight had begun release of cluster images using Ubuntu 18.04 from June 27, 2021. We recommend our customers who are running clusters using Ubuntu 16.04 is to rebuild their clusters with the latest HDInsight images by 30 November 2022.

For more information on how to check Ubuntu version of cluster, see here

1. Execute the command “lsb_release -a” in the terminal.

2. If the value for “Description” property in output is “Ubuntu 16.04 LTS”, then this update is applicable to the cluster.

Bug fixes

• Support for Availability Zones selection for Kafka and HBase (write access) clusters.

Open-source bug fixes

Hive bug fixes