apache/spark-kubernetes-operator 0.2.0

on GitHub

Apache Spark™ K8s Operator is a subproject of Apache Spark and aims to extend K8s resource manager to manage Apache Spark applications via Operator Pattern. This release includes Apache Spark K8s Operator 0.2.0 (App Version) with Helm chart 1.0.0 (Chart Version).

Notable Features

• Support K8s v1.30, v1.31, v1.32, and v1.33
• Support Apache Spark 3.5 and 4.0 (RC7)
• Support SparkApp and SparkCluster CRDs
  • sparkapplications.spark.apache.org (v1alpha1)
  • sparkclusters.spark.apache.org (v1alpha1)
• Support HPA for SparkCluster
• Support Artifact Hub
  • https://artifacthub.io/packages/helm/spark-kubernetes-operator/spark-kubernetes-operator/
• Provide various examples

Docker Image

docker pull apache/spark-kubernetes-operator:0.2.0

Install Helm Chart

$ helm repo add spark https://apache.github.io/spark-kubernetes-operator
$ helm repo update
$ helm install spark spark-kubernetes-operator/spark-kubernetes-operator
$ helm list
NAME 	NAMESPACE	REVISION	UPDATED                            	STATUS  	CHART                          	APP VERSION
spark	default  	1       	2025-05-20 11:39:27.72567 -0700 PDT	deployed	spark-kubernetes-operator-1.0.0	0.2.0

Full Changelog:

• https://github.com/apache/spark-kubernetes-operator/commits/0.2.0

Resolved Issues

• [SPARK-52001] Bump version to 0.2.0-SNAPSHOT in main branch
• [SPARK-52003] Improve publish_snapshot_dockerhub GitHub Action job to support branches
• [SPARK-52004] Improve publish_snapshot_chart GitHub Action job to support branches
• [SPARK-52028] Update README.md with YuniKorn 1.6.3
• [SPARK-52043] Upgrade DropWizard to 4.2.30
• [SPARK-52044] Upgrade log4j2 to 2.24.3
• [SPARK-52045] Upgrade junit to 5.12.2
• [SPARK-52105] Provide Helm Chart repository
• [SPARK-52106] Update Helm Chart to use apache/spark-kubernetes-operator for image.repository
• [SPARK-52107] Add workflow_dispatch to pages.yml to support manual trigger
• [SPARK-52108] Enable GitHub Pages
• [SPARK-52115] Document Helm Chart
• [SPARK-52117] Add LICENSE, NOTICE, README.md to Helm Chart
• [SPARK-52118] Enforce Helm Chart configuration value verification
• [SPARK-52133] Set Helm chart version to 1.0.0-dev
• [SPARK-52135] Remove powermock-core dependency
• [SPARK-52136] Upgrade checkstyle to 10.23.1
• [SPARK-52137] Upgrade pmd to 7.13.0
• [SPARK-52139] Fix Helm chart icon to use PNG file
• [SPARK-52154] Update .asf.yaml to show Spark K8s operator website
• [SPARK-52161] Add home, sources, maintainers, keywords metadata to Chart.yaml
• [SPARK-52173] Fix OwnerReference Patching for Spark Application Driver Resources
• [SPARK-52190] Revise Helm chart README
• [SPARK-52194] Add spark-connect-server.yaml example
• [SPARK-52199] Remove Spark 3.4.3 bindings in spark-versions E2E test
• [SPARK-52200] Make StatusRecorder.objectMapper static
• [SPARK-52201] Revisit PMD annotations
• [SPARK-52202] Reduce PMD exclusion rules
• [SPARK-52206] Update GitHub Action YAML file in branch-0.2
• [SPARK-52209] Fix operatorContainer.env type
• [SPARK-52210] Fix watched-namespaces E2E test