apache/spark-kubernetes-operator 0.6.0

on GitHub

Apache Spark™ K8s Operator is a subproject of Apache Spark and aims to extend K8s resource manager to manage Apache Spark applications via Operator Pattern. This release includes Apache Spark K8s Operator 0.6.0 (App Version) with Helm chart 1.4.0 (Chart Version).

Notable Features

• Support K8s v1.32, v1.33, and v1.34
• Support Apache Spark 3.5, 4.0, 4.1.0-preview3
• Support SparkApp and SparkCluster CRDs
  • sparkapplications.spark.apache.org (v1)
  • sparkclusters.spark.apache.org (v1)
• Support HPA for SparkCluster
• Support Artifact Hub
  • https://artifacthub.io/packages/helm/spark-kubernetes-operator/spark-kubernetes-operator/
• Provide various examples

Docker Image

docker pull apache/spark-kubernetes-operator:0.6.0

Install Helm Chart

$ helm repo add spark https://apache.github.io/spark-kubernetes-operator
$ helm repo update
$ helm install spark spark-kubernetes-operator/spark-kubernetes-operator
$ helm list
NAME   NAMESPACE  REVISION UPDATED                              STATUS   CHART                           APP VERSION
spark  default    1        2025-11-07 10:31:35.82152 -0800 PST	deployed spark-kubernetes-operator-1.4.0 0.6.0

Full Changelog:

• https://github.com/apache/spark-kubernetes-operator/commits/0.6.0

Resolved Issues

• SPARK-53725 Bump version to 0.6.0-SNAPSHOT in main branch
• SPARK-53726 Release Spark Kubernetes Operator 0.6.0
• SPARK-53739 Add pi-preview-with-eventlog.yaml example
• SPARK-53740 Update (pi|spark-history-server)-preview.yaml to use 4.1.0-preview2
• SPARK-53767 Upgrade Dropwizard metrics to 4.2.33
• SPARK-53768 Upgrade lombok to 1.18.42
• SPARK-53769 Enable Annotation Processing during Java compilation
• SPARK-53770 Upgrade mockito to 5.20.0
• SPARK-53778 Use JDK25 Gradle Image as builder images
• SPARK-53780 spark-submission-worker should depend on io.fabric8 directly
• SPARK-53781 Exclude Spark's transitive dependencies consistently across modules
• SPARK-53782 Use slf4j-api as a direct dependency
• SPARK-53783 Use log4j2 instead of log4j
• SPARK-53787 Upgrade Spark to 4.1.0-preview2
• SPARK-53813 Recover Publish Snapshot Chart GitHub Action Job by using GitHash
• SPARK-53814 Use Java List.of instead of Collections.(empty|singleton)List
• SPARK-53815 Remove branch-0.4 from daily publish_snapshot_* GitHub Action jobs
• SPARK-53817 Use o.a.s.util.Pair instead of o.a.commons.lang3.tuple.Pair
• SPARK-53818 Improve SentinelManager.toString by JEP-280 instead of ToStringBuilder
• SPARK-53820 Introduce o.a.s.k8s.operator.utils.StringUtils
• SPARK-53821 Ban org.apache.commons.lang3 package
• SPARK-53824 Ban org.apache.commons.collections4 package
• SPARK-53825 Use Java MessageDigest instead of org.apache.commons.codec
• SPARK-53826 Use Java 25 in GitHub Action jobs consistently
• SPARK-53827 Exclude commons-(codec|compress) and com.ibm.icu transitive dependencies
• SPARK-53828 Exclude org/apache/spark/ui/static/** from Spark operator jar
• SPARK-53830 Exclude .idea directory from Spotless check
• SPARK-53851 Set io.netty.noUnsafe to true to avoid JEP-498 warnings
• SPARK-53852 Add spark.kubernetes.driver.pod.excludedFeatureSteps usage example
• SPARK-53853 Add Example section in operators.md
• SPARK-53863 Remove Java 17 requirement from deploy.gradle
• SPARK-53869 Support multiple files in pyFiles
• SPARK-53871 Upgrade JUnit to 6.0.0
• SPARK-53872 Revisit JUnit assert usage in test cases
• SPARK-53874 SparkAppDriverConf should respect sparkVersion of SparkApplication CRD
• SPARK-53899 Update Operator SDK to 5.1.4
• SPARK-53909 Fix RBAC to allow Spark driver to create StatefulSet
• SPARK-53910 Add StatefulSet-based SparkApp example
• SPARK-53911 Support SPARK_VERSION placeholder in container image names
• SPARK-53912 Add cluster-preview.yaml and spark-connect-server-preview.yaml
• SPARK-53932 Add FAIR schedule examples with Spark (Connect|Thrift) Servers
• SPARK-53933 Add Apache Iceberg example
• SPARK-53983 Increase s.k.o.reconciler.foregroundRequestTimeoutSeconds to 60s
• SPARK-53984 Use CRD v1 instead of v1beta1 in benchmark script
• SPARK-53985 Suppress StatusRecorder warning messages
• SPARK-53987 Upgrade JaCoCo to 0.8.14 for official Java 25 support
• SPARK-53988 Upgrade Spotless to 8.0.0
• SPARK-53989 Optimize sparkapps.sh to use kubectl delete --all
• SPARK-53990 Use Java (Map|Set).of instead of Collections.(empty|singleton)(Set|Map)
• SPARK-53992 Add SparkOperatorConfManager.getAll method
• SPARK-53993 Support spark.logConf configuration
• SPARK-53994 Exclude KerberosConfDriverFeatureStep during benchmarking
• SPARK-54010 Support restart counter in Spark app
• SPARK-54070 Add spark.logConf configuration to operator docs
• SPARK-54073 Improve ConfOptionDocGenerator to generate a sorted doc by config key
• SPARK-54081 Add word-count-preview.yaml Example
• SPARK-54082 Upgrade Spark to 4.1.0-preview3
• SPARK-54092 Use Java-friendly KubernetesClientUtils APIs
• SPARK-54097 Upgrade Gradle to 9.2.0
• SPARK-54107 Use 4.1.0-preview3-java21-scala image for preview examples
• SPARK-54120 Update assertGeneratedCRDMatchesHelmChart to include diff
• SPARK-54167 Update GitHub Action YAML file in branch-0.6