4.21.0 (2023-08-09)
Features Added
- Spark 3.4 support: - See PR 35176.
- Added a new configuration setting
spark.cosmos.write.bulk.initialBatchSizeto allow specifying the initial micro batch size for bulk operations. The batch size will be tuned automatically based on the throttling rate afterwards - by default it starts initially with 100 documents per batch. This can lead to exceeding the requested throughput when using throughput control in the first few seconds of a Spark job. This usually isn't a problem - but if there is the desire to avoid this, reducing the initial micro batch size - for example setting it to1- would avoid the initial spike in RU/s usage. - See PR 36068 - Added new strategy
ItemBulkUpdateto allow patch with more than 10 columns - See PR 35977
Bugs Fixed
- Fixed schema reference issue for empty array - See PR 35746