4.6.1 (2022-02-11)
Key Bug Fixes
- Fixed a regression introduced in 4.5.0 that could result in returning incomplete query results when Executors are under high CPU load. - See PR 26991
New Features
- Added support for reading from a Cosmos table without schema inference and maintaining system properties
_ts
and_etag
when writing this data to another Cosmos container. This is helpful when moving data from one container to another without always consistent schema of the documents in the source container. - See PR 26820 - Added support for new
spark.cosmos.write.strategy
valueItemOverwriteIfNotModified
, which will allow only updating documents that haven't been modified since reading them (optimistic concurrency). - See PR 26847 - Added support for correlating queries executed via the Cosmos Spark connector with service-telemetry based on the
correlationActivityId
. - See PR 26908 - Improved direct transport configuration in Spark to reduce number of transient failures under very high CPU/memory pressure on Spark Executors. - See PR 27021