apache/fory v0.11.0 on GitHub

Highlights

Add Dart Serialization support
Chunk-based map serialization for Java/Python with performance optimizations and XLang support
Cross-language homogeneous collection serialization between Java/Python
Refactor XLang type system and serialization for fory go
Performance improvements in Python (dict/tuple/list serialization, C-API optimization)
Row format evolution with support for Optional, Records, nested beans, and interface synthesis
Unify Java and XLang object serialization with cross-language compatibility for enums, time, arrays, and more
Renamed Apache Fury to Apache Fory

Breaking Changes

Renamed Apache Fury to Apache Fory, this transition impacts all project components including code repositories, maven groupId, jar names, package names, documentation, and communication channels and the source release name (and location), see Rename Announcement for more details.

Features

feat(java): ReplaceResolveSerializer deep copy by @zhaommmmomo in #1925
perf(python): Enhance the Python benchmark script by @penguin-wwy in #1926
perf(python): Improve tuple and list serializer performance by @penguin-wwy in #1933
feat(python): Implement collection serialization protocol by @penguin-wwy in #1942
perf(python): Pre-allocate size for the dictionary by @penguin-wwy in #1949
feat(java): use varint for jdk compatible serializers by @chaokunyang in #1960
feat(java): configurable buffer size limit by @theigl in #1963
perf(python): Directly access the key-value pairs of a dict by @penguin-wwy in #1970
feat(python): Hardcoding metastring into passable parameters by @pandalee99 in #1987
feat(java/python): new xlang type system spec implementation by @chaokunyang in #1690
feat(c++): Support the UTF-8 to UTF-16 with SIMD by @pandalee99 in #1990
feat(python): support latin1/utf16 string encoding in python by @chaokunyang in #1997
feat(c++): add simd accelerated c++ ascii/latin1 check funcion by @chaokunyang in #1999
perf(python): get object dict for faster field read/write by @chaokunyang in #2003
perf(python): optimize bytes buffer creation by @chaokunyang in #2008
feat(java): make 4 bytes utf16 size header optional for utf8 encoding by @chaokunyang in #2010
feat(ci): support building python on windows by @An-DJ in #1885
feat(java): Chunk by chunk predictive map serialization protocol by @Hen1ng in #1722
feat(java): support streaming encode/decode to/from buffer for row format by @chaokunyang in #2024
feat(c++): Add benchmark for performance evaluation by @pandalee99 in #2023
feat(java): new implementation and protocol refine for chunk based map serialization by @chaokunyang in #2025
feat(c++): BenchMark will be a Module by @pandalee99 in #2028
feat(java): jit support for chunk based map serialization by @chaokunyang in #2027
feat(java): Add basicMultiConfigFury dataprovider and add basic map unit test use the dataProvider by @Hen1ng in #2032
feat(python): unify type system between python and xlang serialization in pyfury by @chaokunyang in #2034
feat(java): deserialize one pojo into another type by @orisgarno in #2012
feat(python): chunk based map serialization for python by @chaokunyang in #2038
feat(python): add macos pyfury ci by @chaokunyang in #2041
feat(python): automatic release pyfury on macos and windows by @chaokunyang in #2045
feat(javascript): optimize string serializer by @theweipeng in #2043
feat(java): zstd meta compressor by @orisgarno in #2042
feat(python): support 3.13 by @penguin-wwy in #2046
feat(java): support register type by name in java by @chaokunyang in #2053
feat(spec): remove polymorphic from type id by @chaokunyang in #2054
feat(java): Add fastpath for collection/map serialize and deserialize by @Hen1ng in #2050
perf(python): upgrade pyarrow version to 15.0.0 by @pandalee99 in #2056
feat(python): chunk based map serialization in pure python by @pandalee99 in #2037
build(bazel): Extracts compile_commands.json by @penguin-wwy in #2059
feat(javascript): simplify the definition of object type description by @theweipeng in #2058
perf(java): Optimize Computational Efficiency of MetaStringEncoder::encodeGeneric by @LouisLou2 in #2072
feat(javascript): Impl xlang by @theweipeng in #2075
perf(python): Optimize sequence buffer writing via the c-api by @penguin-wwy in #2078
feat(javascript): Speed up string serializer by @theweipeng in #2083
feat(java): extra entry in disallowed.txt by @pjfanning in #2098
perf(java): Refactor ThreadPoolFury to improve performance by @moooonk in #2092
perf(java): Improve performance by using System.arraycopy to copy between byte arrays by @Cybermaxke in #2101
feat(java): use sha256 to check disallowed.txt tamper by @chaokunyang in #2102
perf(java): Refactor field sorting in StructSerializer to cache transformed field names and avoid redundant computation by @LouisLou2 in #2091
feat(java): support passed tracking ref meta when building serializers by @chaokunyang in #2113
feat(java): support nested bean in array/collection/map for row format by @chaokunyang in #2116
feat(java): support inconsistent registration by name/id by @chaokunyang in #2120
feat: xlang map chunk serialization between java/python by @chaokunyang in #2127
feat: xlang homogeneous collection serialization between java/python by @chaokunyang in #2130
feat(chore): bump version support kotlin by @chaokunyang in #2134
feat(dart): Add Fury Dart Support by @LouisLou2 in #2112
refactor(java): refactor object serializer for unifying xlang/java serialization in java by @chaokunyang in #2139
refactor(java): move methods from object serializer to abstract object serializer by @chaokunyang in #2140
feat(java): unify java and xlang object serialization by @chaokunyang in #2146
feat(java): support enum/time/array final types in xlang serialization by @chaokunyang in #2164
feat(java): FuryField annotation hints for struct serialization by @Hen1ng in #2036
feat(kotlin): add kotlin ci by @pandalee99 in #2176
perf(kotlin): Add multiple versions of kotlin CI by @pandalee99 in #2184
feat(dart): add test_config.yaml and apply TestConfig for environment-specific settings by @LouisLou2 in #2183
feat: add Dart to Language enums across all implementations by @LouisLou2 in #2187
feat(go): fury go implements new xlang by @pandalee99 in #2198
feat(java): add protobuf serializer for message and byte string by @chaokunyang in #2213
feat(spec): update type meta encoding by @chaokunyang in #2216
feat(java): type meta encoding for xlang in java by @chaokunyang in #2197
feat(java): support trackingRef in furyField by @Hen1ng in #2168
feat(java): add DescriptorBuilder for easy build and copying Descriptor by @Hen1ng in #2229
feat(java): Support furyField nullable in codeGen pattern by @Hen1ng in #2191
feat(java): row encoder supports custom types and collections by @stevenschlansker in #2243
test(benchmark): add msgpack serialization/deserialization to benchmark by @Kn0688 in #2242
feat(go): fury-go implements adaptation and optimization for new xlang by @pandalee99 in #2230
perf(c++): Evaluate the implementation effect &&simdutf performs partial vectorization by @pandalee99 in #2033
feat(java): fury row encoder now supports implementing interfaces with simple value type by @stevenschlansker in #2250
feat(java): row format supports Optional by @stevenschlansker in #2254
feat(java): row format supports Record types by @stevenschlansker in #2256
perf(go): Optimize the type registration system && Reduce useless code by @pandalee99 in #2262
feat(java): support meta shared serialization for xlang in java by @chaokunyang in #2276
feat(python): replace "_class" prefix with "_type" in metashare by @urlyy in #2290
feat(java): Adding explanatory message to deserializeNonexistentClass misconfiguration in Config by @drse in #2300
feat(java): row encoder supports synthesizing interfaces nested inside of records by @stevenschlansker in #2304
feat(java): Support customized serializer for abstract or interface. by @CherishCai in #2311
feat(java): row encoder supports custom rewriting values without changing their type by @stevenschlansker in #2305
feat(java): support multi-dimensional array field serialization in xlang meta shared mode by @OmCheeLin in #2314
feat(python): bump cython to 3.1.1 by @penguin-wwy in #2318

Bug Fix

fix(doc): set compressString default value to false by @chaokunyang in #1913
fix(java): Move schema caching to unsafe trait to avoid issues when using non-inferred schema. by @wywen in #1944
fix(java): ClassLoaderFuryPooled#setFactoryCallback cannot effect old Fury by @MrChang0 in #1946
fix(java): fix find constructor error in generated serializer class caused by duplicated class classloading for Fury by @chaokunyang in #1948
fix(c++): fix bazel install by @chaokunyang in #1979
fix(java): Fix flakiness in ExpressionVisitorTest#testTraverseExpression by @AmitPr in #1968
fix(java): only print warn message if scopedMetaShareEnabled is true … by @zhfeng in #1985
fix(java): Fix the issue caused by not using readCompressedBytesString during deserialization when string compression is enabled. by @Aliothmoon in #1991
fix(java): Compatible mode on de/serialize api failed to deserialize by @orisgarno in #1996
fix(java): chunk map serialize an error by @Hen1ng in #2030
fix(java): fix duplicate entry write at max chunk size bound by @chaokunyang in #2040
fix(java): Remove duplicate maven dependency by @Hen1ng in #2044
fix(python): Fix missing parameters and old API names by @pandalee99 in #2049
fix(java): fix read null chunk out of bound by @chaokunyang in #2065
fix(java): Fix error with MemoryBuffer::readBytesAsInt64 when not in LITTLE_ENDIAN mode by @LouisLou2 in #2069
fix(java): fix read primitives error on fill buffer bound by @chaokunyang in #2064
fix(java): java.util.Date and its subclasses are mutable by @moooonk in #2076
fix(java): fix ImmutableCollections$SubList duplicate registration by @w-yfan in #2074
fix(java): Modify some mistake by @Hen1ng in #2086
fix(python): inconsistent struct hash calculation between Java and Python by @LouisLou2 in #2108
fix(java): fix serialization npe of collection with all null elems by @chaokunyang in #2111
fix(java): fix not null value flag by @chaokunyang in #2114
fix(java): use registered id to sort fields by @chaokunyang in #2115
fix(java): fix disallowed.txt check in windows by @chaokunyang in #2128
fix(java): fix nested chunk map serialization error when generics exists by @chaokunyang in #2136
fix(java): fix DisallowedList calculate hash in Windows by @moooonk in #2142
fix(java): fix fury logger log exception by @chaokunyang in #2153
fix(java): fix xlang container field deserialization type error by @chaokunyang in #2161
fix(java): fix nested map chunk serialization codegen by @chaokunyang in #2172
fix(kotlin): fix kotlin compile error by @chaokunyang in #2173
fix(dart): fix inappropriate naming and align test titles by @LouisLou2 in #2180
fix(dart): replace string-based type selection with enum in FixedNum factory method by @LouisLou2 in #2185
fix(java): ensure readVarUint36Small reads full bits regardless of remaining buffer size by @LouisLou2 in #2179
test(java): add test for fix of readVarUint36Small behavior by @LouisLou2 in #2186
fix(python): fix pyfury build using pyproject.toml by @chaokunyang in #2206
fix(java): ensure FuryObjectInputStream.read never returns 0 when length>0 #2204 by @X-czh in #2205
fix(python): TimestampSerializer fails on Windows for naive datetimes near epoch by @LouisLou2 in #2209
fix(java): Fix empty string processing in MetaStringBytes by @LouisLou2 in #2212
fix(java): fix field super class missing in compatible mode by @chaokunyang in #2214
fix(java): mark fury-test-core as test dep in fury extensions by @chaokunyang in #2231
fix(java): MemoryBuffer getRemainingBytes can return unexpected results on a slice by @stevenschlansker in #2237
fix(java): use serialization binding by @Hen1ng in #2241
fix(C++): Fix reversed parameters and incorrect calls in string_util benchmark by @lzaeh in #2246
fix(java): fix row type enum properties by @stevenschlansker in #2258
fix(java): fix generics when write/read null map kv chunk by @chaokunyang in #2261
fix(java): row encoder incorrectly interprets type parameters as cycles by @stevenschlansker in #2265
fix(scala): Ensure singleton classes are initialized before reading by @creddy in #2267
fix(go): fix metastringbytes inconsistency issue by @lzaeh in #2269
fix(java): descriptors for beans should not include static methods by @stevenschlansker in #2281
fix(java): raise exception when registering invalid serializer for Map/List by @OmCheeLin in #2291
fix(java): xlang test skip needs correct import in newer python versions by @stevenschlansker in #2296
fix(java): row encoder interface synthesis handle (Optional) null correctly by @stevenschlansker in #2306
fix(java): Use (long, long, byte) key for MetaStringBytes cache to prevent collisions by @LouisLou2 in #2308
fix(python): update python benchmark script by @penguin-wwy in #2315

Other Improvements

chore(ci): Migrate the ci from macOS 12 to macOS 13 by @penguin-wwy in #1927
chore: bump release version to 0.9.0 by @chaokunyang in #1940
chore(Rust): fix lint problems by @theweipeng in #1962
docs(java): add object mapping example and tests by @chaokunyang in #1974
chore(python): drop py3.7 support by @chaokunyang in #1981
chore(c++): Function inlining and code tuning by @pandalee99 in #2002
style(cpp): rename logging level with "FURY_" prefix by @An-DJ in #2016
docs(java): update java serialization schema compatibility doc by @chaokunyang in #2047
docs: Update to latest version by @theweipeng in #2080
Update snapshot version by @chaokunyang in #2081
chore: remove required_linear_history by @chaokunyang in #2122
chore: fix license issue by @chaokunyang in #2137
chore(java): use the SHA256_HASH field value directly. by @LiangliangSui in #2144
chore(java): Update the content that needs to be corrected when reading the code. by @LiangliangSui in #2143
chore: bump release version to 0.10.1 by @chaokunyang in #2151
chore(dart): remove unused dart/fury-test/lib/main.dart by @LouisLou2 in #2181
chore: bump release version to 0.10.2 by @LiangliangSui in #2207
chore(java): reduce java test log by @chaokunyang in #2217
chore(.asf.yaml): enable rebase-and-merge button by @chaokunyang in #2220
chore(github): fix YAML warnings about spaces before comments by @stevenschlansker in #2225
chore(java): janino 3.1.12 by @stevenschlansker in #2226
chore(java): jdk 24 build support by @stevenschlansker in #2224
chore(javascript): remove nodejs 12 support by @chaokunyang in #2233
chore(java): gitignore testng output by @stevenschlansker in #2238
chore: fix clippy warning by @theweipeng in #2240
chore: Delete the redundant comments and add the document by @pandalee99 in #2244
chore: bump release version to 0.10.3 by @pandalee99 in #2252
chore: rename apache fury to apache fory by @chaokunyang in #2263
chore: keep fury for maven artifact by @chaokunyang in #2264
chore(java): skip xlang tests when pyfory is not available by @stevenschlansker in #2266
chore(java): Use crossLanguage instead of language in Fory by @CherishCai in #2274
chore(java): fix xlang test skip in the case where there is no Python at all by @stevenschlansker in #2280
chore(java): gitignore test-output from integration_tests as well as java by @stevenschlansker in #2282
chore: update fory rename in README.md by @chaokunyang in #2292
chore: Some Fory changes in README.md by @pjfanning in #2295
chore: update fory logo by @chaokunyang in #2298
chore(java): skip find customized serializer when absClassInfo is empty by @CherishCai in #2316

New Contributors

@theigl made their first contribution in #1963
@AmitPr made their first contribution in #1968
@zhfeng made their first contribution in #1985
@Hen1ng made their first contribution in #1722
@LouisLou2 made their first contribution in #2069
@moooonk made their first contribution in #2076
@w-yfan made their first contribution in #2074
@Cybermaxke made their first contribution in #2101
@X-czh made their first contribution in #2205
@stevenschlansker made their first contribution in #2225
@Kn0688 made their first contribution in #2242
@lzaeh made their first contribution in #2246
@creddy made their first contribution in #2267
@CherishCai made their first contribution in #2274
@OmCheeLin made their first contribution in #2291
@drse made their first contribution in #2300

Full Changelog: v0.10.3...v0.11.0-rc2