This release improves performance for JSON, adds support for the new protobuf edition 2026, and adds an option to allow depending on protobuf-py generated code published as a package, such as a BSR generated SDK, along with other miscellaneous enhancements.
JSON parse accepts duplicate fields
Most JSON parsers accept objects which include duplicate keys, with the last key winning, and the spec for ProtoJSON has also documented this behavior as recommended for some time. However, conformance tests did not allow this behavior and would fail, so per our policy of full conformance, we had to implement rejection of duplicate fields, which can be unintuitive and have poor performance. We have worked with the protobuf team to update conformance tests to match the spec, so they pass for implementations accepting duplicate keys. With that, we have also updated our implementation to accept duplicate fields, with last-in-wins semantics.
The most obvious benefit of this is improved performance for all JSON objects as we do not have to track keys anymore to find duplicates. With this, our benchmarks show consistently higher performance than Pydantic across. The gap is not large, generally less than 10% - performance is not a reason to pick Protobuf over Pydantic but we hope that users can enjoy the benefits of the Protobuf ecosystem without worrying about regressions.
Edition 2026 support
protobuf-py now supports the latest Protobuf Edition 2026. The notable new feature is the ability to customize the JSON string for enum values, not just field names as before. This should allow even more existing JSON schemas to be mappable to proto, for example when migrating to Connect.
edition = "2026";
import "google/protobuf/json_enumvalue_options.proto";
enum Season {
SEASON_UNSPECIFIED = 0;
SEASON_SPRING = 1 [(pb.enumvalue.json).string = "primavera"];
SEASON_SUMMER = 2 [(pb.enumvalue.json).string = "estate"];
SEASON_FALL = 3;
}
message JsonEnumNames {
Season season_field = 1;
}rewrite_imports option for protoc plugins
protoc-gen-py and other plugins based on protobuf-py by default use relative imports for proto dependencies. This conveniently allows outputting code to any location, for example a package called gen. But it prevented being able to reference proto dependencies you may not be generating, for example those in a BSR generated SDK dependency. The workaround for this has to always generate for proto dependencies using include_imports, accepting potential duplication of gencode.
This release adds a flexible rewrite_imports option that can be used to map default relative imports to absolute imports for referencing other packages. For example, if using protovalidate, you can replace rewrite_imports with a dependency on the bufbuild-protovalidate-bufbuild-py generated SDK and point imports at it.
plugins:
- local: protoc-gen-py
out: src/gen
opt: rewrite_imports=./buf/validate/**/*_pb.py:We will continue to explore options to allow this to be more automatic without glob expressions but hope this enables more convenient usage of protobuf-py by enabling the use of published packages.
📈 Enhancements
- Implement eq in native by @anuraaga in #62
- Remove handling of duplicates in JSON parsing by @anuraaga in #56
- Add DescFile dependencies to registry automatically by @anuraaga in #64
- Wire in typing_extensions and set Message generic parameter default by @anuraaga in #79
- Add edition 2026 support and pure-python depth limit to JSON parse by @anuraaga in #73
- Accept any buffer for from_binary by @anuraaga in #85
- Add rewrite_imports option by @anuraaga in #70
Full Changelog: v0.3.0...v0.4.0