Embedded-AMS/EmbeddedProto 4.0.0b1

on GitHub

Version 4.0.0 is the largest release of Embedded Proto so far. It changes how the library is installed, adds maps and streaming, and supports the new protobuf editions. Your proto files and the code you wrote against generated messages keep working. What you have to change is listed on the page about migrating from 3.x to 4.0. This page lists what you gain.

Installation and code generation

• Install with pip or uv. Embedded Proto is a package on PyPI, and the protobuf compiler comes along with it. Installing protoc yourself, and matching its version to the Python library, is a thing of the past. The git submodule keeps working for those who prefer it. See installation.
• The embeddedproto command. One command generates your code. It runs protoc, adds the plugin and the include folders for you, and tells you where the C++ sources are with --cpp-src-location. See generating source code.
• An options file. The size of repeated, string, bytes and map fields can be set from a JSON file, without editing a proto file which is shared with other teams. The same file sets the extension of the generated headers. See options file.
• A header only library. There is nothing to compile besides your own code and the generated headers. The headers moved into a folder of their own, included as <EmbeddedProto/Name.h>, or all at once as <EmbeddedProto.h>. See installation.
• A version check. Each generated file records the version of the plugin which made it. Compiling it against a library of another major version is a compile error, instead of a mystery at run time.
• Your license header. With a commercial license you set a token once, and the plugin puts your license header on top of every generated file. Without a token the generated file states that its template code is licensed under the GPLv3. See installation.
• A clearer dual license. The license itself has not changed, its wording has. The open source version used to be described as for non-commercial projects, which the GPLv3 never required. The dividing line is now stated as it always was in the license text: open source products use Embedded Proto for free under the GPLv3, closed source products need a commercial license. The repository gained a LICENSE file which says so, every source file carries an SPDX license identifier, and the README lists the third-party software used during code generation and the fact that none of it ends up in your product. See the pricing FAQ.

New in the proto file

• Maps. A map<K, V> field is stored as a fixed number of entries and comes with the lookup functions you expect. See maps.
• Any. A google.protobuf.Any field holds a message of a type not fixed by your proto file, as a type url and bytes with a fixed length. See Any.
• Callback storage. A repeated, string, bytes or map field can be streamed through callbacks instead of stored in the message, for data larger than your RAM. See callback storage.
• Custom storage. The storage class of a field can be replaced with one of your own. See custom storage.
• Protobuf editions. Files with edition = "2023" or "2024" are supported, including delimited messages which serialize in a single pass. See protobuf editions.

New in the library

• Serializing in parts. A message can be serialized into, and deserialized from, a buffer smaller than the message. See serialization.
• Fixed-width fields as one block. A field of fixed32, fixed64, float or double, single or repeated, is copied to and from the buffer in a single call on little-endian targets. See packed fixed-width fields.
• Null terminated strings. With the NULL_TERMINATED_STRINGS define every string field reserves room for a terminator, so get_const() is always a valid C string. See strings.
• Resuming after an error. When deserialize() returns before the end of a message, a second call continues where it stopped. See deserialization.
• Faster nested messages. The size of a nested message is no longer calculated twice when serializing, which made deeply nested messages slow.

Smaller improvements

• A single element of a repeated field can be removed with erase(index).
• An enum field has a mutable_ function, like the other scalar fields.
• A repeated, string or bytes field with a maximum length of zero takes no storage, and indexing it stays in bounds.
• A string field compares with a C string using ==, and fits() tells you beforehand whether a C string fits.
• A ReadBufferFixedSize can be constructed from a list of bytes, which is handy in unit tests.
• A custom read buffer can pop a block of bytes at once, and a custom write buffer is asked to push one.
• The interface destructors are no longer virtual, which keeps malloc and free out of your firmware. See code size.