Bug Fixes
-
assure excessive amounts of newlines can't lead to amplification.
Now we won't read more than 1024 newlines in a row, which leads to
a protection from specifically crafted configuration files which
can amplify themselves when large amounts of edits happen on them.If somebody where to create a lot of sections based on one that
has a huge amount of newlines before it, this whitespace would
be retained with each new section, causing huge files to be created
in memory that cause great delays when writing the file back
and re-reading it.Maybe there would have been a way to avoid copying excessive amounts
of whitespace when altering a section, or maybe one could also
have adjusted the fuzz-test that found it [1].
This would, however, have been much harder and time-consuming to
implement for dubious value.
Bug Fixes (BREAKING)
-
greatly reduce peak memory usage when reading files.
Previously it was possible to cause a memory consumption of 2.5GB
by merely reading a specifically tuned configuration file sized a little
less than 1MB, a massive 2500x amplification.Data structures have been optimized and premature optimizations removed
in favor of much more efficient memory usage.Now the same file causes a 36MB peak memory usage, still quite a lot,
but also ~70x less than before.
Commit Statistics
- 11 commits contributed to the release over the course of 20 calendar days.
- 20 days passed between releases.
- 2 commits were understood as conventional.
- 0 issues like '(#ID)' were seen in commit messages
Commit Details
view details
- Uncategorized
- Prepare changelogs prior to release (6a2e0be)
- Limit the amount of work done to avoid the creation of huge file when fuzzing the section API (4138902)
- Release gix-trace v0.1.7, gix-features v0.37.2, gix-commitgraph v0.23.2, gix-traverse v0.36.2, gix-index v0.28.2 (b6c04c8)
- Assure excessive amounts of newlines can't lead to amplification. (b382f76)
- Merge branch 'fix-fuzzer' (17a81c7)
- Add test for memory usage in worse-case scenario (6bb407f)
- Greatly reduce peak memory usage when reading files. (c71d16e)
- Optimize gix-config fuzzer performance (36f4d92)
- Merge branch 'fuzz-gix-config' (34e4a16)
- Add gix_config::File fuzzer dictionary (2420547)
- Fuzz more of mutable gix_config::File API (9cdb461)