Bug fixes
-
Fixed the C extension reading freed memory on free-threaded builds when a
list handed to :py:meth:~multidict.MultiDict.update,
:py:meth:~multidict.MultiDict.extend, :py:meth:~multidict.MultiDict.merge
or the :py:class:~multidict.MultiDictand :py:class:~multidict.CIMultiDict
constructors, a[key, value]item inside any iterable handed to them, or a
list tested withinagainst :py:meth:~multidict.MultiDict.items, is
changed by another thread; a call that catches the list shrinking under it
now raises :py:exc:RuntimeError-- by :user:rodrigobnogueira.Related issues and pull requests on GitHub:
#1437. -
Fixed a data race on the free-threaded build where a retired hash table's
reader count used relaxed atomics, letting a lock-freeget()/getone()/
__getitem__()read race a concurrent free of that table. The reader-exit
decrement and the drain's free check now use release/acquire ordering
instead -- by :user:asvetlov.Related issues and pull requests on GitHub:
#1481. -
Fixed a free-threaded build bug where two threads calling
update(),
merge(), or__setitem__()on the same key at the same time could lose
the key entirely instead of just racing on which value wins. A decref of the
replaced value could transiently suspend the writer's critical section,
letting a second writer for the same key observe the first writer's
in-progress entry as absent and, once both settled, mistake it for a stale
duplicate and delete it. Every such decref is now deferred until the writer
has released its critical section, so the window can no longer open.
setdefault()had an unrelated instance of the same blind spot (it could
insert a duplicate rather than recognizing an in-flight key), fixed alongside
it -- by :user:asvetlov.Related issues and pull requests on GitHub:
#1483. -
Fixed a free-threaded build bug where
getall()and theitems()/
keys()/values()equality path could raiseKeyErroror report a
present, never-deleted key as missing. A concurrentupdate()/extend()/
__setitem__()call can have its critical section transiently suspended
(a decref triggering a blocking allocator call) while an entry is marked as
part of its own bookkeeping; a reader landing in that window used to treat
the mark as "not found" instead of "still there, in flight" -- by
:user:asvetlov.Related issues and pull requests on GitHub:
#1484. -
Fixed a reference leak in the C extension where
operand | md.items()
andmd.items() - operandleaked one key and one value reference per
element ofoperand, letting a large operand grow memory without bound
(:gh:GHSA-54p9-h82j-f925 <aio-libs/multidict/security/advisories/GHSA-54p9-h82j-f925>)
-- by :user:asvetlov.The issue was reported by :user:
waydeshi.Related commits on GitHub:
:commit:350b4a0. -
Fixed a segmentation fault on the standard (non-free-threaded) C extension
build when a value type's__del__released the GIL (for example by
callingtime.sleep()) whileupdate(),merge(),__setitem__(),
__delitem__(),pop(),popone(), orpopall()was dropping a
replaced or removed value.Py_BEGIN_CRITICAL_SECTIONcompiles to a no-op
on this build, so nothing else was stopping a second thread from mutating the
very sameMultiDictconcurrently once the GIL was released mid-mutation.
Every such decref is now deferred until the mutation has fully finished, the
same technique already used to close the analogous free-threaded-build race
-- by :user:asvetlov.Related issues and pull requests on GitHub:
#1489. -
Fixed the C extension reading freed memory while iterating a
:py:class:~multidict.CIMultiDictwhose keys are plain :py:class:str.
Converting such a key to :py:class:~multidict.istrcould run Python code
(a :py:class:strsubclass's__str__or__del__) or, on free-threaded
builds, suspend the iterator's critical section, after which the iterator read
the entry again even though a concurrent mutation could already have freed it.
As part of the fix, :py:meth:~multidict.MultiDict.copyand re-initializing
from another multidict now assign a new version in the C extension instead of
reusing the source's, matching the pure Python implementation
-- by :user:asvetlov.Related issues and pull requests on GitHub:
#1496. -
Fixed a use-after-free on the free-threaded build where a lock-free
get(),[]orincould read a hash table that a concurrent
resize had just retired and another reader was freeing
-- by :user:asvetlov.Related issues and pull requests on GitHub:
#1497.
Contributor-facing changes
-
Removed a redundant
includeand a duplicatedexcludeline from
MANIFEST.in; sdist contents are unchanged -- by :user:asvetlov.Related issues and pull requests on GitHub:
#1478. -
Added
.claudeignorefile -- by :user:asvetlovRelated issues and pull requests on GitHub:
#1479. -
Scaled up the pure-Python
pop(),popitem(),__delitem__(),
add()and item-insertion benchmarks to do more work per measurement.
Repeated CodSpeed runs on the same commit showed these particular
benchmarks flagged as dominated by syscalls, understating their real
cost and adding noise to the reported values; a larger working set
amortizes that overhead -- by :user:asvetlov.Related issues and pull requests on GitHub:
#1485. -
Replaced deprecated instrumentation codspeed mode with simulation -- by :user:
asvetlovRelated issues and pull requests on GitHub:
#1493. -
Reorganized the mutating benchmarks (item insertion,
update(),
add()of the same key,pop(),popitem(),clear(),
__delitem__()and__setitem__()) to copy a fresh multidict and
apply the operation in a loop, like theadd()andextend()
benchmarks already do. The insertion,update()andclear()
benchmarks previously mutated a single multidict shared across
rounds, so only the first round measured the intended operation; the
rest did a single copy per round, letting per-round overhead dominate
-- by :user:asvetlov.Related issues and pull requests on GitHub:
#1494. -
The
repr()and view inequality benchmarks were updated to repeat their
operation in a loop, like the other benchmarks, and the CodSpeed benchmark
job was moved to Python 3.14 -- by :user:asvetlov.Related issues and pull requests on GitHub:
#1498. -
Dropped
-Ifrom the AddressSanitizer test command inAGENTS.md
and in the CI job. It implies-E, which made Python ignore
PYTHONMALLOC=malloc, so small hash tables were still served from
pymallocarenas where use-after-free went undetected
-- by :user:asvetlov.Related issues and pull requests on GitHub:
#1499. -
The CI/CD workflow was updated to stop superseded runs of the same pull request
when a new commit is pushed; runs onmaster, release branches, tags,
the merge queue, and the daily schedule are never interrupted
-- by :user:asvetlov.Related issues and pull requests on GitHub:
#1500. -
The release job was changed to upload distributions and their signatures
to the GitHub Release one file at a time, skipping assets that were
already attached and retrying after a pause, so that a parallel upload
burst no longer tripped the GitHub secondary rate limit
-- by :user:asvetlov.Related issues and pull requests on GitHub:
#1503.
Miscellaneous internal changes
-
Corrected several comments in the free-threaded C extension that
attributed critical-section suspension to a blockingPyMem_Malloc()
call; allocation alone never suspends an acquired critical section, and
the real risk at those sites is a decref running
a finalizer or weakref callback. Also dropped a retry loop in
md_clone_from_ht()that guarded against the same, non-existent
allocation-triggered suspension -- by :user:asvetlov.Related issues and pull requests on GitHub:
#1486. -
Changed the free-threaded build's deferred decref buffer, used by
update()and__setitem__(), to a chain of fixed-size blocks
with a large inline first block instead of a small inline array that
was reallocated on growth -- by :user:asvetlov.Related issues and pull requests on GitHub:
#1501.