Highlights
🐛 No more "File descriptor N is used by transport"
A connection closed while it was still starting up (TLS handshake in progress) could leave its fd half-owned: the socket was closed, but the asyncio transport still claimed the fd number. The next socket to reuse that number failed with RuntimeError: File descriptor N is used by transport, and in a long-running process, a pytest session for example, every later connection could break.
ServiceConnection was reworked so this can't happen: a connection is bound to its streams when it is created, TLS is negotiated at creation, and the stream transport is the only owner of the socket. Closing during the TLS upgrade, a send or a receive now cancels cleanly. A TCP connection that finishes connecting just as its caller is cancelled is closed instead of leaked.
For library users: ServiceConnection is a low-level class, and its API changed. The sync methods, start(), ssl_start() and abort() are gone, and DTXConnection.from_socket was removed. Services built on LockdownService and the service providers are unaffected.
🐛 Image mounting on pre-iOS 14 devices
On iOS 13 and earlier, the image mounter serves one host connection at a time, and mounter auto-mount and other mounter commands left their connection open, so the next mounter call hung. Every mounter connection is now closed after use. mounter list falls back to LookupImage on devices without CopyDevices, the developer disk image downloads into the per-user cache instead of a root-owned Xcode bundle, and --xcode accepts a directory. doctor gives up on an image mounter that doesn't answer after 10 s instead of waiting forever.
🐛 Fixes
- The pre-iOS 14 DVT TLS handshake (complete it, then speak plaintext) now runs over the connection's own streams and waits for the close when it fails.
- A service connection whose SSL start fails is closed.
- Background tasks are started with
asyncio.create_task()and named, so they show up recognizably when debugging.
🧪 Socket hygiene in the test suite
Every test, device tests included, now fails if it leaves a socket, transport, loop watch or fd open, on every platform. Module- and session-scoped fixtures are checked at their own teardown. Device tests take --udid to pick the device.
pytest -m device --udid <UDID>What's Changed
- dcdb3ca tests: Run device tests under socket_hygiene (#2103) (@doronz88)
- 8b91047 tests: Select the test device with --udid (#2103) (@doronz88)
- b3fde3f tests: Check module- and session-scoped fixtures for leaked sockets (#2103) (@doronz88)
- ff7d72f tests: Track every socket a test creates in socket_hygiene (#2103) (@doronz88)
- 8ec51cd ci: Remove the CodeQL workflow (#2102) (@doronz88)
- 84e8c16 tests: Inspect sockets in socket_hygiene without opening an fd (#2101) (@doronz88)
- 222ca96 service_connection: Close a TCP dial that completes as its caller is cancelled (#2097) (@doronz88)
- 0108805 tests: Run every device-free test under socket_hygiene (#2097) (@doronz88)
- 4bdd1b0 service_connection: Wait for the close when the TLS strip fails (#2096) (@doronz88)
- 20c0f56 userspace_tunnel: Say how the tunnel's read task really ends (#2100) (@doronz88)
- 0e34b0f doctor: Give up on an image mounter that does not answer (#2099) (@doronz88)
- 4c6c501 mounter: Fix image mounting and listing on pre-iOS 14 devices (#2099) (@doronz88)
- 49de1f7 maintenance: Start tasks with asyncio.create_task(), not ensure_future() (#2098) (@doronz88)
- fb8e850 service_connection: Do the pre-iOS 14 TLS strip over the bound streams (#2096) (@doronz88)
- 5d5d4dc tests: Fail socket tests that leave an fd, transport or loop watch behind (#2095) (@doronz88)
- b9e01df dtx: Remove the unused DTXConnection.from_socket (#2095) (@doronz88)
- 5ccfb09 service_connection: Bind on creation and negotiate TLS there (#2095) (@doronz88)
- d2d3c73 lockdown, dtx: Close a service connection whose SSL start fails (#2094) (@doronz88)
- 89d21d8 service_connection: Cancel an in-flight start on close instead of closing its fd (#2094) (@doronz88)
Full Changelog: v11.42.0...v11.43.0