🐛 Fixes
Fix potential telemetry deadlock (PR #7142)
The tracing_subscriber crate uses RwLocks to manage access to a Span's Extensions. Deadlocks are possible when
multiple threads access this lock, including with reentrant locks:
// Thread 1 | // Thread 2
let _rg1 = lock.read(); |
| // will block
| let _wg = lock.write();
// may deadlock |
let _rg2 = lock.read(); |
This fix removes an opportunity for reentrant locking while extracting a Datadog identifier.
There is also a potential for deadlocks when the root and active spans' Extensions are acquired at the same time, if
multiple threads are attempting to access those Extensions but in a different order. This fix removes a few cases
where multiple spans' Extensions are acquired at the same time.
By @carodewig in #7142
Connection shutdown timeout (PR #7058)
When a connection is closed we call graceful_shutdown on hyper and then await for the connection to close.
Hyper 0.x has various issues around shutdown that may result in us waiting for extended periods for the connection to eventually be closed.
This PR introduces a configurable timeout from the termination signal to actual termination, defaulted to 60 seconds. The connection is forcibly terminated after the timeout is reached.
To configure, set the option in router yaml. It accepts human time durations:
supergraph:
connection_shutdown_timeout: 60s
Note that even after connections have been terminated the router will still hang onto pipelines if early_cancel has not been configured to true. The router is trying to complete the request.
Users can either set early_cancel to true
supergraph:
early_cancel: true
AND/OR use traffic shaping timeouts:
traffic_shaping:
router:
timeout: 60s
By @BrynCooke in #7058
Fix crash when an invalid query plan is generated (PR #7214)
When an invalid query plan is generated, the router could panic and crash.
This could happen if there are gaps in the GraphQL validation implementation.
Now, even if there are unresolved gaps, the router will handle it gracefully and reject the request.
By @goto-bus-stop in #7214
Improve Error Message for Invalid JWT Header Values (PR #7121)
Enhanced parsing error messages for JWT Authorization header values now provide developers with clear, actionable feedback while ensuring that no sensitive data is exposed.
Examples of the updated error messages:
- Header Value: '<invalid value>' is not correctly formatted. prefix should be 'Bearer'
+ Value of 'authorization' JWT header should be prefixed with 'Bearer'- Header Value: 'Bearer' is not correctly formatted. Missing JWT
+ Value of 'authorization' JWT header has only 'Bearer' prefix but no JWT tokenBy @IvanGoncharov in #7121