benoitc/gunicorn 27.0.0

gunicorn 27.0.0

on GitHub

Breaking Changes

• Reject with 400 a request whose Transfer-Encoding does not end with
  chunked, such as gzip or identity alone, with or without
  Content-Length (RFC 9112 §6.3). The WSGI workers used to read such a body
  until the connection closed and the ASGI worker treated it as empty. Clients
  that send Transfer-Encoding: identity must stop sending it.

Bug Fixes

• Reject whitespace between a header name and the colon (RFC 9112 §5.1).
• Trim only spaces and tabs around Transfer-Encoding values (RFC 9110 §5.6.3)
  (#3364).
• Reject a Content-Length that is not plain digits (RFC 9110 §8.6).
• Reject chunk data not followed by CRLF (RFC 9112 §7.1).
• Reject control characters in the request path (RFC 9112 §3.2)
  (#3371).
• Return 414 URI Too Long for a request line over limit_request_line
  (RFC 9112 §3)
  (#2487).
• ASGI: never write more body bytes than the app's Content-Length, and close
  the connection when the body is shorter (RFC 9112 §6.3).
• ASGI: close the connection when the app returns before finishing its
  response (RFC 9112 §6.3).
• ASGI: reject response header names and values that are not valid
  (RFC 9110 §5.1, §5.5).
• WSGI: reject a response status that is not three digits followed by a
  valid reason phrase (RFC 9112 §4).
• ASGI: read the body a HEAD request declares instead of serving it as the
  next request (RFC 9112 §6.3).
• ASGI: stop buffering a request line, header, chunk-size line or trailer
  that never ends; the usual request limits now apply before the line
  terminator arrives, and trailers count against limit_request_fields.
• ASGI: apply forwarded_allow_ips to HTTP/1 requests as WSGI and HTTP/2
  already do: secure_scheme_headers set the scheme only from trusted
  proxies, and header names with underscores follow header_map.
• Ignore one empty line before the request line, as the fast parser
  already does (RFC 9112 §2.2).
• ASGI: stop gunicorn when the app reports a failed lifespan startup,
  instead of serving requests or rebooting the worker forever (ASGI
  lifespan spec) (#3778).
• ASGI: close idle kept-alive connections after keepalive seconds, as
  the other workers do (#3734).
• ASGI: give each request a shallow copy of the lifespan state, so a value
  one request stores in scope["state"] is not seen by the next one (ASGI
  lifespan spec).
• ASGI: deliver websocket.disconnect as soon as the client closes the
  connection, instead of after the close timeout (RFC 6455 §5.5.1).
• ASGI: close a WebSocket with 1009 when a message is larger than the new
  websocket_max_message_size setting (default 16 MiB), and with 1002 for a
  control frame over 125 bytes or fragmented (RFC 6455 §5.5, §7.4.1).
• ASGI: stop reading from a WebSocket while unread data or messages pile up,
  until the app catches up.
• gthread: serve a request that arrived while the previous request's unread
  body was drained, instead of leaving it until the keepalive timeout
  (#3770).
• gthread: idle keep-alive connections no longer hold up a graceful
  shutdown, and the worker keeps reporting to the arbiter while it waits, so
  it is not killed during max_requests restarts.
• Close the connection when reading a request body fails, instead of keeping
  it open with the bytes that followed the body lost.
• Don't call listen() again on a socket that already listens, such as one
  passed by systemd socket activation; SELinux can deny that call.
• With reuse_port on a platform without SO_REUSEPORT, workers now use the
  master's sockets instead of failing to bind.
• StatsD: send metrics over IPv6 when statsd_host is a numeric IPv6
  address; they used to be dropped silently.
• Read request bodies of known length in large steps instead of 8 KiB at a
  time. Large uploads are much faster, most of all while other threads keep
  the worker busy (#2596).
• Read chunked request bodies in large steps too, up to the size each chunk
  announces (#2596).
• inotify reloader: reload when the target of a symlinked file changes
  (#3243).
• Docs: fix the worker types link in the worker_class setting.
• Docs: explain that timeout watches each worker's heartbeat, which idle
  workers keep updating.
• Close the connection after a CONNECT request, whatever the application
  answers; what follows is never served as another request (RFC 9931 §8).
• WSGI: answer 500 instead of 400 when the application sends an invalid
  response header, and leave the header value out of the error.
• Join repeated request headers in linear time when building the WSGI
  environ; many copies of one header used to take time growing with the
  square of their number.
• Reject a request path with raw non-ASCII bytes; they must be
  percent-encoded (RFC 9112 §3.2, RFC 3986 §2.1).
• Dirty arbiter: create its PID file in the temporary directory with mode
  0600, and only use it to stop an orphaned arbiter when this user created
  it and the process is ours. A file left there by someone else is ignored.
• ASGI: stop reading from the socket while more than 256 KiB of request body
  or pipelined data waits for the application, and resume as it reads; uWSGI
  request bodies are read as the application asks for them.
• HTTP/2: close the connection with ENHANCE_YOUR_CALM when a peer sends more
  than 1000 PING, SETTINGS or empty DATA frames in 10 seconds, as already done
  for RST_STREAM floods.
• HTTP/2, sync workers: close the connection with ENHANCE_YOUR_CALM when more
  than 10000 frames pile up while a response waits for window credit.
• WebSocket: fail the connection on frames RFC 6455 forbids: a new message
  inside a fragmented one, reserved opcodes, lengths not minimally encoded,
  malformed close frames (1002), and text or close reasons that are not UTF-8
  (1007) (RFC 6455 §5.2, §5.4, §5.5.1, §7.4, §8.1). An empty client close is
  answered with 1000 instead of the reserved 1005.
• WebSocket: refuse a handshake without Sec-WebSocket-Version: 13 with 426
  (and the supported version), and one whose key is not 16 bytes of base64
  with 400 (RFC 6455 §4.2.1, §4.2.2).
• WebSocket: refuse a handshake the application closes or never accepts with
  an HTTP 403 (500 if it raised), as ASGI requires, instead of writing a
  WebSocket close frame on a connection that never switched protocols.
• WebSocket: wait while the client is not reading before sending a message,
  as HTTP responses already do, so a slow client cannot grow the worker's
  write buffer.
• WebSocket: send() after the connection closed raises an OSError
  subclass, as ASGI 2.4 asks; it is still a RuntimeError, so existing
  handlers keep working, and gunicorn no longer logs it as an error. HTTP
  sends after a disconnect stay silently ignored.
• Dirty apps: bound every stream read by the idle timeout and the total
  timeout, and drop the connection when one fires, so a late chunk is never
  read by the next call. A sync stream's idle timeout is no longer replaced
  by a fixed 5 seconds.
• ASGI: answer Expect: 100-continue with 100 Continue when the app first
  reads the body, so clients such as curl no longer wait about a second before
  sending an upload (RFC 9110 §10.1.1).
• ASGI: answer a request the fast parser rejects with 400 and close the
  connection; it used to be dropped without a response, with a traceback
  logged for each one (RFC 9112 §3, §5.1).
• ASGI: put the standard reason phrase in error responses, such as 400 Bad Request, and the error text in the body; the text, which can quote the
  request, used to be written into the status line (RFC 9112 §4).
• ASGI: answer 501 to a transfer coding gunicorn does not support, such as
  foo, chunked, as the WSGI workers do (RFC 9112 §6.1).
• ASGI: accept whitespace between a chunk size and its extension, such as 1 ;a, as the WSGI workers do (RFC 9112 §7.1.1).
• ASGI: copy a bytearray or memoryview response body when sending it; an
  app that reused the buffer after send returned could have the changed
  bytes sent.
• Reject a line ended by a bare LF at once with 400; with the Python parsers,
  a head or trailer section ended that way kept the connection waiting until a
  limit or the timeout (RFC 9112 §2.2).
• Reject a chunk extension that is not a name with an optional token or
  quoted-string value, such as one with a control character or an unclosed
  quote (RFC 9112 §7.1.1).
• WSGI: with the fast parser, answer 414 to a request line longer than
  limit_request_line that follows an empty line, as the Python parser does
  (RFC 9112 §2.2, §3).

Changes

• Require gunicorn_h1c 0.6.14 for the fast extra. It parses requests faster,
  reads a chunked body in time proportional to its size, and accepts and
  rejects the same requests as gunicorn's Python parsers, whatever way their
  bytes are split across reads.
• ASGI: do less work per request: settings and forwarded-header trust are read
  once per connection, a request is built in one pass over its headers, their
  text copy is only made when something reads it, and response bodies of 64
  KiB or more are sent without being copied. With one worker, small requests
  are about a third faster and streamed responses over 50% faster, and large
  responses use much less memory under load.

Full changelog: https://github.com/benoitc/gunicorn/blob/master/docs/content/news.md