fluent/fluentd v1.20.0

Fluentd v1.20.0

on GitHub

Enhancement

• #5090 Replace Yajl with built-in JSON for simple cases
• #5262 out_forward: add send_keepalive_packet option to enable TCP keepalive (SO_KEEPALIVE)
• #5320 in_exec: add command_timeout option to limit child process execution time
• #5383 Replace Yajl with JSON for handling JSON stream
  • When an input stream ends in the middle of a JSON document, Fluentd now logs a warning and discards the incomplete tail instead of raising ParseError.
• #5390 system_config: add umask option to <system> directive
• #5449 parser_syslog: accept a space after RFC3164 priority
• #5528 in_tail: add opt-in enable_dir_watcher and dir_watcher_limit to discover new files earlier by watching directories
• #5529 in_tail: add opt-in worker-thread parsing via num_threads to offload parsing from the event loop
  • The default num_threads 1 keeps the previous synchronous behavior.
  • In worker mode, monitor_agent also reports worker_pending_batch_count, worker_pending_bytes and worker_parse_error_count.
• #5473 buffer: add evacuate_limit_size to keep evacuated chunks from filling the disk
  • This option sets an approximate limit on the total size of the chunk files evacuated when the retry limit is reached, including the ones evacuated before. Chunks that do not fit are purged without being evacuated.
  • The default nil keeps the previous behavior. Set 0 to disable evacuation.
• #5411 in_forward: accept metadata on encapsulated timestamps sent by Fluent Bit v5.0.5 or later
• #5464 Enable Ruby JIT by default
  • The system configuration enable_jit now defaults to true.
  • JIT increases memory usage of each worker process. Set enable_jit false in the <system> section to restore the previous behavior.
  • This has no effect on Windows because YJIT is not supported there.
• #5370 fluent-plugin-generate: add .gitignore file
• Performance improvements
  • #5281
  • #5288
  • #5289
  • #5344
  • #5353
  • #5354
  • #5357
  • #5287
  • #5278
  • #5567

Bug Fix

• #5239 http_server helper: Fix IPv6 bind address support in URI construction
  • Fixed URI::InvalidURIError when binding to IPv6 addresses (e.g., ::, ::1)
  • IPv6 addresses are now properly bracketed in URIs per RFC 3986 (e.g., http://[::]:24231)
  • Handles pre-bracketed addresses correctly to avoid double-bracketing
  • Affects all plugins using http_server helper with IPv6 bind addresses
• #5476 out_file: create symlinks for all chunks sharing the latest timekey
• #5420 time: fix TimeFormatter returning nil for the Unix epoch
• #5499 in_tail: fix log duplication when a rotated file is appended during rotate_wait with follow_inodes
• #5513 in_tail: fix NameError raised when flushing buffered lines with emit_unmatched_lines true
• #5515 in_tail: fix unread lines never being collected when a throttled file is removed from the watch list
• #5530 in_tail: fix shutdown errors and timeouts caused by an ongoing watcher refresh
• #5551 in_tail: fix closed stream error when file rotation overlaps with shutdown
• #5542 in_tail: use a monotonic clock for multiline flush timers so that they are not affected by wall-clock adjustments
• #5481 buffer: fix a race that could make the buffer stage_size gauge transiently negative while a write and a chunk enqueue overlap
• #5517 buffer: fix an enqueued unstaged file chunk being re-staged and flushed twice, which caused closed stream / ENOENT errors and leaked buffer size gauges
• #5553 parser_msgpack: fix leftover data from one parse call leaking into the next
• #5523 http_server helper: fix HTTP endpoints never recovering after a bind failure
  • The plugin now fails to start, and the worker is restarted until the port is available.
• #5526 root_agent: fix emit_records and emit_size never counted for inputs with @label
• #5522 supervisor: fix graceful config reload failing for YAML configuration
  • It also fixes <<: !include of a file that contains a mapping.

Misc

• #5465 Drop Ruby 3.2 support
• #5484 out_http: remove the Net::HTTP extra_chain_cert monkey patch
• #5400 gem: add aarch64-mingw-ucrt gem
• #5429 gem: add json gem (>= 2.21) as runtime dependency and remove Refinements workaround
• in_tail: internal refactoring
  • #5518
  • #5520
  • #5521
  • #5527
  • #5554

Contributors to this release (Alphabetical order)

• Akanksha Trehun
• Akash Kumar
• Alexander Urieles
• Arpit Jain
• Aviral Garg
• Byun Jincheol
• ChrisJr404
• Christopher Pruijsen
• Daijiro Fukuda
• Gagan H R
• Hiroshi Hatake
• Ian Driver
• Jesse Awan
• JSap0914
• Kentaro Hayashi
• Mehrdad Biukian
• Olexandr88
• onice
• Orgad Shaneh
• SEONGHYUN HONG
• Shizuo Fujita
• SimonYang
• somaz
• spairy
• srilumpa
• Sutou Kouhei
• Takuro Ashie
• Vedant Madane