github syslog-ng/syslog-ng syslog-ng-3.36.1

2 years ago

3.36.1

Highlights

  • system() source: added basic support for reading macOS system logs

    The current implementation processes the output of the original macOS syslogd:
    /var/log/system.log.
    (#3710)

  • $(values) and $(names): these new template functions can be used to
    query a list of name-value pairs in the current message. The list of name
    value pairs queried are specified by a value-pairs expression, just like
    with $(format-json).

    Examples:

    This expression sets the JSON array values to contain the list of SDATA
    values, while the JSON array names would contain the associated names, in
    the same order.

    $(format-json values=list($(values .SDATA.*)) names=list($(names .SDATA.*)))

    The resulting name-value pairs are always sorted by their key, regardless of
    the argument order.
    (#3911)

  • rename(): added a new rewrite rule, called rename()

    Example usage:

    rewrite {
      rename( "renamed-from" "renamed-to" );
    };
    

    (#3841)

Features

  • network() drivers: added TLS keylog support

    syslog-ng dumps TLS secrets for a given source/destination, which can be used for
    debugging purposes to decrypt data with, for example, Wireshark.

    This should be used for debugging purposes only!

    Example usage:

    source tls_source{
      network(
          port(1234)
          transport("tls"),
          tls(
            key-file("/path/to/server_key.pem"),
            cert-file("/path/to/server_cert.pem"),
            ca-dir("/path/to/ca/")
            keylog-file("/path/to/keylog_file")
          )
      );
    };
    

    (#3792)

  • tls() block: added option for restricting TLS 1.3 ciphers

    The network(), syslog(), and the http() modules now support specifying TLS 1.3 cipher suites,

    Example usage:

    network(
      transport("tls")
      tls(
        pkcs12-file("test.p12")
        cipher-suite(
          tls12-and-older("ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256"),
          tls13("TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384")
        )
      )
    );
    

    tls12-and-older() can be used to specify TLS v1.2-and-older ciphers,
    tls13() can be used for TLS v1.3 ciphers only.

    Note: The old cipher-suite("list:of:ciphers") option restricts only the TLS v1.2-and-older cipher suite
    for backward compatibility.
    (#3907)

  • file() destination: added a new option: symlink-as()

    This feature allows one to maintain a persistent symlink to a log file when a
    template is used (for example: /var/log/cron -> /var/log/cron.${YEAR}${MONTH}).

    Example usage:

    destination d_file_cron {
      file("/var/log/cron.${YEAR}${MONTH}" symlink-as("/var/log/cron"));
    };
    

    From a functional perspective, the symlink-as file inherits both
    create-dirs and file ownership from its file destination (permissions are not
    applicable to symlinks, at least on linux).

    The symlink is adjusted at the time a new destination file is opened (in the
    example above, if ${YEAR} or ${MONTH} changes).

    Although not specific to time macros, that's where the usefulness is. If the
    template contains something like ${PROGRAM} or ${HOST}, the configuration wouldn't
    necessarily be invalid, but you'd get an ever-changing symlink of dubious
    usefulness.
    (#3855)

  • flags(no-rfc3164-fallback): added a new flag to sources that parse
    incoming syslog data and operate in RFC5424 mode (e.g. syslog-protocol is
    also set). With the new flag the automatic fallback to RFC3164 format
    is disabled. In this case if the parsing in RFC5424 fails, the
    syslog parser would result in an error message. In the case of
    syslog-parser(drop-invalid(yes)), the message would be dropped.
    (#3891)

  • syslog-format: accept ISO timestamps that incorrectly use a space instead of
    a 'T' to delimit the date from the time portion. For example, a
    "2021-01-01T12:12:12" timestamp is well formed according to RFC5424 (which
    uses a subset of ISO8601, see https://datatracker.ietf.org/doc/html/rfc5424#section-6.2.3).
    Some systems simply use a space instead of a 'T'. The same format is
    accepted for both RFC3164 (e.g. udp(), tcp() and network() sources) and
    RFC5424 (e.g. syslog() source).
    (#3893)

  • transport(text-with-nuls): added a new transport mechanism for
    the network() driver that allows NUL characters within the message.

    Note: syslog-ng does not support embedded NUL characters everywhere, so it is
    recommended that you also use flags(no-multi-line) that causes NUL
    characters to be replaced by space.
    (#3913)

Bugfixes

  • filter: fixed the not operator in filter expressions (regression in v3.35.1)

    Reusing a filter that contains the not operator more than once, or
    referencing a complex expression containing not might have caused invalid results
    in the previous syslog-ng version (v3.35.1). This has been fixed.
    (#3863)

  • throttle() filter: support negation
    (#3863)

  • disk-buffer(): fixed a crash which could happen in very rare cases, while a corrupted disk-buffer was getting replaced
    (#3845)

  • disk-buffer(): fixed a memory leak issue and inconsistent buffer handling in rare cases
    (#3887)

  • disk-buffer(): fixed underflowing queued stats counter
    (#3887)

  • disk-buffer(): fixed queued stats were not adjusted when a disk-buffer became corrupt
    (#3851)

  • disk-buffer(): fixed a disk-buffer corruption issue

    A completely filled and then emptied disk-buffer may have been recognised as corrupt.
    (#3874)

  • amqp(): fixed a minor error reporting problem.
    (#3869)

  • amqp(): syslog-ng now drops messages that are too large to send
    (#3869)

  • amqp(): fixed a crash, which happened with librabbitmq v0.9.0 or v0.10.0, while using the tls() block.
    (#3929)

  • file() source: fixed invalid buffer handling when encoding() is used

    A bug has been fixed that - under rare circumstances - could cause message
    duplication or partial message loss when non-fixed length or less known
    fixed-length encodings are used.
    (#3892)

  • syslog-ng: fixed a SIGSEGV triggered by an incorrectly formatted "CONFIG"
    command, received on the syslog-ng control socket. The only known
    implementation of the control protocol is syslog-ng-ctl itself, which always
    sends a correct command, but anyone with access to the UNIX domain socket
    syslog-ng.ctl (root only by default) can trigger a crash.
    (#3900)

  • credit-card-mask(): fixed visa, mastercard and jcb card regex pattern
    (#3853)

  • cisco-parser(): allow a leading dot in the timestamp (not synced clocks)
    (#3843)

Notes to developers

  • plugins: we have made it easier to implement filter plugins

    An example can be found under modules/rate-limit-filter.
    (#3866)

  • dev-utils: various fixes for the plugin skeleton generator script
    (#3866)

Other changes

  • The syslog-ng Docker image
    is now automatically tagged and pushed to Docker Hub after each release
    (#3870)
  • throttle() filter: renamed to rate-limit()
    (#3866)
  • python: support Python 3.10
    (#3865)
  • java: upgraded from old log4j v1.x line to log4j v2.17.2
    (#3861)
    (#3927)

Credits

syslog-ng is developed as a community project, and as such it relies
on volunteers, to do the work necessarily to produce syslog-ng.

Reporting bugs, testing changes, writing code or simply providing
feedback are all important contributions, so please if you are a user
of syslog-ng, contribute.

We would like to thank the following people for their contribution:

Andras Mitzki, Andrea Biardi, Attila Szakacs, Balazs Scheidler,
Balázs Barkó, Benedek Cserhati, Gabor Nagy, Janos SZIGETVARI,
Laszlo Budai, Laszlo Szemere, László Várady, Mikel Olasagasti Uranga,
Norbert Takacs, Parrag Szilárd, Peter Kokai, Szilárd Parrag,
Zoltan Pallagi, Stanislav Osipov, Yash Mathne

Don't miss a new syslog-ng release

NewReleases is sending notifications on new releases.