Overview

Version 7.0 is a major release with new enhancements, bugfixes, and several backwards incompatibilities mentioned in the sections below.

I'd also like to introduce @brenuart as a new maintainer! logstash-logback-encoder now has a bus factor of 2. Welcome Bertrand!

We've moved the logstash-logback-encoder repository to the new logfellow GitHub organization. See #700 for more info.

Enhancements

• #364 #365 #377 Switch FastDateFormat to DateTimeFormatter (@J-Jimmy, @philsttr)
• #461 #472 #630 Reduce memory allocations by writing directly into the output stream and reusing Jackson JsonGenerators between invocations. These improvements should greatly reduce pressure on the garbage collector when logging at high rate. (@brenuart)
• #465 Remove support for logback 1.1 (@philsttr)
• #470 Remove deprecated functionality
  • #537 Remove deprecated contextMap and jsonMessage providers (@philsttr)
  • #538 Remove deprecated SSLLogstashTcpSocketAppender (@philsttr)
  • #539 Remove deprecated JsonFactoryDecorator.decorate(MappingJsonFactory) (@philsttr)
  • #540 Remove deprecated LogstashTcpEncoder (@philsttr)
  • #541 Remove deprecated AsyncDisruptorAppender.threadNamePrefix (@philsttr)
  • #542 Remove deprecated LogstashSocketAppender (@philsttr)
  • #543 Remove deprecated includeCallerInfo (@philsttr)
  • #544 Remove deprecated writeVersionAsString (@philsttr)
  • #545 Remove LogstashTcpSocketAppender.remoteHost/port (@philsttr)
• #496 Allow customizing pattern of "message" field for AccessEvents (@philsttr)
• #506 Documentation does not mention StackHashJsonProvider (@philsttr)
• #511 Avoid calling 'toArray' with pre-sized array argument (@ArthurGazizov)
• #559 #565 #619 Allow async appenders to block when buffer is full instead of dropping event (@brenuart)
• #567 #570 Create Disruptor with a ThreadFactory instead of ExecutorService (@brenuart )
• #568 #571 Release/clear LogEvent early before end of batch. Thanks to this change space becomes available in the RingBuffer as soon as an event is successfully async processed. This should reduce the time a logging thread must wait to enqueue its log event when the ring buffer is at or close to maximum capacity. (@brenuart)
• #575 Move servlet-api to test scope (@philsttr)
• #576 Eager validation of configuration parameters (@brenuart)
• #613 #617 Refresh LogstashBasicMarker implementation and remove synchronisation (@brenuart)
• #616 LogstashMarker#equals() and #hashCode() are unstable (@brenuart)
• #619 #620 Implement exponential backoff for the retry strategy (@brenuart)
• #630 #631 #646 #667 #670 Implement a pool of reusable JsonGenerator (@brenuart)
• #636 LogstashMarker refactoring (@brenuart)
• #649 #329 Drop event when Encoder fails to encode it before it becomes a "poison" event (@brenuart)
• 903b77a Be more resilient to exceptions thrown by listeners when firing events (@brenuart)
• #663 New JSON providers have been added to the AccessCompositeJsonProvider:
  • #660 threadName: Name of the thread from which the event was logged (@brenuart).
  • #661 sequence: Output an incrementing sequence number for every log event (@brenuart).
  • #662 uuid: Outputs random UUID as field value. Handy to provide unique identifier for log lines (@brenuart).
• #650 #655 FormattedTimestampJsonProvider#setTimeZone defaults to GMT if the zoneId is not a valid zone id (@brenuart)
• #660 Allow ThreadNameJsonProvider to be used with IAccessEvent (@brenuart)
• #661 Allow SequenceJsonProvider to be used with IAccessEvent (@brenuart)
• #679 #680 #681 jsonproviders no auto start (@brenuart)
• #686 #691 PatternJsonProvider now detects errors in pattern at configuration time instead of silently ignoring the field at runtime. The provider emits an ERROR status at startup when it encounters an invalid Logback pattern layout and reverts to producing nothing at runtime.
• #688 Enhance performance of AbstractPatternJsonProvider (@brenuart)
• #690 #696 Allow masking of multiple substrings (@philsttr)
• #697 Add a #asBoolean() pattern operation (@brenuart)

Bug fixes

• #400 MaskingJsonGeneratorDecorator masks only complete string (@philsttr)
• #463 Markers examples in JavaDoc makes use of deprecated constructs (@philsttr)
• #474 CompositeJsonEncoder throws NullPointerException when not started (@philsttr)
• #475 CompositeJsonFormatter must set Context on JsonProviders BEFORE setting the JsonFactory (@philsttr)
• #507 README maven logger should be runtime, not compile (@paulvi, @philsttr)
• #520 Reference to YourKit in Readme.md is broken (@philsttr)
• #527 #529 Fix marker aggregation when multiple empty markers are aggregated (@philsttr)
• #560 #564 LogstashTcpSocketAppenderTest.testConnectOnPrimary is not stable (@brenuart)
• #566 Shutting down the Disruptor with pending events in the buffer causes high CPU usage (@brenuart)
• #569 TCP connection not closed when Disruptor fails to shutdown within the grace period (@brenuart)
• #610 Documentation uses deprecated links to Logstash documentation (@brenuart)
• #623 Attempt to fix unstable AsyncDisruptorAppenderTest#testEventHandlerCalled test case (@brenuart)
• #642 #643 ObjectFieldsAppendingMarker may use wrong unwrapping JsonSerializer when multiple ObjectMapper are used (@brenuart)
• #658 #659 LogstashAccessEncoder#fieldNames does not change the field name used by ContextJsonProvider (@brenuart)
• #687 Fix rendering of providers table (@aqt)

In addition, a longtime bug in logback (LOGBACK-1326) that affected logstash-logback-encoder (#232) has been fixed in logback 1.2.5!

Deprecations

• #567 The AsyncDisruptorAppender base class does not expose any ExecutorService anymore. Sub-classes should create their own executor if needed, potentially reusing the exposed ThreadFactory.
• #608 Configuring the AsyncDisruptorAppender in SINGLE mode is now deprecated and will be removed in future versions. This mode provides some performance improvements when only one thread is ever appending to the appender but may lead to unpredictable behaviour if not the case.
• #624 Using the queueSize property to set the size of the ring buffer of the TCP async appender is now deprecated. Use ringBufferSize instead.
• #633 #634 Setting secondaryConnectionTTL directly on the AbstractLogstashTcpSocketAppender is now deprecated. You should instead set the property on the connection strategy itself. Example:

  <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
      <connectionStrategy>
          <preferPrimary>
              <secondaryConnectionTTL>5 minutes</secondaryConnectionTTL>
          </preferPrimary>
      </connectionStrategy>
  </appender>

• #661 net.logstash.logback.composite.loggingevent.SequenceJsonProvider is deprecated in favour of net.logstash.logback.composite.SequenceJsonProvider.
• #662 net.logstash.logback.composite.loggingevent.UuidProvider is deprecated in favour of net.logstash.logback.composite.UuidJsonProvider.
• #674 Default implementation of AppenderListener and TcpAppenderListener interfaces are now deprecated in favour of the default methods provided by the interfaces themselves.

Project improvements

• #459 Reduce maven verbosity during ci builds (@sullis)
• #460 Use ubuntu-20.04 instead of latest during ci builds (@sullis)
• #464 Add CodeQL Analysis workflow (@philsttr)
• #517 #629 #654 #665 Switch from java 15 to 17 in build matrix (@philsttr, @brenuart)
• #468 #469 Configure M2E to ignore executions of the maven-license-plugin (@brenuart)
• #478 #547 #548 #556 #561 #562 #563 introduce checkstyle (@brenuart @philsttr)
• #468 #469 #549 #550 license improvements (@brenuart)
• #558 Remove dependency-reduced-pom.xml as part of a normal “mvn clean” invocation (@brenuart)
• #553 #554 Maven Shade plugin complains about duplicate/overlapping classes/resources (@brenuart)
• #588 Validate javadoc during builds (@brenuart)
• #582 #595 Update javadoc link to Jackson packages (@brenuart)
• #583 maven-release-plugin:2.5.3 requires Maven 3.0.4
• #584 #676 Update Maven version used by the wrapper from 3.6.1 to 3.8.3.
• #585 #587 Additional enforcer rule to verify no dependencies have a bytecode higher than 1.8 (@brenuart)
• #588 #594 Fix javadoc issue and produce javadoc artifact during normal builds (@brenaurt)
• #590 #596 Prevent duplicate generation of sources and javadocs artefacts during a release build (@brenuart)
• #597 #599 Setup Github "cache" action (@brenuart)
• #598 #600 Don’t trigger build on changes in Markdown files (@brenuart)
• #604 Upload Surefire reports for later troubleshooting (@brenuart)
• #666 Separate build output directory for Maven CLI and Eclipse (@brenuart)
• #673 Use AssertJ instead of Junit asserts (@brenuart)
• #695 Run tests against Jackson 2.12 to ensure runtime compatibility (@philsttr)
• Add CONTRIBUTING.md (@philsttr)
• Add CODE_OF_CONDUCT.md (@philsttr)
• Add PROCESS.md (@philsttr)
• Configure no-response bot (@philsttr)
• Rename master branch to main (@philsttr)

Dependency version changes

Backwards Incompatibilities

• #364 #365 #377 FastDateFormat switched to DateTimeFormatter. Date patterns are slightly different
• #400 Value-based regex masking now masks all occurrences of the regex match within a string value, rather than matching the whole string value. To revert to the previous behavior, update the regex to include the beginning of line (^) and end-of-line markers ($)
• #465 Support for logback 1.1 has been removed. The minimum logback version is now 1.2.0
• #470 All previously deprecated functionality has been removed. See the PRs associated with #470 for replacements to the removed functionality.
• #567 #570 AsyncDisruptorAppender no longer has an ExecutorService. Subclasses must create/use their own ExecuterService if needed.
• #605 #611 AbstractLogstashTcpSocketAppender#setWriteTimeout(Duration) does not accept null values anymore. Use a positive value to enable the write timeout feature. A timeout of zero disables it.
• #606 The default keep alive message used by the TCP Async appenders is now set to UNIX line ending (\n) instead of being platform dependent. To revert to previous behaviour, set the property keepAliveMessage to SYSTE%.
• #616 Two LogstashMarker used to be equal if they had the same name and their references (child Markers) were equal. This means that adding a child marker to an existing one will break the equal and hashCode method and may lead to unpredicted behaviour when used in a Set or used as a key in a Map for example. To avoid those potential issues, the equals and hashCode methods have been changed to test equality on the Marker's name only - just like what SLF4J BasicMarker does. To summarise, LogstashMarker now behave exactly the same as SLF4J BasicMarker: two markers are equal if they have the same name.
• #637 Visibility of LogstashBasicMarker has been reduced to package protected. People using this class as a base class for their own SLF4J Marker implementation should implement their own.
• #651 Subclasses AsyncDisruptorAppender must now implement createEventHandler
• #663 Refactored JsonProvider hierarchy is not binary compatible. It is source compatible, however.
• #696 Previously, only the first value masker to return a masked value was used. Now all value maskers are used