• Abstract away notion of topics groups (until now it was just an array)
• Optimize how jobs queue is closed. Since we enqueue jobs only from the listeners, we can safely close jobs queue once listeners are done. By extracting this responsibility from listeners, we remove corner cases and race conditions. Note here: for non-blocking jobs we do wait for them to finish while running the poll. This ensures, that for async jobs that are long-living, we do not reach max.poll.interval.
• Shutdown jobs are executed in workers to align all the jobs behaviours.
• Shutdown jobs are always blocking.
• Notion of ListenersBatch was introduced similar to WorkersBatch to abstract this concept.
• Change default shutdown_timeout to be more than max_wait_time not to cause forced shutdown when no messages are being received from Kafka.
• Abstract away scheduling of revocation and shutdown jobs for both default and pro schedulers
• Introduce a second (internal) messages buffer to distinguish between raw messages buffer and karafka messages buffer
• Move messages and their metadata remap process to the listener thread to allow for their inline usage
• Change how we wait in the shutdown phase, so shutdown jobs can still use Kafka connection even if they run for a longer period of time. This will prevent us from being kicked out from the group early.
• Introduce validation that ensures, that shutdown_timeout is more than max_wait_time. This will prevent users from ending up with a config that could lead to frequent forceful shutdowns.