redpanda-data/connect v3.47.0

on GitHub

For installation instructions check out the getting started guide.

Added

• CLI flag max-token-length added to the blobl subcommand.
• Go Plugins API: Plugin components can now be configured seamlessly like native components, meaning the namespace plugin is no longer required and configuration fields can be placed within the namespace of the plugin itself. Note that the old style (within plugin) is still supported.
• The http_client input fields url and headers now support interpolation functions that access metadata and contents of the last received message.
• Rate limit resources now emit checked, limited and error metrics.
• A new experimental plugins API is available for early adopters, and can be found at ./public/x/service.
• A new experimental template system is available for early adopters, examples can be found in ./template.
• New beta Bloblang method bloblang for executing dynamic mappings.
• All http components now support a beta jwt authentication mechanism.
• New experimental schema_registry_decode processor.
• New Bloblang method parse_duration for parsing duration strings into an integer.
• New experimental twitter_search input.
• New field args_mapping added to the sql processor and output for mapping explicitly typed arguments.
• Added format csv to the unarchive processor.
• The redis processor now supports incrby operations.
• New experimental discord input and output.
• The http_server input now adds a metadata field http_server_verb.
• New Bloblang methods parse_yaml and format_yaml.
• CLI flag env-file added to Benthos for parsing dotenv files.
• New mssql SQL driver for the sql processor and output.
• New POST endpoint /resources/{type}/{id} added to Benthos streams mode for dynamically mutating resource configs.

Changed

• Go Plugins API: The Bloblang ArgSpec now returns a public error type ArgError.
• Components that support glob paths (file, csv, etc) now also support super globs (double asterisk).
• The aws_kinesis input is now stable.
• The gcp_cloud_storage input and output are now beta.
• The kinesis input is now deprecated.
• Go Plugins API: the minimum version of Go required is now 1.16.

Fixed

• Fixed a rare panic caused when executing a workflow resource processor that references branch resources across parallel threads.
• The mqtt input with multiple topics now works with brokers that would previously error on multiple subscriptions.
• Fixed initialisation of components configured as resources that reference other resources, where under certain circumstances the components would fail to obtain a true reference to the target resource. This fix makes it so that resources are accessed only when used, which will also make it possible to introduce dynamic resources in future.
• The streams mode endpoint /streams/{id}/stats should now work again provided the default manager is used.

The full change log can be found here.