processone/ejabberd 22.05 on GitHub

A new ejabberd release is finally here! ejabberd 22.05 includes five months of work, 200 commits, including many improvements (MQTT, MUC, PubSub, ...) and bug fixes.

Improved MQTT, MUC, and ConverseJS integration
New installers and container
Support Erlang/OTP 25

When upgrading from the previous version please notice: there are minor changes in SQL schemas, the included rebar and rebar3 binaries require Erlang/OTP 22 or higher, and make rel uses different paths. There are no breaking changes in configuration, and only one change in commands API.

A more detailed explanation of those topics and other features:

New Indexes in SQL for MUC

Two new indexes were added to optimize MUC. Those indexes can be added in the database before upgrading to 22.05, that will not affect older versions.

To update an existing database, depending on the schema used to create it:

MySQL (mysql.sql or mysql.new.sql):

CREATE INDEX i_muc_room_host_created_at ON muc_room(host(75), created_at);
CREATE INDEX i_muc_room_subscribers_jid USING BTREE ON muc_room_subscribers(jid);

PostgreSQL (pg.sql or pg.new.sql):

CREATE INDEX i_muc_room_host_created_at ON muc_room USING btree (host, created_at);
CREATE INDEX i_muc_room_subscribers_jid ON muc_room_subscribers USING btree (jid);

SQLite (lite.sql or lite.new.sql):

CREATE INDEX i_muc_room_host_created_at ON muc_room (host, created_at);
CREATE INDEX i_muc_room_subscribers_jid ON muc_room_subscribers(jid);

MS SQL (mssql.sql):

CREATE INDEX [muc_room_host_created_at] ON [muc_registered] (host, nick)
    WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON);
CREATE INDEX [muc_room_subscribers_jid] ON [muc_room_subscribers] (jid);

Fixes in PostgreSQL New Schema

If you moved your PostgreSQL database from old to new schema using mod_admin_update_sql or the update_sql API command, be aware that those methods forgot to perform some updates.

To fix an existing PostgreSQL database schema, apply those changes manually:

ALTER TABLE archive DROP CONSTRAINT i_archive_sh_peer;
ALTER TABLE archive DROP CONSTRAINT i_archive_sh_bare_peer;
CREATE INDEX i_archive_sh_username_peer ON archive USING btree (server_host, username, peer);
CREATE INDEX i_archive_sh_username_bare_peer ON archive USING btree (server_host, username, bare_peer);

DROP TABLE carboncopy;

ALTER TABLE push_session DROP CONSTRAINT i_push_session_susn;
CREATE UNIQUE INDEX i_push_session_susn ON push_session USING btree (server_host, username, service, node);

ALTER TABLE mix_pam DROP CONSTRAINT i_mix_pam;
ALTER TABLE mix_pam DROP CONSTRAINT i_mix_pam_us;
CREATE UNIQUE INDEX i_mix_pam ON mix_pam (username, server_host, channel, service);
CREATE INDEX i_mix_pam_us ON mix_pam (username, server_host);

ALTER TABLE route DROP CONSTRAINT i_route;
CREATE UNIQUE INDEX i_route ON route USING btree (domain, server_host, node, pid);

ALTER TABLE mqtt_pub DROP CONSTRAINT i_mqtt_topic;
CREATE UNIQUE INDEX i_mqtt_topic_server ON mqtt_pub (topic, server_host);

API Changes

The oauth_revoke_token API command has changed its returned result. Check oauth_revoke_token documentation.

API Batch Alternatives

If you use the command delete_old_messages periodically and noticed it can bring your system to an undesirable state with high CPU and memory consumption...

Now you can use delete_old_messages_batch, which performs the operation in batches, by setting the number of messages to delete per batch and the desired rate of messages to delete per minute.

Two companion commands are added: delete_old_messages_status to check the status of the batch operation, and abort_delete_old_messages to abort the batch process.

There are also new equivalent commands to delete old MAM messages.

Erlang/OTP and Elixir

From now, Erlang/OTP 25 is supported. As that's a brand new version, for stable deployments you may prefer to use 24.3 or other lower version.

Notice that ejabberd can be compiled with Erlang as old as 19.3, but the rebar and rebar3 binaries included with ejabberd 22.05 require at least Erlang 22. This means that, to compile ejabberd 22.05 with those tools using an Erlang version between 19.3 and 21.3, you should get yourself a compatible rebar/rebar3 binary. If your operating system doesn't provide a suitable one, you can download the old ones: rebar from ejabberd 21.12 and rebar3 from ejabberd 21.12.

Regarding Elixir supported versions:

Elixir 1.4 or higher is supported for compilation, but:
Elixir 1.10 is required to build OTP releases (make rel and make dev)
Elixir 1.11 is required to run make relive
Elixir lower than 1.11.4 requires Erlang lower than 24 to build OTP releases

mod_conversejs

mod_conversejs was introduced in ejabberd 21.12 to serve a simple page for the Converse.js XMPP web browser client.

Several improvements in mod_conversejs now allow a simpler configuration, and more customization at the same time:

The options now support the @HOST@ keyword
The options now support auto, which uses local or remote Converse files
The Converse's auth and register options are set based on ejabberd's configuration
default_domain option now has @HOST@ as default value, not the first defined vhost
conversejs_options: New option to setup additional options for Converse
conversejs_resources: New option to serve converse.js files (no need to setup an additional web server)

For example, if you downloaded Converse, now you can setup WebSocket, mod_conversejs, and serve Converse without additional web server, in an encrypted port, as simple as:

listen:
  -
    port: 443
    module: ejabberd_http
    tls: true
    request_handlers:
      /websocket: ejabberd_http_ws
      /conversejs: mod_conversejs

modules:
  mod_conversejs:
    conversejs_resources: "/home/ejabberd/conversejs-9.0.0/package/dist"

With that configuration, Converse is available in https://localhost/conversejs

More details in the mod_conversejs documentation.

New Installers

For many years, the release of a new ejabberd source code package was accompanied with binary installers, built using InstallBuilder and CEAN, and available in the ProcessOne Downloads page.

Since this ejabberd 22.05, there are new installers that use a completely different build method:

they are built using the tools provided in PR 3781
they use the most recent stable dependencies
they are available for linux/amd64 and linux/arm64 architectures
they are built automatically using the Installers Workflow
for stable releases, they are available for download in the ejabberd GitHub Releases
they are built also for every commit in master branch, and available for download in the results of Installers Workflow
if the installer is ran by root, it installs in /opt/ejabberd* and setups systemd service
if ran by a regular user, it asks installation path

However, compared to the old installers, those new installers:

do not ask for domain: now you must edit ejabberd.yml and set the hosts option
do not register the first Jabber account and grant admin rights: you must do it yourself

Please give those new installers a try, and comment any problem, improvement or ideas.

New Container Image

In addition to the ejabberd/ecs Docker container image published in Docker Hub, there is a new container image published in ejabberd GitHub Packages.

Its usage is similar to the ejabberd/ecs image, with some benefits and changes worth noting:

it's available for linux/amd64 and linux/arm64 architectures
it's built also for master branch, in addition to the stable ejabberd releases
it includes less customizations to the base ejabberd compared to ejabberd/ecs
it stores data in /opt/ejabberd/ instead of /home/ejabberd/

See its documentation in CONTAINER.

If you used previous images from that GitHub Packages registry please note: until now they were identical to the ones in Docker Hub, but the new 22.05 image is slightly different: it stores data in /opt/ejabberd/ instead of /home/ejabberd/. You can update the paths to the container volumes in this new image, or switch to Docker Hub to continue using the old same images.

Source Code Package

Until now, the source code package available in the ProcessOne Downloads page was prepared manually together with the binary installers. Now all this is automated in GitHub, and the new source code package is simply the same one available in GitHub Tags.

The differences are:

instead of tgz it's now named tar.gz
it contains the .gitignore file
it lacks the configure and aclocal.m4 files

The compilation instructions are slightly improved and moved to a separate file: COMPILE.md

New `make relive`

This new make relive is similar to ejabberdctl live, but without requiring to install or build an OTP release: compile and start ejabberd immediately!

Quickly put:

Prepare it with: ./autogen.sh && ./configure --with-rebar=./rebar3 && make
Or use this if you installed Elixir: ./autogen.sh && ./configure --with-rebar=mix && make
Start without installing (it recompiles when necessary): make relive
It stores config, database and logs in _build/relive/
There you can find the well-known script: _build/relive/ejabberdctl
In that erlang shell, recompile source code and reload at runtime: ejabberd_admin:update().

Please note, when make relive uses Elixir's Mix instead of Rebar3, it requires Elixir 1.11.0 or higher.

New GitHub Workflows

As you may notice while reading these release notes, there are new github workflows to build and publish the new installers and the container images, in addition to the Common Tests suite.

The last added workflow is Runtime. The Runtime workflow ensures that ejabberd compiles with Erlang/OTP 19.3 up to 25, using rebar, rebar3 and several Elixir versions. It also checks an OTP release can be built, started, register an account, and stop ejabberd.

See its source code runtime.yml and its results.

If you have troubles compiling ejabberd, check if those results reproduce your problem, and also see the steps used to compile and start ejabberd using Ubuntu.

Translations Updates

The German, Portuguese, Portuguese (Brazil), Spanish and Catalan translations are updated and completed. The French translation was greatly improved and updated too.

Documentation Improvements

Some sections in the ejabberd Documentation are improved:

ChangeLog

Core

C2S: Don't expect that socket will be available in c2s_terminated hook
Event handling process hook tracing
Guard against erlang:system_info(logical_processors) not always returning a number
domain_balancing: Allow for specifying type only, without specifying component_number

MQTT

Add TLS certificate authentication for MQTT connections
Fix login when generating client id, keep connection record (#3593)
Pass property name as expected in mqtt_codec (fixes login using MQTT 5)
Support MQTT subscriptions spread over the cluster (#3750)

MUC

Attach meta field with real jid to mucsub subscription events
Handle user removal
Stop empty MUC rooms 30 seconds after creation
default_room_options: Update options configurable
subscribe_room_many_max_users: New option in mod_muc_admin

`mod_conversejs`

Improved options to support @HOST@ and auto values
Set auth and register options based on ejabberd configuration
conversejs_options: New option
conversejs_resources: New option

PubSub

mod_pubsub: Allow for limiting item_expire value
mod_pubsub: Unsubscribe JID on whitelist removal
node_pep: Add config-node and multi-items features (#3714)

SQL

Improve compatibility with various db engine versions
Sync old-to-new schema script with reality (#3790)
Slight improvement in MSSQL testing support, but not yet complete

Other Modules

auth_jwt: Checking if an user is active in SM for a JWT authenticated user (#3795)
mod_configure: Implement Get List of Registered/Online Users from XEP-0133
mod_host_meta: New module to serve host-meta files, see XEP-0156
mod_mam: Store all mucsub notifications not only message notifications
mod_ping: Delete ping timer if resource is gone after the ping has been sent
mod_ping: Don't send ping if resource is gone
mod_push: Fix notifications for pending sessions (XEP-0198)
mod_push: Keep push session ID on session resume
mod_shared_roster: Adjust special group cache size
mod_shared_roster: Normalize JID on unset_presence (#3752)
mod_stun_disco: Fix parsing of IPv6 listeners

Dependencies

autoconf: Supported from 2.59 to the new 2.71
fast_tls: Update to 1.1.14 to support OpenSSL 3
jiffy: Update to 1.1.1 to support Erlang/OTP 25.0-rc1
luerl: Update to 1.0.0, now available in hex.pm
lager: This dependency is used only when Erlang is older than 22
rebar2: Updated binary to work from Erlang/OTP 22 to 25
rebar3: Updated binary to work from Erlang/OTP 22 to 25
make update: Fix when used with rebar 3.18

Compile

mix release: Copy include/ files for ejabberd, deps and otp, in mix.exs
rebar3 release: Fix ERTS path in ejabberdctl
configure.ac: Set default ejabberd version number when not using git
mix.exs: Move some dependencies as optional
mix.exs: No need to use Distillery, Elixir has built-in support for OTP releases (#3788)
tools/make-binaries: New script for building Linux binaries
tools/make-installers: New script for building command line installers

Start

New make relive similar to ejabberdctl live without installing
ejabberdctl: Fix some warnings detected by ShellCheck
ejabberdctl: Mention in the help: etop, ping and started/stopped
make rel: Switch to paths: conf/, database/, logs/
mix.exs: Add -boot and -boot_var in ejabberdctl instead of adding vm.args
tools/captcha.sh: Fix some warnings detected by ShellCheck

Commands

Accept more types of ejabberdctl commands arguments as JSON-encoded
delete_old_mam_messages_batch: New command with rate limit
delete_old_messages_batch: New command with rate limit
get_room_occupants_number: Don't request the whole MUC room state (#3684, #1964)
get_vcard: Add support for MUC room vCard
oauth_revoke_token: Add support to work with all backends
room_unused_*: Optimize commands in SQL by reusing created_at
rooms_unused_...: Let get_all_rooms handle global argument (#3726)
stop|restart: Terminate ejabberd_sm before everything else to ensure sessions closing (#3641)
subscribe_room_many: New command

Translations

Updated Catalan
Updated French
Updated German
Updated Portuguese
Updated Portuguese (Brazil)
Updated Spanish

Workflows

CI: Publish CT logs and Cover on failure to an external GH Pages repo
CI: Test shell scripts using ShellCheck (#3738)
Container: New workflow to build and publish containers
Installers: Add job to create draft release
Installers: New workflow to build binary packages
Runtime: New workflow to test compilation, rel, starting and ejabberdctl

Full Changelog

21.12...22.05

ejabberd 22.05 download & feedback

As usual, the release is tagged in the Git source code repository on GitHub.

The source package and installers are now available in GitHub Release / Tags. To check the *.asc signature files, see How to verify ProcessOne downloads integrity.

The Docker image is in Docker Hub, and a new Container image at GitHub Packages.

If you suspect that you've found a bug, please search or fill a bug report on GitHub Issues.

18 May: Updated installers (run, deb, rpm)

New versions, with -2 in the name, of the installers are available with some fixes. See details in the recent commits in https://github.com/processone/ejabberd/commits/master/tools

processone/ejabberd 22.05 on GitHub

New Indexes in SQL for MUC

Fixes in PostgreSQL New Schema

API Changes

API Batch Alternatives

Erlang/OTP and Elixir

mod_conversejs

New Installers

New Container Image

Source Code Package

New make relive

New GitHub Workflows

Translations Updates

Documentation Improvements

ChangeLog

Core

MQTT

MUC

mod_conversejs

PubSub

SQL

Other Modules

Dependencies

Compile

Start

Commands

Translations

Workflows

Full Changelog

ejabberd 22.05 download & feedback

18 May: Updated installers (run, deb, rpm)

processone/ejabberd 22.05
on GitHub

New `make relive`

`mod_conversejs`