Horizontal Scalability
An extension to the Mnesia database was introduced, named 'Mria'.
EMQX nodes can continue to form a cluster (named 'core') exactly as in previous versions.
In version 5, Mria allows more nodes to join the cluster as 'replicants'.
The difference is, replicant nodes apply changes in the routing table (and configurations etc.) asynchronously, and the replicant nodes can be scaled up and down without affecting the data consistency.
This reduces the networking overheads of forming a full-mesh cluster.
In our most recent test, we managed to achieve 100 million MQTT clients connecting to a 23 nodes Mria cluster.
It's a 3 times larger cluster size compared to a typical version 4 cluster,
and 10 times more capacity comparing to our previous 10 million achievement
Reliability
With the help of the Mria clustering topology, the risk of suffering a split-view of the cluster due to network partitioning, and improves cluster stability in general.
- Only core nodes are involved in transactional writes
- Data is fully replicated to replicant nodes so they can read from local copy (i.e., from RAM)
- While core nodes are fully functional MQTT broker, one can also opt to dedicate them as pure data base nodes
The new Dashboard
Version 5 comes with a fresh new design for the dashboard. Powered by vue.js at the frontend, and OpenAPI at the backend, it provides the most easy-to-use MQTT broker management UI.
Key improvements compared to version 4:
- Access control (authentication and authorization) management
- Improved rule engine and action management UI
- Visual aid view to display data flow e.g., from MQTT topics to destination data sink
- Online config update
- Gateway/Extension management
- More diagnosis tools, such as slow subscriptions and online tracing
Typesafe Config
Starting from version 5, EMQX will use HOCON for the configuration syntax.
With a schema on top, the configs are now type-checked.
HOCON supports Nginx-like configuration layouts, and provides a native array syntax.
But don't worry, the old cuttlefish syntax is still supported if you prefer to keep everything flat like in version 4.
The EMQX team have put huge effort into providing sane-default values, as a result, the default configuration file is now less than 100 lines including comments.
Provided side-by-side, there is a full-blown example configuration (generated from the schema) to help us quickly find examples to copy from.
Operability
In version 5, the interface descriptions for the Dashboard-enabled management APIs follow the OpenAPI specification 3.0. Behind the scene, the specification is actually generated from the configuration schema, the same schema which type-checks the files.
Now we have a single source of schema, which is used to guard both the configuration file and HTTP interface, also to generate config and API documents.
Another nice thing about Swagger, is that it comes with the Swagger-UI in which we can click a "try it out" button to test the API directly from the web browser, or copy-paste the curl command example to a console to test it out.
Another major change in operability compared to version 4 is that the changes are persisted back on disk as configuration file (in HOCON syntax).
The immediate benefit from unifying the config file interface and HTTP API interface is the hot-configuration.
Hot-configuration was a feature only available in EMQX Enterprise edition, it allows us to change a lot of configurations from Dashboard or through the APIs at runtime without having to restart the service.
In enterprise edition version 4, the hot-values were stored in the database, but not in config files, which was not very convenient for users who want to change the overridden values from the config file interface.
Observability
The dashboard comes with more detailed monitoring metrics. We can now view up to 7 days of historical metrics data on Dashboard, and integrate with Prometheus with one click. Add log tracking and slow subscription diagnostic tools to effectively improve the user experience when troubleshooting and diagnosing abnormal client behaviors.
Another enhancement in observability is the introduction of structured logging, now most of the logs emitted from EMQX have a msg field. The text of this filed is an underscore-separated words, to make it more search-friendly for humans but also helps the log indexing tools to index the logs.
Data Integration (The Old Rule Engine)
As you may notice from the dashboard, the old 'Rule Engine' has been renamed to 'Data integration'.
It includes two major functionalities: rules and data bridges.
Rule is a data processing language in SQL syntax (in addition a jq language support), which helps to filter and transform IoT messages.
Data bridge provides the ability to ingest data into EMQX or export data outside of EMQX.
Through the dashboard, users can now clearly see how IoT data is processed with rules and how the data flows from/to external data services.
Another important change: now rules and actions are configs (in config files), but not data in Mnesia (the built-in database, which is quite opaque for non-Erlang developers), meaning we'll be able to configure rules with various deployment/orchestration tools a lot easier.
Access Control
Now we can manage authentication and authorization from the dashboard without having to restart the broker.
Previously provided as plugins, authentication and authorization (ACL) configurations are scattered in different files, and changes will often require a restart of the broker to take effect.
In version 5, all the commonly used security configurations are grouped together. There is also a user management UI provided in the dashboard, allowing you to manage users and access rules on the fly.
We can even configure different authentication rules per listener.
Gateway
The gateway is re-implemented with a unified design language, providing independent management interfaces and security authentication capabilities for various protocols with different client attributes and life cycles, allowing users to access in a more native way. Since each gateway can be configured with its own independent authentication, authentication information of different gateway devices can now be isolated from each other to meet the needs of more scenarios.
MQTT over QUIC
QUIC RFC 9000, the next generation transport layer for the internet, brings amazing opportunities (and challenges too) to IoT.
EMQX team cannot wait to start experimenting MQTT on top of it.
Coming with the 5.0 release, one can configure a QUIC listener to experiment with MQTT over QUIC.
Extension
EMQX 5.0 keeps and to a certain extent enhances the ability of plugin extensions. The plugins can now be compiled, distributed as a standalone package, and then uploaded on the Dashboard to complete the installation for the entire cluster, so they do not have to repeat the steps for each node.
Hello Elixir
EMQX 5 is still a rebar3 project, but it now compiles with mix, if you know Elixir, you know what it means. Happy hacking.