Migration guide from 0.13 to 0.14
Breaking changes
Communication protocol has been changed
The Moleculer communication protocol has been changed. The new protocol version is 4
.
It means the new Moleculer 0.14 nodes can't communicate with old <= 0.13 nodes.
Validation settings changed
The validation: true
broker options was removed to follow other module configuration. Use validator
option, instead.
Enable validation with built-in validator (default option)
const broker = new ServiceBroker({
validator: true
});
Disable validation/validator
const broker = new ServiceBroker({
validator: false
});
Use custom validation
const broker = new ServiceBroker({
validator: new MyCustomValidator()
});
The broker.use
removed
The broker.use
has been deprecated in version 0.13 and now it is removed. Use middleware: []
broker options to define middlewares.
loading middleware after the broker has started is no longer available.
The $node.health
response changed
The $node.health
action's response has been changed. The transit
property is removed. To get transit metrics, use the new $node.metrics
internal action.
Middleware shorthand definition is dropped
In previous versions you could define middleware which wraps the localAction
hook with a simple Function
.
In version 0.14 this legacy shorthand is dropped. When you define a middleware as a Function
, the middleware handler will call it as an initialization and pass the ServiceBroker instance as a parameter.
Old shorthand middleware definition as a Function
const MyMiddleware = function(next, action) {
return ctx => next(ctx);
};
const broker = new ServiceBroker({
middlewares: [MyMiddleware]
});
New middleware definition as a Function
const MyMiddleware = function(broker) {
// Create a custom named logger
const myLogger = broker.getLogger("MY-LOGGER");
return {
localAction: function(next, action) {
return ctx => {
myLogger.info(`${action.name} has been called`);
return next(ctx);
}
}
}
};
const broker = new ServiceBroker({
middlewares: [MyMiddleware]
});
The localEvent
middleware hook signature changed
Old signature
// my-middleware.js
module.exports = {
// Wrap local event handlers
localEvent(next, event) {
return (payload, sender, event) => {
return next(payload, sender, event);
};
},
};
New context-based signature
// my-middleware.js
module.exports = {
// Wrap local event handlers
localEvent(next, event) {
return (ctx) => {
return next(ctx);
};
},
};
New
Context-based events
The new 0.14 version comes context-based event handler. It is very useful when you are using event-driven architecture and you would like to tracing the event. The Event Context is same as Action Context. They are the same properties except a few new properties related to the event.
It doesn't mean you should rewrite all existing event handlers. Moleculer detects the signature if your event handler. If it finds that the signature is "user.created(ctx) { ... }
, it will call it with Event Context. If not, it will call with old arguments & the 4th argument will be the Event Context, like "user.created"(payload, sender, eventName, ctx) {...}
Use Context-based event handler & emit a nested event
module.exports = {
name: "accounts",
events: {
"user.created"(ctx) {
console.log("Payload:", ctx.params);
console.log("Sender:", ctx.nodeID);
console.log("We have also metadata:", ctx.meta);
console.log("The called event name:", ctx.eventName);
ctx.emit("accounts.created", { user: ctx.params.user });
}
}
};
New built-in metrics
Moleculer v0.14 comes with a brand-new and entirely rewritten metrics module. It is now a built-in module. It collects a lot of internal Moleculer & process metric values. You can easily define your custom metrics. There are several built-in metrics reporters like Console
, Prometheus
, Datadog
, ...etc.
Multiple reporters can be defined.
Enable metrics & define console reporter
const broker = new ServiceBroker({
metrics: {
enabled: true,
reporter: [
"Console"
]
}
});
Define custom metrics
// posts.service.js
module.exports = {
name: "posts",
actions: {
get(ctx) {
// Update metrics
this.broker.metrics.increment("posts.get.total");
return posts;
}
},
created() {
// Register new custom metrics
this.broker.metrics.register({ type: "counter", name: "posts.get.total" });
}
};
Enable metrics & define Prometheus reporter with filtering
const broker = new ServiceBroker({
metrics: {
enabled: true,
reporter: [
{
type: "Prometheus",
options: {
port: 3030,
includes: ["moleculer.**"],
excludes: ["moleculer.transit.**"]
}
}
]
}
});
Supported metric types
-
counter
- A counter is a cumulative metric that represents a single monotonically increasing counter whose value can only increase or be reset to zero. For example, you can use a counter to represent the number of requests served, tasks completed, or errors. -
gauge
- A gauge is a metric that represents a single numerical value that can arbitrarily go up and down. Gauges are typically used for measured values like current memory usage, but also "counts" that can go up and down, like the number of concurrent requests. -
histogram
- A histogram samples observations (usually things like request durations or response sizes) and counts them in configurable buckets. It also provides a sum of all observed values and calculates configurable quantiles over a sliding time window. -
info
- An info is a single string or number value like process arguments, hostname or version numbers.
Internal metrics
Process metrics
process.arguments
(info)process.pid
(info)process.ppid
(info)process.eventloop.lag.min
(gauge)process.eventloop.lag.avg
(gauge)process.eventloop.lag.max
(gauge)process.eventloop.lag.count
(gauge)process.memory.heap.size.total
(gauge)process.memory.heap.size.used
(gauge)process.memory.rss
(gauge)process.memory.external
(gauge)process.memory.heap.space.size.total
(gauge)process.memory.heap.space.size.used
(gauge)process.memory.heap.space.size.available
(gauge)process.memory.heap.space.size.physical
(gauge)process.memory.heap.stat.heap.size.total
(gauge)process.memory.heap.stat.executable.size.total
(gauge)process.memory.heap.stat.physical.size.total
(gauge)process.memory.heap.stat.available.size.total
(gauge)process.memory.heap.stat.used.heap.size
(gauge)process.memory.heap.stat.heap.size.limit
(gauge)process.memory.heap.stat.mallocated.memory
(gauge)process.memory.heap.stat.peak.mallocated.memory
(gauge)process.memory.heap.stat.zap.garbage
(gauge)process.uptime
(gauge)process.internal.active.handles
(gauge)process.internal.active.requests
(gauge)process.versions.node
(info)process.gc.time
(gauge)process.gc.total.time
(gauge)process.gc.executed.total
(gauge)
OS metrics
os.memory.free
(gauge)os.memory.total
(gauge)os.uptime
(gauge)os.type
(info)os.release
(info)os.hostname
(info)os.arch
(info)os.platform
(info)os.user.uid
(info)os.user.gid
(info)os.user.username
(info)os.user.homedir
(info)os.network.address
(info)os.network.mac
(info)os.datetime.unix
(gauge)os.datetime.iso
(info)os.datetime.utc
(info)os.datetime.tz.offset
(gauge)os.cpu.load.1
(gauge)os.cpu.load.5
(gauge)os.cpu.load.15
(gauge)os.cpu.utilization
(gauge)os.cpu.user
(gauge)os.cpu.system
(gauge)os.cpu.total
(gauge)os.cpu.info.model
(info)os.cpu.info.speed
(gauge)os.cpu.info.times.user
(gauge)os.cpu.info.times.sys
(gauge)
Moleculer metrics
moleculer.node.type
(info)moleculer.node.versions.moleculer
(info)moleculer.node.versions.protocol
(info)moleculer.broker.namespace
(info)moleculer.broker.started
(gauge)moleculer.broker.local.services.total
(gauge)moleculer.broker.middlewares.total
(gauge)moleculer.registry.nodes.total
(gauge)moleculer.registry.nodes.online.total
(gauge)moleculer.registry.services.total
(gauge)moleculer.registry.service.endpoints.total
(gauge)moleculer.registry.actions.total
(gauge)moleculer.registry.action.endpoints.total
(gauge)moleculer.registry.events.total
(gauge)moleculer.registry.event.endpoints.total
(gauge)moleculer.request.bulkhead.inflight
(gauge)moleculer.request.timeout.total
(counter)moleculer.request.retry.attempts.total
(counter)moleculer.request.fallback.total
(counter)moleculer.request.total
(counter)moleculer.request.active
(gauge)moleculer.request.error.total
(counter)moleculer.request.time
(histogram)moleculer.request.levels
(counter)moleculer.event.emit.total
(counter)moleculer.event.broadcast.total
(counter)moleculer.event.broadcast-local.total
(counter)moleculer.event.received.total
(counter)moleculer.transit.publish.total
(counter)moleculer.transit.receive.total
(counter)moleculer.transit.requests.active
(gauge)moleculer.transit.streams.send.active
(gauge)moleculer.transporter.packets.sent.total
(counter)moleculer.transporter.packets.sent.bytes
(counter)moleculer.transporter.packets.received.total
(counter)moleculer.transporter.packets.received.bytes
(counter)
Built-in reporters
All reporters have the following options:
{
includes: null,
excludes: null,
metricNamePrefix: null,
metricNameSuffix: null,
metricNameFormatter: null,
labelNameFormatter: null
}
Console reporter
This is a debugging reporter which prints metrics to the console periodically.
const broker = new ServiceBroker({
metrics: {
enabled: true,
reporter: [
{
type: "Console",
options: {
interval: 5 * 1000,
logger: null,
colors: true,
onlyChanges: true
}
}
]
}
});
CSV reporter
CSV reporter saves changed to CSV file.
const broker = new ServiceBroker({
metrics: {
enabled: true,
reporter: [
{
type: "CSV",
options: {
folder: "./reports/metrics",
delimiter: ",",
rowDelimiter: "\n",
mode: MODE_METRIC, // MODE_METRIC, MODE_LABEL
types: null,
interval: 5 * 1000,
filenameFormatter: null,
rowFormatter: null,
}
}
]
}
});
Datadog reporter
Datadog reporter sends metrics to the Datadog server.
const broker = new ServiceBroker({
metrics: {
enabled: true,
reporter: [
{
type: "Datadog",
options: {
host: "my-host",
apiVersion: "v1",
path: "/series",
apiKey: process.env.DATADOG_API_KEY,
defaultLabels: (registry) => ({
namespace: registry.broker.namespace,
nodeID: registry.broker.nodeID
}),
interval: 10 * 1000
}
}
]
}
});
Event reporter
Event reporter sends Moleculer events with metric values.
const broker = new ServiceBroker({
metrics: {
enabled: true,
reporter: [
{
type: "Event",
options: {
eventName: "$metrics.snapshot",
broadcast: false,
groups: null,
onlyChanges: false,
interval: 5 * 1000,
}
}
]
}
});
Prometheus reporter
Prometheus reporter publishes metrics in Prometheus format. The Prometheus server can collect them. Default port is 3030
.
const broker = new ServiceBroker({
metrics: {
enabled: true,
reporter: [
{
type: "Prometheus",
options: {
port: 3030,
path: "/metrics",
defaultLabels: registry => ({
namespace: registry.broker.namespace,
nodeID: registry.broker.nodeID
})
}
}
]
}
});
StatsD reporter
The StatsD reporter sends metric values to StatsD server via UDP.
const broker = new ServiceBroker({
metrics: {
enabled: true,
reporter: [
{
type: "StatsD",
options: {
protocol: "udp",
host: "localhost",
port: 8125,
maxPayloadSize: 1300,
}
}
]
}
});
New tracing feature
An enhanced tracing middleware has been implemented in version 0.14. It support several exporters, custom tracing spans and integration with instrumentation libraries (like dd-trace
).
Enable tracing
const broker = new ServiceBroker({
tracing: true
});
Tracing with console exporter
const broker = new ServiceBroker({
tracing: {
enabled: true,
exporter: [
{
type: "Console",
options: {
width: 80,
colors: true,
}
}
]
}
});
Tracing with Zipkin exporter
const broker = new ServiceBroker({
tracing: {
enabled: true,
exporter: [
{
type: "Zipkin",
options: {
baseURL: "http://zipkin-server:9411",
}
}
]
}
});
Add context values to span tags
In action defintion you can define which Context params or meta values want to add to the span tags.
Example
// posts.service.js
module.exports = {
name: "posts",
actions: {
get: {
tracing: {
// Add `ctx.params.id` and `ctx.meta.loggedIn.username` values
// to tracing span tags.
tags: {
params: ["id"],
meta: ["loggedIn.username"],
response: ["id", "title"] // add data to tags from the action response.
},
async handler(ctx) {
// ...
}
}
}
});
Example with all properties of params without meta (actually it is the default)
// posts.service.js
module.exports = {
name: "posts",
actions: {
get: {
tracing: {
// Add all params without meta
tags: {
params: true,
meta: false,
},
async handler(ctx) {
// ...
}
}
}
});
Example with custom function
Please note, the tags
function will be called two times in case of success execution. First with ctx
, and second times with ctx
& response
as the response of action call.
// posts.service.js
module.exports = {
name: "posts",
actions: {
get: {
tracing: {
tags(ctx, response) {
return {
params: ctx.params,
meta: ctx.meta,
custom: {
a: 5
},
response
};
}
},
async handler(ctx) {
// ...
}
}
}
});
Example with all properties of params in event definition
// posts.service.js
module.exports = {
name: "posts",
events: {
"user.created": {
tracing: {
// Add all params without meta
tags: {
params: true,
meta: false,
},
async handler(ctx) {
// ...
}
}
}
});
Built-in exporters
Console exporter
This is a debugging exporter which prints the full local trace to the console.
Please note that it can't follow remote calls, only locals.
const broker = new ServiceBroker({
tracing: {
enabled: true,
exporter: [
{
type: "Console",
options: {
logger: null,
colors: true,
width: 100,
gaugeWidth: 40
}
}
]
}
});
Datadog exporter
Datadog exporter sends tracing data to Datadog server via dd-trace
. It is able to merge tracing spans between instrumented Node.js modules and Moleculer modules.
TODO screenshot
const broker = new ServiceBroker({
tracing: {
enabled: true,
exporter: [
{
type: "Datadog",
options: {
agentUrl: process.env.DD_AGENT_URL || "http://localhost:8126",
env: process.env.DD_ENVIRONMENT || null,
samplingPriority: "AUTO_KEEP",
defaultTags: null,
tracerOptions: null,
}
}
]
}
});
To use this exporter, install the
dd-trace
module withnpm install dd-trace --save
command.
Event exporter
Event exporter sends Moleculer events ($tracing.spans
) with tracing data.
const broker = new ServiceBroker({
tracing: {
enabled: true,
exporter: [
{
type: "Event",
options: {
eventName: "$tracing.spans",
sendStartSpan: false,
sendFinishSpan: true,
broadcast: false,
groups: null,
/** @type {Number} Batch send time interval. */
interval: 5,
spanConverter: null,
/** @type {Object?} Default span tags */
defaultTags: null
}
}
]
}
});
Event (legacy) exporter
This is another event exporter which sends legacy moleculer events (metrics.trace.span.start
& metrics.trace.span.finish
). It is compatible with <= 0.13 Moleculer metrics trace events.
const broker = new ServiceBroker({
tracing: {
enabled: true,
exporter: [
"EventLegacy"
]
}
});
Jaeger exporter
Jaeger exporter sends tracing spans information to a Jaeger server.
const broker = new ServiceBroker({
tracing: {
enabled: true,
exporter: [
{
type: "Jaeger",
options: {
/** @type {String?} HTTP Reporter endpoint. If set, HTTP Reporter will be used. */
endpoint: null,
/** @type {String} UDP Sender host option. */
host: "127.0.0.1",
/** @type {Number?} UDP Sender port option. */
port: 6832,
/** @type {Object?} Sampler configuration. */
sampler: {
/** @type {String?} Sampler type */
type: "Const",
/** @type: {Object?} Sampler specific options. */
options: {}
},
/** @type {Object?} Additional options for `Jaeger.Tracer` */
tracerOptions: {},
/** @type {Object?} Default span tags */
defaultTags: null
}
}
]
}
});
To use this exporter, install the
jaeger-client
module withnpm install jaeger-client --save
command.
Zipkin exporter
Zipkin exporter sends tracing spans information to a Zipkin server.
const broker = new ServiceBroker({
tracing: {
enabled: true,
exporter: [
{
type: "Zipkin",
options: {
/** @type {String} Base URL for Zipkin server. */
baseURL: process.env.ZIPKIN_URL || "http://localhost:9411",
/** @type {Number} Batch send time interval. */
interval: 5,
/** @type {Object} Additional payload options. */
payloadOptions: {
/** @type {Boolean} Set `debug` property in v2 payload. */
debug: false,
/** @type {Boolean} Set `shared` property in v2 payload. */
shared: false
},
/** @type {Object?} Default span tags */
defaultTags: null
}
}
]
}
});
Custom tracing spans
// posts.service.js
module.exports = {
name: "posts",
actions: {
async find(ctx) {
const span1 = ctx.startSpan("get data from DB", {
tags: {
...ctx.params
}
});
const data = await this.getDataFromDB(ctx.params);
span1.finish();
const span2 = ctx.startSpan("populating");
const res = await this.populate(data);
span2.finish();
return res;
}
}
};
Caller action
There is a new caller
property in Context. It contains the action name of the caller when you use ctx.call
in action handlers.
broker2.createService({
name: "greeter",
actions: {
hello(ctx) {
this.logger.info(`This action is called from '${ctx.caller}' on '${ctx.nodeID}'`);
}
}
});
NodeID conflict handling
Having remote nodes with same nodeID
in the same namespace
can cause communication problems. In v0.14 ServiceBroker checks the nodeIDs of remote nodes. If some node has the same nodeID, the broker will throw a fatal error and stop the process.
Sharding built-in strategy
There is a new built-in shard invocation strategy. It uses a key value from context params or meta to route the request a specific node. It means the same key value will be route to the same node.
Example with shard key as name
param in context
const broker = new ServiceBroker({
registry: {
strategy: "Shard",
strategyOptions: {
shardKey: "name"
}
}
});
Example with shard key as user.id
meta value in context
const broker = new ServiceBroker({
registry: {
strategy: "Shard",
strategyOptions: {
shardKey: "#user.id"
}
}
});
All available options of Shard strategy
const broker = new ServiceBroker({
registry: {
strategy: "Shard",
strategyOptions: {
shardKey: "#user.id",
vnodes: 10,
ringSize: 1000,
cacheSize: 1000
}
}
});
Extending internal services
Now the internal services can be extended. You can define mixin schema for every internal service under internalServices
broker option.
// moleculer.config.js
module.exports = {
nodeID: "node-1",
logger: true,
internalServices: {
$node: {
actions: {
// Call as `$node.hello`
hello(ctx) {
return `Hello Moleculer!`;
}
}
}
}
};
Action hook inside action definition
Sometimes it's better to define action hooks inside action definition instead of service hooks
property.
broker.createService({
name: "greeter",
hooks: {
before: {
"*"(ctx) {
broker.logger.info(chalk.cyan("Before all hook"));
},
hello(ctx) {
broker.logger.info(chalk.magenta(" Before hook"));
}
},
after: {
"*"(ctx, res) {
broker.logger.info(chalk.cyan("After all hook"));
return res;
},
hello(ctx, res) {
broker.logger.info(chalk.magenta(" After hook"));
return res;
}
},
},
actions: {
hello: {
hooks: {
before(ctx) {
broker.logger.info(chalk.yellow.bold(" Before action hook"));
},
after(ctx, res) {
broker.logger.info(chalk.yellow.bold(" After action hook"));
return res;
}
},
handler(ctx) {
broker.logger.info(chalk.green.bold(" Action handler"));
return `Hello ${ctx.params.name}`;
}
}
}
});
Output
INFO - Before all hook
INFO - Before hook
INFO - Before action hook
INFO - Action handler
INFO - After action hook
INFO - After hook
INFO - After all hook
Metadata in broker options
There is a new metadata
property in broker options to store custom values. You can use the metadata
property in your custom middlewares or strategies.
const broker2 = new ServiceBroker({
nodeID: "broker-2",
transporter: "NATS",
metadata: {
region: "eu-west1"
}
});
This information is available in response of $node.list
action.
Enhanced hot-reload feature
In v0.14 the built-in hot-reload feature was entirely rewritten. Now, it can detect dependency-graph between service files and other loaded (with require
) files. This means that the hot-reload mechanism now watches the service files and their dependencies. Every time a file change is detected the hot-reload mechanism will track the affected services and will restart them.
New middleware hooks
There are some new middleware hooks.
registerLocalService
It's called before registering a local service instance.
Signature
// my-middleware.js
module.exports = {
registerLocalService(next) {
return (svc) => {
return next(svc);
};
}
}
serviceCreating
It's called before a local service instance creating. At this point the service mixins are resolved, so the service schema is merged completely.
Signature
// my-middleware.js
module.exports = {
serviceCreating(service, schema) {
// Modify schema
schema.myProp = "John";
}
}
transitPublish
It's called before communication packet publishing.
Signature
// my-middleware.js
module.exports = {
transitPublish(next) {
return (packet) => {
return next(packet);
};
},
}
transitMessageHandler
It's called before transit receives & parses an incoming message
Signature
// my-middleware.js
module.exports = {
transitMessageHandler(next) {
return (cmd, packet) => {
return next(cmd, packet);
};
}
}
transporterSend
It's called before transporter send a communication packet (after serialization). Use it to encrypt or compress the packet buffer.
Signature
// my-middleware.js
module.exports = {
transporterSend(next) {
return (topic, data, meta) => {
// Do something with data
return next(topic, data, meta);
};
}
}
transporterReceive
It's called after transporter received a communication packet (before serialization). Use it to decrypt or decompress the packet buffer.
Signature
// my-middleware.js
module.exports = {
transporterReceive(next) {
return (cmd, data, s) => {
// Do something with data
return next(cmd, data, s);
};
}
}
New built-in middlewares
Encryption
AES encryption middleware protects all inter-services communications that use the transporter module.
This middleware uses built-in Node crypto
library.
const { Middlewares } = require("moleculer");
// Create broker
const broker = new ServiceBroker({
middlewares: [
Middlewares.Transmit.Encryption("secret-password", "aes-256-cbc", initVector) // "aes-256-cbc" is the default
]
});
Compression
Compression middleware reduces the size of messages that go through the transporter module.
This middleware uses built-in Node zlib
library.
const { Middlewares } = require("moleculer");
// Create broker
const broker = new ServiceBroker({
middlewares: [
Middlewares.Transmit.Compression("deflate") // or "deflateRaw" or "gzip"
]
});
Transit Logger
Transit logger middleware allows to easily track the messages that are exchanged between services.
const { Middlewares } = require("moleculer");
// Create broker
const broker = new ServiceBroker({
middlewares: [
Middlewares.Debugging.TransitLogger({
logPacketData: false,
folder: null,
colors: {
send: "magenta",
receive: "blue"
},
packetFilter: ["HEARTBEAT"]
})
]
});
Action Logger
Action Logger middleware tracks "how" service actions were executed.
const { Middlewares } = require("moleculer");
// Create broker
const broker = new ServiceBroker({
middlewares: [
Middlewares.Debugging.ActionLogger({
logParams: true,
logResponse: true,
folder: null,
colors: {
send: "magenta",
receive: "blue"
},
whitelist: ["**"]
})
]
});
Load middlewares by names
To load built-in middlewares, use its names in middleware
broker option.
const { Middlewares } = require("moleculer");
// Extend with custom middlewares
Middlewares.MyCustom = {
created(broker) {
broker.logger.info("My custom middleware is created!");
}
};
const broker1 = new ServiceBroker({
logger: true,
middlewares: [
// Load by middleware name
"MyCustom"
]
});
Global error handler
There is a new global error handler in ServiceBroker. It can be defined in broker options as errorHandler(err, info)
.
It catches unhandled errors in action & event handlers.
Catch, handle & log the error
const broker = new ServiceBroker({
errorHandler(err, info) {
this.logger.warn("Error handled:", err);
}
});
Catch & throw further the error
const broker = new ServiceBroker({
errorHandler(err, info) {
this.logger.warn("Error handled:", err);
throw err; // Throw further
}
});
The
info
object contains the broker and the service instances, the current context and the action or the event definition.
Async storage for current context
ServiceBroker has a continuous local storage in order to store the current context. It means you don't need to always pass the ctx
from actions to service methods. You can get it with this.currentContext
.
// greeter.service.js
module.exports = {
name: "greeter",
actions: {
hello(ctx) {
return this.Promise.resolve()
.then(() => this.doSomething());
}
},
methods: {
doSomething() {
const ctx = this.currentContext;
return ctx.call("other.service");
}
}
});
Buffer
supporting improved in serializers
In earlier version, if request, response or event data was a Buffer
, the schema-based serializers convert it to JSON string which was not very efficient. In this version all schema-based serializers (ProtoBuf, Avro, Thrift) can detect the type of data & convert it based on the best option and send always as binary data.
Other notable changes
- Kafka transporter upgrade to support kafka-node@4.
- rename
ctx.metrics
toctx.tracing
. broker.hotReloadService
method has been removed.- new
hasEventListener
&getEventListeners
broker method. - new
uidGenerator
broker options to overwrite the default UUID generator code.