SwiftMQ Documentation

SwiftMQ Documentation

  • Client
  • CE
  • UR
  • HA
  • Javadocs
  • Release Notes

›Swiftlets

SwiftMQ Client

  • Getting Started
  • JNDI Client
  • JMS Client
  • AMQP 1.0 Client
  • Filetransfer Client

SwiftMQ CE

  • Getting Started
  • Installation
  • Upgrade
  • Software Architecture
  • Administration

    • Preconfig
    • CLI Command Line Interface
    • JMX Administration
    • CLI Message Interface
    • routerconfig.xml Watch Dog
    • System Properties

    Swiftlets

    • AMQP Swiftlet
    • Authentication Swiftlet
    • Deploy Swiftlet
    • JMS Swiftlet
    • JNDI Swiftlet
    • Log Swiftlet
    • Management Swiftlet
    • MQTT Swiftlet
    • Network Swiftlet
    • Queue Manager Swiftlet
    • Routing Swiftlet
    • Scheduler Swiftlet
    • Store Swiftlet
    • Streams Swiftlet
    • Threadpool Swiftlet
    • Timer Swiftlet
    • Topic Manager Swiftlet
    • Trace Swiftlet
    • XA Resource Manager Swiftlet

    APIs

    • CLI Admin API

    How To

    • TLS Configuration
    • HTTP Tunneling Configuration
    • Upgrade SwiftMQ Router and Clients

    In Depth

    • Versioning
    • File Store

SwiftMQ UR

  • Getting Started
  • License
  • Installation
  • Upgrade
  • Administration

    • SwiftMQ Explorer

    Swiftlets

    • JDBC Authentication Swiftlet
    • FileCache Swiftlet
    • JMS Application Container Swiftlet
    • JMS XA Swiftlet
    • JDBC Store Swiftlet
    • AMQP Bridge Extension Swiftlet
    • JMS Bridge Extension Swiftlet
    • JavaMail Bridge Extension Swiftlet
    • Replicator Extension Swiftlet

SwiftMQ HA

  • Getting Started
  • License
  • Installation
  • Upgrade
  • HA Introduction
  • HA Deployment
  • Administration
  • JNDI/JMS under HA
  • Routing under HA
  • Advanced Configuration
  • HA Test Suite

MQTT Swiftlet

MQTT stands for MQ Telemetry Transport. It is a publish/subscribe, extremely simple and lightweight messaging protocol, designed for constrained devices and low-bandwidth, high-latency or unreliable networks. The design principles are to minimise network bandwidth and device resource requirements whilst also attempting to ensure reliability and some degree of assurance of delivery. These principles also turn out to make the protocol ideal of the emerging "machine-to-machine" (M2M) or "Internet of Things" world of connected devices, and for mobile applications where bandwidth and battery power are at a premium.

(Description from mqtt.org)

Features

The MQTT Swiftlet provides a full-featured MQTT 3.1/3.1.1 broker implementation. The following MQTT features are supported:

  • QoS 0 (at most once), QoS 1 (at least once), QoS 2 (exactly once).
  • Topic wildcard filters.
  • Retained messages.
  • Last Will and Testament.
  • Clean and persistent sessions.
  • Session takeover.
  • Packet replay on reconnect.
  • Keep Alive.

Additional features provided by the MQTT Swiftlet:

  • Full interoperability between MQTT, AMQP 1.0/0.9.1 and JMS clients.
  • Topic order guarantees for all QoS levels.
  • Persistent session timeout and automatic cleanup.
  • Administrative deletion of persistent sessions.
  • Plain and TLS connections.

Each feature is explained in more detail in the following chapters.

Quality of Service (QoS)

In MQTT the publisher of a message decides about the importance of a message by specifying the QoS in the publish package. The QoS levels are as follows:

QoS Meaning Description
0 At Most Once Messages can be lost.
1 At Least Once Messages can't be lost but duplicates may occur.
2 Exactly Once Messages are delivered once and only once.

The interaction between publisher and MQTT Swiftlet takes place according to the specified QoS level in the publish packet. A subscriber has it's own QoS specified per subscription. This QoS level is the maximum level the subscriber supports. The QoS level for each message is compared to the QoS of the subscriber and may be downgraded, if necessary. For example, if the publisher has specified QoS 2 and the subscriber only has QoS 1, the message is handled according to QoS 1. On the contrary, if the publisher specifies QoS 0 and the subscriber has QoS 2, QoS 0 is used.

Topic Wildcard Filters

In MQTT topics are structured as a hierarchy, for example devices/device1/temp. The foward slash / is the delimiter. A subscriber can define a topic filter that is applied to the messages which are delivered to the subscription if the filter matches the topic under which the messages have been published.

MQTT defines 2 wildcard characters: + addresses all messages of a hierarchy node and # addresses all messages of a node and all sub-nodes, thus # can only be specified as the last character of a filter.

MQTT topics are mapped to SwiftMQ topic predicates internally. SwiftMQ topics are a bit different as the topic delimiter is a . such as devices.device1.temp and the whole topic name can be a SQL LIKE predicate where the underscore _ matches any single character and the percent sign % any string. SwiftMQ does not allow to specify a wildcard for the root node of a topic hierarchy.

In consequence there are some limitation for MQTT topic filters:

MQTT Topic Filter SwiftMQ Topic Predicate Valid Description
+ - No Filter at the root node is not allowed.
# - No Filter at the root node is not allowed.
+/+/temp - No Filter at the root node is not allowed.
/devices/+/temp devices.%.temp Yes
/devices/# devices.% Yes

Retained Messages

A publisher can flag a message as "retain". The last retained message per topic is sent to new subscriptions of that topic as the very first message. This is a useful feature to e.g. store and communicate the last state sent by a device without the need that the device is currently connected.

The MQTT Swiftlet stores retained messages in memory. So after a reboot of a SwiftMQ Router or a failover of a SwiftMQ High Availability Router these retained messages are lost.

Last Will and Testament (LWT)

A MQTT client can specify a LWT message in the connect packet. This message will be sent to a topic when the client unexpectedly disconnects without sending a disconnect packet. For example, the network connection is lost or the client is disconnected administratively via SwiftMQ Explorer/CLI.

LWT messages can also be marked as "retain". See the previous chapter.

Sessions

In MQTT a session represents the state of a connection. It contains:

  • All topic subscriptions.
  • A replay log of unacknowledged packets.

A MQTT client specifies a "clean session" flag in the connect packet. If that is set, a previous persistent session including all active subscriptions is deleted and a new (clean) session is created. This clean session is temporary and has a life time of that of the connection. All subscriptions of a clean session are non-durable (backed by a temporary queue).

If the "clean session" flag is not set, the session is persistent. All subscriptions of a persistent session are durable (backed by a durable subscriber queue) and the messages are stored persistently. If a client disconnects, the session is stored persistently (survives a reboot and failover). If a client reconnects and there is a persistent session stored for this client id, the replay log is first sent to the client to finish unacknowledged packets (the client does the same) and the subscriptions are active without the need to subscribe. Message delivery starts immediately.

In case the persistent session is associated with a connection with the same client id (e.g. the previous TCP connection is still alive due to half open sockets), this active connection is closed by the MQTT Swiftlet and the new connection is associated with the persistent session.

Keep Alive

A MQTT client can send ping request packets. This will be answered by a ping response packet by the MQTT Swiftlet. Each MQTT connection has a keepalive timer. If a connection did not sent a packet within 1.5 times of the value of the keepalive field in the connect packet, the connection will be closed by the MQTT Swiftlet.

Interoperability between MQTT, AMQP, JMS

SwiftMQ provides full interoperability between MQTT, AMQP and JMS clients.

The payload of a MQTT message is a stream of bytes. A message published to a topic is internally converted to a JMS BytesMessage. A subscription receives a JMS BytesMessage and converts it into a MQTT publish packet.

The resulting JMS BytesMessage from a MQTT publish contains the following message properties:

Name Type Value
JMSXUserID String Username of the Publisher.
JMS_SWIFTMQ_CID String Client id of the Publisher.
JMS_SWIFTMQ_MQTT_PUBQOS Integer QoS level of the Publisher.

These properties can be used by JMS/AMQP clients in message selectors. The properties have only informational purpose, except JMS_SWIFTMQ_MQTT_PUBQOS which is used by MQTT subscribers to determine the resulting QoS for the subscription. If JMS_SWIFTMQ_MQTT_PUBQOS is not set, the subscriber uses the QoS of the subscription.

Message Order Guarantees

The MQTT Swiftlet uses the pub/sub sub system of the SwiftMQ Router. Subscriptions are backed by durable or non-durable queues which guarantee message order independent of the QoS level.

Persistent Session Timeout and Cleanup

Persistent sessions have a configurable timeout (default: 1 week). If they were not associated with a connection during this time, they are deleted including all subscriptions.

Plain and TLS Connectiions

Per default, the MQTT Swiftlet has 2 configured listeners. One for plain TCP connections and one for TLS connections. TLS connections are served by Java JSSE included in the Java distribution.

Please have a look at this chapter how to configure the SwiftMQ Router. Configuration of the MQTT client is client specific.

Management

Listener Configuration

An MQTT listener listens on a specific port and accepts MQTT connections. Per default the MQTT Swiftlet defines 2 MQTT listeners:

  • Port 1883: Listener for plain MQTT connections.
  • Port 8883: Listener for TLS MQTT connections.

A listener is configured by a connection template which is referenced by its name. See next section.

A new listener on a different port can be created by selecting the Listeners entity, right click, Create a new Entity:

Connection Templates

Connection templates are pre-configured templates which are referenced from MQTT listeners. They are located under the Declarations entity:

Connection template default does not need to be created because it refers to a connection template with all default values. The above tls connection template is a default template with a different socket factory (JSSESocketFactory for TLS).

A new connection template can be created by selecting the Connection Templates entity, right click, Create a new Entity:

Session Timeout

The session timeout is the time of inactivity of a persistent session where it is not associated with a connection. Sessions and their subscriptions are deleted when they reach the timeout. The timeout is in hours (default: 168 hours = 1 week) and can be configured here:

Administratively delete an active MQTT Connection

MQTT connections can be deleted administratively this way:

Administratively delete a persistent Session

Persistent sessions can be deleted administratively this way:

The session to delete must not be associated with a connection, otherwise an error is shown.

Monitoring

Monitoring MQTT Connections

MQTT connections are located under the Usage section of the MQTT Swiftlet. Select it, right click, Show Entity Table is an alternate view of the same information:

Start some MQTT clients and expand the Usage section of the MQTT Swiftlet. The information shown is down to the subscriptions and message transfer per subscription can be oberserved:

To view it in an Entity Table frame, select the Subscriptions node, right click, Show Entity Table:

Monitoring Queues

Another view can be opened on the Queue Manager Swiftlet's Usage section. It shows the message throughput of the queues. Clean sessions use non-durable subscriber queues (tmp$...) while persistent sessions use durable subscriber queues (<clientid>$<number>) By selecting a queue and pushing the first button in the upper left corner, a message viewer will be opened and the queue messages can be inspected:

Monitoring Threadpools

The MQTT Swiftlet uses a single threadpool:

  • mqtt.connection: Performs connection tasks and outbound writes.

The threadpool (and all others) can be oberserved by opening an Entity Table frame on the Threadpool Swiftlet's Usage section:

Tracing and Debugging

A SwiftMQ Router has an integrated tracing facility, the Trace Swiftlet. Kernel Swiftlets such as the MQTT Swiftlet are using the kernel trace space to send trace output (if the trace space is enabled) about the internal processing of the MQTT Swiftlet. The trace output goes to <swiftmqinstall>/data/trace/mqtt.trace by default. Tracing can be enabled/disabled dynamically through SwiftMQ admin tools.

Another trace space protocol is used to send information about incoming and outgoing MQTT packets and the output of the packet decoder. This is the more interesting trace space for debugging.

Trace space kernel contains various predicate entries for different Kernel Swiftlets. Every entry can be enabled (default) or disabled. The MQTT Swiftlet predicate is in entry 22 by default. To get trace output, enable the kernel trace space and make sure, predicate 22 is enabled too. Be aware that kernel tracing produces huge output and slows down the router.

To get MQTT protocol information only, change the single predicate of the protocol space to mqtt. Then enable the protocol space. The trace output goes to <swiftmqinstall>/data/trace/protocol.trace by default.

The following snippet shows how to enable the above tracing directly in the routerconfig.xml to have tracing enabled on startup:

<swiftlet name="sys$trace">
    <spaces>
        <space name="kernel" enabled="true">
            <predicates>
                ...
                <predicate name="22" filename="../data/trace/mqtt.trace" value="sys$mqtt"/>
                ...
            </predicates>
        </space>
        <space name="protocol" enabled="true">
            <predicates>
                <predicate name="1" filename="../data/trace/protocol.trace" value="mqtt"/>
            </predicates>
        </space>
        ...
    </spaces>
</swiftlet>

Configuration

The configuration of the MQTT Swiftlet is defined within the element

      <swiftlet name="sys$mqtt" .../>

of the router's configuration file. One can use the SwiftMQ Exlorer or CLI for configuration as well. They both save into that file.

Attributes of Element "swiftlet"

Definition

Attribute Type Mandatory Description
collect-interval java.lang.Long No Collect Interval Messages/Sec
session-timeout java.lang.Long No Time in hours after which a unused Session is deleted

Values

Attribute Values
collect-interval Default: 1000
session-timeout Default: 168

Element "declarations", Parent Element: "swiftlet"

Declarations Section.

Element List "connection-templates", Parent Element: "declarations"

Templates for Connections. This element list contains zero or more "connection-template" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Connection Template
socketfactory-class java.lang.String No Socketfactory Class
use-tcp-no-delay java.lang.Boolean No Use Tcp No Delay
idle-timeout java.lang.Long No Inactivity timeout (ms) after which a Connection is disconnected
max-message-size java.lang.Integer No Maximum Message Size
reject-disconnect-delay java.lang.Long No Time (ms) after which a rejected Connection is closed
router-input-buffer-size java.lang.Integer No Router Network Input Buffer Size
router-input-extend-size java.lang.Integer No Router Network Input Extend Size
router-output-buffer-size java.lang.Integer No Router Network Output Buffer Size
router-output-extend-size java.lang.Integer No Router Network Output Extend Size

Values

Attribute Values
socketfactory-class Default: com.swiftmq.net.PlainSocketFactory
use-tcp-no-delay Default: true
idle-timeout Default: 90000
max-message-size Min: 0
Max: 2147483647
Default: 10485760
reject-disconnect-delay Min: 1000
Default: 5000
router-input-buffer-size Min: 65536
Default: 131072
router-input-extend-size Min: 65536
Default: 65536
router-output-buffer-size Min: 65536
Default: 131072
router-output-extend-size Min: 65536
Default: 65536

Element List "listeners", Parent Element: "swiftlet"

Listener Definitions. This element list contains zero or more "listener" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Listener
bindaddress java.lang.String No Listener Bind IP Address
port java.lang.Integer Yes Listener Port
max-connections java.lang.Integer Yes Maximum Connections for Listener
connection-template java.lang.String Yes Connection Template to use

Values

Attribute Values
bindaddress
port Default: 1883
max-connections Default: -1
connection-template Default: default

Element List "host-access-list", Parent Element: "listener"

Host Access List. This element list contains zero or more "host-access-entry" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Host Access Entry

Element "usage", Parent Element: "swiftlet"

Live Usage.

Element List "connections", Parent Element: "usage"

Active MQTT Connections. This element list contains zero or more "usage" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Active MQTT Connection
client-id java.lang.String No Client Id
connect-time java.lang.String No Connect Time
msgs-received java.lang.Integer No Messages/Sec received from Connection
msgs-sent java.lang.Integer No Messages/Sec sent to Connection
total-received java.lang.Integer No Total Number of Messages received from Connection
total-sent java.lang.Integer No Total Number of Messages sent to Connection
username java.lang.String No User Name
mqtt-protlevel java.lang.Integer No MQTT Protocol Level [4 = 3.1.1]

Values

Attribute Values
client-id
connect-time
msgs-received Default: 0
msgs-sent Default: 0
total-received Default: 0
total-sent Default: 0
username
mqtt-protlevel

Element "MQTT Session", Parent Element: "usage"

MQTT Session.

Definition

Attribute Type Mandatory Description
persistent java.lang.Boolean No Persistent Session

Values

Attribute Values
persistent Default: false

Element List "subscriptions", Parent Element: "MQTT Session"

Topic Subscriptions. This element list contains zero or more "Topic Subscription" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Topic Subscription
swiftmq-topic java.lang.String No Translated SwiftMQ Topic Name
qos java.lang.Integer No Quality of Service
msgs-received java.lang.Integer No Messages/Sec received from Subscription
total-received java.lang.Integer No Total Number of Messages received from Subscription

Values

Attribute Values
swiftmq-topic
qos Min: 0
Max: 2
Default: 0
msgs-received Default: 0
total-received Default: 0

Element List "sessions", Parent Element: "usage"

Persistent MQTT Sessions. This element list contains zero or more "MQTT Session" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this MQTT Session
associated java.lang.Boolean No Associated with a Connection
persistent java.lang.Boolean No Persistent Session

Values

Attribute Values
associated Default: false
persistent Default: false

Element List "subscriptions", Parent Element: "MQTT Session"

Topic Subscriptions. This element list contains zero or more "Topic Subscription" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Topic Subscription
swiftmq-topic java.lang.String No Translated SwiftMQ Topic Name
qos java.lang.Integer No Quality of Service

Values

Attribute Values
swiftmq-topic
qos Min: 0
Max: 2
Default: 0
← Management SwiftletNetwork Swiftlet →
  • Features
    • Quality of Service (QoS)
    • Topic Wildcard Filters
    • Retained Messages
    • Last Will and Testament (LWT)
    • Sessions
    • Keep Alive
    • Interoperability between MQTT, AMQP, JMS
    • Message Order Guarantees
    • Persistent Session Timeout and Cleanup
    • Plain and TLS Connectiions
  • Management
    • Listener Configuration
    • Connection Templates
    • Session Timeout
    • Administratively delete an active MQTT Connection
    • Administratively delete a persistent Session
  • Monitoring
    • Monitoring MQTT Connections
    • Monitoring Queues
    • Monitoring Threadpools
    • Tracing and Debugging
  • Configuration
Copyright © 2020 IIT Software GmbH