SwiftMQ Documentation

SwiftMQ Documentation

  • Client
  • CE
  • UR
  • HA
  • Javadocs
  • Release Notes

›Swiftlets

SwiftMQ Client

  • Getting Started
  • JNDI Client
  • JMS Client
  • AMQP 1.0 Client
  • Filetransfer Client

SwiftMQ CE

  • Getting Started
  • Installation
  • Upgrade
  • Software Architecture
  • Administration

    • Preconfig
    • CLI Command Line Interface
    • JMX Administration
    • CLI Message Interface
    • routerconfig.xml Watch Dog
    • System Properties

    Swiftlets

    • AMQP Swiftlet
    • Authentication Swiftlet
    • Deploy Swiftlet
    • JMS Swiftlet
    • JNDI Swiftlet
    • Log Swiftlet
    • Management Swiftlet
    • MQTT Swiftlet
    • Network Swiftlet
    • Queue Manager Swiftlet
    • Routing Swiftlet
    • Scheduler Swiftlet
    • Store Swiftlet
    • Streams Swiftlet
    • Threadpool Swiftlet
    • Timer Swiftlet
    • Topic Manager Swiftlet
    • Trace Swiftlet
    • XA Resource Manager Swiftlet

    APIs

    • CLI Admin API

    How To

    • TLS Configuration
    • HTTP Tunneling Configuration
    • Upgrade SwiftMQ Router and Clients

    In Depth

    • Versioning
    • File Store

SwiftMQ UR

  • Getting Started
  • License
  • Installation
  • Upgrade
  • Administration

    • SwiftMQ Explorer

    Swiftlets

    • JDBC Authentication Swiftlet
    • FileCache Swiftlet
    • JMS Application Container Swiftlet
    • JMS XA Swiftlet
    • JDBC Store Swiftlet
    • AMQP Bridge Extension Swiftlet
    • JMS Bridge Extension Swiftlet
    • JavaMail Bridge Extension Swiftlet
    • Replicator Extension Swiftlet

SwiftMQ HA

  • Getting Started
  • License
  • Installation
  • Upgrade
  • HA Introduction
  • HA Deployment
  • Administration
  • JNDI/JMS under HA
  • Routing under HA
  • Advanced Configuration
  • HA Test Suite

Replicator Extension Swiftlet

The Replicator Swiftlet supports the replication of files and directories via a SwiftMQ router network from arbitrary definable sources to arbitrary definable sinks.

A replication source supervises a source directory and uses SwiftMQ's Filetransfer API to replicate the content and subsequent changes into an associated file cache on the local or a remote router. Any number of replication sinks can be defined anywhere within the SwiftMQ Universal Router Network. They synchronized the file cache with a local directory and get subsequent updates from the replication source via a topic. The result is an exact and up-to-date copy of the replication source, fully automatic.

The Replicator Swiftlet is well-suited for the replication of configuration files or entire websites, the distribution of software, the centralized storage of log-files and much more.

Replication Sources

Create a File Cache

A file cache needs to be created on the local or any remote router. This file cache will be associated with a replication source which replicates its files into it.

Assume we want to replicate log files, so we create a new file cache at the local router and name it logs:

Because the replication source will not use expiration, it can be disabled at the file cache declaration by setting it to -1.

Create a Replication Source

To create a replication source, go to node Replication Sources and do a right click Create a New Entity:

A new frame appears where the new replication source can be declared:

Field description:

Attribute Meaning
Name Some unique name of your choice.
Check Interval The interval in which the directory is checked for changes.
Control File A registry file where the replication source stores links to transferred files. This should be stored outside of the source directory.
Directory The directory which shall be replicated.
Enabled Activates/deactivates the replication source.
File Cache Name The name of the associated file cache.
File Cache Router Name The router name of the associated file cache.
Recursive Enables/disables the replication of subdirectories.
Update Topic The name of the topic where updates events published to.

The replication source starts replication into the associated file cache if it is enabled.

Replication Sinks

To create a replication sink, go to node Replication Sinks and do a right click Create a New Entity:

A new frame appears where the new replication sink can be declared:

Field description:

Attribute Meaning
Name Some unique name of your choice.
Check Interval The interval in which the topic is checked for changes or a retry of a cache sync takes place if the previous sync failed or if there was any other error.
Control File A registry file where the replication sink stores links to transferred files. This should be stored outside of the sink directory.
Directory The destination directory.
Enabled Activates/deactivates the replication sink.
File Cache Name The name of the associated file cache.
File Cache Router Name The router name of the associated file cache.
Update Topic The name of the topic where updates events are received from.

The replication sink starts replication from the associated file cache into the destination directory if it is enabled. It starts with synchronizing with the file cache and subsequently receives update events from the replication source on the update topic.

Event Generation

A replication sink can be configured to generate appropriate events and send it as a JMS TextMessage containing the event as an XML document to configured queues and/or topics. This is useful in situations where a file should be processed once it has arrived, for example. In that case, a simple JMS client would listen on a particular queue (or topic) and would perform particular actions (e.g. process an arrived file) on the events it receives.

Event generation is configured per replication sink. There is an entity list with name Event Sinks. Select it and perform a right mouse click. Select Create a new Entity:

A dialog appears where you can specify the destination, persistence mode and where you can select which type of events you like to send to that event sink:

An event is send as a JMS TextMessage. The text is an XML document like the following example:

Example:

<?xml version="1.0" encoding="UTF-8"?>
<replication-event
  event-type="deleted"
  sink="customer-orders"
  sink-path="/usr/local/orders"
  path="/"
  filename="PO-63446.pdf"
  modification-time="1040245494822"/>

Field description:

Attribute Meaning
event-type The event type. Possible values are "created" or "deleted".
sink The name of the replication sink.
sink-path The path of the sink as specified in the replication sink configuration.
path The path of the file/directory that caused the event. The path is relative to the "sink-path". A value of "/" is equal to the "sink-path" itself. The path seperator character is always "/".
filename The name of the file/directory that caused the event.
modification-time The modification time of the file at the replication source. This attribute is just informational.

Note that events are generated after the associated operation is completed. For example, an event created for a file is generated after the file has been completely received by the replication sink.

Job Scheduling

The Replicator Swiftlet registers jobs in job group Replicator at the Scheduler Swiftlet:

These jobs can be scheduled via the Scheduler Swiftlet to run at specific times or in intervals, based on calenders and so on.

Source

The Source job activates replication source on job start and deactivates it on job stop. The name of the replication source must be specified as parameter. The job requires to set a maximum runtime to deactivate the source. Otherwise it would be active forever.

The implementation of the Source job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a replication source which should run as a job should be initially disabled.

Parameter Mandatory Description
Source Name Yes Name of the replication source.

Sink

The Sink job activates replication source on job start and deactivates it on job stop. The name of the replication sink must be specified as parameter. The job requires to set a maximum runtime to deactivate the source. Otherwise it would be active forever.

The implementation of the Sink job is quite simple. It sets the enabled attribute to true on job start and to false on job stop. Therefore, a replication sink which should run as a job should be initially disabled.

Parameter Mandatory Description
Sink Name Yes Name of the replication sink.

Installation

The installation takes place via 'hot deployment'. See the documentation of the Deploy Swiftlet for further information.

Configuration

The configuration of the Replicator Extension Swiftlet is defined within the element

      <swiftlet name="xt$replicator" .../>

of the router's configuration file. One can use the SwiftMQ Exlorer or CLI for configuration as well. They both save into that file.

Element List "sources", Parent Element: "swiftlet"

Replication Source Definitions. This element list contains zero or more "source" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Replication Source
filecache-routername java.lang.String Yes File Cache Router Name
filecache-name java.lang.String Yes File Cache Name
ivm-username java.lang.String No IntraVM Connection User Name
ivm-password java.lang.String No IntraVM Connection Password
enabled java.lang.Boolean No Enabled
controlfile java.lang.String Yes Control File
directory java.lang.String Yes Directory
recursive java.lang.Boolean No Recursive
check-interval java.lang.Long Yes Check Interval (ms)
update-topic java.lang.String Yes Update Topic

Values

Attribute Values
filecache-routername
filecache-name
ivm-username
ivm-password
enabled Default: false
controlfile
directory
recursive Default: true
check-interval Min: 1000
Default: 60000
update-topic

Element List "sinks", Parent Element: "swiftlet"

Replication Sink Definitions. This element list contains zero or more "sink" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Replication Sink
filecache-routername java.lang.String Yes File Cache Router Name
filecache-name java.lang.String Yes File Cache Name
ivm-username java.lang.String No IntraVM Connection User Name
ivm-password java.lang.String No IntraVM Connection Password
enabled java.lang.Boolean No Enabled
controlfile java.lang.String Yes Control File
directory java.lang.String Yes Directory
check-interval java.lang.Long Yes Check Interval (ms)
update-topic java.lang.String Yes Update Topic

Values

Attribute Values
filecache-routername
filecache-name
ivm-username
ivm-password
enabled Default: false
controlfile
directory
check-interval Min: 1000
Default: 60000
update-topic

Element List "event-sinks", Parent Element: "sink"

Event Sinks. This element list contains zero or more "event-sink" elements with this template definition:

Definition

Attribute Type Mandatory Description
name java.lang.String Yes Name of this Event Sink
destination-name java.lang.String Yes Name of the Queue/Topic
destination-type java.lang.String Yes Destination Type
delivery-mode java.lang.String Yes Delivery Mode
event-file-created java.lang.Boolean No Event File Created
event-file-deleted java.lang.Boolean No Event File Deleted

Values

Attribute Values
destination-name
destination-type Choice: queue topic
Default: queue
delivery-mode Choice: non_persistent persistent
Default: non_persistent
event-file-created Default: true
event-file-deleted Default: true

CLI Commands During Deployment

This Swiftlet performs the following CLI commands during deployment:

cc /sys$threadpool/pools
new replicator
cc /sys$threadpool/pools/replicator/threads
new extension.xt$replicator.%

CLI Commands During Undeployment

This Swiftlet performs the following CLI commands during undeployment:

cc /sys$threadpool/pools
delete replicator
← JavaMail Bridge Extension SwiftletGetting Started →
  • Replication Sources
    • Create a File Cache
    • Create a Replication Source
  • Replication Sinks
  • Event Generation
  • Job Scheduling
    • Source
    • Sink
  • Installation
  • Configuration
Copyright © 2020 IIT Software GmbH