xBus

Further Concepts

This section will explain some important concepts of the xBus. Understanding them is very important for the ability to successfully configure the xBus.

 Transactions

As you can see in Data Flows, two or more technical resources (receiver and sender) are involved in processing a message. Precautions must be taken to deal with the failure of accessing resources. What happens for example, when a message shall be sent to two destinations and one of the senders cannot deliver the message?

The solution are transactions. A transaction includes all actions from receiving the message until all senders have done their work. It ensures a consistent state even in case of errors:

  • In case of an error, the chain of processing the message will be stopped, i.e. no further senders will be called. Instead of it a rollback procedure will be called on all previously activated receivers and senders to revoke the effect of their operation.
  • If all processes finish successfully, a commit is performed on all receivers and senders to finalize the operation.

Unfortunately, not all receivers and senders are capable of supporting this transaction mechanism. Some technical protocols do not support transactions. It is very simple when accessing message queues or databases, because the used API's support transactions. For reading and writing files, a transaction mechanism has been realized by working with temporary files. When sending messages via HTTP, no transaction is possible, because the HTTP protocol does not support transactions. The following table explains, which receivers and senders are transaction-aware and how they react during commit and rollback:

Module Commit Rollback
FilexxxReceiver(Thread)

FTPReceiver

The commit action depends on the parameter System_xxx_FinalResolution:

Preserve: The file will not be touched.
Delete: The file will be deleted.
Rename: The file will be renamed by adding a timestamp to its end.
The rollback action depends on the parameter System_xxx_OnError:

Preserve: The file will not be touched.
Delete: The file will be deleted, its content can be stored in the Deleted Message Store.
Rename: The file will be renamed by adding a timestamp to its end.
FilexxxSender

FTPSender

The temporay file which has been written will be renamed to the filename as found in the configuration. The temporay file which has been written will be deleted.
MQReceiverThread The successful processed message on top of the queue will be deleted by calling a commit on the message queue server. The rollback action depends on the parameter System_xxx_OnError:

Preserve: Rollback is called on the message queue server, causing the message to stay on top of the queue. After a configurable timeout, the xBus will make a new try to process this message.
Delete: The message will be deleted, its content can be stored in the Deleted Message Store.
MQSender The writing of the message to the message queue will be fixed. Only after commit the message is visible in the queue. The writing of the message to the message queue will be undone. Consumers will see no change in the queue.
DatabaseSender Update, Insert and Delete statements will be committed to the database. A rollback will be performed on the database server for all Update, Insert and Delete of this processing.
POP3xxxReceiver(Thread) The message will be deleted in the INBOX. The rollback action depends on the parameter System_xxx_OnError:

Preserve: The message will stay in the INBOX. After a configurable timeout, the xBus will make a new try to process this message.
Delete: The message will be deleted, its content can be stored in the Deleted Message Store.
AS400xxxReceiver The commit action depends on the parameter System_xxx_FinalResolution:

Preserve: The file and its members will not be touched.
Delete, DeleteFile: The complete file with all its members will be deleted.
DeleteMember: The member will be deleted.
Rename: The member will be renamed to "SVTimestamp", e.g. SV051231.
CallProgram: A program on the AS/400 will be called. The name of the program is configurable.
No action, the file and its members will not be touched.

 Plugin Mechanism

The plugin mechanism consists of three features:

  • All modules like senders, receivers, messages, transformers are not hard linked inside the xBus. Which specific module is used during the processing of a message depends solely on the configuration. The only requirement for these modules is, that they implement special interfaces.
  • There is a flexible way how xBus loads Java classes and configuration files during runtime:
    • A xBus specific class loader dynamically loads all JAR files found in the directories XBUS_HOME/lib, XBUS_HOME/lib/runtime and XBUS_HOME/plugin/lib. This enables you to create your own functions by building your own modules, storing them in a JAR file and putting the JAR file in one of these directories. They are automatically loaded on request.
      The Java CLASSPATH must contain only the one JAR file for starting the xBus: XBUS_HOME/lib/xbus-bootstrap.jar.
    • When loading configuration files like standard.conf, xbus.conf or mapping.conf merging is performed:
      E.g. interfaces are usually specified in XBUS_HOME/etc/standard.conf. However, e.g. all files named XBUS_HOME/etc/standard*.conf and XBUS_HOME/plugin/etc/standard*.conf are also loaded at startup and all found files are merged together as the standard configuration.
  • Similar to the configuration, the messages traced by exceptions are loaded in a dynamical way too. All files named XBUS_HOME/etc/errors*_en.properties and XBUS_HOME/plugin/etc/errors*_en.properties are merged together.

One should be aware, that configuration files and message files which are merged together, must not have any entries with duplicate keys. The xBus will be terminated with an error message, if duplicate keys are found when merging the files.

 General transmitted Datatypes

Two categories of transfered data are available in the xBus:

  • Strings
  • Objects

The receiver/sender modules of the technical layer can process either Strings or Objects. For example a FileSender writes the data as a String into a text file. The message modules of the protocol layer are able to deal with either Strings or Objects or both. An XML message for example has a String representation (the human-readable XML string) and an Object representation (a DOM-tree as specified by the W3C).

This leads to a dependency between receiver/sender modules on the one hand and message modules on the other hand. A PDF-file with possible combinations lists also the format of the transferred String or Object.

 Additional Address Information

Most systems need some kind of address, where to read the date from or write it into: HTTP connections have an URL, files a filename etc. This address is stored as part of the system definition in the configuration.

However, a fixed address is often not flexible enough. Think of an installation, where different dealers use a central server for their interfaces. Every dealer shall have its own directory for interface files. With fixed addresses you would need lots of system definitions for every dealer. In contrast the configuration is much easier when variables, like the dealernumber, are part of the filenames.

References to the variables in the configuration are of format $variable name$

.
Such references are replaced by the variable's value when processing the configuration entry.

Two types of variables are implemented:

  • Global variables, which are defined in the xBus core.
    $XBUS_HOME$
    The xBus home directory. Its value is set in the call to the integration layer as Java parameter.
    $VARIABLE_variable name$
    Configuration-internal variable, used to factor out common parts of file parts etc. The value is set within the configuration itself.
    $TIMESTAMP$
    Generates a timestamp - the same format as for renaming files.
    $WILDCARD$
    A wildcard in a file name - but exactly in the file name not in a folder name within the file path. Its value is set in a FileReciver. Afterwards it may be used in any configuration entry. In all those instances it is replaced be the value determined by the concrete name of the received file.
    $WILDCARD$ is also understood as a parameter in a HTTP request by HTTPParameterSender and HTTPParameterReceiver.
  • Project dependent variables. A Java class, implementing a special interface, can be written during a project. This class will contain functionalities to replace variables with actual values. The list of the allowed values is defined in a standard*.conf file. the specified data is used
    • to generate allowed address strings like file paths
    • map values within the specified tupels of values for different variables
    • execute a broadcast to all specified addresses
    This mechanism is referred to as "Additional Address".
Most of the receiver and sender modules support such variables as part of the address information:
  • Receivers with variables in the address will receive messages from every resource addressed. The messages are processed one after the other. The additional address information, which variables are replaced by which values, is stored in the message module.
  • Senders with variables in the address will replace the variables with the values stored in the message object and send the message to the resulting address.
The configuration of system definitions page contains the list of available variables .