xBus

Transformation of Record Type Interface Files

The transformation of record type structured interface files is implemented in a very general way. A certain transformation for a specific situation is simply achieved by configuring the system. How to do this in detail is explained in the chapter XML/XSL for RecordTypeMessages. Please read the Record Type structured File section for a common understanding of this term.

Evidently such a powerful configuration mechanism needs some elements and rules to consider. This section will explain which elements are involved and how they are related.

This shows how the transformation works. Even the internal format for the interface data is described to enable the reader to understand the XSL transformation scripts.

 External View of the Transformation

External view on the transformation

The xBus is responsible for transporting interface data from source systems to destination systems. Besides the necessary routing this may involve some data transformation, too.

In case of interface files structured by record types this transformation is carried out by a general mechanism. The system is just configured due to the specific transformation tasks. Mainly, three elements are necessary to described (and configure) the transformation:

  • The file structure description for the source file.
  • The file structure description for the destination file.
  • The XSL transformation style sheet.

The record types and their order within a respective interface file are described with the help of XML. InterfaceSpec.dtd declares the tags for such a structure description and how to use them. (Some of the applicable rules are mentioned as comments because things like existence interdependencies cannot be specified as DTD expressions.) Using a DTD is possible because the XML tags to use are fixed.

 Internal View of the Transformation

Internal view on the transformation

The instance in the xBus reading the source interface file is a Receiver object. It does the pure file handling and stores the file content into a string or a list of byte arrays. This step does not use the file structure description at all.

Within the xBus, Message objects are responsible for transporting data. For records which need a transformation, this task is done by objects of the RecordTypeMessage class. Thus the interface data is embodied into such an object. In that moment the string or byte array is parsed according to the respective XML file structure description. The class to perform the parsing is called RecordTypeParser.

The intermediate format for the record processing is XML. It was chosen to use of the XSLT transformer for record transformations.

The two different XML formats for interface files must be well-distinguished:

  • One XML file describes the interface file structure. This is the file mentioned above. It is ruled by the DTD InterfaceSpec.dtd.
  • The other XML structure contains the interface file content after parsing. The XSL transformation style sheet operates on this. The structure is never stored as XML file.
    The field names of the structure description are used as tags for structuring the contents. This solution was chosen to keep the XSL style sheets lean. As consequence, there is no DTD for the XML structures storing the interface data because their tags are dynamical. Their format is explained in the section Intermediate XML representation.

 Intermediate XML Representation

The following is the structure of the DOM tree used to store the interface content internally. XSL transformation style sheets operate upon this structure.

The sketched structure is to interpret according to following rules:

  • Tags given in plain style appear literally in the DOM tree. (e.g. "<Records>")
  • Leading and trailing '+' indicate the place-holders in the description.
  • Tags given in italic style are place-holders for interface, record and field names from the interface file description which serve as tags. (e.g. "<+FieldName+>")
  • Value place-holders are shown in bold. They indicate where the field values read from the interface file are placed into the DOM tree. ("+FieldValue+")

<+InterfaceName+>

<+HeaderName+> <+FieldName+>+FieldValue+</+FieldName+>
...
</+HeaderName+>
<Records> <RecordGroup> <+RecordType+> <+FieldName+>+FieldValue+</+FieldName+>
...
</+RecordType+>
...
</RecordGroup>
...
</Records>
<+TrailerName+> <+FieldName+>+FieldValue+</+FieldName+>
...
</+TrailerName+>
<+InterfaceName+>

All read records are represented in the DOM tree in their original file order.
Header and trailer are optional. The respective interface file description shows if they are present.
The <RecordGroup> structure is only used if records of different types are grouped together in the interface file (RecordOrder=Structured for the <Lines> specification in the structure description, cf. the configuration chapter). Otherwise records are listed immediately below the <Records> level.

There is no DTD for interface content DOM tree since the used tags depend on the structure of the interface file (tags indicated by '+' and italic style above). But the structure can be derived from the structure description taking the name attributes for interface type, the header, the record types, the fields and the trailer as tags to replace the shown place-holders.

 When to use Record Type Transformation

Typical tasks performed by the record transformation are:

  • Changing the field order in the records.
  • Changing the record structure by combining fields from different records.
  • Changing the record structure by distributing fields from one into several records.
  • Grouping of fields in sense of string concatenation.
  • Splitting field values.
  • Shortening or extending field lengths.
  • Calculating "derived" fields.
  • Executing value mappings to adopt codes the destination system encoding.
  • Deleting undesired characters.

Tip: Do not use RecordTypeMessage data transmission if you can transfer the data without any structure change. Simpler transmission methods are more efficient because they do not involve parsing the content of single field values. Additionally you have to set up the structure descriptions and the transformation style sheet.

Tip: If you need only to change single fields to satisfy the destination system's needs, you may concatenate unchanged fields in the structure descriptions and the transformation style sheet. This is advisable if performance is more important than easy-to-read configuration files.