Configuring a Record Transformation

The chapter Records Transformation explains all elements within the transformation.

Here all configuration details are listed:

standard.conf Entries
XML Interface File Descriptions
XSL Transformation Style Sheets

 standard.conf Entries

The chapter System Definitions explains the single keys in detail.

Entries for the Source Interface - plus those necessary for the specific receiver

Chapter Section Key Value
System source interface name Message RecordTypeMessage
System source interface name Receiver Receiving result Text or Object for string- or byte-array-oriented parsing
System source interface name DescriptionFile name of the source structure description XML file without path
System source interface name RecordLineLength only necessary if records are to be identified by counting bytes instead of reading until line breaks
System source interface name LineCutting true or false (default false), only necessary if records are to be identified by line breaks and ending blanks may be cut from the record lines

Evidently the message has to be "RecordTypeMessage".

The record type transformation works based on file receivers with read data into strings and based on byte array receivers. Switching the Receiver determines if the interface content accessed in a string- or byte-array-oriented way. Internally the interface content is stored in a DOM tree instead of a string. (For information on the different receivers, please refer to the chapter Technical Layer.)

The last 3 keys are really particular to interfaces with record transformation (and byte array data transfer without transformation).

The DescriptionFile entry tells the system where to find the description of the source interface data structure. This structure description is essential for parsing. All XML interface descriptions are stored in the directory etc/InterfaceDescriptions within the xBus home directory. Information about its structure are listed in the section XML interface file descriptions.

The RecordLineLength instructs the parser to count bytes to separate the records. The receiver must produce a ByteArrayList to use this. The default for recognising the records is searching for line breaks. This default behivour is changed by setting the parameter to the line length in bytes. All records must have the same length in this case.

The LineCutting instructs the parser fill ending fields if ending blanks are cut off in the source if it is set to true. The default value is false. The records must be identified by line breaks to use the feature.

Entries for the Destination Interface

Chapter Section Key Value
System destination interface name Message RecordTypeMessage
System destination interface name Sender the sender class to use
System destination interface name DescriptionFile name of the destination structure description XML file without path

Most configuration entries are just like for the source interface - only describing destination features. The only new is the sender key. It specifies which method to use for writing to the destination system. Usually for RecordTypeMessages this is a sender for writing files. The record type transformation works with senders which write data in a string-oriented or a byte array oriented manner. Switching the Sender selects between both alternatives. Receiver and sender may even differ in this aspect. Reading byte arrays and writing strings is as well possible as the contrary. (For information on the different senders, please refer to the chapter Technical Layer.)

The DescriptionFile entry key is once again the only particular to interfaces with record transformation. Even it is not necessary for byte array data transfer without transformation. It tells the system where to find the description of the destination interface data structure. This structure description is essential for serializing the internally used DOM tree. All XML interface descriptions are stored in the directory etc/InterfaceDescriptions within the xBus home directory. Information about its structure are listed in the section XML interface file descriptions.

Entries for the Routing

Chapter Section Key Value
Router source interface name.function Invoken destination interface name
Router source interface name.function Distributen destination interface name

RecordTypeMessages can only be used with invoke distribute routing. In the first case the pesponse part of the message will be populated with the transformation result, in the second the request part.

Entries for the Transformation

Chapter Section Key Value
XSLTStylesheet source interface name destination interface name XSL style sheet name
Transformer RecordTypeMessage RecordTypeMessage RecordTypeTransformer

The transformation class is always RecordTypeTransfomer. It performs the transformation by application of a XSL style sheet to the internally used DOM tree of the source data. Which style sheet to choose is determined by the described key. The XSL file is specified by its name without any path because all style sheets are stored in the directory etc/xsl within the xBus home directory. Information about its structure are listed in the section XSL transformation style sheets.

Entries for the XML Parser

Chapter Section Key Value
ParserSettings RecordTypeMessage IgnoringComments true
ParserSettings RecordTypeMessage IgnoringElementContentWhitespace true
ParserSettings RecordTypeMessage XMLValidating false
ParserSettings RecordTypeMessage CheckingActivated false

The parser settings determine how the interface description documents are parsed. The first two parameters have to be "true" because the RecordTypeParser and RecordTypeSerializer classes are not capable of treating DOM tree nodes without substantial content, i.e. comment nodes and text nodes with formating whitespace sequences. The last two parameters should be set to "false" to avoid checking the interface descriptions again and again. Activating the checks is interesting during developing new interface descriptions in XML. See also Configuration of parser options.

 XML Interface File Descriptions

The structure of the source and the destination file is described in XML. The respective file sticks to the data type definition InterfaceSpec.dtd:


<?xml version="1.0" encoding="UTF-8"?>

<!ELEMENT InterfaceSpec ( Header?, Lines, RecordTypes, Trailer?, Groups? ) >
<!-- Root element -->

<!ELEMENT Header ( (Field | Group)+ ) >
<!-- File header -->

<!ELEMENT Lines ( RecordGroup+ | RecordType+ ) >
<!-- List of all record types and specification of their order.
If <RecordOrder>="Structured", records are grouped, thus the first level below <Lines> is the list of record groups. In this case the record types are listed for each group one level below.
Otherwise <Lines> driectly contains the list of all records - no record groups are present. -->
<!ATTLIST Lines RecordIdentification (TypeIdentifier | RecordOrder) #REQUIRED >
<!-- <RecordIdentification> may be
"TypeIdentifier" - The records are identified by a contained id field.
"RecordOrder" - The record type is determined by the line order. Works only with <RecordOrder>="Ordered"/"Structured". -->
<!ATTLIST Lines IdentifierPos NMTOKEN #IMPLIED >
<!-- Necessary for <RecordIdentification>="TypeIdentifier":
Position in row. Counting starts at 0. -->
<!ATTLIST Lines IdentifierLength NMTOKEN #IMPLIED >
<!-- Necessary for <RecordIdentification>="TypeIdentifier" -->
<!ATTLIST Lines RecordOrder (Arbitrary | Ordered | Structured) #REQUIRED >
<!-- <RecordOrder> may be
"Arbitrary" - The records are in arbitrary order. Works only with <RecordIdentification>="TypeIdentifier".
"Ordered" - The records are ordered due to the record type order in the <Lines> list. Record types may be optional. Together with <RecordIdentification>="RecordOrder" all optional record types must have an existance indicator if the structure contains more than one record type, cf. RecordType attributes.
"Structured" - Records form groups and the interface content can be described as a list of such groups. In this case the possible groups are specified one level below <Lines>. The record types are listed for each group another level deeper. Their order within the groups is defined by the corresponding <RecordType> specification. Record types - despite of the first one - may be optional. The first record type for each group must have cardinality "1". Together with <RecordIdentification>="RecordOrder" all optional record types must have an existance indicator, cf. RecordType attributes. Groups of records appear in arbitrary order and cardinality in the interface content. Only if <RecordIdentification>="TypeIdentifier" groups of different structure may be defined - otherwise there may be only one record group specification. -->

<!ELEMENT RecordGroup ( RecordType+ ) >
<!-- Used if <RecordOrder>="Structured" to define the structuring groups.
There may only be several group specifications if <RecordIdentification>="TypeIdentifier" beacuse otherwise the different group types cannot be distinguished.-->

<!ELEMENT RecordType ( #PCDATA ) >
<!-- Declaration of a record type -->
<!ATTLIST RecordType Identifier CDATA #IMPLIED >
<!-- Necessary if <RecordIdentification>="TypeIdentifier".
Specifies the used record type identifier value. -->
<!ATTLIST RecordType IdentifierLow NMTOKEN #IMPLIED >
<!ATTLIST RecordType IdentifierHigh NMTOKEN #IMPLIED >
<!-- If <RecordIdentification>="TypeIdentifier"
but records are numbered in their identifier field, these two attributes are used to specify the first and the last possible id. The order relation is defined as the lexicographical string order. Used record identifiers must be ascending and between the two limits but need not to be subsequent. -->
<!ATTLIST RecordType Occurrences CDATA #REQUIRED >
<!-- Declares the number of occurrences
- within the whole interface file (<RecordOrder>="Ordered" or "Arbitrary") - within a record group (RecordOrder=Structured). -->
<!ATTLIST RecordType ExistIndicatorField NMTOKEN #IMPLIED >
<!ATTLIST RecordType ExistIndicatorValue CDATA #IMPLIED >
<!-- Necessary if a specific field value in the preceding record
indicates that a record of this type follows. (Value set <=> Record follows) In particular essential if <RecordIdentification>="RecordOrder" and this record type is optional.-->

<!ELEMENT RecordTypes ( RecordTypeSpec+ ) >
<!-- Structure descriptions of all record types -->

<!ELEMENT RecordTypeSpec ( (Field | Group)+ ) >
<!-- Structure description for a single record type. -->

<!ELEMENT Trailer ( (Field | Group)+ ) >
<!-- File trailer -->

<!-- Occurrence of a group in a record.
A group constists of fields which occurr repeatedly in this structure. -->

<!ELEMENT Groups ( GroupSpec+ ) >
<!-- Structure description of all field groups occurring in the records. -->

<!ELEMENT GroupSpec ( Field+ ) >
<!-- Structure description of a single field group. -->

<!-- Specification of a single field -->
<!ATTLIST Field Format ( alpha | blank | const | date | num ) #REQUIRED >
<!-- Necessary if <Format>="const".
Specifies the constant value. -->
<!-- Necessary if <Format>="date".
Format strings like "yyMMdd" should be used which obey the java.text.SimpleDateFormat conventions. -->
<!ATTLIST Field DecimalPoint ( comma | dot ) #IMPLIED >
<!-- Necesssary if <Format>="num" and float values may be specified. -->

The root node in the interface description is labeled <InterfaceSpec>. Its name attribute contains the name of the specified interface. For the destination system this name must coincide with the interface name in the standard.conf and the root tag of the internally used destination DOM tree - given by the invoked XSL style sheet. For the source system it will be used as the root tag of the internally used source DOM tree.

The interface description may contain the five information blocks listed as children of the root note. The <Lines> and the <RecordTypes> sections are mandatory because they explain the record structure in the interface file.

The <Lines> section lists all record types which may be contained in the file. Additionally it states how record types can be identified in the file. For detailed information about this issue, cf. the comments in the DTD and the corresponding tips section.

The structure of all record types is given in the <RecordTypes> section. Thus for each record type its fields are listed. If a group of fields occurs multiple times, it may referred as group on each occurrence. In this case, its structure - i.e. contained fields - is explained in the <Groups> section. Thus the details for fields are found in the <RecordTypes> section and in the <Groups> section. A field within a group is only specified in the corresponding group specification. Its occurrences in record types are defined by references to this group.

Besides the <RecordTypes> section and the <Groups> section, field specifications occur within the <Header> or <Trailer> specification. These two sections are optional because an interface file may not have a header or a trailer.

Field names are arbitrary as long as they are legal XML identifiers. Only in case of existence identifiers indicating the type of the next record they must be unique.

Fields may have different data types. The record type parser will check if read data conforms to this. The types determine how values will be expanded to fit a certain field length in serialization. The types are

Strings. No type-specific checks are applied during They are trimmed during processing. For expanding them to a certain field length blanks are added at the end.
Record fillers which only contain blanks. Other characters in them will issue an error. Fillers are ignored in the record type transformation. Parsing just checks them and serializing just adds them.
Constant values which are checked against their specified default. They are often used to identify record types during parsing. Into the transformation result they are directly inserted. Thus no length correction is applicable.
Values of date fields must confirm to the specified pattern. Any modification of the date format should be done in the transformation XSL style sheet. Length correction during serializing is not applicable for date fields.
Numerical values may be integers or floating point numbers. In the later case the decimal point style must be chosen between using a comma or a dot. The number digits may be preceded by the number's sign '-' or '+'. The parser and the serializer check if values in numerical fields are numbers in this sense. Additionally they accept empty and completely blank strings. If necessary to fit the field length, the serializer cuts off zeros - in the case of integers leading zeros (behind the sign), in the case of floating point numbers first leading zeros (behind the sign before the decimal point) and then trailing zeros after the decimal point. If the value length is to be extended, the serializer insert leading zeros (behind the sign).

The XML parser used in the xBus may be configured to check the interface file descriptions against the DTD listed above. But some more rules apply to the interface descriptions:

  • All records types declared in the <Lines> section have to be specified in the <RecordTypes> section.
  • All records types described in the <RecordTypes> section must be declared in the <Lines> section.
  • Record types must have unique names within the description of an interface file.
  • Record type names must be legal XML tag identifier.
  • In any situation the record type must be uniquely determined when reading a record.
  • If the record type order is arbitrary, the record must contain a type identifier to identify them.
  • Different record groups are only allowed if the record types are recognized by an identifier.
  • If the record type is recognized by identifiers, the identifier position and length must be specified by the corresponding attributes in the <Lines> section. The position has to be a positive integer (>=0), the length a strictly positive integer (>0). Furthermore the length has to be identical for all used identifiers.
  • If the record type is recognized by identifiers, all record type declarations must define an identifier value or an interval of such values. Evidently, the identifier must be unique within the interface structure description. In particular, identifier ranges may not contain values used for other record types.
  • If the records are ordered by their type or by repeating groups, each record type declaration needs a cardinality constraint by the Occurences attribute in the <Lines> section. The specified cardinality must be a single strictly positive integer or a valid interval.
  • The first record type within a record group must have cardinality 1 to identify the group start.
  • If a field contains values to indicate the type of the following record, all these values must have the same length and this length must coincide with the length of the referred Field within all possibly preceding record types. (Note that each value is defined in the declaration of the following record type.)
  • Fields referred as existence indicator for a record type must be unique in each possibly preceding record type.
  • If the record types are not given by identifiers, each optional record type must declare an existence identifier - thus a field and a value in the preceding record to decide about the following record type. Optional record types are those with cardinality intervals.
  • Attribute groups must be uniquely named.
  • Each group reference in a record type definition must refer to a group definition in the <Groups> section.
  • Field lengths have to be strictly positive integers.
  • Specifications for date fields must declare the date format.
  • Specifications for constant fields must declare the constant value. This value must have the same length as the field.
The xBus can be configured to check all these rules. This serves mainly debugging purposes. Once the interface descriptions are established, there is no need for checking them again and again. For simplicity the checking option applies to all interface descriptions.

Hints for producing XML structure descriptions are listed in Tips for XML structure descriptions.

 XSL Transformation Style Sheets

The transformation operates on the internally used DOM tree which is constructed by the parser. Its tags are made up by the element names in the structure description - of the source interface for the pre-transformation status and of the destination interface for the transformation result. The chapter Record type transformation explains this structure in detail.

For configuring a record type transformation one constraint must be observed to get the xBus working:

The root tag of the destination DOM tree must be identical to the used destination interface name in the standard.conf and the root tag name attribute in the destination XML file description.

Hints for producing transformation stylesheets are listed in Tips for XSL Transformation Style Sheets