Architecture

The Open Integration Engine (OIE) is an open source integration engine for healthcare. It receives messages from one system, filters and transforms them under rules you define, and delivers them to other systems, in the formats those systems speak (HL7, X12, EDI, DICOM, XML, among others).

Core concepts

Channels

Channels are the fundamental building blocks of OIE. A channel is an interface you configure to receive data from a source, take actions on that data (filter, transform, extract), and send it out to zero or more destinations.

Each channel has general properties including a unique ID, name, and description. Channels can also be configured with code template library links, library resources, deploy/start dependencies (to control deployment order), attachment handler settings, message storage and pruning settings, and custom metadata columns.

Data types

A data type is the format of the message a channel is handling: HL7 v2.x, X12, DICOM, JSON, and so on. It is also the instruction for reading that format. The inbound data type parses the incoming message into an internal representation that filters and transformers can work with. The outbound data type serializes that representation into the format that leaves the channel.

The internal representation varies by data type. For example, HL7 v2.x data is parsed into an E4X XML object, JSON data becomes a JavaScript Object, and Raw data remains a Java String, unparsed.

Supported data types include:

• Delimited Text
• DICOM
• EDI / X12
• HL7 v2.x
• HL7 v3.x
• JSON
• NCPDP
• Raw
• XML

See Data Types and Formats for details.

Connectors

A channel consists of multiple connectors:

• Source Connector. Gets data into the channel. Every channel has exactly one.
• Destination Connectors. Sends data out. A channel is configured with at least one, and may have several. A given message may still reach none of them, if the source filter rejects it or destination set filtering removes them all. Each has an enabled flag and a wait for previous destination setting that decides its chain membership.

A connector's type decides its configuration. A TCP Listener asks for a local address and port, an HTTP Listener for a base context path, a Database Reader for a query and a polling schedule. The fields belong to the type, and no two types offer the same set, so picking the type is most of the configuration decision.

Every source connector also carries a common block of source settings, though not all of them apply to every type:

• a source queue, which when on accepts the message and responds immediately rather than waiting for processing to finish. Turning it on enables a queue buffer size and changes what the response can be set to
• a response selector, choosing which response goes back
• batch processing, splitting one inbound payload into several messages. Offered only by types that support batching
• max processing threads, controlling how many messages the channel processes at once

The attachment handler is a channel setting rather than a connector one, but it acts on data as it arrives: it pulls large or repeated content out of the message and stores it once, leaving a reference behind. On traffic carrying images, documents, or any payload that recurs across messages, this can cut storage dramatically.

For example, you may receive data over HTTP, then write it out to a file and insert pieces of it into a database.

See Channels, and Source Connectors and Destination Connectors for what each type exposes.

Filters

A filter decides whether a message should proceed to the next step or not. A filter returns true (accepted) or false (filtered). Filters are comprised of multiple rules, joined by AND or OR operators.

• If the source connector filters out a message, it will not be processed by any destination connectors.
• If a destination connector filters out a message, other destinations may still process it.

Transformers

A transformer modifies a message, converts it from one format to another, and extracts pieces for later use. A transformer has an inbound data type and an outbound data type (e.g., HL7 v2.x to JSON). Transformers are comprised of multiple steps, each of which modifies or extracts data.

See Filters and Transformers for details.

Response transformers

Every destination can have a response transformer, which runs on what the receiving system sends back. Use it to read an acknowledgement, decide whether the send really succeeded, and set the status the message ends up with. It is the only transformer that runs after a connector rather than before one.

Channel scripts

Four special scripts are associated with each channel:

• Deploy Script. Runs once right before a channel is deployed.
• Preprocessor Script. Runs once for every message, after the attachment handler has optionally extracted data, but before the source filter/transformer. Used to modify the incoming message.
• Post Processor Script. Runs once for every message, after the source connector and all destinations have completed (excluding asynchronous processes like the destination queue). Has access to responses from all executed destinations and can return a custom response for the source connector to use.
• Undeploy Script. Runs once after the channel has been stopped and its connectors undeployed.

How a message moves

Destination 2 follows destination 1 in the diagram because it waits for the previous destination, which is the default. With that setting off, each destination starts a chain of its own and the chains run concurrently.

Neither end of that diagram has to be outside OIE. A Channel Reader source receives from Channel Writers on other channels, a Channel Writer destination hands the message to another channel on the same engine, and a script can dispatch to one through router.

Each box in the diagram is a step, and every step after the first is optional or conditional:

1. The source connector receives the message.
2. The attachment handler, if one is configured, pulls content out of the message and leaves a reference in its place.
3. The preprocessor script can modify the raw message before anything parses it.
4. The source filter decides whether the message goes on, and the source transformer converts and extracts from it. A message filtered here reaches no destination.
5. For each destination: its filter and transformer run, then its connector processes the message, which may or may not result in a send.
6. That destination's response transformer handles whatever comes back.
7. The postprocessor script runs once every destination has finished, and can build the response itself.
8. If something is waiting on the other end, the response goes back to it. A listener source such as an HTTP, TCP, or Web Service Listener returns it to the caller. A polling source such as a Database Reader or File Reader has no caller, so the response is selected and stored on the message but sent nowhere.

Four pages explain the mechanisms behind those steps:

• Message Lifecycle and Storage, the processing path, statuses, content types, and what each storage mode keeps
• Queueing, the source queue and destination queues
• Threading and Ordering, which thread does the work, chains, and response selection
• Deploy, Start, and Channel State, what deploy does and what each state means

The Glossary collects the terms these pages use.

Administration interfaces

• Administrator. A Java Swing desktop client for managing channels, viewing messages, configuring settings, and monitoring the system. Connects to the server over HTTPS using a RESTful API. Requires a compatible launcher; see Desktop Administrator for the available options.
• Web Administrator. A browser-based administrator installed as the Web Support extension and served by the engine's own embedded Jetty. Read/write against the same REST API, so it and the desktop client can be used against one engine at the same time. Requires 4.6.0.
• Command Line Interface (CLI). A terminal-based client for scripting and headless administration. Uses the same REST API as the Administrator.

Extensions

OIE uses a plugin architecture where connectors, data types, and other features are loaded as extensions. Extension types include:

• Connectors. Source and destination communication protocols
• Data Type Plugins. Message format parsers and serializers
• Server Plugins. Background services and event handlers
• Transmission Mode Providers. Frame-level protocol handling (e.g., MLLP)

Extensions are loaded from the extensions directory when the engine starts. An extension can ship a desktop UI, a web UI, or both; the web administrator serves the web side. A growing set of open source extensions is available beyond those bundled with the engine. See the Plugin Guide, and openintegrationengine.org/plugins for the community catalog.