HL7 Programming using Java
Introduction
Building custom applications using the HL7 standard can be a very daunting task for a beginner. I have built a number of custom HL7 server applications over the years and couldn’t have done it without the help of others in the field through my many discussions with them over the Internet. As I got more proficient, I also received a number of emails from other developers/customers asking the same questions that I did when I was getting started on this standard. Therefore, in the spirit of sharing, I decided to put together an example/tutorial illustrating all the minimal basics any programmer dealing with the HL7 V2 standard should know when getting started.
This is part of my HL7 article series. Before we get started on this tutorial, have a quick look at my earlier article titled “A Very Short Introduction to the HL7 2.x Standard”. Please note that this tutorial assumes you know Java or any equivalent object-oriented language. A basic understanding of network as well as thread programming will be useful but is not necessary.
“A problem well stated is a problem half solved.” ~ Charles Kettering
A couple of things to note before we proceed. Although the HL7 standard itself does not recommend any specific protocol to use for message communication, most HL7 systems communicate using the Minimum Lower Layer Protocol. This tutorial will show you how to build custom HL7 servers in Java using this protocol. Advanced topics such as robust error handling, error logging and notifications, message translation as well as data persistence are not covered in this tutorial. However, I do plan to write another tutorial on those topics soon.
What is Minimum Lower Layer Protocol?
Minimum Lower Layer Protocol (often shortened to “MLLP”) is the most popular protocol used for transmitting HL7 messages using TCP/IP. Because TCP/IP transmits information as a continuous stream of bytes, a wrapping protocol is required for communications code to be able to recognize the start and the end of each message. MLLP is often used to address this requirement typically using non-printable characters to serve as wrapping characters around the core HL7 message information. These wrapping characters help envelope the HL7 message payload effectively as a "block" of data which is then transmitted to the receiving system. The receiving system then unwraps the message payload, parses the message content and then acknowledges the receipt of this message by returning a response message (known as as "ACK" or a "NACK" message in HL7) by wrapping its message content as well. In MLLP-based systems, the source system shall typically not send any new messages until the acknowledgement message of a previously transmitted message is received.
Something to keep in mind is that the MLLP Block that I described above is framed by the use of characters that have single-byte values. These characters have to agreed upon by the sites exchanging information since the data in the message content can potentially conflict with the byte values used for framing the MLLP block required to carry the message. Also,the use of multi-byte character encodings such as UTF-16 or UTF-32 is not ideal in these situations since this may result in byte values of the data which may equal the MLLP framing characters leading to MLLP enveloping-related errors. So,use of only single byte value encoding such as UTF-8, ISO-8859-x, etc is recommended. Please see official HL7 documentation for more on this area.
What are Sockets?
You can think of sockets as an abstraction of “terminals” of a connection between two machines transmitting information through either TCP or the UDP protocols. Most modern languages such as Java, C# and Ruby provide socket-programming support through a rich library of classes and/or interfaces. This reduces the burden of application programmers having to deal with lower layers of the OSI (Open Systems Interconnect) model of the ISO (International Standards Organization) standard. When using sockets, the programmer is working with the “Session” layer, and therefore shielded from having to deal with real-time data communication services provided by the lower layer of the OSI stack. Java provides excellent support for socket programming (through the java.net package). We will be using the “Socket” and the “ServerSocket” classes for the examples to be illustrated below.
Tools you need
- JDK 1.4 SDK or higher
- Eclipse or any other Java IDE (or even a text editor)
- You can also find all the code demonstrated in this tutorial on GitHub here
Getting Started
We will take small steps so all the core elements are covered
-
Build a simple TCP/IP client capable of transmission and receipt of information.
-
Build a simple TCP/IP server capable of receiving and echo-ing of received information
-
Build simple threading support to handle many concurrent connections from clients
-
Add MLLP and message acknowledgement features into the server
-
Write a small TCP/IP client that will transmit a dummy MLLP-wrapped HL7 message
-
Modify the server class to build MLLP-based parsing functionality
-
Modify the server class to build a simple acknowledgment functionality
-
Put it all together
Step 1 of 8 - Build a simple TCP/IP client capable of transmission and receipt of information
Here is a really simple TCP client that we will use for understanding how sockets work. We will use this client to primarily test our server application through the various steps.
package com.saravanansubramanian;
import java.net.*;
import java.io.*;
public class SimpleTCPEchoClient {
public static void main(String[] args) throws IOException {
String testMessage = "This is a test message that the client will transmit";
byte[] byteBuffer = testMessage.getBytes();
// Create socket that is connected to a server running on the same machine on port 1080
Socket socket = new Socket("localhost", 1080);
System.out.println("Connected to Server");
InputStream in = socket.getInputStream();
OutputStream out = socket.getOutputStream();
// Send the message to the server
out.write(byteBuffer);
in.read(byteBuffer);
System.out.println("Message received from Server: " + new String(byteBuffer));
// Close the socket and its streams
socket.close();
}
}As you can see from the code, the “Socket” class is used to establish a connection with another listener application running locally, listening on port 1080. Once a connection is established, one can read and write through this connection by using the input and output streams. When we are done, we simply close the socket.
I’ve learned that people will forget what you said, people will forget what you did, but people will never forget how you made them feel. ~ Maya Angelou
Tip: In industrial strength applications, if one cannot establish a connection right away, or if an existing connection is dropped for unforeseen reasons (if the server application crashes, or is shut down accidentally), client applications should retry a certain number of times before giving up. I will not be illustrating this functionality as it is not in the scope of this tutorial; however, it is a very useful feature to include in your HL7 application.
Step 2 of 8 - Build a simple TCP/IP server capable of receiving and echo-ing of received information
Now that we have seen a TCP client example in the previous step, here is a simple TCP server that can accept a connection from the client through a specified port, receive any information transmitted by it, and simply echo it back. It will continue running forever in a loop. We typically build server applications in Java using the “ServerSocket” class. This version of our server can accept and processing only one connection at a time. Once a client connection is accepted by the server, we can read and write through the input and output streams provided by this connection.
package com.saravanansubramanian;
import java.net.*;
import java.io.*;
public class SimpleTCPEchoServer {
private static final int BUFFER_SIZE = 200;
public static void main(String[] args) throws IOException {
ServerSocket serverSocket = new ServerSocket(1080);
int receivedMessageSize;
byte[] receivedByeBuffer = new byte[BUFFER_SIZE];
while (true) {
Socket clientSocket = serverSocket.accept(); // Get client connection
System.out.println("Handling client at " +
clientSocket.getInetAddress().getHostAddress() + " through port " +
clientSocket.getPort());
InputStream in = clientSocket.getInputStream();
OutputStream out = clientSocket.getOutputStream();
receivedMessageSize = in.read(receivedByeBuffer);
out.write(receivedByeBuffer, 0, receivedMessageSize);
clientSocket.close(); // Close the socket. We are done serving this client
}
}
}Tip: In production HL7 server applications, you will have to ensure that the server does not crash by implementing rugged error handling as well as “auto-restart” feature in case the server does crash. Also, you will want to send error notifications through emails in case manual intervention is required quickly or write to event logs of the operating system.
Step 3 of 8 - Build simple threading support to handle many concurrent connections from clients
The client and server examples illustrated so far are extremely simplified examples to help you understand the basics of socket communications. The server we have implemented is capable of handling only one connection at a time. To handle multiple connections, you will have to use threads. Threads are very easy to implement in Java. But, they are also very easy to misuse. When implementing threads in Java, you will have to ensure that the various threads don’t interfere with one another’s work. This is because threads share the same memory. In this modified version of the server, we instantiate a new thread, and pass every incoming connection that is accepted by the server to its own ConnectionHandler class. This will ensure that the server is quickly ready to accept the next client waiting to connect to the server without any noticeable delay.
package com.saravanansubramanian;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.BindException;
import java.net.ServerSocket;
import java.net.Socket;
public class SimpleThreadedEchoServer {
private int listenPort;
public SimpleThreadedEchoServer(int aListenPort) {
listenPort = aListenPort;
}
public static void main(String[] args) {
SimpleThreadedEchoServer server = new SimpleThreadedEchoServer(1080);
server.acceptIncomingConnections();
}
private void acceptIncomingConnections() {
try {
ServerSocket server = new ServerSocket(listenPort, 5); //Accept up to 5 clients in the queue
System.out.println("Server has been started");
Socket clientSocket = null;
while (true) {
clientSocket = server.accept();
handleIncomingConnection(clientSocket);
}
} catch (BindException e) {
System.out.println("Unable to bind to port " + listenPort);
} catch (IOException e) {
System.out.println("Unable to instantiate a ServerSocket on port: " + listenPort);
}
}
protected void handleIncomingConnection(Socket aConnectionToHandle) {
new Thread(new ConnectionHandler(aConnectionToHandle)).start();
}
private static class ConnectionHandler implements Runnable {
private Socket connection;
private int receivedMessageSize;
private byte[] receivedByeBuffer = new byte[BUFFER_SIZE];
private static final int BUFFER_SIZE = 32;
public ConnectionHandler(Socket aClientSocket) {
connection = aClientSocket;
}
public void run() {
try {
System.out.println("Handling client at " + connection.getInetAddress().getHostAddress()
+ " on port " + connection.getPort());
InputStream in = connection.getInputStream();
OutputStream out = connection.getOutputStream();
receivedMessageSize = in.read(receivedByeBuffer);
out.write(receivedByeBuffer, 0, receivedMessageSize);
connection.close(); // Close the socket. We are done serving this client
} catch (IOException e) {
//In real-life, do something about this exception
System.out.println("Error handling a client: " + e);
}
}
}
}Tip: In many HL7 systems in use around the world, multiple clients may attempt to connect to a server at the same time (either on different ports, or on the same port). The server application must be able to accept all these connections without too much delay, and then receive, process and acknowledge any information received from each client.
Step 4 of 8 - Add MLLP and message acknowledgement features into the server
Even though the examples illustrated so far allow the client and server to transmit simple text messages back and forth, this will not be enough to communicate with HL7 systems. Communication with HL7 systems will require the use of MLLP protocol and message acknowledgment features. In this step, we will take our existing server and add code to enable MLLP support. We will also implement functionality in the server to create and transmit a simple acknowledgement message back to the client application.
More on MLLP
At the start of this tutorial, I described the MLLP protocol as a wrapping protocol where the core HL7 message is enveloped in special characters to signal the start and end of transmission of message information to the receiving application. When dealing with MLLP, there are essentially three character groups that you will configure and use. They are start block, end block and segment separator. HL7 messages transmitted using the MLLP protocol are prefixed with a start block character, message segments are terminated with a segment separator, and the messages themselves are then terminated with two characters namely an end block and a carriage return character. Most often, the character typically used to signify start block is a VT (vertical tab), which is ASCII 11. The FS (file separator), ASCII 28, is used to signify end block, and CR (ASCII 13), is used for carriage return.
Here is an example of an observation request HL7 message “enveloped” for transmission using the MLLP-related wrapping characters:
<SB>MSH^~\&199809091533ORU^R01856369D2.2<CR>
PID139385000343456<CR>
ORC123112312RAD19980909-0005<CR><EB><CR>Step 5 of 8 - Write a small TCP/IP client that will transmit a dummy MLLP-wrapped HL7 message
Here is the original TCP client modified to transmit a sample HL7 message. Notice how the test message is wrapped in the special characters as explained earlier (highlighted in bold).
package com.saravanansubramanian;
import java.net.*;
import java.io.*;
public class SimpleMLLPBasedTCPClient {
private static final char END_OF_BLOCK = '\u001c';
private static final char START_OF_BLOCK = '\u000b';
private static final char CARRIAGE_RETURN = 13;
public static void main(String[] args) throws IOException {
// Create a socket to connect to server running locally on port 1080
Socket socket = new Socket("localhost", 1080);
System.out.println("Connected to Server");
StringBuffer testHL7MessageToTransmit = new StringBuffer();
testHL7MessageToTransmit.append(START_OF_BLOCK)
.append("MSH|^~\\&|AcmeHIS|StJohn|CATH|StJohn|20061019172719||ORM^O01|MSGID12349876|P|2.3")
.append(CARRIAGE_RETURN)
.append("PID|||20301||Durden^Tyler^^^Mr.||19700312|M|||88 Punchward Dr.^^Los Angeles^CA^11221^USA|||||||")
.append(CARRIAGE_RETURN)
.append("PV1||O|OP^^||||4652^Paulson^Robert|||OP|||||||||9|||||||||||||||||||||||||20061019172717|20061019172718")
.append(CARRIAGE_RETURN)
.append("ORC|NW|20061019172719")
.append(CARRIAGE_RETURN)
.append("OBR|1|20061019172719||76770^Ultrasound: retroperitoneal^C4|||12349876")
.append(CARRIAGE_RETURN)
.append(END_OF_BLOCK)
.append(CARRIAGE_RETURN);
InputStream in = socket.getInputStream();
OutputStream out = socket.getOutputStream();
// Send the MLLP-wrapped HL7 message to the server
out.write(testHL7MessageToTransmit.toString().getBytes());
byte[] byteBuffer = new byte[200];
in.read(byteBuffer);
System.out.println("Received from Server: " + new String(byteBuffer));
// Close the socket and its streams
socket.close();
}
}Step 6 of 8 - Modify the server class to build MLLP-based parsing functionality
In this step, we will add a method to the server (inside the Connection Handler Class) to handle the parsing of the stream of characters received from the client using the socket stream. The gist of this parsing routine is that it will expect MLLP wrapping characters in the correct order/sequence. If not, it will throw an exception and close the connection with the client.
public String getMessage(InputStream anInputStream) throws IOException {
boolean end_of_message = false;
StringBuffer parsedMessage = new StringBuffer();
int characterReceived = 0;
try {
characterReceived = anInputStream.read();
} catch (SocketException e) {
System.out
.println("Unable to read from socket stream. "
+ "Connection may have been closed: " + e.getMessage());
return null;
}
if (characterReceived == END_OF_TRANSMISSION) {
return null;
}
if (characterReceived != START_OF_BLOCK) {
throw new RuntimeException(
"Start of block character has not been received");
}
while (!end_of_message) {
characterReceived = anInputStream.read();
if (characterReceived == END_OF_TRANSMISSION) {
throw new RuntimeException(
"Message terminated without end of message character");
}
if (characterReceived == END_OF_BLOCK) {
characterReceived = anInputStream.read();
if (characterReceived != CARRIAGE_RETURN) {
throw new RuntimeException(
"End of message character must be followed by a carriage return character");
}
end_of_message = true;
} else {
parsedMessage.append((char) characterReceived);
}
}
return parsedMessage.toString();
}Step 7 of 8 - Modify the server class to build a simple acknowledgement functionality
Next, we will add two other methods to the server (again, inside the Connection Handler Class) to build simple HL7 message acknowledgements to transmit back to the client. The first method is to help build a basic message acknowledgement functionality. The method shown below is a verify simplified example to aid in the understanding how acknowledgement messages can be constructed. In reality, you will re-transmit other fields in the MSH and MSA segments such as sending facility, sending application, version of the application, and other useful information.
private String getSimpleAcknowledgementMessage(String aParsedHL7Message) {
if (aParsedHL7Message == null)
throw new RuntimeException("Invalid HL7 message for parsing operation. Please check your inputs");
String messageControlID = getMessageControlID(aParsedHL7Message);
StringBuffer ackMessage = new StringBuffer();
ackMessage = ackMessage.append(START_OF_BLOCK)
.append("MSH|^~\\&|||||||ACK||P|2.2")
.append(CARRIAGE_RETURN)
.append("MSA|AA|")
.append(messageControlID)
.append(CARRIAGE_RETURN)
.append(END_OF_BLOCK)
.append(CARRIAGE_RETURN);
return ackMessage.toString();
}The second supporting method is to help parse the message control id of the received message that will need to be re-transmitted in the acknowledgement message. Some portions of this second routine below may seem like an overkill, but I do this deliberately to illustrate how fields inside a HL7 message can be parsed using the field delimiter (once you can parse one field, you can apply the same concept to do the others). You may also want to raise an exception when the message control id is not found.
private String getMessageControlID(String aParsedHL7Message) {
int fieldCount = 0;
StringTokenizer tokenizer = new StringTokenizer(aParsedHL7Message, FIELD_DELIMITER);
while (tokenizer.hasMoreElements())
{
String token = tokenizer.nextToken();
fieldCount++;
if (fieldCount == MESSAGE_CONTROL_ID_LOCATION){
return token;
}
}
return "";
}Step 8 of 8 - Put it all together
We are done. We have essentially seen all the bits and pieces needed to assemble a custom HL7 application from scratch if you needed to. Often, instead of re-inventing the wheel, a HL7 programmer will use either freeware or commercial toolkits to build his/her HL7 messaging application. However, even when dealing with toolkits, a basic understanding of sockets, the MLLP protocol as well as message acknowledgement fundamentals that were covered in this tutorial is required. I am planning on writing a whole series of articles on HL7 programming using a popular HL7 library called "HAPI". So, watch for that on my blog soon.
You can find the full source code used in this tutorial on GitHub here
Conclusion
Building robust HL7 systems capable of running 24/7 and unattended is not easy. For most programmers, once they get past the initial socket-programming hurdles, the bulk of the issues in HL7 2.x-related programming will be around supporting any parsing, translating, formatting and storage requirements. With HL7 3.0* standard starting to come into use increasingly, these issues should go slowly away. However, HL7 2.x standard isn’t going anywhere for now. We should expect it to stay at least for 10 more years before they are fully retired. Until then, we will have to continue our quest to build more elegant and sturdier solutions for HL7 messaging.
Tip: You will want to design the system so that the MLLP character groupings are configurable. This way, the customer can change them to whatever they require them to be. You should also permit some configuration around the message acknowledgment functionality such as information that is transmitted about the processing system and its location (goes in the “MSH” segment). Also, a configuration to indicate whether the system is in “test” mode versus “production mode”. In the past, things that I have made configurable also include message control id starting number or number pattern, email addresses and error log locations (for error notifications), as well as connection retry attempts.
* - HL7 3.0 (RIM) never came to achieve the wide adoption that many people imagined to occur in the early 2000s and beyond including the replacement of V2-based message interfaces. Please see my article on the V3 standard for more information on what this standard attempted to do, its strengths, weaknesses and its legacy in the now long history of healthcare informatics.