HL7 Programming using Java and HAPI - Message Validation

Introduction

This is part of my HL7 article series. Before we get started on this tutorial, have a quick look at my earlier article titled “A Very Short Introduction to the HL7 2.x Standard”. So far in this series of articles, we have looked at how to create, send, receive and parse HL7 messages using the HAPI HL7 library. However, the ultimate goal in any message exchange especially in healthcare environments is to ensure that the information is semantically understood to the highest degree possible by the various parties involved. This ensures that no erroneous misinterpretation of the data arises. In some situations, errors could even mean life or death for the patients involved. Achieving technical and semantic interoperability is not easy, especially in the HL7 2.x standard which has a lot of optionality built-in. However, this optionality is also the cause of many issues since a lot of data is either not sent when it should be, or the data is sent in a different part of the message than where the data is expected by the receiving party. Additional features (including Z-segments which we looked at in a previous tutorial) can cause even more opportunities for confusion as the two systems cannot be certain what to expect when messages are communicated between one another unless the specifications are drawn out and agreed upon well in advance. To solve these challenges a number of approaches are often utilized within HL7 message processing systems to ensure that any message interfaces conform to strict specifications that are agreed upon by the various parties involved, and are correctly validated during any exchange of message information between these parties. HAPI provides a number of features in this area, and we will explore some of them in this tutorial.

Tools for Tutorial

Overview of HL7 Message Validation Mechanisms in HAPI

At a very high level, the HAPI library enables us to build three "levels" of message validation functionality. They are basic message validation, custom message validation and validation by the use of conformance profiles. The basic message validation functionality that HAPI offers by default ensures that any HL7 message that we are either sending or receiving conforms to some basic rules around both length as well as optionality permitted around things such as segment groups, segments and primitive data types as specified by the HL7 message standard. However, this alone will not be sufficient as no two environments are ever the same and always require site specific customizations. Specific workflows in many healthcare environments always require additional data constraints to be in place, and these constraints need to be applied over and beyond what the HL7 2.x standard specifies. This is where custom message validation and conformance profiles come in. Custom message validation is basically an enhancement to the basic validation offered by HAPI, and is simply enabled through the extensibility offered by the message validation framework itself. It helps us bolt-on any additional rules for validation during message processing by overriding the default behaviour offered by message validation-related classes available within the library. They can for example help constrict the definition of a message specification in a manner that removes the optionality constructs and the general rules around the processing of a HL7 2.x message within a specific clinical workflow. The parties involved here can then codify these rules into their message exchange systems. This is easier said than done especially when this information needs to be exchanged in a unambiguous manner to assist in the development and testing of the message workflows independently first before the actual integration is activated. Conformance profiles (specifically known as "Message Profiles" in HL7 2.x) come to our rescue here by enabling the sharing of any stringent message specifications involving specific workflows to an open registry for everyone to use. Vendors and implementors can download these specifications and test their systems against them in isolation well in advance of any actual messaging implementation. This approach greatly assists in the goal of achieving or atleast moving towards "out of the box interoperability" saving a lot of money, stress and time for the implementors. In this tutorial, we will look at basic as well as custom message validation mechanisms available in the HAPI framework. I will cover the topic of conformance profiles as well as the features HAPI offers in that area in the next tutorial in this series.

Basic Message Validation

In our earlier tutorials on using the HAPI library, we looked at a special class called HAPIContext when needing to create parsers and connections, etc. This same class also plays an important role in the message validation functionality provided by HAPI. Any instance of HAPI Context works closely with a large family of cooperating classes and interfaces to help provide both out of the box as well as customized message validations within your HL7 application. You can also turn off message validation completely if you desire as well although this is not a recommended practice.

Because HAPI validation framework is so complex and it provides so many options for customization and configuration, it is nearly impossible to describe the behavior of each and every one of the classes and interfaces that play a role in the overall process in a short tutorial. I highly recommend that you review the source code of the HAPI library to get a deeper understanding of how this all works. However, I will explain just enough here so you can hopefully get a big picture understanding of how it all appears to work under the covers. Essentially, every HAPIContext has a special attribute called ParserConfiguration which plays a key role during the message parsing process. The parser configuration object has many methods that help control the type of "laxity" or looseness that is permitted during the parsing of a HL7 message. Methods on the parser configuration class help control parsing behavior such as the following:

  • Whether the message validation should be activated or not (it is turned on by default)
  • How to deal with unrecognized message versions, segments, fields, etc during the parsing process
  • Behaviour around how the parser should encode segments and fields even when if no data content is present

“A book is like a garden, carried in the pocket.” ~ Chinese Proverb

The HAPI context class also has another attribute called ValidationContext which manages a collection of codified rules for handling messages, fields and primitive types to be used during the parsing process. These rules themselves are created under the covers through the use of a special class called ValidationRuleBuilder. During the message parsing process (and when validation is turned on), the rules held in the validation context are applied against the actual message data by the use of a visitor pattern enabled by classes that implement an interface called MessageVisitor. These classes help traverse the entire message structure, and the rules held in the validation context of the message parser instance are applied against the various elements presents within the message structure to ensure rule compliance. A special exception class called ValidationException is used to signal any validation failures during this validation process.

The exception handling during the validation process can also be configured and controlled through the use of factory classes that implement a special interface called ValidationExceptionHandlerFactory which can be also configured on the HAPI context instance. Classes that implement this interface help produce instances of classes that implement another interface namely ValidationExceptionHandler which enable fine grained control of behavior on aspects such as what needs to happen before validation, during validation, and also when validation succeeds or fails. As you can see, the validation framework is quite complex, and shows the many aspects one must consider when building a message processing framework especially one that can scale and support the numerous versions, message types and trigger events that are part of the overall HL7 2.x standard. I think we have had enough theory for now. Let us look at some code.

    package com.saravanansubramanian.hapihl7tutorial.validation;

    import ca.uhn.hl7v2.DefaultHapiContext;
    import ca.uhn.hl7v2.HL7Exception;
    import ca.uhn.hl7v2.HapiContext;
    import ca.uhn.hl7v2.parser.PipeParser;
    import ca.uhn.hl7v2.validation.ValidationContext;
    import ca.uhn.hl7v2.validation.impl.ValidationContextFactory;

    public class HapiParserBasicMessageValidationDemo {

        private static HapiContext context = new DefaultHapiContext();

        public static void main(String[] args) {

            //We will look at four scenarios

            parsingValidMessage_Scenario1();

            parsingInvalidAttribute_Scenario2();

            parsingInvalidMessageWithValidationTurnedOff_Scenario3();
        }

        private static void parsingValidMessage_Scenario1() {

            String aValidAcknowledgementMessage = "MSH|^~\\&|SENDING_APPLICATION|SENDING_FACILITY|"
                    + "RECEIVING_APPLICATION|RECEIVING_FACILITY|"
                    + "20110614075841||ACK|1407511|P|2.4||||||\r\n" +
                    "MSA|AA|1407511|Success||";

            context.setValidationContext((ValidationContext)ValidationContextFactory.defaultValidation());

            try {
                PipeParser parser = context.getPipeParser();
                parser.parse(aValidAcknowledgementMessage);
                System.out.println("Scenario 1. Successfully validated a correct HL7 acknowledgement message");
            } catch (HL7Exception e) {
                //In real-life, do something about this exception
                System.out.println("Scenario 1. Validation failed during parsing:" + e.getMessage());
            }
        }

        private static void parsingInvalidAttribute_Scenario2() {

            //intentionally create bad data in the MSH-6 field which is a TS (Timestamp) data type
            String anInvalidAcknowledgementMessage = "MSH|^~\\&|SENDING_APPLICATION|SENDING_FACILITY|"
                    + "RECEIVING_APPLICATION|RECEIVING_FACILITY|"
                    + "AAAAAAAAAAAAA||ACK|1407511|P|2.4||||||\r\n" +
                    "MSA|AA|1407511|Success||";

            try {
                PipeParser parser = context.getPipeParser();
                parser.parse(anInvalidAcknowledgementMessage);
                System.out.println("Scenario 2. The code show not get here as the validation will fail");
            } catch (HL7Exception e) {
                //In real-life, do something about this exception
                System.out.println("Scenario 2. Validation failed during parsing:" + e.getMessage());
            }
        }

        private static void parsingInvalidMessageWithValidationTurnedOff_Scenario3() {

            //intentionally create bad data in the MSH-6 field which is a TS (Timestamp) data type
            String anInvalidAcknowledgementMessage = "MSH|^~\\&|SENDING_APPLICATION|SENDING_FACILITY|"
                    + "RECEIVING_APPLICATION|RECEIVING_FACILITY|"
                    + "AAAAAAAAAAAAA||ACK|1407511|P|2.4||||||\r\n" +
                    "MSA|AA|1407511|Success||";

            //now turn off validation off
            context.getParserConfiguration().setValidating(false);

            try {
                PipeParser parser = context.getPipeParser();
                parser.parse(anInvalidAcknowledgementMessage);
                System.out.println("Scenario 3. By disabling validation, we successfully parsed an invalid HL7 message");
            } catch (HL7Exception e) {
                //In real-life, do something about this exception
                System.out.println("Scenario 3. Validation failed during parsing:" + e.getMessage());
            }
        }

    }

The program console output from running our program is shown below. The first scenario illustrates the successful validation of a correct HL7 message. In the second scenario, we intentionally set the value in the MSH-6 field to a non-date time value (here the value is 'AAAAAAAAAAAAA'). This should trigger the validation exception to be thrown. In the third scenario, we turn off message validation completely. We are then able to parse the invalid message successfully. Please exercise extreme caution when turning off validation of your parser unless it is necessary for special circumstances. If you do have special customizations on your messages, and you require customized validations as a result, I will show how to address that scenario next.


Scenario 1. Successfully validated a correct HL7 acknowledgement message
Scenario 2. Validation failed during parsing:ca.uhn.hl7v2.validation.ValidationException: Validation failed: Primitive value 'AAAAAAAAAAAAA' requires to be empty or a HL7 datetime string at MSH-6
Scenario 3. By disabling validation, we successfully parsed an invalid HL7 message

Implementing Custom Message Validation

In many HL7 implementations, you will often see the use of message customizations over and beyond what is specified by the HL7 standard. These customizations are often enabled by any number of ways including the following:

  • By using Z-segments (See my previous tutorial on message parsing for more information)
  • By extending what are known as "code sets" in HL7. These are enabled by using lookup tables provided by the HL7 standard itself, or by external standard bodies such as ISO and sites may extend the values in these tables or create their own lookup tables
  • By modifying the rules optionality on certain fields that are not considered by the HL7 standard. For instance, PV1-3 (Assigned Patient Location) is not considered mandatory by HL7, but you and your partner system may want to make this mandatory in any message exchange.

Let us look at how to enable message validation in a fictional scenario where the PV1-3 is made mandatory using the HAPIContext class that we are already familiar with as well as by using ValidationRuleBuilder class that I had described earlier.

    package com.saravanansubramanian.hapihl7tutorial.validation;

    import ca.uhn.hl7v2.DefaultHapiContext;
    import ca.uhn.hl7v2.HL7Exception;
    import ca.uhn.hl7v2.HapiContext;
    import ca.uhn.hl7v2.Version;
    import ca.uhn.hl7v2.parser.PipeParser;
    import ca.uhn.hl7v2.validation.builder.support.DefaultValidationBuilder;

    public class HapiParserCustomMessageValidationDemo {

        private static HapiContext context = new DefaultHapiContext();

        public static void main(String[] args) {

            String adtMessage
            = "MSH|^~\\&|SENDING_APPLICATION|SENDING_FACILITY|"
                + "RECEIVING_APPLICATION|RECEIVING_FACILITY|20110613083617|"
                + "|ADT^A04|934576120110613083617|P|2.3||||\r\n" +
                "EVN|A04|20110613083617|||\r\n" +
                "PID|1||135769||MOUSE^MICKEY^||19281118|M|||"
                + "123 Main St.^^Lake Buena Vista^FL^32830||"
                + "(407)939-1289|||||1719|99999999||||||||||||||||||||\r\n" +
                "PV1|1|O|||||7^Disney^Walt^^MD^^^^|";

            //specify an override for our default validation behavior by injecting our own extension
            context.setValidationRuleBuilder(new OurSpecialMessageValidationBuilderClass());

            try {
                PipeParser parser = context.getPipeParser();
                parser.parse(adtMessage);
                System.out.println("Code should not get here");
            } catch (HL7Exception e) {
                //In real-life, do something about this exception
                System.out.println("Validation failed as expected during parsing since PV1-3 is now mandatory");
                System.out.println("Validation Message: " + e.getMessage());
            }
        }


    }

    @SuppressWarnings("serial")
    class OurSpecialMessageValidationBuilderClass extends DefaultValidationBuilder{

        @Override
        protected void configure() {
            super.configure();
            forVersion(Version.V23)
                .message("ADT", "A04")
                .terser("PV1-3", not(empty()));
        }
    }

Running the program above should result in the output as shown below in your console (the field PV1-3 should now be mandatory in our case). We also enabled this message validation for a specific set of criteria only (for ADT A04 messages belonging to HL7 V2.3 version only) and so all other default behavior around message validation will remain unaffected.


Validation failed as expected during parsing since PV1-3 is now mandatory
Validation Message: Validation failed:  'null' requires to be not empty at PV1-3(0)-1-1

Conclusion

This concludes a quick tutorial on the basic message validation mechanisms available within the HAPI library. However, the basic validation mechanisms covered here still have a number of limitations in terms of how they are often agreed on and complied with independently so that development and testing times are reduced. This is often a major consideration in large integration efforts involving multiple systems and implementors. The data constraints for the workflows involved need to be easily shared using standardized templates that can provide a precise definition of the data exchanged between applications in a common format. We will look at how these are enabled by using "conformance profiles" in our next tutorial. See you then!

* - Sometimes, I use a new term that I came up with myself as there was either no formal definition of a concept in the official documentation, or because I feel that the concept is better explained using this new terminology. When in doubt, always consult the official HAPI and HL7 documentation for final reference.