Author: Joe Welsh - Splunk
Version/Date: 1.13 July 14, 2020
Supported product(s):
This add-on supports logs containing HL7 v2.X data.
Source type(s): This add-on will process data that is source-typed
as "hl7_v2".
Input requirements: N/A
To be used with HL7 Dashboard Examples App for Splunk ( )
Configuration
To use this add-on, manually configure the data input with the following input.conf
properties and the add-on will optimize the data automatically.
sourcetype = hl7_v2
These properties will also be utilized by the underlying searches and eventtypes .
The value from this Add-on is derived by it's ability to:
If the logs come from Cloverleaf's Interface engine there are additional extractions for thread_id, client ip, and client port.
Notes:
*The datetime format can vary for each HL7 message. In lieu of using TIME_FORMAT inside props.conf
a custom datetime.xml was created inside this app. The extracted groups correspond to those specified
in the extracted attribute for define element. The groups (year, month, day, etc.) are specified in
Splunk's default datetime.xml in /etc/system/datetime.xml.
**Repeating Segments will be placed inside multi-value fields.
i.e. If there are four NTE segments in the message there will be one multi-value NTE field with four values.
The subsequent HL7 fields will only be extracted for the first segment in repeating segments.
The reference files used for this Add-on can be found in the cloverleaf_logs directory.
The samples directory contains files used to generate sample messages. If you install Splunk's Eventgen App v7.2+ [https://splunkbase.splunk.com/app/1924/#/overview] these samples will be replayed and put in your main index.
You'll also need to modify splunkUser and splunkPass in eventgen.conf to match credentials on your system.
HL7_Segment_Template transformation uses MV_ADD = true to aggregate repeating segments. However, you ultimately need to construct a mv field using the makemv SPL command to split them out for use. However, I was unable to split it using the newline character as a delimiter, needed to add an anchor to the end of each line first.
So, take this sample message with repeating DG1 segments:
MSH|^~\&|EPIC|MGH||MGH|201503241909|OHEDSCRIBE|ADT^A08|725467|T|2.3|||||||||
EVN|A08|20150324190937||REG_UPDATE|^EMERGENCY^SCRIBE^OH^^^^^OHSA^^^^^MGH||
PID|1||3100068701^^^EPI^MR||MGHTEST^EDSCRIBE||19600324|M|||^^^^^US^P|||||||6100215419|999-99-9999|||||||||||N||
PD1|||MARION GENERAL HOSPITAL^^1016100|||||||||||||||
PV1|1|E|ED^ED1^01^MGH^R^^^^^^DEPID||||15802^BOEHMER^DAVID^C^^^^^STARPROV^^^^STARPROV|||Emergency||||||||EMERGENCY|6100215419|SELF||||||||||||||||||||||^^^MGH^^^^^^^||20150324190300||||||27150830003
PV2||GENERAL||||||||||||||||||||N|||||||||||||||||||||||||||
ZPV|||||||||||||20150324190832||||||||
AL1|1|Drug Class|45202^NO KNOWN ALLERGIES^||||||
DG1|1||52^Headache^HRV|Headache||
DG1|2||53^Neck^NCK|Neck||
DG1|3||54^Ear^EAR|Outer Ear||
GT1|1|100046662|MGHTEST^EDSCRIBE^^||^^^^^US|||19600324|M|P/F|SLF|999-99-9999|||||||||||||||||||||||||||||||||||||
ZMP|1|||||N||||^^^^|||^^^^|||||^^^^||||^^^^||Not eligible for Medicare||||^^^^|||||^^^^|0|||^^^^|||||||||||||||||||||||||||||
T_50005_1T_50005_1{CONNID 0} {IPVERSION 4} {CLIENTIP 172.17.154.197} {CLIENTPORT 63143}
HL7_Segment_Template transform, with MV_ADD = true, collected ALL the DG1 segments and put the aggregated values as the single value for DG1:
1||52^Headache^HRV|Headache||
2||53^Neck^NCK|Neck||
3||54^Ear^EAR|Outer Ear||
Then I ran this SPL: sourcetype="hl7_v2" DG1=*| rex mode=sed field=DG1 "s/$/:/g"
which tacks on a ":" delimiter to end of the line and produces:
1||52^Headache^HRV|Headache||:
2||53^Neck^NCK|Neck||:
3||54^Ear^EAR|Outer Ear||:
This SPL then combines the three segments into single line, using end of line as anchor:
sourcetype="hl7_v2" DG1=* | rex mode=sed field=DG1 "s/$/:/g" | makemv delim="$" DG1
1||52^Headache^HRV|Headache||:2||53^Neck^NCK|Neck||:3||54^Ear^EAR|Outer Ear||:
Then I employ the split function multiple times within an eval to split DG1, and use mvindex to extract the particular iterated DG1 segment:
sourcetype="hl7_v2" DG1=* | rex mode=sed field=DG1 "s/$/:/g" | makemv delim="$" DG1| eval s1=mvindex(split('DG1', ":"),0), s2=mvindex(split('DG1', ":"),1), s3=mvindex(split('DG1', ":"),2)
If you're not seeing extracted field in your searches try adding "| fields *" to the end of your search.
-Now supports Splunk 8.x
-Will be Splunk Cloud Certified
-Removed pre-built panels and dashboards. Will create a new, HL7 Analytics App to be used with Add-on
-Changed permissions required for Splunk Cloud Vetting
find . -type d -exec chmod 0755 {} \;
find . -type f -exec chmod 0644 {} \;
-Updated macros to work properly in Splunk 7.x
-Updated eventgen.conf to ensure it works with latest Eventgen in Splunkbase: https://splunkbase.splunk.com/app/1924/
-Ensured all three pre-built panels work as a result of all changes
-Updated Message Type Lookup ( hl7_v2_message_types.csv). Please use Message_Type_Description in lieu of previous Message_Type in your searches.
=== Splunk TA HL7 (Add-On) ===
Author: Joe Welsh - Splunk
Version/Date: 1.05 August 25, 2016
*Note: This app was previously Add-on for HL7
Supported product(s):
This add-on supports logs containing HL7 v2.X data.
Source type(s): This add-on will process data that is source-typed
as "hl7_v2".
Input requirements: N/A
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.