Hunk App for HBase

Overview

The Hunk App for HBase is a Streaming External Results Provider enabling import of data from HBase. This application will perform a scan on a HBase table and stream the data to Hunk for analysis. The data returned from HBase is streamed as JSON documents, one document per row.

HBase Version

This release of the Hunk App for HBase has been compiled and tested with HBase 1.1.2. For other versions please contact us.

Configuration

Provider Configuration

The Hunk App for HBase provides the usual Hunk ERP parameters:

vix.mode = stream
vix.command =  java
vix.command.arg.1 = -Xmx512m
vix.command.arg.2 = -classpath
vix.command.arg.3 = $SPLUNK_HOME/bin/jars/SplunkMR-h1.jar:$SPLUNK_HOME/etc/apps/hbase-erp/bin/hbase-erp-1.0.0.jar:$SPLUNK_HOME/etc/apps/hbase-erp/bin/lib/*
vix.command.arg.4 = com.splunk.erp.hbase.HBaseERP
vix.splunk.search.debug = 0

Additionally you can set HBase-specific parameters using vix.<name> = <value>

e.g.

vix.hbase.zookeeper.quorum = localhost
vix.zookeeper.znode.parent = /hbase-unsecure

Examples of these have already been set in the provided indexes.conf.

Finally, you can change the log level with vix.loglevel and one of these threshold values: ALL, TRACE, DEBUG, INFO, WARN, ERROR, FATAL or OFF.

Virtual Index Configuration

To customize the index, you need at a minimum to provide the following parameters: * vix.hbase.table: the table you want to scan * vix.hbase.family: the column family to include in the scan

There are some optional settings you can also provide: * vix.hbase.column.<name>: data type for a column (bigdecimal, boolean, double, float, integer, long, short, string). If not specified the default data type is string * vix.hbase.datetime.column: name of the HBase column that contains the event time in Splunk * vix.hbase.datetime.format: timestamp format * vix.hbase.max.days.hence: maximum number of days in the future from the current date that an extracted date can be valid

These can be set in the Hunk UI (Settings > Virtual indexes > Virtual Indexes > Index), or in the indexes.conf file.

If the vix.hbase.datetime.column is left blank, there will be no event time. Hunk does try to infer the event time from the data, so if your data has dates, Hunk may use that as the event time.
Similarly if you do not specify the vix.hbase.datetime.format, Hunk will try to infer it. If you do have vix.hbase.datetime.column set, setting vix.hbase.datetime.format will result in better performance.
For more information about the Hunk date format see this documentation page

If you do not set vix.hbase.max.days.hence events that occur in the future may be rolled up to an event time of today (or this month). The data will still be present, but the inferred event date will be incorrect.

You can create multiple Virtual Indices for the same Provider (and, thus, the same HBase client connection) by clicking the New Virtual Index button. Make sure to select the proper Provider (e.g., the hbase provider that comes with this application).

Searching HBase through Hunk

To search a virtual index in Hunk type index=<virtual index name> in the search box, e.g. index=mytable.

You can use any pipe command just like you would normally do with Hunk, e.g. index=mytable | top limit=20 date