What is InfluxDB Connect?
A splunk app that facilitates sending data from Splunk to InfluxDB(an open source time series database)
Where should InfluxDB Connect reside?
On the Splunk Search Head
How does it work?
- User will create/setup influxDB instances by providing an instance name, hostname/ipaddress, port, database, influxdb access credentials (user and password). This is facilitated via influx_configs.
- Once instances are available, user will setup scheduled searches in the required format to extract data from Splunk. This is facilitated via influx_searches.
- The scheduled searches will trigger a python script that will read the dispatch and port the data to the respective influxDB instance.
- The influxDB instances can be managed at influxdb_instances.
InfluxDB Data structure
- Data in InfluxDB resides within Databases. In Influxdb_Connect each instance directly points to an influxdb database.
- The data in influxdb databases can be further spread across various “measurements”. In influxdb_connect the “measurement” can be specified when you setup your searches under influx_searches.
- If a measurement is not existent in the influxdb database (influxdb_connect instance), then it will be automatically created after the first run of the search.
- Each measurement consists of TimeStamp, Values and Tags.
- From a splunk data export standpoint, the result of the splunk search needs to be presented in terms of timestamp, values and tags. This is accomplished per the syntax below.
Splunk Search Syntax for Influxdb_Connect
- Time needs to passed on in unix format (milliseconds) and prefixed with “ts_”
- All fields that needs to be identified as values need to be prefixed with “val_”
- All fields that are strings and need to be identified as values need to be prefixed with “valst_”
- Ideal to make sure your results are sorted by time
Eg:
index=internal earliest=-1m latest=@m per_index_thruput series="*"|rename kbps as val_kbps, eps as val_eps, ev as val_ev|eval ts_time = _time * 1000|table ts_time,series,host,val_kbps,val_eps,val_ev|sort ts_time|head 10
https Support
- From version 1.7 writes to influxdb on https is supported.
- Include the prtotocol and port information under influx_configs tab as https:port. Eg. https:8089
- Providing just the port will be processed as http. Eg: 8086
App Release Status
This is the first release, tested with splunk 6.3 and influxDB v0.12 on Linux and Windows. The functionality is limited to running formatted searches and exporting the data to InfluxDB instance. The app has also been tested successfully with influxDB relay server.
NOTE
- InfluxDB host should be accessible from the splunk host on the provided port.
- InfluxDB access credentials should have write privileges on the respective databases.
- Currently the app does not support sending data to influxdb over ssl
- All time on InfluxDB side is stored in UTC.
- The validations are very basic (ensure that proper data is entered while configuring the app)
- When saving searches, ensure any search artifacts related to the search are available from within influxdb_connect app
- Do not rename the saved searches created by the app
Troubleshooting
- The app logs information to influxdbmod.log (located at SPLUNK_HOME/var/log/splunk).
- For any questions or help with issues, please post on “Splunk Answers” I will check daily and will try to address them at the earliest possible.
