Amazon S3 Uploader for Splunk
Overview
Details
See the Details and Installation tabs for more information.
Amazon S3 Uploader for Splunk
This Splunk add-on delivers an alert action that uploads search results to an Amazon (AWS) S3 bucket.
Object keys
Object keys uniquely identify objects in an Amazon S3 bucket. Although all UTF-8 characters are allowed, some should be avoided. Forward slashes can be used to mimic a directory structure. Please refer to the official documentation for additional information.
Supported file types
The output format is inferred from the object key extension. Only CSV (.csv), gzip-compressed CSV (.csv.gzip), and JSON (.json) files are supported.
Multivalue fields are treated differently based on the output format: in JSON they are stored as an array, while in CSV they are in a single entry, separated by their delimiter (by default a newline). For example, the search
| makeresults | eval test=split("value1,value2", ",") | fields - _*
produces the CSV
test
"value1
value2"
whereas the search
| makeresults | eval test="value1,value2" | makemv delim="," test | fields - _*
produces the CSV
test
"value1,value2"
Both searches generate the same JSON {"test": ["value1", "value2"]}.
Timestamp
The user-provided object key is passed to Python's datetime.strftime() function, which encodes the time the search started.
Format codes are extremely similar to Splunk's, please refer to the official documentation.
Configuration
The add-on must be configured prior to its use. The setup is performed through a configuration page with tabs dedicated to accounts, web proxy, and logging level.
Accounts
Unless Boto3's default authentication method is used, it is necessary to configure at least an AWS IAM user with write privileges on the bucket. Temporary security credentials (which have a session token in addition to the access key ID and the secret access key) are supported, but should be used for test purposes only.
Use the account Boto3 to use Boto3's default authentication method. Please refer to the official documentation for the order in which Boto3 searches for credentials. If this method is account, then manually specifying a role has no effect.
Proxy
HTTP and HTTPS proxy servers, both authenticated and unauthenticated, are supported.
All communications with AWS servers use HTTPS. The option Disable SSL verification disables checks on the server's certificate, it should not be used unless necessary (e.g., the proxy performs TLS bridging with certificates signed by an untrusted CA).
Logging
The log file is stored in $SPLUNK_HOME/var/log/splunk/log/amazon_s3_uploader_modalert.csv. The default logging level is INFO, but it can be increased or decreased from the configuration dashboard.
Release Notes
Version 1.1.1
Compliance with the new Splunk Cloud requirements.
Version 1.1.0
Add support for Splunk Cloud and Boto3's automatic credentials retrieval.