Security Onion App for Splunk software
Overview
Details
Security Onion:
Security Onion (http://securityonion.blogspot.com/) is a Linux distribution for intrusion detection and network security monitoring. Security Onion for Splunk is designed to run on a Security Onion server, providing an alternative method for correlating events and incorporating field extractions and reporting for Sguil, Bro IDS and OSSEC. (For a detailed description of the application with screenshots: http://eyeis.net/2012/04/splunking-the-onion/.)
Overview:
Splunk for Security Onion provides several dashboards and search interfaces for correlating Sguil, OSSEC and Bro IDS log events.
Required Splunk Apps:
- Sideview Utils - http://splunk-base.splunk.com/apps/36405/sideview-utils
- Splunk for OSSEC - Splunk v4 version -http://splunk-base.splunk.com/apps/22285/splunk-for-ossec-splunk-v4-version
- Geo Location Lookup Script (powered by MAXMIND) - http://splunk-base.splunk.com/apps/22282/geo-location-lookup-script-powered-by-maxmind
- Google Maps - http://splunk-base.splunk.com/apps/22365/google-maps
- Splunk Visualizations - http://splunk-base.splunk.com/apps/22283/splunk-visualizations
Prerequisites:
A Security Onion server and a Splunk 5.x installation. (Note: when installing Splunk on a Security Onion system there will likely be a port conflict for the Splunk web server. Port 81 as an alternate port should be a safe alternative.)
Setup Splunk:
Download from www.splunk.com. Install via terminal command:
sudo dpkg -i splunk-5.*.deb
When install completes, we need to start Splunk for the first time:
cd /opt/splunk/bin
sudo ./splunk start
After accepting the agreement, you'll have to pick an alternate port for the Splunk web interface since the default is in use. You'll see the following and when prompted to change, choose yes, then specify port 81:
Checking prerequisites...
Checking http port [8000](http://splunkbase.splunk.com/wiki/8000): already bound
ERROR: The http port [8000](http://splunkbase.splunk.com/wiki/8000) is already bound. Splunk needs to use this port.
Would you like to change ports? [y/n](http://splunkbase.splunk.com/wiki/y:n): y
Enter a new http port: 81
Setting http to port: 81
Checking http port [81](http://splunkbase.splunk.com/wiki/81): open
Checking mgmt port [8089](http://splunkbase.splunk.com/wiki/8089): open
Checking configuration... Done.
Splunk's built-in way to run at boot time is (as root): $SPLUNK_HOME/bin/splunk enable boot-start with an optional user=foo at the end to run as a certain user. It installs proper boot scripts for your OS so you don't need to worry about editing /etc/rc.* files:
/opt/splunk/bin/splunk enable boot-start
Install Required Splunkbase Apps:
Open Firefox and browse to http://localhost:81 (or if you used an alternate port change accordingly). Change your password, then click App > Find More Apps from the menu in the upper right corner. Find the following apps and install them.
- Install Splunk for OSSEC - Splunk v4 version
- Install Geo Location Lookup Script (powered by MAXMIND)
- Install Google MapsInstall Splunk Visualizations
- Install Sideview Utils
- Install Security Onion for Splunk
Configure Bro IDS Inputs:
Depending on how much traffic your sensor monitors, you may need to leave some of the Bro inputs disabled to avoid maxing out your license. The following are the sourcetypes configured for each Bro IDS log data input:
- bro_communication - Bro sensor communications
- bro_conn - Connections
- bro_dns - DNS requests
- bro_dpd - Dynamic Protocol Detection
- bro_ftp - FTP activity
- bro_http - HTTP traffic
- bro_irc - IRC activity
- bro_known_certs - Certificates seen
- bro_known_hosts - Hosts seen
- bro_known_services - Services detected
- bro_notice - You definitely want this one enabled
- bro_smtp - SMTP activity
- bro_smtp_entities - SMTP
- bro_software - Software versions detected (incl. vulnerable versions)
- bro_ssh - SSH activity
- bro_ssl - SSL activity
- bro_syslog - Syslog activity
- bro_weird - Anomalous events
Using Splunk for Security Onion:
I've standardized the source and destination IP fields in the Bro IDS and Sguil log field extractions so "src_ip" and "dest_ip" are consistent across events.
Log file data inputs are file specific (as opposed to monitoring the entire /nsm/bro/logs/current/ directory) primarily for granular control over what gets splunked. Depending on your licensing you may need to scale back certain logs and this provides an easy way to do so.
SOstat Monitoring:
The SOstat monitoring scripts are configured to run at various intervals (in seconds). The default settings are very conservative,
executing most scripts once a day and status scripts 3 times a day. The default configuration is:
Sourcetype - Script - Interval (seconds)
- sostat_disk - /opt/splunk/etc/apps/securityonion/bin/disk.sh - 86400
- sostat_ifconfig - /opt/splunk/etc/apps/securityonion/bin/ifconfig.sh - 86400
- sostat_nsm_log_archive - /opt/splunk/etc/apps/securityonion/bin/nsm_log_archive.sh - 86400
- sostat_nsm_sensor_ps-status - /opt/splunk/etc/apps/securityonion/bin/nsm_sensor_ps-status.sh - 21600
- sostat_nsm_server_ps-status - /opt/splunk/etc/apps/securityonion/bin/nsm_server_ps-status.sh - 21600
- sostat_nsm_sguil_uncategorized - /opt/splunk/etc/apps/securityonion/bin/nsm_sguil_uncategorized.sh - 86400
- sostat_top - /opt/splunk/etc/apps/securityonion/bin/top.sh - 21600
The default settings can be modified via Splunk Manager > Data Inputs > Scripts. Click on the script name in the "Command" column to increase/decrease the interval. You can also disable/enable scripts.
Comments or Questions:
For comments, suggestions or questions, feel free to drop me an e-mail: brad@eyeis.net
Hope you enjoy the app!
Brad Shoop
http://eyeis.net
@bradshoop
Release Notes
Version 2.0
IMPORTANT - Security Onion for Splunk 2.0 supports the latest release of Security Onion 12.04. If you are running the older version, Security Onion 10.04, please continue to use version 1.1.7.
Sideview Utils is now a required app, available from Splunkbase.
2.0 contains updated log sources and field extractions for Security Onion 12.04, includes several updated/enhanced dashboards including Overview, IR Search and SOstat, and introduces the new dashboard "Bro(wser)" for reviewing Bro IDS logs.
Version 1.1.7
1.1.7 - Tweaked Sguil indexing to prevent Bro URL data from being duplicated into Splunk via sguild.log. - Monitors dashboard field name drop down selections added to all panels. - General Mining dashboard added panels for Bro SSH logs and Bro HTTP TLDs (top level domains). Also added drop down options for Bro FTP and IRC panels. - Squil Mining has been updated and improved. - Syslog Mining dashboard added for Bro Syslog. - An Event Workflow was added for searching Splunk for events by src_ip.…and last but not least: - CIF Dashboards! See README for details on how to configure CIF integration.
Version 1.1.6
1.1.6 - Sguil mining - added ability to drilldown on source or destinations to timechart of activty by sourcetype for selected IP.
Version 1.1.5
1.1.5 - Performance improvements on several dashboards and fixed a bug in Sguil mining.
Version 1.1.4
1.1.4 - Minor bug fix to IDS rule reference full document lookup.
Version 1.1.3
1.1.3 - Added SOstat IDS Rules: indexes /etc/nsm/rules/*.rules and provides an easy to use interface for referencing rules for tuning. Sort by classtype, category, enabled status, and/or rule source and drilldown on a rule to see it's activity history and the full rule entry complete with Splunk event workflow lookups by BugtraqID, CVE ID and URL.
-
Added VRT reference workflow lookup. For Sguil events with a sig_id, you can now use the Events view workflow menu to view the Snort signature reference document in a new window, when available.
-
Added SOstat ability to sort by sensor to provide
better monitoring of a distributed deployment.
See README and http://eyeis.net/2012/07/security-onion-for-splunk-1-1-3-ids-rule-reference/ for full details.
Version 1.1.2
Workflow field and event search items added for CIF, DShield and Robtex. Most panel
drilldowns have been changed from table views to event listing views. The primary reason
(while less aesthetic) is accessibility to the workflow queries, which will allow you to
efficiently query domains, IPs, and hashes without disrupting your workflow and visiblity
(i.e., results spawn in new windows/tabs).
Version 1.1.1
1.1.1 - Minor update to get src_ip into Known Knowns and fix a typo. Also tweaked sguild inputs to better support non-standard timezone setting environments.
Version 1.1
http://eyeis.net/2012/06/security-onion-1-1-for-splunk/
I've added an input for Bro's capture_loss.log which now displays on the SOstat Security Onion monitor in a time chart paired with Snort packet loss. To enable this log in Bro edit:
/usr/local/share/bro/site/local.bro
and add the following:
@load misc/capture-loss
You'll have to check and install Bro for the change to get loaded.
sudo broctl
check
install
exit
and you're done. It takes a few before the first logged event will show so give it a bit before you worry if it's working.
I also tweaked the sguild inputs to exclude "{URL" events. This data is already being consumed via bro_http so it should cut down on the licensing volume.
Monitors Dashboard
- Returned misc-activity to the Sguil panel.
- Added date/time and raw event to drill down display for the FTP Args panel.
GeoIP
- A drop down list has been added to GeoIP allowing you to search GeoIP by sourcetype which should reduce query times for more targeted views. The map also now includes drill down capabilities with results appearing below the map when selected.
Mining
- Added drill down to the time chart panels for HTTP and SMTP mining
- Added a PADS dashboard (similar to HTTP and SMTP mining) searchable by Name, Classification, Source IP, Source Port, Destination IP, Destination Port, Protocol, and Severity.
- Added a Known Knowns dashboard. Includes: Known Services; Known Software searchable by Name and Type; and Known Certs searchable by Country, Common Name, Certificate Issuer Subject, Location, Organization, Organizational Unit, Port Number, State, and Certificate Subject.
PADS
- Created an event type for PADS in addition to the PADS Mining dashboard.
SOstat
- Updated SOstat SO to include Bro capture loss in addition to Snort packet loss.
Version 1.0
See README or check http://eyeis.net/2012/05/security-onion-1-0-for-splunk/
Version 0.9
Bro's http.log will be using a log file per interface in mutli-interface sensors. This update adds a data input to capture them.
Version 0.8
Added CIF (Collective Intelligence Framework: http://code.google.com/p/collective-intelligence-framework/) query capability to the field and event menus in Splunk. For the links to work you will need to update the workflow actions for your CIF server and a valid API key. Edit the workflow via Splunk Manager > Fields > Workflow action from the Security Onion app context and you should see two CIF entries.
Version 0.7
SOstat update to include folder sizes. Mining menu additions: HTTP mining and SMTP mining.
Version 0.6
Added Bro stats.log monitoring to SOstat Server/Service Status panel (sourcetype = bro_stats). Added Bro irc.log monitoring to Mining dashboard (sourcteype = bro_irc). Added Bro smtp_entities filename monitoring to Mining dashboard (sourcetype = bro_smtp_entites)