icon/x Created with Sketch.

Splunk Cookie Policy

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website. Learn more (including how to update your settings) here.
Accept Cookie Policy

We are working on something new...

A Fresh New Splunkbase
We are designing a New Splunkbase to improve search and discoverability of apps. Check out our new and improved features like Categories and Collections. New Splunkbase is currently in preview mode, as it is under active development. We welcome you to navigate New Splunkbase and give us feedback.

Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading S.o.S - Splunk on Splunk
SHA256 checksum (sos-splunk-on-splunk_321.tgz) 6237d7807b6168c9a9206fb534e0eb089c9c0c17a46437b0a0473d4d6f3c7f7b SHA256 checksum (sos-splunk-on-splunk_32.tgz) 518aa6b3e810962a837b507df54a800b8c0708ca2d9859d52e8557d30565d4ec SHA256 checksum (sos-splunk-on-splunk_310.tgz) ce58ec1e9a5cf1be90a939502ace9c39843a3c0bd5c7777524a510ff9583a168 SHA256 checksum (sos-splunk-on-splunk_301.tgz) cca2e12cf90153a9b8f190ee5f3bd708c4558b6435e8b99579898ffec3ffa4cd SHA256 checksum (sos-splunk-on-splunk_30.tgz) dd082e13ebcb6a445e02fbeee0e4eff90a5e6c05deeaed0445b8301bb32e4818 SHA256 checksum (sos-splunk-on-splunk_231.tgz) 7e4fac4703cb20544a601c54bc3373fd5f4d7c35fce46f2b4389cfd2116efdf9 SHA256 checksum (sos-splunk-on-splunk_230.tgz) 521edf486d23bb05dc7cd560167d1b1ade33b6333dfc76b0cb6c03d0ed1b0ff4 SHA256 checksum (sos-splunk-on-splunk_220.tgz) a2165e3a3357e81ead2cf8e4293401f4cc0f9ab6a777491e1016a9bf3d70f1e1 SHA256 checksum (sos-splunk-on-splunk_210.tgz) ab16158a9690232d3dcaca4a4e5fb525b94e72f89dbfd72599e22a0e397b41d9 SHA256 checksum (sos-splunk-on-splunk_200.tgz) e79e7b8f241f9c7cf7571f4aaee28baf83c2be646658c6085ee09d1080a0f4a8 SHA256 checksum (sos-splunk-on-splunk_10.tgz) 22acdf535ce91116c5ea5125bce2277ee9c953a261dc1bad15eec0a523858e94
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate


S.o.S - Splunk on Splunk

Splunk Labs
This app has been archived. Learn more about app archiving.
This app is NOT supported by Splunk. Please read about what that means for you here.
On April 8, 2019, this app has been deprecated and reached its End of Life on July 7, 2019. For more information about the end of availability and support for this app, see https://www.splunk.com/blog/2019/03/18/end-of-availability-splunk-built-apps-and-add-ons.html?April.

IMPORTANT: As of Splunk Enterprise 6.3, the S.o.S app is End of Life. Its functionality has been replaced and superseded by the Distributed Management Console, a feature that is included with Splunk Enterprise as of version 6.2. We recommend that you migrate from S.o.S to the DMC for all your Splunk monitoring and introspection needs at your earliest convenience.


Splunk on Splunk (S.o.S) is an app that turns Splunk's diagnostic tools inward to analyze and troubleshoot problems in your Splunk environment. It contains views and tools that allow you to do the following:
* View, search and compare Splunk configuration files.
* Detect and expose errors and anomalies in your installation, including inspection of crash logs.
* Measure indexing performance and expose event processing bottlenecks.
* View details of scheduler and user-driven search activity.
* Analyze data volume metrics captured by Splunk.

IMPORTANT: As of Splunk Enterprise 6.3, the S.o.S app is End of Life.

Its functionality has been replaced and superseded by the Distributed Management Console, a feature that is included with Splunk Enterprise as of version 6.2. We recommend that you migrate from S.o.S to the DMC for all your Splunk monitoring and introspection needs at your earliest convenience.

Splunk on Splunk installation instructions for Splunk 5.x/6.x

Please consult Splunk Answers to learn about best practices to deploy the S.o.S app in a distributed environment.

1) If Sideview Utils version 1.1.7 (or later) is not installed, please install or update it before installing S.o.S.

Download Sideview Utils from Splunkbase

2) Install the S.o.S app:

If you have a distributed search environment, make sure you install S.o.S on the search-head(s) only. From the search-head the app can discover search-peers present in the distributed deployment.

3) Download and install the Splunk on Splunk add-on (S.o.S TA) on search peers to provide resource usage information to S.o.S.

This add-on provides data inputs that gather memory and CPU usage for Splunk Web, splunkd, and search processes as well as other system resource information. For more information, see the README file available with the S.o.S TAs.

Note: You do not need to install the S.o.S add-on on a Splunk instance were the S.o.S app is already installed. The S.o.S app ships with the same data inputs.

Release Notes

Version 3.2.1
Dec. 29, 2014

Bugs fixed in version 3.2.1

  • [SOS-11] Fixed an issue where ps_sos.ps1 would log many "WriteError" exceptions to splunkd.log and insert incorrect values in its events.

  • [SOS-12] Fixed an issue where the "Security Health Check" view would fail to show results on a Splunk Enterprise 6.2 instance.

  • [SOS-19] Retired the "Bucket information" panel in the "Cluster Master View" as it was dispatching potentially unsafe rest-based searches against the Cluster Master's buckets endpoint.

  • [SOS-39] The securityinfo.py search command - and by extension, the "Security Health Check" view - now appropriately scopes its results to the instance picked by the user.

  • [SOS-40] Fixed an issue where the "cluster" command would fail to show event cluster counts due to a change in internal behavior.

  • [SOS-113][SOS-117][SOS-141] Forwarder instances will no longer be listed in the "Host" pulldown of the "Search Usage Patterns, "Scheduler Activity" and "Search Activity" views.

Version 3.2
May 6, 2014
  • Full support for Splunk Enterprise 6.1

  • NEW VIEW: Search > Search Activity
    Provides deep insight into instance-scoped search workload, expressed as search concurrency, resource usage or aggregate search time. These metrics can be grouped by various relevant search properties: mode (historical vs. real-time), type (ad hoc vs. scheduled), user...

  • NEW VIEW: Resource Usage > Indexes Disk Usage and Properties
    Allows a deployment-wide or instance-scoped view of index disk usage and other properties. Can be scoped to one or all indexes.

  • NEW VIEW: Deployment Status > Warnings and Errors > Security Health Check
    A series of checks against security settings in your Splunk Enterprise installation.

  • NEW VIEW: Indexing > Index Replication > Cluster Service Activity
    Shows service activity in a Cluster in great detail, allowing to better understand maintenance and repair operations undertaken by the Cluster Master and its peers.

  • 24 bugs fixed! See the RELEASE-NOTES file for full details.

Version 3.1.0
Sept. 30, 2013
  • New features for the Deployment Topology view
    Data overlays for instance status and resource usage (CPU/Memory).

  • NEW VIEW - Search > Search-head Pooling Performance
    Check the usage and performance of the NFS shared storage device central to search-head pooling deployments. Compare performance metrics both at the storage (NFS) and application (Splunk) levels.

  • NEW VIEW - Indexing > Metrics > License Usage - Today
    Get a license usage report for the current day and a history of license warnings for the current license window. (Applies to Splunk 4.3.x and 5.x only)

  • NEW VIEW - Indexing > Metrics > License Usage - Last 30 Days
    Get a daily license usage report for the past 30 days and break it down by pool, indexer, source, sourcetype or host. (Applies to Splunk 4.3.x and 5.x only)

  • NEW VIEW - Indexing > Index Replication > Bucket Fix-up Activity
    Monitor the status and progress of bucket fix-up operations in a cluster.

  • 10 bugs fixed! See the README file for full details.

Version 3.0.1
June 9, 2013

Bugs fixed in version 3.0.1

  • [SUP-723] Fixed an issue where scheduled searches "sos_splunk_instances_info" and "sos_refresh_splunk_servers_cache" would run several times per minute instead of at their scheduled time on a pooled search-head running Splunk 5.0.3. Note that the root cause of this problem is core Splunk bug SPL-68970.

  • [SUP-720] Fixed an issue where the Home view would be caught in a reload loop after S.o.S was installed or upgraded on a pooled search-head running Splunk 5.0.3.

  • [SUP-716] File $SPLUNK_HOME/var/log/splunk/sos_ftr.log is now explicitly sourcetyped.

  • [SUP-715] Our invocations of the "btool" command with the "--debug" flag no longer cause logs to be appended to $SPLUNK_HOME/var/log/splunk/btool.log.

  • [SUP-701] Fixed an issue where the Data Inputs > Tailing Processor view would fail to display when scoped to instances running Windows, showing instead an error banner stating "Invalid header received from stream generating script tpstatusquery".

Version 3.0
May 6, 2013

Bugs fixed in version 3.0

  • [SUP-692] Fixed an issue where the in-product app browser wouldn't be scoped
    to the Sideview Utils app during the installation workflow.

  • [SUP-668] There is now a scheduled search populating the "splunk_forwarders_cache.csv" lookup table with forwarder information.

  • [SUP-657] Added a spec file describing the "splunk_servers_cache.csv" lookup table.

  • [SUP-630] Created a macro to qualify searches based on their search ID.

  • [SUP-627] Fixed an issue where the ps_sos.sh scripted input would no longer print out full process arguments when executed by Splunk 5.x on Solaris.

  • [SUP-619] Metrics: Fixed an issue where the license usage chart would improperly show a "license_audit" pool for a license self-master.

  • [SUP-616] Fixed an issue with the ps_sos.ps1 scripted input where memory usage would sometimes be recorded as a negative value.

  • [SUP-596] Metrics: Fixed an issue where the license usage chart would not show multiple pools.

  • [SUP-578] Retired the "Distributed Searches Memory Usage" view.

  • [SUP-573] A new scripted input is now available to monitor the I/O usage of pooled search-heads on the shared NFS device: nfs-iostat_sos.py

  • [SUP-565] Fixed an issue where the ps_sos.ps1 scripted input would not run on an instance part of a search-head pool.

  • [SUP-541] Updated the app icon.

  • [SUP-540] Updated the app screenshot displayed on Splunkbase.

  • [SUP-530] Splunk File Descriptor Usage: The time stamp of the data sample used to populate the view is now shown.

  • [SUP-475] Dispatch Directory Inspector: Added a search box to filter results.

  • [SUP-474] Dispatch Directory Inspector: Added some statistical aggregations at the top of the view.

Version 2.3.1
Dec. 6, 2012

Bugs fixed in version 2.3.1

  • [SUP-606] Splunk CPU/Memory Usage: Resolved a problem where the memory usage charts would fail to report the memory usage of certain search processes.

  • [SUP-600] Metrics: Fixed an issue with the license reporting panel, which would show inaccurate numbers when multiple license pools are defined.

  • [SUP-599] Resolved a problem where the host "tag" for instances listed in the "Server to query" pulldown would not be properly determined on Splunk 5.x.

  • [SUP-595] Indexing Performance: Fixed an issue where no data points would be drawn when "Last 15mn" is selected from the time picker.

  • [SUP-589] Data Inputs Overview: Fixed an issue where this view would show no results when running on Splunk 5.x.

  • [SUP-587] Splunk CPU/Memory Usage: Renamed the "splunkd" series to "splunkd service".

  • [SUP-585] Metrics: Ensured that internal indexes and sourcetypes are no longer excluded from indexing volume reports.

  • [SUP-584] Metrics: Fixed an issue where excessive division for indexing volume metrics would lead to inaccurate reporting.

  • [SUP-583] Metrics: Fixed an issue where outgoing network throughput would be inaccurate by one order of magnitude when a split-by clause was used.

  • [SUP-582] Fixed an issue where an improper value for the "count" parameter of the "rest" command would cause a red error banner.

  • [SUP-558] Added an outputs.conf file with configuration that, if enabled, ensures that _internal events are forwarded from search-head to indexers.

  • [SUP-556] Fixed an issue where the "level" parameter of the Messages module would cause a red error banner on certain versions of Splunk.

  • [SUP-555] Resolved an issue where the "Server to query" pulldown on the Home view was not sorting hosts properly.

  • [SUP-554] Forwarders are now excluded by the searches of the Distributed Indexing view.

  • [SUP-547] Added a panel to the Indexing Performance view to expose subtask- level CPU time usage metrics for the indexer pipe which are new in 5.x.

  • [SUP-545] Adapted the searches against events generated by the ps_sos.* scripted inputs to the new splunkd process command line format in 5.x.

  • [SUP-527] Updated the build2version.csv lookup with information for the latest Splunk releases.

Version 2.3.0
Aug. 29, 2012

Bugs fixed in version 2.3

  • [SUP-538] Inputs Overview: Fixed a bug where the drilldown to file monitor input details would break due to a regular expression not supporting Windows paths.

  • [SUP-537] Home: Fixed a bug that caused the search powering the "A glimpse of your Splunk instance" panel to mismatch field values across hosts.

  • [SUP-532] Configuration File Comparator: General uncluttering and visual sanitization of this view.

  • [SUP-528] Distributed Indexing Performance: Set the height of the charts to a sensible default value.

  • [SUP-526] Scheduler Activity: Fixed wrong total execution count reported in the "Scheduler Activity" and "Execution Count by App/SavedSearch Name" panels.

  • [SUP-524] Scheduler Activity: Fixed a field extraction that was causing a NULL series to appear in the "Execution Count by App/SavedSearch Name" panel.

  • [SUP-521] Splunk CPU/Memory Resource Usage: Updated the search strings in the in-view help.

  • [SUP-507] Documented the search strings used for the Data Inputs Overview and Dispatch Directory Inspector in the in-view help.

  • [SUP-505] Fixed a typo in the lsof_sos.sh scripted input.

  • [SUP-503] Entries in the "Server to query" pulldown are now sorted based on the role of the Splunk instance: search-heads > search peers > forwarders.

  • [SUP-478] In the Errors view, improved chart readability by moving legends underneath the charting area.

Version 2.2.0
July 14, 2012
Version 2.1.0
Jan. 11, 2012

2 bugs and 4 new features in this version! Check the CHANGELOG file for details.

Version 2.0.0
Dec. 16, 2011

New features for 2.0:

Centralized Splunk instance troubleshooting
Tracking Splunk resource usage
Improved searches and data representation
Improved help panels and troubleshooting documentation
Improved visual theme

Version 1.0
Aug. 15, 2011

Subscribe Share

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
Splunk, Splunk>,Turn Data Into Doing, Data-to-Everything, and D2E are trademarks or registered trademarks of Splunk Inc. in the United States and other countries. All other brand names,product names,or trademarks belong to their respective owners.