Requirements

Splunk versions:
Splunk 4.0 and higher

Platforms:
Windows Vista, 7, 2008 with Powershell 1.0 installed

SCOM versions:
Tested on SCOM 2007 R2 and later. The add-on has no known dependencies on R2, so it should work with R1. The legacy MOM environment is not supported.

We recommend that the add-on be installed on the SCOM root server.

Powershell requirements

Powershell 1.0 must be installed and the SCOM Powershell snap-in must be registered. Registration happens automatically when the SCOM operations console is installed.

NOTE: By default Powershell has a security policy enabled that restricts execution of user scripts from the command line. To overcome this, we change the execution policy to the less restricted RemoteSigned mode each time the Powershell script is called. See http://ss64.com/ps/set-executionpolicy.html for more information.

Support

This add-on is community suported. This means that there's no formal support from Splunk available for this add-on, but free support is available on Splunk Answers and questions, bug reports, and feature requests will be addressed on a best-effort basis by the add-on's author or other community members.

• Ask a question or file a bug - http://answers.splunk.com/questions/ask?tags=app-scom
• List of recent questions - http://answers.splunk.com/questions/tagged/app-scom
• RSS feed for recent questions - http://answers.splunk.com/feeds/tag/app-scom

Installation

1. Install Splunk if it's not already installed.
2. Using the download button on this page, download and save the scom.tar.gz file.
3. Decompress the add-on (it's compressed in .tar.gz format so you'll need a tool like 7Zip to decompress) and place the extracted scom folder into %PROGRAMFILES%\Splunk\etc\apps
4. Start or restart your Splunk instance (you can use the splunk restart command or the GUI)
5. Navigate to the Splunk GUI (usually http://localhost:8000/) and log in if needed
6. Select Manage Apps from the App menu, OR go to Manager>Apps.
7. Open Splunk Add-on For SCOM.
8. Fill in the user settings on the add-on's setup page.

Configuration

The add-on provides a setup page to configure SCOM-specific settings like SCOM hostname, polling interval, and inputs. This info is then written to bin\scom_client.xml. The powershell scripted input reads its configuration from this file each time it starts up.

You can select whether to enable or disable forwarding for all events and/or alerts from the SCOM database. If the checkbox is selected, the script will request all events/alerts starting with the very first event/alert and forward those to Splunk. When the script reaches the most recent event/alert, it will switch to polling mode; in polling mode, the script waits for new events/alerts and forwards them as soon as they become available. No events/alerts filtering is available through the setup screen-- you can either get all events/alerts or none.

How it works

• SCOM -> Splunk event and alert data forwarding is done using two PowerShell comandlets, which are part of OpsMgr Powershell library ("Snap-in") that ships with the OpsMgr. Once executed in the Powershell environment, these comandlets (Get-Alert and Get-Event) print out events and alerts to the console.
• We implement a powershell script scom_client.ps1 that:
  • Imports the SCOM snap-in
  • Reads config data (SCOM hostname, selected type of input(alerts and/or events) and SCOM polling interval)
  • Establishes a connection with local or remote SCOM server
  • Enters a loop polling SCOM for events starting with the latest imported event
  • Formats the output to look more Splunk-friendly and prints it to the console
• The script keeps track of the timestamp and GUID of the last event forwarded by saving this information to a file. If the script is stopped or interrupted, it will resume indexing from where it was when it got interrupted.

Performance

Our tests showed a throughput of about 3,000 events/min on a Xeon 3Ghz/4Gb Win2008 with full SCOM and Splunk installed at the same machine. The Powershell client process consumed around 3Gb of RAM while 800,000 events were being forwarded; the average CPU usage was 25%.

Remote execution

As mentioned above, we recommend that the add-on be executed on the same box as the SCOM root server. If you do need to run it remotely:

• The add-on can only run on Windows Vista, 7, or 2008, with Powershell 1.0 installed.
• If you want to run it remotely, it's recommended that the SCOM operations console be installed on the remote box to make sure the Powershell snap-in is registered in GAC. Alternatively, a few steps need to be taken as described here: http://derekhar.blogspot.com/2007/07/operation-manager-command-shell-on-any.html
• You need to add the user account under which the script will be running to the SCOM Administrators list. To do this, in the SCOM management console go to Administration -> Security / User Roles, right-click on Operations Manager Administrators, and select Properties. Click Add and select the group that the remote user belongs to.

Field names

Mapping of SCOM field names to Splunk field names is hardcoded in the scom_client.ps1 script, but can be easily modified as needed in that script.

Below is the full field mapping for alerts:

SCOM
Splunk

----------------------------------------------------
--------------------

TimeRaised
_time

MonitoringObjectDisplayName
ComputerName

Severity
Severity

Priority
Priority

Category
Category

ResolutionState
ResolutionState

Id
Id

ProblemId
ProblemId

Owner
Owner

LastModifiedBy
LastModifiedBy

Description
Message

Events also have their fields mapped:

SCOM
Splunk

----------------------------------------------------
--------------------

TimeGenerated
_time

Channel
LogName

PublisherName
SourceName

Number
EventCode

CategoryId
EventType

LevelId
Type

MonitoringObjectDisplayName
ComputerName

User
User

CategoryId
TaskCategory

Id
Id

Description
Message