This app will download email messages from an IMAP account/mailbox and index it into Splunk. Each email message will be treated as a separate splunk event.
This is an update to the "Splunk for IMAP" v1.20 app. It appears the Splunk employee who created the original app is not able to maintain it. Its last update was in 2009. So I decided (with permission of Splunk) to write an update! It cannot be named the same due to Splunk development naming rules, but this app can be considered the new version going forward; and thus started it at version 2.0. The original app will still work fine for Splunk v5 users and below.
You can help make IMAP Mailbox better by contributing: https://github.com/IMAPMailbox/IMAPmailbox.git
This section is to install on a centralized splunk setup. Look further down for distributed splunk design instructions.
By default, the IMAP app will create a new Splunk index named "mail". This is controlled by: default/indexes.conf. If you want the IMAP output to go to the default Splunk index:
Instructions if you are currently using Splunk for IMAP v1.20 and want to upgrade to this new app.
Install this app as instructions above, it will install in a new location than the original app.
Note: If you have any users with private saved searches, dashboards, alerts, then you will need to change their user app directory name so it appears for this app.
This app uses the same index "mail" as the original app, so all past indexed data will still exist.
For those who are running a distributed Splunk design or HA: ie separate forwarders, search heads, indexers, etc... Please follow these directions. Depending on your design YMMV. Please see this link for more instructions: [http://docs.splunk.com/Documentation/AddOns/released/Overview/Installingadd-ons]
Install this App on your Search head(s).
Disable the inputs: =- inputs.conf
[script://./bin/get_imap_email.py]
disabled = true
[script://.\bin\get_imap_email.py]
disabled = true
Disable the indexes: =- indexes.conf
[mail]
disabled = true
Install this App on your indexer(s).
Disable the inputs: =-inputs.conf
[script://./bin/get_imap_email.py]
disabled = true
[script://.\bin\get_imap_email.py]
disabled = true
Enable the index and replication: =- indexes.conf
[mail]
disabled = false
repFactor = auto
Disable app visibility: =- app.conf
[ui]
is_visible = false
Enable inputs on ONE of your systems.
Make sure that only ONE system is running the inputs for the IMAP app otherwise you will get email duplications. I personally recommend installing and running the inputs on a heavy forwarder.
It is recommeneded that your store your IMAP user password and splunk admin password encrypted in your local/imap.conf file.
source $SPLUNK_HOME/bin/setSplunkEnv
bin/genpass.sh
Message headers are indexed as key-value pairs, for example:
From = "User Name <user@emailcom>"
Subject = "This is sooo cool"
This makes it easy to generate reports from the email indexed in Splunk.
For example:
index=mail | top From
Also, note the quotes around the field values. This makes it easy to perform searches 'where' or regexes. For example, if you want to find all your email that was sent by any Will, do the following:
index=mail From="Will*"
Instead of searching with the "index=mail" tag as examples show above. I recommend using the macro imap_index
instead. If you change your index or even don't use one, this macro will still keep your searches working correctly.
If you have bugs or suggestions please contact pj@dysan.net.
What if your Exchange Admin does not want to enable imap? I suggest using the davmail server proxy, http://davmail.sourceforge.net/. It will convert IMAP requests to MAPI for exchange. You can load davmail on your local splunk server, or on any other server. Just point your imap.conf to the correct server and port. It will pass through the imap credentials to Exchange.
Some hints on troubleshooting:
Run the get imap script manually to look for mail issues.
sudo su - splunk
source $SPLUNK_HOME/bin/setSplunkEnv
$SPLUNK_HOME/bin/splunk cmd python bin/get_imap_email.py --debug
Check that the email message is plain text. Rich Text and HTML emails will not index by default inless you update imap.conf.
Google your error messages. :)
Duplicate Emails: If using a service like Gmail and you are getting a two copies or more of an email, this is normally because the email is being found twice on the server; once in the INBOX and the second in All Mail. Solution is to limit what folders you are checking. I suggest using:
This will download just your new email in your inbox and not the archived messages, which is why this is happening.
In my case I forward all unix root system messages to one mailbox that I index from. Then I can watch for errors and create alerts, such as cron failures.
I've signed up for mailing lists to my indexed email address and create reports and alerts based on only certain content I'm interested in looking for.
Just minor updates.
Just minor updates.
App completely updated for Splunk 6.0
Just minor updates.
- Added a TA for distributed Splunk designs.
Just minor updates.
App completely updated for Splunk 6.0
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.