icon/x Created with Sketch.

Splunk Cookie Policy

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website. Learn more (including how to update your settings) here.
Accept Cookie Policy

We are working on something new...

A Fresh New Splunkbase
We are designing a New Splunkbase to improve search and discoverability of apps. Check out our new and improved features like Categories and Collections. New Splunkbase is currently in preview mode, as it is under active development. We welcome you to navigate New Splunkbase and give us feedback.

Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading Shuttl
SHA256 checksum (shuttl_084.tgz) 86a393416c562ed1cf77c1434f9243f958a7da474b9675d711f218589b3b2637 SHA256 checksum (shuttl_0831.tgz) 0dae588d44aeda24ac1aa327eb5097b940e0e516ee49d1861dab6b6bd33c38ff SHA256 checksum (shuttl_081.tgz) 12b261fd918d8f64994c1d65ae30eeba8a214eeb521cf8f8a96bb5c1ff3cf19e SHA256 checksum (shuttl_080.tgz) 3bc39063fe3060cfdac4d29b2c3011ce42c9b72ef4422f298f8f938ff5b00e42 SHA256 checksum (shuttl_072.tgz) a503a8d76c9d5181ab6835352be11f5a99ea8462d48ff9141265ad1617acf9cd SHA256 checksum (shuttl_063.tgz) 0cf557fc428a061e52142f5bb780af6476a4ce29f43a8fd6a9fb1e76bf6f6da6 SHA256 checksum (shuttl_062.tgz) 609bcf7e22618bbbf5664304df184a6ded782ba006f8858752d9def1c1ef902a SHA256 checksum (shuttl_0611.tgz) 4a1d225017c96fafdd6b1bc728903ea121330009fb50728292000f60e4456303
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

splunk

Shuttl

This app has been archived. Learn more about app archiving.
This app is NOT supported by Splunk. Please read about what that means for you here.
Overview
Details
Big Data archiving and bulk data movement for Splunk. Supports HDFS, NFS, Amazon S3, and Amazon Glacier.

Shuttl provides data archive management for Splunk. It supports backend storage solutions such as: ApacheHDFS, Amazon S3, or NFS attached storage. Shuttl works on the bucket level, and leverages the standard Splunk mechanism for archiving data based on total data size or time expiration. Use of Shuttl eliminates the need for Splunk users to implement their own homegrown solution for bulk-moving data to storage backends.

In addition to Archiving, Shuttl is useful for both compliance needs of data retention, as well as improving performance of Splunk. Shuttl also supports archiving the data in CSV format, and therefore, when data is moved to HDFS, it opens up the data to other tools such as Apache Hive and Hadoop Map Reduce to do further data processing and analysis.

For more information see the following blog articles:

Source code is available here: https://github.com/splunk/splunk-shuttl

Quickstart Guide is available here: https://github.com/splunk/splunk-shuttl/wiki/Quickstart-Guide

Setup video is available here: http://www.youtube.com/watch?v=OP7IYNVR5ms

For feedback, please email shuttl-dev at splunk.com.

Release Notes

Version 0.8.4
May 9, 2013

Now comes with both coldToFrozen AND warmToCold transfer retries!
This means that all failed transfers/shuttl's will be retried periodically (default every 60 secs), instead of "everytime any bucket is shuttl'ed".

User happiness is expected to go up by 137%!

Version 0.8.3.1
May 2, 2013

Automatically retries to transfer coldToFrozen failures every 60 secs (configurable)

Version 0.8.1
March 12, 2013
  • Fixed bug where if an indexer (which is a search peer) does not have Shuttl installed, the resulting error would result in missing values in the UI.
  • Further tests on Hadoop 1.1.1
Version 0.8.0
March 1, 2013

Splunk Shuttl 0.8.0 release is another major milestone release.

The main new feature is distributed Shuttl operations from the Splunk Search Head. This means that all Shuttl operations will operate on the entire cluster, and not on a per-indexer basis.

Actions include:
Listing of archived buckets
Thawing of archived buckets
* Flushing of archived buckets

There is no special configuration that needs to be done on the Search Head. Shuttl when installed on a Search Head will query the Search Head for all Search Peers (the Search Peers should all have Shuttl installed of course), and all Shuttl operations will be distributed operations.

Extensive testing has been done for distributed failure scenarios. However, if problems are encountered, please report them ASAP for them to be addressed.

Happy Shuttling!

Version 0.7.2
Jan. 2, 2013

Shuttl version 0.7.2 represents a significant leap in functionality. Thanks to our robust user base for the feature request ideas. Now, Shuttl is easier to setup, supports a new backend (Glacier), reduced archive latency, and full support for the latest Splunk 5.0.

In summary, new features include:

  • Amazon Glacier storage! Amazon Glacier is an extremely low-cost storage service that provides secure and durable storage for data archiving and backup.
  • Splunk 5.0 Clustering Support. Since Splunk replicates buckets, when archiving, any archiving script will by default get duplicate buckets. Shuttl will dedup buckets and not waste storage space.
  • Shuttl now will move data when buckets are moved to cold. This means significantly reduced latency times, and data that is shuttled is also still available for search by Splunk!
  • New simpler and easier configuration! and a script to test out backends, to verify the configuration is correct, without going through the whole exercise of indexing data and forcing a bucket roll.
Version 0.6.3
Sept. 12, 2012
  • Shuttl tested on CDH3. There is a wirelevel incompatibility with CDH and Apache, so there's documents on how to configure Shuttl to work with CDH3 on the github wiki. https://github.com/splunk/splunk-shuttl/wiki/System-Requirements

  • Fix spurious error message when a thaw request is issued while thaw is in progress. No duplicates happen, and all requests are serviced.

  • Simplified the failed bucket transfer dashboard.

  • When doing thaw operation, the UI can "return" and list a table, before the thaw action completes - Fix is to do thaw operation asynchronously, so the UI returns immediately, while job happens in background.

  • Amazon S3 tested, and works with no code change - slight display issue for bucket sizes in the UI, bug filed.

Version 0.6.2
Sept. 7, 2012

Updates:
* Support for Flushing of Thawed Splunk Buckets via the UI

Known Issues being worked on:
Quirks in the Thaw request UI
S3 is not yet tested
* Failed buckets dashboard may report false failures

Version 0.6.1.1
Sept. 7, 2012

Subscribe Share

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
Splunk, Splunk>,Turn Data Into Doing, Data-to-Everything, and D2E are trademarks or registered trademarks of Splunk Inc. in the United States and other countries. All other brand names,product names,or trademarks belong to their respective owners.