For DSDL - formerly known as Deep Learning Toolkit (DLTK) - you find documentation on https://docs.splunk.com/Documentation/DSDL. Feel free to open issues, report bugs or raise feature requests on https://github.com/splunk/splunk-mltk-container-docker. You can also post your questions and engage with answers on community.splunk.com. Thanks for your collaboration!
Extend the app with custom MLTK Containers: if you want to rebuild the existing MLTK Container images or want to build your own custom images navigate to https://github.com/splunk/splunk-mltk-container-docker
Q: When I launch a container the first time, I cannot access Jupyter Lab.
A: The selected container image will be downloaded from dockerhub automatically in the background when you launch a container for the first time. Depending on your network this can take a while to download the docker image for the first time as the image sizes range from 2-12 GB. Please allow for some time to get the images initially pulled from Dockerhub. You can check which docker images are available locally by running docker images on your CLI.
Q: When I run DLTK 3.5 (or later) locally my browser showes insecure connection.
A: From DLTK 3.5 onwards the container images use HTTPS by default with self signed certificates for the data transfer related api endpoints and Jupyter Lab. Many browsers show "insecure connection" warnings and some allow to suppress that for localhost connections used during development. For production use, please work with your administrators to secure your setup and build containers with your own certificates as needed or use more advanced container environment setups.
Q: The example dashboards show no results or throw errors.
A: First, ensure that the right container image is downloaded and up and running for the specific example (e.g. TensorFlow examples require a TensorFlow container). Secondly, ensure that you have verified the associated notebook code exists in Juypter Lab and you have explicitly saved the notebook again (hit save button). By doing this, a python module is saved automatically generated (located in the /app/model folder in Juypter) which is needed to run the examples and populate the dashboards. Lastly, please check if MLTK's app permissions are set to global so that DLTK can use the lookup files used in most of the examples.
Q: Containers suddenly get stopped in about 1 minute after start.
A: Most likely you have two or more DLTK apps installed and configured to use the same container environment. In DLTK 3.x versions there is a scheduled search (MLTK Container Sync) that ensures synchronization of running containers and associated models for the DLTK app. If more than one DLTK app is running, this can cause synchronization collisions and therefore containers get stopped. When using DLTK 3.x please connect each DLTK app in a 1:1 relationship with your Docker and/or Kubernetes environment.
Q: After a version update I see the error "unable to read JSON response" when running a DLTK related search.
A: This may indicate that some part of the local configuration of DLTK is out of sync. The issue was reported to be resolved by opening the setup dashboard with the existing settings and just "test and save" again to reconfirm the configuration.
Q: Where are my notebooks stored in the docker environment?
A: By default, there are 2 docker volumes automatically mounted for persistance in your docker environment. Those volumes are named "mltk-container-app" and "mltk-container-notebooks". You can verify by running docker volume ls on your CLI. Important note: from DLTK version 3.1 onwards there is a new default volume called "mltk-container-data" - see migration notes below.
Q: What is the password for Jupyter Lab?
A: Please have a look at the Model Development Guide page in the Deep Learning Toolkit app. We recommed you set an own strong password.
Q: When I use conda or pip install in a running container the imported libraries can not be accessed
A: For added security reasons the default user in a DLTK container is not the root user and therefore libraries are installed under the user dltk which can cause issues in using them in Juypter. We recommend to use the build scripts and modify your Dockerfile and built your own container image. This way you also have the libraries installed persistently and you don't need to reinstall them when you restart the container the next time.
Due to the addition of Kubernetes there was a change and addition made to the way how volumes behave. From 3.1 on and with the use of the golden image the container directory /srv is the default and notebooks and app code is in there. In earlier versions there were two docker volumes mounted into /srv/app and /srv/notebooks which will be mapped into a backup folder in Jupyter from version 3.1 on. For migration simply copy your notebooks from the backup folder back into the notebooks and app folder in case those are empty.
Updated container images for:
- Golden Image CPU
- Golden Image GPU
- Rapids 0.17
- Spark 3.0.1
With improvements for:
- Model management with MLflow
- Integrated GIT version control in Jupyter Lab
- HTTPS for api and jupyter endpoints
Newly added algorithm examples:
- Matrix profiles with STUMPY
- Changepoint Detection
- Multivariate LSTM Regressor
Added algorithm examples for
- Support Vector Regressor with grid search in scikit-learn
- Causal inference
Updated Golden Image GPU to version 3.4.1
Example for Process Mining with PM4Py
Other minor small fixes
Graphics:
- Background graphics in content overview page
- Docker + Kubernetes status green highlight in setup dashboard
- Content UI icons refresh
New docker images for Spark and Rapids:
- Spark Image (Experimental)
- Rapids Image (Experimental)
Content updates:
- Correlation Matrix and seaborn plot embedding
- DGA datashader example
- Spark GradientBoosting (non distributed, local client only)
- Spark Hello World / Barebone / Pi
- Spark FP Growth
- Spark ALS Recommender System
- Rapids Graph example
- Rapids UMAP example
Other:
- Passwords.conf add for missing kubernetes field
- Search head cluster config replication (Thank you Martin!)
- Return dataframe of arbitrary/changed shapes
This version is only compatible with Splunk 8.0+ and MLTK 5.0+
If you run on Splunk 7.x and MLTK 4.x please select, download and install Deep Learning Toolkit version 2.3.0
This version is only compatible with Splunk 7.x and MLTK 4.x only.
If you run on Splunk 8.0+ and MLTK 5.0+ please select, download and install Deep Learning Toolkit version 3.0.0
As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.