Script of Scripts (SoS) is a file format for the inclusion of scripts in multiple languages into a single executable script, an interactive environment and notebook tool for working with different scripts, and a workflow engine for the execution of workflows consisting with scripts in different languages. It is designed for data scienticists and bioinformatics who routinely work with scripts in different languages such as R, Python, Perl, and bash.

As a file format that help you record all your analysis into a single script with minimal effort, SoS allows you to include scripts in different languages literally into a .sos files, and execute it as a single workflow. The basis of this format is demonstrated in our Quick start guide.

As an interactive environment and notebook tool that promotes literate programming, SoS allows you to perform and record your analysis in different languages in a single Jupyter notebook, with seamless integration of multiple Jupyter kernels (e.g. python, and R). You can check out example notebooks and create new ones from our public Jupyter server.

As a workflow engine, SoS helps you oraganize your commands and scripts in different languages into readable workflows that can be easily understood and modified by others. The workflows can be specified in both forward (step by step), makefile (dependency rules) and even a mixture of both styles. Beacuse of a great emphasis on readability, SoS is an easy-to-use alternative to specialized workflow systems such as CWL which makes it an ideal tool for the creation and maintainance of workflows that need to be frequently updated and shared with others.

SoS is released under GPL3. It supports Linux and Mac OSX systems and requires Python version 3.3 or higher. You can install the latest released version using command

% pip3 install sos

or compile the latest git version with commands

% git clone https://github.com/BoPeng/SOS.git
% cd SOS
% pip3 install sos
% python3 setup.py install

Note that

• You might need to use command pip and python instead of pip3 and python3 if you have python 3 as the default python installation.
• If command sos is not found after installation, you will need to add paths such as /Library/Frameworks/Python.framework/Versions/3.4/bin/ to $PATH or create symbolic links of sos and sos-runner commands in /usr/local/bin.

The following packages are needed if you are using SoS with R

• R version 3.2 or later (for IRKernel)
• IRKernel R kernel for Jupyter
• Python feather-format module for exchanging data frames between SoS/Python DataFrame and R data.frame. This module is not installed by default because it need to compile from source or install from conda.

If you are using docker, you can run SoS directly using command

% docker run -it mdabioinfo/sos:latest /bin/bash

to enter a command prompt with sos command. More usefully, you can start a Jupyter server with R and IRkernel, Julia, Python, and SoS kernels, and many Python and R modules for data sciencists using command

% docker run -d -p 8888:8888 mdabioinfo/sos-notebook

After the docker is running in the background, you can start a browser and start working with a complete SoS environment with URL

http://localhost:8888

You can even use this docker image for your daily data analysis if you make your local directory available to the Jupyter server using command

% docker run -d -p 8888:8888 -v $HOME:/home/jovyan/work  mdabioinfo/sos-notebook

This command mounts your home directory ($HOME) to the docker machine but you can specify any local directory. This container is hosted at our public Jupyter server from which you can open our sample notebooks and create your own notebooks without installing anything.

Please find more information on SoS wiki, or use the following links directly:

• Quick start guide
• Tutorial and example of Jupyter notebooks.
• A presentation about SoS (updated on Apr. 7th, 2016 for version 0.5.7, already quite outdated given the rapid evolution of SoS)
• Complete documentation
• Tutorials on using SOS with iPython, Jupyter, and Spyder.
• A complex RNASeq data analysis workflow written in SoS.