Skip to content

heydenberk/s3nb

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

66 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

S3-backed notebook manager for IPython

Setup

  1. Install:

    from pypi:

    pip install s3nb

    from source with pip:

    pip install git+https://github.com/monetate/s3nb

    or from source the old fashioned way:

    git clone git@github.com:monetate/s3nb.git
    cd s3nb
    python ./setup.py install
  2. Configure

    # set this - notebooks will be stored relative to this uri
    S3_NOTEBOOK_URI=s3://path/to/notebooks/
    
    # and this
    IPYTHON_MAJOR_VERSION=2
    
    # optionally set this - checkpoints will be stored locally, relative to this path (for IPython 3)
    CHECKPOINT_ROOT_DIR=~/.checkpoints
    
    # optionally set this
    PROFILE=s3nbserver
    
    # shouldn't need to edit beyond this point
    
    ## IPython 2.x
    IPYNB_MANAGER=S3NotebookManager
    IPYNB_MANAGER_CFG=notebook_manager_class
    
    ## IPython 3.x
    if [ $IPYTHON_MAJOR_VERSION == 3 ]; then
        IPYNB_MANAGER=S3ContentsManager
        IPYNB_MANAGER_CFG=contents_manager_class
    fi
    
    IPYTHONDIR=${IPYTHONDIR:-$HOME/.ipython}
    PROFILE_DIR=${IPYTHONDIR}/profile_${PROFILE}
    
    if [ ! -d $PROFILE_DIR ]; then
        ipython profile create $PROFILE
        IPYNB_CONFIG=${PROFILE_DIR}/ipython_notebook_config.py
        mv $IPYNB_CONFIG $IPYNB_CONFIG.orig
        cat > $IPYNB_CONFIG <<EOF
    c = get_config()
    c.NotebookApp.${IPYNB_MANAGER_CFG} = 's3nb.${IPYNB_MANAGER}'
    c.${IPYNB_MANAGER}.s3_base_uri = '$S3_NOTEBOOK_URI'
    EOF
    fi
    
    
    if [ $IPYTHON_MAJOR_VERSION == 3 ]; then
        echo "c.S3ContentsManager.checkpoints_kwargs = {'root_dir': '${CHECKPOINT_ROOT_DIR}'}"  >> ${IPYNB_CONFIG}
    fi
  3. If you haven't already, configure AWS variables for boto. Follow these instructions.

  4. Run

    ipython notebook --profile=s3nbserver

Development

  1. Provision a virtual machine with vagrant up
  2. Create an IPython profile with make configure -e S3_BASE_URI=YOUR_BUCKET
  3. Share you AWS credentials with the virtual machine with make configure -e AWS_USER=YOUR_USER
  4. Run the notebook server with make run

About

S3-backed notebook manager for IPython 2.x and 3.x

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 95.3%
  • Makefile 4.1%
  • Shell 0.6%