Key components:

1. Programming lanurage: Python3
2. Interactive computing: Jupyter
3. Task Dependency and run: Snakemake
4. Package management: Conda
5. Deploy the environment: Docker
6. web based X server: Xpra
7. Image to Object: ImageAI
8. AI framework: GYM
9. App test auto: Appium

Steps to create jupyterlab server:

1. download git by git clone https://github.com/chengyehwang/jupyterlab
2. cd jupyterlab
3. download docker image: make pull
4. run jupyterlab: make run
5. copy the link http://xxxx:8888#?token=yyyy and open by browser

Key features:

1. working directory shared between docker and host

   • install your tools in host and process by yourself

2. jupyterlab server is running on docker and extensions are enabled

   • jupytext for git diff, show table of content, show variables for debug

3. python3 packages: pandas, plotly are ready

   • data processing (table process, show chart)

4. cython is ready for speed

   • speedup kernel function by C level
   • compile and load module when needed

5. switch between interactive and command line

   • jupyterlab for interactive "make run"
   • ipython for command line "make cmd"

6. Debugging by jupyterlab-debugger

   • normal mode by kernel "Python 3"
   • debugging mode by kernel "xpython"

7. Use Jupytext in Jupyterlab

   • new notebook "Python 3" -> file Untitled.ipynb
   • File/Save Notebook As -> your-file-name.py
   • or Rename Notebook... -> your-file-name.py
   • Access (open / save) your-file-name.py by Notebook

8. Image export

   • Plotly to image kaleido

9. Design Pattern

   • input / output by json
   • Data source
     • Database
     • Remote files SSHFS
     • S3 server minio
   • Process - interactive
     • Pandas DataFrame
     • Plotly Chart
     • show table QGrid
   • Report
     • Excel: openpyxl
     • PPT: python-pptx

10. reduce image size

    • Based on Docker Ubuntu
    • Based on Miniconda
    • Add extra Ubuntu packages and Conda packages by Docker command

11. easy to customize your packages

    • Create a customized docker
      • Fork git (https://github.com/chengyehwang/jupyterlab)
      • Refine Dockerfile and shell scripts *.sh
      • "make build" to build new image
    • Run time
      • wget / vim / git / make is installed
      • "make shell" to enter container
    • trade-off between package coverage and image size
      • database: sqlite, postgresql, or mongo

12. use Makefile to maintain commands

    • run process:
      • make pull: get image from docker hub
      • make run: run jupyterlab server in container
      • make cmd: run ipython demo.py in container
      • make stop: stop and kill all docker jobs
    • develop process:
      • make build: build docker image
      • make shell: run shell in container
      • make push: push image to docker hub
      • make clean: remove images
      • make image: list images

13. demo.py is provided to demo:

    • jupytext to save as .py for diff
    • pandas to handle table and show
    • plotly to show chart
    • jupyterlab-toc to show table of contents

14. ports & service

    • 8888: jupyterlab - computing
    • 8080: snakeviz - debug
    • 6080: xpra - X11 server
    • 9000: minio - storage image

15. embedded dev

    • termux
    • proot to ubuntu 20.04 (arm64)
    • python 3.7.X / pip
    • jupyterlab server (3.0.X) on phone
    • browser access by PC
    • camera by termux-api
    • NN by tensorflow (2.4)
    • Object detect by imageai
    • UI by dash
    • QR code scan by google lens for login
    • fdroidserver for apk build

16. Step-by-Step Install

    • install termux apk
    • install termux-api apk
    • launch termux
    • apt install git
    • git clone https://github.com/chengyehwang/jupyterlab
    • cd jupyterlab/termux
    • ./jupyter.sh -U # install proot ubuntu 20.04 & auto by .bash_profile
    • ./jupyter.sh -u # launch proot ubuntu 20.04
    • cd /data/data/com.termux/files/home/jupyterlab/termux
    • ./jupyter.sh -i # install packages
    • ./jupyter.sh -D # auto by .bash_profile
    • ./jupyter.sh -d # run dash

17. ToDo

    • Termux fork and maintain by github
    • flow to build proot image file (apt + pip ready)
    • Termux GUI webkit -> access localhost:8050 by webkit + javascript
    • enable GPU on Termux, tensorflow + GPU
    • BotAI