Skip to content

dadidange/dl_bio_example

 
 

Repository files navigation

dl_bio_example

Download the dataset on: https://www.kaggle.com/prasunroy/natural-images Unzip and remove the data folder (for some reason the data are saved two times).

Files:

docker/docker_build.sh

Script to build a Docker-Image based on a Dockerfile in the same folder. See Section "Docker" for further information.

docker/Dockerfile

A file containing all the dependencies and information on the folder structure, etc. of the project. This file must be named "Dockerfile"!

config.py

Contains parameters for launching the run_training.py

docker_run.sh

Script to launch a Docker-Container based on the image that is generated by the docker_build.sh script. See Section "Docker" for further information.

run_training.py

The starting point of the project. Execute this file to launch the training process.

train_interfaces.py

A supplementary file for the training process.

Docker

Upon execution of the docker_build.sh script, a Docker-Image will be created. This image will be based on the Dockerfile. Henceforth, you will always have to rebuild the image when your dependencies change or are updated (like for example the DLBio-repository) and you wish to use these updates. Execute the docker_run.sh script to launch a Docker-Container in your terminal. This container will be based on the Docker-image that was created by the docker_build.sh. You can run Files the same way as you would in any other terminal.

More on docker with vs-code: https://moodle.uni-luebeck.de/mod/forum/discuss.php?d=33479

Debugging:

The Debugger has two options. Select "Docker: Python - General" to start debugging the project from its main file. Select "Docker: Python - Current" to start debugging the currently in the editor selected file. More configurations can be added or present configurations can be changed in the ./vscode/launch.json and ./vscode/tasks.json.

Visdom

Visdom eases to display data remotely. For general infos see their git: visdom.

Here it is used to display the feature representation of a trained model. The represantation is saved in exe_save_embeddings.py and plotted in plot_embeddings.py.

Visdom can also be used for visualizing images and plots. Start the visdom server from the command line with visdom and add something to it for example like this:

import visdom
from datasets.ds_mnist import get_dataloader

dl = get_dataloader(False, batch_size=16)	
image = dl.dataset.data[0]

vis = visdom.Visdom()
vis.image(image)

Installation

You can install the stable version via pip but I would recommend installing it from source to get the current features (See https://github.com/fossasia/visdom#setup).

For plotting the embeddings this has to be done anyway at the moment (march 21). Because the vis.embeddings uses this package for the t-sne algorithm (https://github.com/lvdmaaten/bhtsne) you will probably have to install this as well. For this, clone the repo into visdom/py/visdom/extra_deps and follow the installation steps here: https://github.com/lvdmaaten/bhtsne#installation

Tensorboard

Tensorboard can be used to display training experiments. Infos at https://www.tensorflow.org/tensorboard and here for using it in pytorch. Tensor behaves similarly to a jupyter notebook.

The information which will be used by tensorboard can be stored during or after an experiment, see 'exe_log_tb.py' or tensorboard_test.py. For run_training this can be activated with --log_tb. By default the data will be stored in runs/_debug and can be visualized by 'tensorboard --logdir=runs' if tensorboard is installed.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Jupyter Notebook 72.4%
  • Python 25.6%
  • Dockerfile 1.7%
  • Shell 0.3%