Download the dataset on: https://www.kaggle.com/prasunroy/natural-images Unzip and remove the data folder (for some reason the data are saved two times).
Script to build a Docker-Image based on a Dockerfile in the same folder. See Section "Docker" for further information.
A file containing all the dependencies and information on the folder structure, etc. of the project. This file must be named "Dockerfile"!
Contains parameters for launching the run_training.py
Script to launch a Docker-Container based on the image that is generated by the docker_build.sh script. See Section "Docker" for further information.
The starting point of the project. Execute this file to launch the training process.
A supplementary file for the training process.
Upon execution of the docker_build.sh script, a Docker-Image will be created. This image will be based on the Dockerfile. Henceforth, you will always have to rebuild the image when your dependencies change or are updated (like for example the DLBio-repository) and you wish to use these updates. Execute the docker_run.sh script to launch a Docker-Container in your terminal. This container will be based on the Docker-image that was created by the docker_build.sh. You can run Files the same way as you would in any other terminal.
More on docker with vs-code: https://moodle.uni-luebeck.de/mod/forum/discuss.php?d=33479
The Debugger has two options. Select "Docker: Python - General" to start debugging the project from its main file. Select "Docker: Python - Current" to start debugging the currently in the editor selected file. More configurations can be added or present configurations can be changed in the ./vscode/launch.json and ./vscode/tasks.json.
Visdom eases to display data remotely. For general infos see their git: visdom.
Here it is used to display the feature representation of a trained model.
The represantation is saved in exe_save_embeddings.py
and plotted in plot_embeddings.py
.
Visdom can also be used for visualizing images and plots.
Start the visdom server from the command line with visdom
and add something to it for example like this:
import visdom
from datasets.ds_mnist import get_dataloader
dl = get_dataloader(False, batch_size=16)
image = dl.dataset.data[0]
vis = visdom.Visdom()
vis.image(image)
You can install the stable version via pip but I would recommend installing it from source to get the current features (See https://github.com/fossasia/visdom#setup).
For plotting the embeddings this has to be done anyway at the moment (march 21).
Because the vis.embeddings
uses this package for the t-sne algorithm (https://github.com/lvdmaaten/bhtsne) you will probably have to install this as well.
For this, clone the repo into visdom/py/visdom/extra_deps and follow the installation steps here: https://github.com/lvdmaaten/bhtsne#installation
Tensorboard can be used to display training experiments. Infos at https://www.tensorflow.org/tensorboard and here for using it in pytorch. Tensor behaves similarly to a jupyter notebook.
The information which will be used by tensorboard can be stored during or after an experiment, see 'exe_log_tb.py' or tensorboard_test.py
.
For run_training
this can be activated with --log_tb
. By default the data will be stored in runs/_debug
and can be visualized by 'tensorboard --logdir=runs' if tensorboard is installed.