- General info
- Content
- Project Folder Structure
- Setup
- Usage
- Results
- TODO
- Current tasks
- Notes
- Testing setup
- Resources
This project contains code for some of the most know types of GAN (Generative Adverserial Network). I am using this repo to play with these types of networks to get better understanding how they work and how to properly train them.
Disclaimer: This repository is more like proof of concept than download and run!
DCGAN - GAN for generating new images from latent vector
WGAN(GC) - GAN for generating new images from latent vector
SRGAN - GAN for upscaling images
- datasets (place for all data that you will want feed to network)
- media (folder with media files of repo)
- modules
- gans (trainers for all GANs are pleced)
- keras_extensions (extesions based on keras functionality)
- models (models and building blocks for models)
- utils (other helper stuff)
- settings (settings for scripts)
- utility_scripts (scripts for data processing and other useful stuff)
pip install -r requirements.txt
- Python3.7
- Tensorflow 2.2.0
- Keras 2.3.1
For GPU acceleration:
- Cuda Toolkit 10.1
- cuDNN for toolkit version
Adjust settings in settings/****_settings.py
Get datasets and place them in dataset directory (Or in directory you set in settings.py)
python preprocess_dataset.py
python train_****.py
After training use
1) python generate_images.py for DCGAN and WGAN
2) python upscale_images.py for SRGAN
(These scripts still needs tweaking because settings for them are hardcoded in them)
Note: These scripts are still in progress of making, some of them may not work!
preprocess_dataset.py - Script for mass rescaling images to target size and optionaly splitting them to training and testing parts
visualize_conv_activations.py - Script for displaying activation of each conv layer as image
show_vgg_structure.py - Script that will print all layers of vgg19 usable for perceptual loss
parse_hr_image.py - Script to parse large images to small ones (WIP)
Note: Some utility scripts have its settings in settings folder
For my dataset ideal pretrain of generator is something around 50k episodes \
- No pretrain, 400k episodes
- 50k pretrain, 400k episodes
- 200k pretrain, 400k episodes
Generator / Discriminator (Critic)
1. mod_srgan_exp / mod_base_9layers
2. mod_srgan_exp_sn / mod_base_9layers_sn
- Implement DCGAN
- Implement WGAN
- Implement SRGAN
- Implement StyleGAN
- Implement custom Tensorboard for logging stats
- Implement learning rate scheduler
- Implement spectral normalization layer
- Test Charbonnier loss instead of MSE loss in SRGAN
- Test Huber loss instead of MSE loss in SRGAN
- Test MAE loss instead of MSE loss in SRGAN
- Test impact of different weights of losses in SRGAN
- Optimize training loop of SRGAN (Too much time is spending of testing performance)
- Implement custom batch maker
- Optimize batch maker to use generator class from keras
- Optimize preprocessing dataset (Too slow)
- Optimize interface scripts with more acessible settings
- Test pretrain effect on results from SRGAN
- Retrain all SRGAN models with single test image with same train settings to properly compare them
- Implement gradient accumulation to "simulate" larger batch
- Testing best working model pairs for WGAN
- Refactoring
- Retraining all SRGAN models
- Testing efects of pretrain on SRGAN model
Testing of Charbonnier loss for SRGAN failed because the values were too different from MSE loss values, maybe more tweaking required and test again.
MAE loss is causing lot of artifacts and image distortions (like color shifting, "image bleedoff", etc) in results from SRGAN. \
Hardware:
Processor: I7-9700KF 4.8GHz
RAM: HyperX Fury RGB 32GB (2x16GB) DDR4 3200MHz
GPU: GIGABYTE GeForce RTX 2080 SUPER 8G
SSD: Intel 660p M.2 2TB SSD NVMe
Editor: PyCharm (always latest version)
https://github.com/mitchelljy/DCGAN-Keras
https://arxiv.org/pdf/1511.06434.pdf
https://machinelearningmastery.com/how-to-code-a-wasserstein-generative-adversarial-network-wgan-from-scratch/
https://arxiv.org/pdf/1704.00028.pdf
https://github.com/LuEE-C/WGAN-GP-with-keras-for-text/blob/master/Exploration/GenerativeAdverserialWGAN-GP.py
https://github.com/kongyanye/cwgan-gp
https://github.com/keras-team/keras-contrib/blob/master/examples/improved_wgan.py
https://github.com/eriklindernoren/Keras-GAN/blob/master/wgan_gp/wgan_gp.py
keras-team/keras#8585
Note: Some of these concepts were used in my implementation of my WGAN
https://arxiv.org/pdf/1609.04802.pdf
https://github.com/deepak112/Keras-SRGAN/blob/master/Network.py
https://medium.com/@birla.deepak26/single-image-super-resolution-using-gans-keras-aca310f33112
https://github.com/MathiasGruber/SRGAN-Keras
https://github.com/idealo/image-super-resolution
https://github.com/JGuillaumin/SuperResGAN-keras
https://github.com/Golbstein/EDSR-Keras
Note: Ideas used for improving SRGAN
https://arxiv.org/pdf/1603.08155.pdf%7C
https://deepai.org/machine-learning-glossary-and-terms/perceptual-loss-function
https://github.com/yuqil/style_transfer_keras/blob/master/vgg/vgg_loss.py
https://machinelearningmastery.com/practical-guide-to-gan-failure-modes/
https://github.com/soumith/ganhacks
https://medium.com/intel-student-ambassadors/tips-on-training-your-gans-faster-and-achieve-better-results-9200354acaa5
https://distill.pub/2016/deconv-checkerboard/
https://stackoverflow.com/questions/55268762/how-to-accumulate-gradients-for-large-batch-sizes-in-keras
keras-team/keras#3556 (comment)