Kaggle Ultrasound Nerve Segmentation competition [Keras]

Enjoy!

1. data.py -- modified create_test_data() to include test_mask into npy for predict evaluation.
   -- resize here to let images input of different sizes possible.
2. u_model.py -- updated Conv2D, BatchNormalization, concatenate according to keras2 API.
3. train.py -- small changes.
4. current.py -- add process and save test_mask.
5. submission.py -- calulate mean dice score for test samples, save predicted images.
6. add file keras_pre_img.py, which can be found at https://www.kaggle.com/hexietufts/easy-to-use-keras-imagedatagenerator

#Install (Ubuntu {14,16}, GPU)

cuDNN required.

###Theano

• http://deeplearning.net/software/theano/install_ubuntu.html#install-ubuntu
• sudo pip install pydot-ng

In ~/.theanorc

[global]
device = gpu0
[dnn]
enabled = True

###Keras

• sudo apt-get install libhdf5-dev
• sudo pip install h5py
• sudo pip install pydot
• sudo pip install nose_parameterized
• sudo pip install keras

In ~/.keras/keras.json (it's very important, the project was running on theano backend, and some issues are possible in case of TensorFlow)

{
    "image_dim_ordering": "th",
    "epsilon": 1e-07,
    "floatx": "float32",
    "backend": "theano"
}

###Python deps

• sudo apt-get install python-opencv
• sudo apt-get install python-sklearn

#Prepare

Place train and test data into '../train' and '../test' folders accordingly.

mkdir np_data
python data.py

#Training

Single model training.

python train.py

Results will be generatated in "res/" folder. res/unet.hdf5 - best model

Generate submission:

python submission.py

Generate predection with a model in res/unet.hdf5

python current.py

#Model

Motivation's explained in my internal pres (slides: http://www.slideshare.net/Eduardyantov/ultrasound-segmentation-kaggle-review)

I used U-net like architecture (http://arxiv.org/abs/1505.04597). Main differences:

• inception blocks instead of VGG like
• Conv with stride instead of MaxPooling 有点意思
• Dropout, p=0.5
• skip connections from encoder to decoder layers with residual blocks
• BatchNorm everywhere
• 2 heads training: auxiliary branch for scoring nerve presence (in the middle of the network), one branch for segmentation
• ELU activation
• sigmoid activation in output
• Adam optimizer, without weight regularization in layers
• Dice coeff loss, average per batch, without smoothing
• output layers - sigmoid activation
• batch_size=64,128 (for GeForce 1080 and Titan X respectively)

Augmentation:

• flip x,y
• random zoom
• random channel shift
• elastic transormation didn't help in this configuration

Augmentation generator (generate augmented data on the fly for each epoch) didn't improve the score. For prediction augmented images were used to calculate the mean score.

Validation:

For some reason validation split by patient (which is proper in this competition) didn't work for me, probably due to bug in the code. So I used random split.

Final prediction uses probability of a nerve presence: p_nerve = (p_score + p_segment)/2, where p_segment based on number of output pixels in the mask.

#Results and technical aspects

• On GPU Titan X an epoch took about 6 minutes. Training early stops at 15-30 epochs.
• For batch_size=64 6Gb GPU memory is required.
• Best single model achieved 0.694 LB score.
• An ensemble of 6 different k-fold ensembles (k=5,6,8) scored 0.70399

#Credits This code was originally based on https://github.com/jocicmarko/ultrasound-nerve-segmentation/