Silver(top 3%) solution for tgs-salt-identification-challenge

I've tried many methods in this competition but mainly focus on seresnext-unet architecture. In the src folder, there are bunch of notebooks using both keras and pytorch to implement the network.

start with simple UNet

I started in unet_baseline.ipynb which defined a straight foward unet. Then tried to fine tune it with deeper layers, se-blocks elu activation, kfolds. Did not get much better results.

Try a dedicated classifier

Since there are partial images with empty masks, it seems there maybe some rule to follow in identifying the empty images. I've tried transfer learning with all keras prebuild CNN classifiers such as DenseNet and InceptionResNetV2 in classifyer.ipynb. They were some what working and after ensemble the results from classifiers and the unet segmentation model, the results improved a little. Just a little.

Try deeper UNets

In the next step, I tried deeper architectures in the backend of UNet with resblocks, inception blocks. The results did not imrpove. The reason for this is covered until the end of this competition that the organizer of this competions made some tricks in the train masks. Talk it later.

Try to train a hybrid model

This is actually an old idea but works here. It was to extract the backend output from previous Unet to make a classifier, then combine the output of the classifier with the segementer to get the mix loss. I tried to implement this idea in keras APIs with 2 loss but found it clumsy and the results nearly not improved. Then I tried the pytorch with newer pretrained se_resnext50 from imagenet which results much better score in leaderboard.

Kfolds with rounds

Then is the repeat training with kfolds and select the best model from the current round and train again. The local CV iou increase a lot from the second round but the LB increase less. I traind 5 folds for 4 rounds and get the best score before the competition ends.

Just before it ends

The kaggle grandmaster Heng CherKeng explained why the label mask is whierd just hours before this competition ends. It is neccesary to try a Mosaics puzzle solving to really uncover this 'secret' and that's what I totally missed. It says if combine one more channel to predict the edge of the salt into the previous model will bring the score in to 30th. But I didn't have time to try it out.

Conclution

There are always new skills to learn from each kaggle competitions and the masters in the platform like Heng CherKeng. Keep fighting!

Name		Name	Last commit message	Last commit date
Latest commit History 6 Commits
models		models
utils		utils
EDA.ipynb		EDA.ipynb
README.md		README.md
average_models.py		average_models.py
blend_masks.py		blend_masks.py
classifyer.ipynb		classifyer.ipynb
classifyer_argu.ipynb		classifyer_argu.ipynb
classifyer_kfold.ipynb		classifyer_kfold.ipynb
classifyer_kfold.py		classifyer_kfold.py
classifyer_kfold_pre.ipynb		classifyer_kfold_pre.ipynb
densenet.ipynb		densenet.ipynb
gen_border.ipynb		gen_border.ipynb
inception.ipynb		inception.ipynb
inception.py		inception.py
inception_unet_depth.ipynb		inception_unet_depth.ipynb
kaggle_util.py		kaggle_util.py
make_preds.py		make_preds.py
models.py		models.py
plot_learning_curv.ipynb		plot_learning_curv.ipynb
predict.ipynb		predict.ipynb
pytorch_3loss.ipynb		pytorch_3loss.ipynb
res_unet_deeper_argu.ipynb		res_unet_deeper_argu.ipynb
res_unet_deeper_csc_argu.ipynb		res_unet_deeper_csc_argu.ipynb
res_unet_deeper_cse.ipynb		res_unet_deeper_cse.ipynb
res_unet_deeper_cse_up.ipynb		res_unet_deeper_cse_up.ipynb
res_unet_deeper_se.ipynb		res_unet_deeper_se.ipynb
res_unet_depth.ipynb		res_unet_depth.ipynb
res_unet_depth_actvalue.ipynb		res_unet_depth_actvalue.ipynb
res_unet_depth_channel.ipynb		res_unet_depth_channel.ipynb
res_unet_depth_channel_cse.ipynb		res_unet_depth_channel_cse.ipynb
res_unet_depth_channel_cse_up.ipynb		res_unet_depth_channel_cse_up.ipynb
res_unet_depth_cse.ipynb		res_unet_depth_cse.ipynb
res_unet_depth_cse_up.ipynb		res_unet_depth_cse_up.ipynb
res_unet_depth_se.ipynb		res_unet_depth_se.ipynb
resunet_sgd.ipynb		resunet_sgd.ipynb
train.py		train.py
train_boundry.py		train_boundry.py
train_fix.py		train_fix.py
train_fix2.py		train_fix2.py
train_mt.py		train_mt.py
train_smt.py		train_smt.py
unet_baseline.ipynb		unet_baseline.ipynb
unet_deeper.ipynb		unet_deeper.ipynb
unet_deeper_3channel.ipynb		unet_deeper_3channel.ipynb
unet_deeper_3loss.ipynb		unet_deeper_3loss.ipynb
unet_deeper_cse_3loss.ipynb		unet_deeper_cse_3loss.ipynb
unet_deeper_depth_3loss.ipynb		unet_deeper_depth_3loss.ipynb
unet_deeper_depth_3loss.py		unet_deeper_depth_3loss.py
unet_deeper_elu_fix_kfold.ipynb		unet_deeper_elu_fix_kfold.ipynb
unet_deeper_elu_kfold.ipynb		unet_deeper_elu_kfold.ipynb
unet_deeper_elu_kfold.py		unet_deeper_elu_kfold.py
unet_deeper_elu_kfold_predict.ipynb		unet_deeper_elu_kfold_predict.ipynb
unet_deeper_elu_kfold_predict_avg.ipynb		unet_deeper_elu_kfold_predict_avg.ipynb
unet_deeper_kfold.ipynb		unet_deeper_kfold.ipynb
unet_deeper_kfold.py		unet_deeper_kfold.py
unet_deeper_kfold_fix.ipynb		unet_deeper_kfold_fix.ipynb
unet_deeper_kfold_fix_predict_avg.ipynb		unet_deeper_kfold_fix_predict_avg.ipynb
unet_deeper_kfold_predict.ipynb		unet_deeper_kfold_predict.ipynb
unet_deeper_kfold_predict_avg.ipynb		unet_deeper_kfold_predict_avg.ipynb
unet_deeper_moredrop_3loss.ipynb		unet_deeper_moredrop_3loss.ipynb
unet_deeper_pad.ipynb		unet_deeper_pad.ipynb
unet_deeper_se_kfold.ipynb		unet_deeper_se_kfold.ipynb
unet_deeper_tune_sgd.ipynb		unet_deeper_tune_sgd.ipynb
unet_res50.ipynb		unet_res50.ipynb
unet_vgg.ipynb		unet_vgg.ipynb
unet_vgg_2loss.ipynb		unet_vgg_2loss.ipynb

kownse/tgs-salt-identification-challenge

Folders and files

Latest commit

History

Repository files navigation