JSPNet: Learning Joint Semantic & Instance Segmentation of Point Clouds via Feature Self-similarity and Cross-task Probability
The code has been tested with Python 3.5 on Ubuntu 16.04.
- TensorFlow 1.4
- h5py
- Download 3D indoor parsing dataset (S3DIS Dataset). Version 1.2 of the dataset is used in this work.
python utils/s3dis_utils/collect_indoor3d_data.py
python utils/s3dis_utils/s3dis_gen_h5.py
cd data && python generate_input_list.py && python generate_train_test_list.py
cd ..- (optional) Prepared HDF5 data for training is available here.
| baseline | S&I module | mIoU | mPre | para |
|---|---|---|---|---|
| JSPNet | w/o | 52.3 | 52.0 | |
| JSPNet | SIFF | 54.7 | 57.9 | |
| JSPNet | PIFF | 55.2 | 60.2 | model |
| JSPNet | SIFF&PIFF | 55.8 | 59.3 | model |
-
Compile TF Operators
Refer to PointNet++
-
Training, Test, and Evaluation
cd models/JSPNet/
ln -s ../../data .
# training
python train.py \
--gpu 0 \
--data_root ./ \
--data_type numpy \
--max_epoch 100 \
--log_dir ../../logs/train_5 \
--input_list data/train_file_list_woArea5.txt
# estimate_mean_ins_size
python estimate_mean_ins_size.py \
--data_root ./ \
--input_list data/train_hdf5_file_list_woArea5.txt \
--out_dir ../../logs/train_5
# test
python test.py \
--gpu 0 \
--data_root ./ \
--data_type hdf5 \
--bandwidth 0.6 \
--num_point 4096 \
--log_dir ../../logs/test_5 \
--model_path ../../logs/train_5/epoch_99.ckpt \
--input_list data/test_hdf5_file_list_Area5.txt
# evaluation
python eval_iou_accuracy.py --log_dir ../../logs/test_5Note: We test on Area5 and train on the rest folds in default. 6 fold CV can be conducted in a similar way.
If our work is useful for your research, please consider citing:
@article{chen2021jspnet,
title={JSPNet: Learning Joint Semantic \& Instance Segmentation of Point Clouds via Feature Self-similarity and Cross-task Probability},
author={Chen, Feng and Wu, Fei and Gao, Guangwei and Ji, Yimu and Xu, Jing and Jiang, Guo-Ping and Jing, Xiao-Yuan},
journal={Pattern Recognition},
pages={108250},
year={2021},
publisher={Elsevier}
}
This code largely benefits from following repositories: ASIS, PointNet++, PointConv, SGPN DiscLoss-tf JSNet Pytorch-Encoding