This repository contains official implementation of Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes in CVPR 2020, by Huanjing Yue, Cong Cao, Lei Liao, Ronghe Chu, and Jingyu Yang.

http://openaccess.thecvf.com/content_CVPR_2020/papers/Yue_Supervised_Raw_Video_Denoising_With_a_Benchmark_Dataset_on_Dynamic_CVPR_2020_paper.pdf
http://openaccess.thecvf.com/content_CVPR_2020/supplemental/Yue_Supervised_Raw_Video_CVPR_2020_supplemental.pdf

https://youtu.be/5za3d81Eiqk

You can download our dataset from MEGA or Baidu Netdisk (key: cdux). We also provide original averaged frame (without applying BM3D) in folder "indoor_raw_noisy", named like "frameXX_clean.tiff". The Bayer pattern of raw data is GBRG, the black level is 240, the white level is 2^12-1. You can apply your ISP to raw data to generate sRGB video denoising data.

• Ubuntu
• Python 3.5
• NVIDIA GPU + CUDA 9.0 + CuDNN 7
• Pytorch 1.0
• Tensorflow 1.5 gpu (only to synthesize raw data)
• Deformable Convolution

  cd ./modules/DCNv2
  bash make.sh
  

• Criss-Cross Attention

  cd ./modules/cc_attention
  python setup.py develop
  

• Download SID dataset. Select raw clean images to generate raw noisy images to train PreDenoising module, select raw clean images and corresponding sRGB clean images to train ISP module.
• Prepare synthesized raw video denoising dataset (SRVD dataset) to pretrain RViDeNet. Please download MOT Challenge dataset and select four videos (02, 09, 10, 11) from train set. To convert sRGB clean videos to raw clean videos, run:

  python sRGB_to_raw.py
  

• To generate raw noisy videos from raw clean videos, run:

  python synthesize_noise.py
  

• Download captured raw video denoising dataset (CRVD dataset) to finetune RViDeNet.

• Please download trained model from Google Drive or Baidu Netdisk (key: xssc).
• Test pretrained RViDeNet on indoor test set (scene 7, 8, 9, 10, 11) of CRVD dataset.

  python test_indoor.py --model pretrain --gpu_id 0  --output_dir ./results/pretrain/ --vis_data True
  

• Test finetuned RViDeNet on indoor test set (scene 7, 8, 9, 10, 11) of CRVD dataset.

  python test_indoor.py --model finetune --gpu_id 0  --output_dir ./results/finetune/ --vis_data True
  

• Train the ISP module. Please run 'convert_to_tiff.py' and 'rename_gt_png.py' to process SID Sony dataset, then run:

  python train_isp.py --gpu_id 0  --num_epochs 770 --patch_size 512
  

• Train the PreDenoising module.

  python train_predenoising.py --gpu_id 0  --num_epochs 700 --patch_size 128
  

• Pretrain RViDeNet on SRVD dataset.

  python train_pretrain.py --gpu_id 0  --num_epochs 33 --patch_size 128 --batch_size 1
  

• Finetune RViDeNet on CRVD dataset.

  python train_finetune.py --gpu_id 0  --num_epochs 70 --patch_size 128 --batch_size 1
  

If you find our dataset or code helpful in your research or work, please cite our paper:

@inproceedings{yue2020supervised,
  title={Supervised Raw Video Denoising with a Benchmark Dataset on Dynamic Scenes},
  author={Yue, Huanjing and Cao, Cong and Liao, Lei and Chu, Ronghe and Yang, Jingyu},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
  year={2020}
}

Our work and implementations are inspired by following projects:
[Unprocessing] (https://github.com/google-research/google-research/tree/master/unprocessing)
[EDVR] (https://github.com/xinntao/EDVR)
[SID] (https://github.com/cchen156/Learning-to-See-in-the-Dark)
[DANet] (https://github.com/junfu1115/DANet)
[CCNet] (https://github.com/speedinghzl/CCNet)