Quantile Regression DQN and C51 DQN I use MSE for qr-dqn. The offline training one is almost like rainbow. Segment tree and replay memory are modified version and the original code is from tusimple To do I'm trying to use quantile huber loss in the future.