Released code for Densely Constrained Depth Estimator for Monocular 3D Object Detection (ECCV22). arxiv Yingyan Li, Yuntao Chen, Jiawei He, Zhaoxiang Zhang

This repo is tested with Ubuntu 16.04, python==3.8, pytorch==1.7.0 and cuda==10.1.

conda create -n dcd python=3.8
conda activate dcd
conda install pytorch==1.7.0 torchvision==0.8.0 cudatoolkit=10.1 -c pytorch
pip install -r requirements.txt

You also need ot build DCNv2 and this project as:

cd DGDE/models/backbone/DCNv2
python setup.py develop
cd ../../..
python setup.py develop

We need KITTI dataset and keypoints annotation Google Drive.

After download them, please organize as:

|DGDE
  |dataset
    |kitti
      |training/
        |calib/
        |image_2/
        |label/
        |ImageSets/
      |testing/
        |calib/
        |image_2/
        |ImageSets/
  |kpts_ann
    |kpts_ann_train.json
    |kpts_ann_val.json

The whole pipeline including 3 parts: a) training DGDE first. b) using DGDE to generate needed data for GMW. c) training GMW and evaluate.

a) training DGDE Training with 2 GPUs.

cd DGDE
CUDA_VISIBLE_DEVICES=0,1 \
python tools/plain_train_net.py --batch_size 8 --config runs/DGDE.yaml \
--output output/DGDE --num_gpus 2 \

b) using DGDE to generate needed data for GMW. Finishing training for DGDE, please generate data on 1 GPU as:

cd DGDE
CUDA_VISIBLE_DEVICES=0 \
python tools/plain_train_net.py --batch_size 8 --config runs/DGDE.yaml \
--output output/DGDE --num_gpus 1 \
--generate_for_GMW \
--ckpt output/DGDE/model_final.pth

after this step, you could see gen_data_train.json and gen_data_infer.json in DGDE/gen_data/

c) training GMW and evaluate.

cd GMW
python -m torch.distributed.launch --master_port 33521 --nproc_per_node=4 \
main.py --log-dir ./logs/GMW \
-b 8 --lr 1e-4 --epoch 100 --val_freq 5 \
--train_data_path ../DGDE/gen_data/gen_data_train.json \
--val_data_path ../DGDE/gen_data/gen_data_infer.json

It will be evaluated periodically. You can also run the following command for evaluation:

python -m torch.distributed.launch --master_port 24281 --nproc_per_node=4 \
main.py --log-dir ./logs/GMW/
-b 36 -e \
--resume logs/GMW/checkpoint_epoch_100.pth.tar

You can also use the pre-trained weights of DGDE,WGM (Google Drive).

The code is mainly based on MonoFlex and BPnP. Thanks for their great work.

If this work is helpful for your research, please consider citing it:

@article{li2022densely,
  title={Densely Constrained Depth Estimator for Monocular 3D Object Detection},
  author={Li, Yingyan and Chen, Yuntao and He, Jiawei and Zhang, Zhaoxiang},
  journal={arXiv e-prints},
  pages={arXiv--2207},
  year={2022}
}