Code Setup and Training

1. Requirements

This codebase was created for and tested on Python 3.8. To install the requirements, please create a virtual environment and install the packages:

python -m venv env

source env/bin/activate

pip install -r requirements.txt

2. Code Structure

The project is organized into the below folders:

• evaluation_code to evaluate the model after adaptation on the Human3.6M test set.

• image_to_latent_encoder, motion_autoencoder, pose_autoencoder, relation_network contain codes for pre-training the respective models required for target adaptation.

• pretrained_weights contains the trained weights for all of the above models.

• target_adaptation code for the adaptation procedure described in the paper.

3. Training Procedure

The adaptation requires pre-trained pose autoencoder, motion autoencoder, relation networks (Flip+InPlane-50 for pose space transformation, Flip+InPlane-50-backward, and slow-backward for motion space transformation), and a source-trained image-to-latent model. The codebase includes pretrained weights for these networks. To re-train any of the models for use in adaptation, please run the corresponding train.py script from the train directory. For example, to train the pose autoencoder, run the below commands:

cd pose_autoencoder

python train.py

For more details, please refer to the README.md file in the codebase.

The adaptation procedure can be applied to a target dataset by running the train.py file in the target adaptation directory. The code includes a sample of H3.6M for running the adaptation.

cd target_adaptation

python train.py

4. Evaluation

The following command can be used to run PA-MPJPE evaluation of the trained model on the H3.6M test set. Evaluation procedure for other target settings will be published soon.

cd evaluation

python eval.py

Citation

If you find our work helpful in your research, please cite our work:

@inproceedings{kundu2021non,

title={Non-Local Latent Relation Distillation for Self-Adaptive 3D Human Pose Estimation},

author={Kundu, Jogendra Nath and Seth, Siddharth and Jamkhandi, Anirudh and YM, Pradyumna and Jampani, Varun and Chakraborty, Anirban and Babu, R Venkatesh},

booktitle={Advances in Neural Information Processing Systems},

year={2021}

}

License

This project is licensed under an [MIT License].

Contact

If you have any queries, please get in touch via email : jogendranathkundu@gmail.com.