Skip to content
/ PSSR Public

Official code for the paper: "Perception and Semantic Aware Regularization for Sequential Confidence Calibration (CVPR2023)"

License

Notifications You must be signed in to change notification settings

husterpzh/PSSR

Repository files navigation

Perception and Semantic Aware Regularization for Sequential Confidence Calibration

This is the official PyTorch implementation for the paper: "Perception and Semantic Aware Regularization for Sequential Confidence Calibration". For more details, please refer to our paper.

Runtime Environment

  • This work was tested with PyTorch 1.9.0, CUDA 11.1, python 3.8
  • Requirements : fastai==1.0.60 lmdb pillow opencv-python tensorboardX editdistance nltk natsort numpy

Datasets

We use lmdb dataset for training and evaluation. The training datasets can be downloaded in clovaai and the ensemble of evalutaion datasets can be downloaded here.

Calibration

1. Get Correlated Sequence

Perception Similarity Sequence

bash get_perc_sim_seq.sh

Semantic Correlation Sequence

CUDA_VISIBLE_DEVICES=0 python abinet/get_sem_cor_seq.py --checkpoint /path/to/checkpoint
  • --checkpoint: path to pretrained model of BCN in ABINet.

2. Calibrate Model With PSSR

CUDA_VISIBLE_DEVICES=0 python train.py \
--train_data data_lmdb_release/training --valid_data data_lmdb_release/test \
--select_data MJ-ST --batch_ratio 0.5-0.5 --lr 0.1 \
--Transformation TPS --FeatureExtraction ResNet --SequenceModeling BiLSTM --Prediction Attn \
--saved_model /path/to/checkpoint --calibrator PSSR --alpha 0.05 --with_vis
  • --saved_model: path to model to be calibrated.
  • --calibrator: select calibration method.
  • --alpha: weight factor for balancing in the Eq. (2) in our paper.
    Note: Detailed argument settings can be found in clovaai.

Evaluation

CUDA_VISIBLE_DEVICES=0 python test.py \
--eval_data data_lmdb_release/test \
--Transformation TPS --FeatureExtraction ResNet --SequenceModeling BiLSTM --Prediction Attn \
--saved_model pretrained_models/TPS-ResNet-BiLSTM-Attn.pth
  • --saved_model: path to calibrated model.

Calibrated Models Download

Download calibrated model by PSSR from this link.

Acknowledgements

This implementation has been based on clovaai and ABINet.

Citation

If you find our work useful in your research, please cite the following:

@inproceedings{peng2023perception,
  title={Perception and Semantic Aware Regularization for Sequential Confidence Calibration},
  author={Peng, Zhenghua and Luo, Yu and Chen, Tianshui and Xu, Keke and Huang, Shuangping},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={10658--10668},
  year={2023}
}

About

Official code for the paper: "Perception and Semantic Aware Regularization for Sequential Confidence Calibration (CVPR2023)"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published