This repo contains the codes for our work 🐼PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs.

The required package can be installed by running the following command.

pip install -r requirements.txt

Firstly, switch to the ScienceWorld workspace:

cd ScienceWorld

0. To directly run experiments with PANDA (You can also run PANDA from scratch by starting from step1).

./run_eval_react_panda.sh
./run_eval_reflexion_panda.sh
./run_eval_saycan_panda.sh

1. Step1: Gather expert trials to construct preferences data.

./gather_trials.sh

2. Step2: PANDA-Learning from the expert preferences.

./panda_learning.sh

3. Step3: Test with PANDA-Insight:

./run_eval_react_panda.sh
./run_eval_reflexion_panda.sh
./run_eval_saycan_panda.sh

Firstly, switch to the ScienceWorld workspace:

cd TweetEval

0. Step0: Download datasets file from cardifnlp/tweeteval and put it in the dataset folder and the expert models from cardifnlp/models and put it in the models folder.

1. Step1: Gather expert trials to construct preferences data.

./gather_trials.sh

2. Step2: PANDA-Learning from the expert preferences.

./panda_learning.sh

3. Step3: Test with PANDA-Insight:

./eval_gpt.sh
./eval_gpt_cot.sh

Our codes for scienceworld are adapted from yuchenlin/SwiftSage. Thanks for their kind open-sourced code.

If you find our project helpful to your research, please consider citing:

@inproceedings{liu2024panda,
  title={PANDA: Preference Adaptation for Enhancing Domain-Specific Abilities of LLMs},
  author={Liu, An and Yang, Zonghan and Zhang, Zhenhe and Hu, Qingyuan and Li, Peng and Yan, Ming and Zhang, Ji and Huang, Fei and Liu, Yang},
  booktitle={Findings of the Association for Computational Linguistics: ACL 2024},
  year={2024}
}