Skip to content
/ geval Public

Code for paper "G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment"

License

Notifications You must be signed in to change notification settings

nlpyang/geval

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Code for paper "G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment" [https://arxiv.org/abs/2303.16634]

Experiments on SummEval dataset

Evaluate fluency on SummEval dataset

python .\gpt4_eval.py --prompt .\prompts\summeval\flu_detailed.txt --save_fp .\results\gpt4_flu_detailed.json --summeval_fp .\data\summeval.json --key XXXXX

Meta Evaluate the G-Eval results

python .\meta_eval_summeval.py --input_fp .\results\gpt4_flu_detailed.json --dimension fluency

Prompts and Evaluation Results

Prompts used to evaluate SummEval are in prompts/summeval

G-eval results on SummEval are in results

About

Code for paper "G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment"

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages