Skip to content

Franck-Dernoncourt/Fair-LLM-Benchmark

 
 

Repository files navigation

Bias and Fairness in Large Language Models: A Survey

Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed

To enable easy use of bias evaluation datasets, we compile publicly-available ones and provide access here. We provide links to the original data sources below. We do not modify any of the datasets, but do remove unrelated material from the original repositories. Please refer to the original works for more detailed documentation.

Dataset Link
BBQ https://github.com/nyu-mll/BBQ
BEC-Pro https://github.com/marionbartl/gender-bias-BERT
Bias NLI https://github.com/sunipa/On-Measuring-and-Mitigating-Biased-Inferences-of-Word-Embeddings
BOLD https://github.com/amazon-science/bold
BUG https://github.com/SLAB-NLP/BUG
CrowS-Pairs https://github.com/nyu-mll/crows-pairs/
Equity Evaluation Corpus https://saifmohammad.com/WebPages/Biases-SA.html
GAP https://github.com/google-research-datasets/gap-coreference
Grep-BiasIR https://github.com/KlaraKrieg/GrepBiasIR
HolisticBias https://github.com/facebookresearch/ResponsibleNLP
HONEST https://github.com/MilaNLProc/honest
PANDA https://github.com/facebookresearch/ResponsibleNLP
RealToxicityPrompts https://toxicdegeneration.allenai.org
RedditBias https://github.com/umanlp/RedditBias
StereoSet https://github.com/McGill-NLP/bias-bench, https://github.com/moinnadeem/stereoset
TrustGPT https://github.com/HowieHwong/TrustGPT
UnQover https://github.com/allenai/unqover
WinoBias https://github.com/uclanlp/corefBias
WinoBias+ https://github.com/vnmssnhv/NeuTralRewriter
WinoGender https://github.com/rudinger/winogender-schemas
WinoQueer https://github.com/katyfelkner/winoqueer

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 93.8%
  • Shell 6.2%