Skip to content

Commit

Permalink
fixing typo
Browse files Browse the repository at this point in the history
  • Loading branch information
mmazeika committed Feb 7, 2024
1 parent 8274140 commit 7193689
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/codebase_structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ The HarmBench code is structured around five key elements:
5. The `data` folder

## 1: Evaluation pipeline scripts
The evaluation pipeline consists of four scripts: `generate_test_cases.py`, `merge_test_cases.py`, `generate_completions.py`, `evaluate_completions.py`. We describe how to use these below in the Quick Start section. At a high level, the first two scripts output a `test_cases.json` dictionary for a specific **experiment**. This dictionary contains a list of test cases for each behavior. The second two scripts run the test cases on a **target model** to compute attack success rate (ASR).
The evaluation pipeline consists of four scripts: `generate_test_cases.py`, `merge_test_cases.py`, `generate_completions.py`, `evaluate_completions.py`. We describe how to use these in the Quick Start section. At a high level, the first two scripts output a `test_cases.json` dictionary for a specific **experiment**. This dictionary contains a list of test cases for each behavior. The second two scripts run the test cases on a **target model** to compute attack success rate (ASR).

## 2: Experiments and target models
Each `test_cases.json` file generated by a red teaming method corresponds to a single **experiment**. Each red teaming method can run different experiments, corresponding to test cases optimized for different target models, or more generally test cases generated with different hyperparameters. By convention we save test cases in `{top_level_results_dir}/{method_name}/{experiment_name}/test_cases/test_cases.json`, so some example save locations could be
Expand Down

0 comments on commit 7193689

Please sign in to comment.