Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add custom_ds_n CLI arg #187

Closed
wants to merge 5 commits into from
Closed

add custom_ds_n CLI arg #187

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
03e463c
add custom_ds_n CLI arg
reaganjlee Apr 13, 2023
c7d6ba3
cleanup comments + typechecking
reaganjlee Apr 13, 2023
f26bac1
cleanup comments + typechecking
reaganjlee Apr 13, 2023
62cb109
Merge branch 'renaming' of https://github.com/EleutherAI/elk into ren…
reaganjlee Apr 13, 2023
b0f6d17
Change to Path types
reaganjlee Apr 13, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cleanup comments + typechecking
  • Loading branch information
@reaganjlee
reaganjlee committed Apr 13, 2023
commit c7d6ba3636b018d50b3d6b513596e3a07ba29905
2 changes: 1 addition & 1 deletion elk/evaluation/evaluate.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ class Eval(Serializable):
num_gpus: int = -1
skip_baseline: bool = False
concatenated_layer_offset: int = 0
custom_ds_n: str = None
custom_ds_n: Optional[str] = None

def execute(self):
datasets = self.data.prompts.datasets
Expand Down
3 changes: 1 addition & 2 deletions elk/run.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ class Run(ABC):
cfg: Union["Elicit", "Eval"]
out_dir: Optional[Path] = None
dataset: DatasetDict = field(init=False)
custom_ds_n: str = None
custom_ds_n: Optional[str] = None

def __post_init__(self):
# Extract the hidden states first if necessary
Expand All @@ -45,7 +45,6 @@ def __post_init__(self):
# Save in a memorably-named directory inside of
# ELK_REPORTER_DIR/<model_name>/<dataset_name>
ds_name = ", ".join(self.cfg.data.prompts.datasets)
print(f"self.custom_root_name: {self.custom_ds_n}")
if self.custom_ds_n:
ds_name = self.custom_ds_n
root = elk_reporter_dir() / self.cfg.data.model / ds_name
Expand Down