Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Select metric choice for visualization #286

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Select metric choice for visualization #286

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
4be2767
add metrics arg in command
reaganjlee Aug 15, 2023
f5864d8
rename var
reaganjlee Aug 17, 2023
2553c6e
allow just one metric at a time
reaganjlee Aug 17, 2023
8ef905a
rename
reaganjlee Aug 17, 2023
54a9310
selectable metric_type support for layer viz
reaganjlee Aug 17, 2023
4deb51b
switch out rest of auroc with metric_type
reaganjlee Aug 17, 2023
0b6abf4
add functionality to readme
reaganjlee Aug 17, 2023
179066c
add functionality to sweep / cleanup
reaganjlee Aug 18, 2023
b0cf33d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 18, 2023
989f00f
line length
reaganjlee Aug 18, 2023
1294170
Merge branch 'select-metric-viz' of https://github.com/EleutherAI/elk…
reaganjlee Aug 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
rename var
  • Loading branch information
@reaganjlee
reaganjlee committed Aug 17, 2023
commit f5864d8b8dacc614c023a8a36dd7f1d9156d7cef
10 changes: 5 additions & 5 deletions elk/plotting/command.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ class Plot:
overwrite: bool = False
"""Whether to overwrite existing plots."""

metrics: list[str] = field(default_factory=list)
metric_types: list[str] = field(default_factory=list)
"""Name of metric to plot"""

def execute(self):
Expand All @@ -38,8 +38,8 @@ def execute(self):
else:
sweep_paths = [root_dir / sweep for sweep in self.sweeps]

if not self.metrics:
self.metrics = ["auroc_estimate"]
if not self.metric_types:
self.metric_types = ["auroc_estimate"]

for sweep_path in sweep_paths:
if not sweep_path.exists():
Expand All @@ -53,6 +53,6 @@ def execute(self):
if self.overwrite:
shutil.rmtree(sweep_path / "viz")

assert len(self.metrics) == 1, "Multiple metrics at a time aren't supported yet. Re-run plot for each metric separately."
assert len(self.metric_types) == 1, "Multiple metrics at a time aren't supported yet. Re-run plot for each metric separately."

visualize_sweep(sweep_path, self.metrics)
visualize_sweep(sweep_path, self.metric_types)