Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prevent invalidation of the hidden state cache when num_gpus changes #182

Merged
merged 2 commits into from
Apr 12, 2023
Merged

Prevent invalidation of the hidden state cache when num_gpus changes #182

merged 2 commits into from
Apr 12, 2023

Conversation

norabelrose
Copy link
Member

@norabelrose norabelrose commented Apr 12, 2023 •
edited
Loading

The fix here is moderately hacky; I had to override create_config_id in _GeneratorConfig to manually remove any trace of the number of GPUs from the generator kwargs before they get hashed.

Fixes #102.

AlexTMallen
AlexTMallen approved these changes Apr 12, 2023
View reviewed changes
Copy link
Collaborator

@AlexTMallen AlexTMallen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM--this works on the UW machine as well

@norabelrose norabelrose merged commit 361fb9b into main Apr 12, 2023
@norabelrose norabelrose deleted the better-caching branch April 12, 2023 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@AlexTMallen AlexTMallen AlexTMallen approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Hidden state cache shouldn't be invalidated when we're using different devices
2 participants
@norabelrose @AlexTMallen