Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

memoize_on_disk can produce filenames that are too long #803

Open
wence- opened this issue Sep 5, 2023 · 1 comment
Open

memoize_on_disk can produce filenames that are too long #803

wence- opened this issue Sep 5, 2023 · 1 comment

Comments

@wence-
Copy link
Contributor

wence- commented Sep 5, 2023

Since #710 (perhaps), memoize_on_disk produces quite long filenames, for example:

.../.cache/pytools/pdict-v4-loopy-memoize-cache-buffer_array-LoopyKeyBuilder.LoopyKeyBuilder-v0-2022.1-islpy2023.1.2-cgen2020.1-8158afdb18f100fb21b6cd86c6178e3c5cd19058-v1-py3.10.12.final.0

Unfortunately, some filesystems have quite short name restrictions. e.g. we had a bug report where this failed because the cache was on an encrypted filesystem where the max filename length is 144 characters (rather than the usual ext4 limit of 255).

Would it make sense to hash the (long) prefix as well as the contents?

@inducer
Copy link
Owner

inducer commented Sep 5, 2023

I'd be open to moving a bunch of that stuff from the directory name to the cache key. Perhaps it would make sense to retain pdict-v4-loopy-buffer_array-3.10? Want to cook up a PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants