Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[catmem] huge page support #966

Open
kyleholohan opened this issue Oct 10, 2023 · 0 comments
Open

[catmem] huge page support #966

kyleholohan opened this issue Oct 10, 2023 · 0 comments
Labels
feature New Feature Request

Comments

@kyleholohan
Copy link
Contributor

kyleholohan commented Oct 10, 2023

Context

Huge pages offer benefits for some high resource use applications. Applications which transmit large amounts of data via shared memory may benefit from using huge pages for IPC. Per the Linux kernel documentation on huge pages, huge pages exist as files mounted under hugetlbfs filesystem. shm_open creates files on a tempfs usually under /dev/shm.

Windows also supports huge pages (called large pages); use of MapViewOfFile3 allows use of the MEM_LARGE_PAGES flag, as well as optional NUMA zone placement through ExtendedParameters.

Proposed Solution

Move the PAL away from usage of shm_create so that files can be put either on tempfs or hugetlbfs. Create a new abstraction for resolving shared memory file names on the appropriate device. Add one implementation for tempfs and one for hugetlbfs.

For Windows in conjunction with #965, add the ability to use MEM_LARGE_PAGES when mapping shared memory.

Alternative Solutions

Adding this feature is a pretty binary choice. Huge pages will likely only offer benefits for a small subset of the application space. This should be driven accordingly.

For implementation, the current PAL implementation could instead be preserved and a hugepage PAL could be added. This would result in some redundant code, but more stability for existing cases.

Edit

I missed a subtlety here: shm_open is the POSIX-compliant interface to shared memory, and huge pages don't seem to exist in the POSIX standard. In this case, preserving the existing PAL may serve value for POSIX-compliant non-Linux targets.

@kyleholohan kyleholohan added the feature New Feature Request label Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New Feature Request
Projects
None yet
Development

No branches or pull requests

2 participants