Add Unified Memory #3494

matyas-streamhpc · 2024-05-22T16:47:20Z

No description provided.

neon60 · 2024-05-23T09:43:22Z

Send some comments via direct message. The short summary:

Add the list of missing CUDA features
Add system requirements for Unified Memory
Add Performance improvements tips
Allocation APIs for System-Allocated Memory? Is this supported?
Tutorial is not needed, example is enough.

neon60 · 2024-05-28T13:20:37Z

@matyas-streamhpc Please rebase your branch on docs/develop

docs/reference/unified_memory.rst

docs/how-to/unified_memory.rst

neon60 · 2024-06-10T19:43:18Z

@matyas-streamhpc rebased the branch.

Rmalavally · 2024-06-13T16:03:15Z

docs/how-to/unified_memory.rst

+high-performance low latency operations, while the GPU is optimized for
+high-throughput (data processed by unit time).
+
+How-to use?


Recommended

Using Unified Memory Management (UMM)

Rmalavally · 2024-06-13T16:04:51Z

docs/how-to/unified_memory.rst

+and a GPU which would require large memory transfers. Here are some areas where
+UMM can be beneficial:
+
+- **Simplification of Memory Management**:


Recommended

Simplification of memory management

UMM can help to simplify the complexities of memory management. This can make it easier for developers to write code without worrying about memory allocation and deallocation details.

Rmalavally · 2024-06-13T16:06:55Z

docs/how-to/unified_memory.rst

+As a positive side effect, the use of UMM can reduce the lines of code,
+thereby improving programming productivity.
+
+In HIP, pinned memory allocations are coherent by default. Pinned memory is


Recommended

In HIP, pinned memory allocations are coherent by default. Pinned memory is host memory mapped into the address space of all GPUs, meaning that the pointer can be used on both host and device. Using pinned memory instead of pageable memory on the host can improve bandwidth.

Rmalavally · 2024-06-13T16:09:47Z

docs/how-to/unified_memory.rst

+pointer can be used on both host and device. Using pinned memory instead of
+pageable memory on the host can lead an improvement in bandwidth.
+
+While UMM can provide numerous benefits, it is also important


Recommended

While UMM can provide numerous benefits, it is important to be aware of the potential performance overhead associated with UMM. You must thoroughly test and profile your code to ensure it is the most suitable choice for your use case.

Rmalavally · 2024-06-13T16:11:54Z

docs/how-to/unified_memory.rst

+
+The following example shows how to use unified memory management with
+``hipMallocManaged()``, function, with ``__managed__`` attribute for static
+allocation and standard ``malloc()`` allocation. The Explicit Memory


Where is EMM presented for comparison? Do you mean in the figure? Consider adding a link to the section, so users don't have to scroll back and forth.

Sorry, I meant in the last tab of the tabset. I updated this section.

Rmalavally · 2024-06-13T16:14:57Z

docs/how-to/unified_memory.rst

+ }
+
+
+Compiler Hints for the Better Performance


Could you help establish context with UMM? Do we mean C++ compiler hints?

These are UMM compiler hints. I updated the header.

Rmalavally · 2024-06-13T16:19:06Z

@matyas-streamhpc @yhuiYH added some comments from a user standpoint. Please feel free to use or ignore them. Let me know if they are not technically applicable or relevant.

matyas-streamhpc · 2024-06-14T12:22:09Z

@matyas-streamhpc @yhuiYH added some comments from a user standpoint. Please feel free to use or ignore them. Let me know if they are not technically applicable or relevant.

@Rmalavally @yhuiYH Thank you very much for your feedback. I updated the page, accordingly.

neon60 · 2024-06-14T14:42:56Z

docs/how-to/unified_memory.rst

+
+❌: **Unsupported**
+
+:sup:`1` Works only with ``XNACK=1``. First GPU access causes recoverable


Add a link to XNACK documentation:
https://rocm.docs.amd.com/en/latest/conceptual/gpu-memory.html#xnack

Nice catch!

neon60 · 2024-06-14T14:48:46Z

Please fix the following rst issues:

HIP\docs\reference\unified_memory_reference.rst:12: WARNING: Duplicate C++ declaration, also defined at reference/unified_memory_reference:10.
Declaration is '.. cpp:function:: hipError_t hipMallocManaged (void **dev_ptr, size_t size, unsigned int flags)'.
HIP\docs\reference/unified_memory_reference.rst:8: CRITICAL: Duplicate ID: "group___memory_m_1gaadf4780d920bb6f5cc755880740ef7dc".
HIP\docs\reference/unified_memory_reference.rst:8: WARNING: Duplicate explicit target name: "group___memory_m_1gaadf4780d920bb6f5cc755880740ef7dc".
HIP\docs\reference/unified_memory_reference.rst:15: WARNING: doxygenfunction: Cannot find function "hipMemoryAdvise" in doxygen xml output for project "HIP 6.2.0 Documentation" from directory: HIP\docs\doxygen\xml
HIP\docs\reference/unified_memory_reference.rst:17: WARNING: doxygenfunction: Cannot find function "hipMemRangeAttribute" in doxygen xml output for project "HIP 6.2.0 Documentation" from directory: HIP\docs\doxygen\xm

Rmalavally · 2024-06-14T15:25:35Z

docs/how-to/unified_memory.rst

+Concept
+=======
+
+In a conventional architectures, both CPUs a GPUs have a dedicated memory,


Thanks, Matyas. Appreciate the prompt fixes.

Some more minor fixes.

Thank you! I applied the recommended fixes.

Rmalavally · 2024-06-14T15:26:32Z

docs/how-to/unified_memory.rst

+Concept
+=======
+
+In a conventional architectures, both CPUs a GPUs have a dedicated memory,


Remove 'a' in

In a conventional architectures

It has been removed in the last commit, but Github UI does not show it, because there was a comment on it.
You can see the latest version here:
https://github.com/ROCm/HIP/pull/3494/files#diff-9ffae9738c016086230e9386a7d0aa1b26bd7a8f4845494ba9952cf0de47c1db

docs/how-to/unified_memory.rst

matyas-streamhpc requested review from neon60 and lpaoletti May 22, 2024 16:47

matyas-streamhpc self-assigned this May 22, 2024

matyas-streamhpc requested review from gandryey, mangupta and rakesroy as code owners May 22, 2024 16:47

matyas-streamhpc force-pushed the unified-memory branch 2 times, most recently from 94f2948 to a610e91 Compare May 22, 2024 19:36

neon60 changed the base branch from develop to docs/develop May 23, 2024 08:49

neon60 force-pushed the docs/develop branch from ecf0b12 to 2a08a20 Compare May 24, 2024 11:22

matyas-streamhpc force-pushed the unified-memory branch 11 times, most recently from 05a8d2a to 4a9a4ac Compare May 31, 2024 21:43

neon60 reviewed Jun 3, 2024

View reviewed changes

neon60 reviewed Jun 5, 2024

View reviewed changes

docs/how-to/unified_memory.rst Outdated Show resolved Hide resolved

matyas-streamhpc force-pushed the unified-memory branch 2 times, most recently from f230126 to 1f7967f Compare June 5, 2024 12:32

neon60 force-pushed the docs/develop branch from 79ff964 to 097af3a Compare June 10, 2024 17:22

neon60 force-pushed the unified-memory branch from 1f7967f to 70673e4 Compare June 10, 2024 19:42

Rmalavally reviewed Jun 13, 2024

View reviewed changes

Rmalavally closed this Jun 13, 2024

Rmalavally reopened this Jun 13, 2024

matyas-streamhpc force-pushed the unified-memory branch from bdd527d to dd845c5 Compare June 14, 2024 12:30

neon60 reviewed Jun 14, 2024

View reviewed changes

Rmalavally reviewed Jun 14, 2024

View reviewed changes

neon60 reviewed Jun 16, 2024

View reviewed changes

docs/how-to/unified_memory.rst Outdated Show resolved Hide resolved

neon60 reviewed Jun 16, 2024

View reviewed changes

docs/how-to/unified_memory.rst Outdated Show resolved Hide resolved

neon60 reviewed Jun 16, 2024

View reviewed changes

docs/how-to/unified_memory.rst Outdated Show resolved Hide resolved

neon60 reviewed Jun 16, 2024

View reviewed changes

docs/how-to/unified_memory.rst Outdated Show resolved Hide resolved

neon60 reviewed Jun 16, 2024

View reviewed changes

docs/how-to/unified_memory.rst Outdated Show resolved Hide resolved

neon60 reviewed Jun 16, 2024

View reviewed changes

docs/how-to/unified_memory.rst Outdated Show resolved Hide resolved

neon60 reviewed Jun 16, 2024

View reviewed changes

docs/how-to/unified_memory.rst Outdated Show resolved Hide resolved

neon60 force-pushed the docs/develop branch from a553751 to 8f70ab4 Compare June 20, 2024 16:31

matyas-streamhpc and others added 6 commits June 20, 2024 18:46

Add Unified Memory

89fc593

Update based on review comments

16b2702

Fixes based on review comments

76121f0

Fix memory reference warning

ff9f57d

Update tone

a117e8a

Fix list styling.

b6c2282

neon60 force-pushed the unified-memory branch from 329410b to b6c2282 Compare June 20, 2024 16:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Unified Memory #3494

Add Unified Memory #3494

matyas-streamhpc commented May 22, 2024

neon60 commented May 23, 2024 •

edited

neon60 commented May 28, 2024

neon60 commented Jun 10, 2024

Rmalavally Jun 13, 2024

matyas-streamhpc Jun 14, 2024

Rmalavally Jun 13, 2024

matyas-streamhpc Jun 14, 2024

Rmalavally Jun 13, 2024

matyas-streamhpc Jun 14, 2024

Rmalavally Jun 13, 2024

matyas-streamhpc Jun 14, 2024

Rmalavally Jun 13, 2024

matyas-streamhpc Jun 14, 2024

Rmalavally Jun 13, 2024

matyas-streamhpc Jun 14, 2024

Rmalavally commented Jun 13, 2024

matyas-streamhpc commented Jun 14, 2024

neon60 Jun 14, 2024

matyas-streamhpc Jun 17, 2024

neon60 commented Jun 14, 2024

Rmalavally Jun 14, 2024

matyas-streamhpc Jun 17, 2024

Rmalavally Jun 14, 2024

matyas-streamhpc Jun 17, 2024


		❌: Unsupported

		:sup:`1` Works only with ``XNACK=1``. First GPU access causes recoverable

Add Unified Memory #3494

Are you sure you want to change the base?

Add Unified Memory #3494

Conversation

matyas-streamhpc commented May 22, 2024

neon60 commented May 23, 2024 • edited

neon60 commented May 28, 2024

neon60 commented Jun 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Rmalavally commented Jun 13, 2024

matyas-streamhpc commented Jun 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neon60 commented Jun 14, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

neon60 commented May 23, 2024 •

edited