-
Notifications
You must be signed in to change notification settings - Fork 96
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
amdgpu and drm drivers version #150
Comments
@kentrussell, Sir? |
If you're using the ROCm install, you can check "dkms status" to return the version of the kernel installation package. If you're using the stock Ubuntu kernel, then that will depend on the version of Ubuntu that you're using. Conversely, "dpkg -l|grep amdgpu-dkms" will also show the version of the kernel+firmware. Lastly, what problems did you have with building from dkms? I've currently got the ROCm kernel installed on UB22.04.1 (6.2 kernel) via dkms without any issue |
Thanks. What about dkms - it wouldn't build if current kernel is ROCK kernel. What about hipMemcpy2DAsync - it seems, that there are some difference between cuda and hip behaviour when processing p2p memory. For example, when active GPU is 1, attempt to copy/memset memory allocated on GPU 0 would crash. At least for gfx900. Moreover it is quite unclear how to enable p2p, and if it ever possible. Currently the code which is working (well, at least code which doesn't triggering SIGSEGV) for cuda users - trigger it on gfx900. If this subject going away from amdgpu - feel fre to close an issue. |
amdgpu-dkms would have all of the P2P features in the kernel enabled, as would the monolithic kernel built from this repo. The stock Ubuntu OS kernel wouldn't have IPC or RDMA support, but for your situation, straight P2P (1 host, 2 GPUs), P2P should be working in the kernel without issue. |
The only what I see in dmesg is WARN_ONCE() which reasonable warn me about large BAR system.
I open issue in HIP with PoC and references. |
Okey, Sir, dmesg logs: descr: attempt to parallel task execution on 2 GPUs, using hiptorch (latest)
|
following this tutorial: https://pytorch.org/tutorials/beginner/blitz/data_parallel_tutorial.html dmesg:
source code of test3.py import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
# Parameters and DataLoaders
input_size = 5
output_size = 2
batch_size = 30
data_size = 100
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
class RandomDataset(Dataset):
def __init__(self, size, length):
self.len = length
self.data = torch.randn(length, size)
def __getitem__(self, index):
return self.data[index]
def __len__(self):
return self.len
rand_loader = DataLoader(dataset=RandomDataset(input_size, data_size),
batch_size=batch_size, shuffle=True)
class Model(nn.Module):
# Our model
def __init__(self, input_size, output_size):
super(Model, self).__init__()
self.fc = nn.Linear(input_size, output_size)
def forward(self, input):
output = self.fc(input)
print("\tIn Model: input size", input.size(),
"output size", output.size())
return output
model = Model(input_size, output_size)
if torch.cuda.device_count() > 1:
print("Let's use", torch.cuda.device_count(), "GPUs!")
# dim = 0 [30, xxx] -> [10, ...], [10, ...], [10, ...] on 3 GPUs
model = nn.DataParallel(model)
model.to(device)
for data in rand_loader:
input = data.to(device)
output = model(input)
print("Outside: input size", input.size(),
"output_size", output.size())
GPUs load during execution:
|
I think it's worth checking with the HIP guys. The fact that you have an address of 0x0 means that there's a bad address being passed in from somewhere. We can work our way down the stack to find it, but I think the HIP guys will have a good initial assessment on why they're using a NULL address: |
Ummm, Sir, I'm really sorry for disturbing you, but I don't know who to ping about this question from HIP team devs, And, maybe it is possible to implement smthn like quick and dirty work-around to this stack of ring-3 bugs, and perform manual r/w op of vram? If there is a way to do so? Kindly appreciated, Sir. @kentrussell |
Hopefully they'll see it and reply. I know that the HIP bug reports tend to take a bit longer to get to, and it's only been 1 workday since you opened the issue. |
Hello.
Thanks for your work.
May I ask, how to enshure, that Im running latest GPU driver?
Question may be strange a bit, but in kmsg I see, that amdgpu dated as 2016.
The kernel from this repo, was olddefconfigured and compiled without keysigns.
Modules were built and installed.
Initramfs was packed, grub was updated, and system (which is remote server) was successfully booted up.
uname -a say, that it running on custom 6.2.8.
I using radeon v340 pro. Twin gpu, 16gb of vram per core. gfx900, vesa 10.
And my ultimate goal to allow cross-device vram access.
At the moment, I got crash somewhere in libhipamd64.so, after entering hipMemcpy2DAsync from userspace.
I spent ~20 hours in attempts to build hipamd, which was moved several times, and now been part of crt. But lack of fresh documentation and my expiriense with HIP in genaral stuck me.
Can you, please, hint me what should I do to:
Btw, as for attempts to build from dkms - I give a try to build on fresh ubuntu installation, on debian, on this kernel too -- no one was build successfully from dkms.
Best regards,
The text was updated successfully, but these errors were encountered: