Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPU Usage 100% at dataloading while not proceeding #139

Open
TintinWuNTUEE opened this issue Jun 6, 2024 · 2 comments
Open

GPU Usage 100% at dataloading while not proceeding #139

TintinWuNTUEE opened this issue Jun 6, 2024 · 2 comments

Comments

@TintinWuNTUEE
Copy link

TintinWuNTUEE commented Jun 6, 2024

Hi, I was trying to run the model via Colab provided at the README, however I encountered two issues,

  1. I was training on the bouncingball scene from dnerf and the GPU usage spiked to 100% while dataloading and the training somehow stucks there...
  2. I proceeded on training on the hellwarrior dataset and getting a loss is nan while training
    (I am using a RTX 4090 with CUDA 11.6 and pytorch 1.13)
    Wondering if there is something I am doing incorrectly, thank you!
@guanjunwu
Copy link
Collaborator

guanjunwu commented Jun 9, 2024

e, most cases happened when your CUDA version not corresponds with your pytorch version.
Is your pytorch version = torch1.13+cu117?

@TintinWuNTUEE
Copy link
Author

I am using torch1.13+cu116, does that make a difference?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants