-
Notifications
You must be signed in to change notification settings - Fork 54
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
train question #59
Comments
Hi,I encounter the same dilemma now, could you share your method for this problem with me? Many thanks for you! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
When I run train.py in kitti6 dataset, I always encounter the following bugs, and I can't find the reason. Can you help me take a look?
ds_train: 3000
ds_val: 82
LOAD NET CHECKPOINT /home/wenbo/code_cy/READ-main/data/logs_offical/path/UNet_stage_0_epoch_net.pth
EPOCH 0
> TRAIN
EVAL MODE IN TRAIN
model parameters: 30193988
running on datasets [0]
model3d {'rgb': array([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.],
...,
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]], dtype=float32), 'normals': None, 'uv2d': array([[0., 0.],
[0., 0.],
[0., 0.],
...,
[0., 0.],
[0., 0.],
[0., 0.]], dtype=float32), 'faces': array([0, 1, 2], dtype=uint32), 'xyz': TrackedArray([[ -21.45231247, 13.24265003, -207.19848633],
[ -21.45231247, 13.24265003, -207.19848633],
[ -21.84580994, 14.13696384, -209.91720581],
...,
[ -30.93203926, 15.21014023, 684.93322754],
[ -30.93203926, 15.21014023, 684.93322754],
[ -30.93203926, 15.21014023, 684.93322754]]), 'xyz_c': TrackedArray([[0.41255814, 0.4299459 , 0.02919084],
[0.41255814, 0.4299459 , 0.02919084],
[0.41 , 0.44497895, 0.02623283],
...,
[0.3509302 , 0.46301866, 0.9998443 ],
[0.3509302 , 0.46301866, 0.9998443 ],
[0.3509302 , 0.46301866, 0.9998443 ]], dtype=float32), 'uv1d': array([ 0, 1, 2, ..., 2547319, 2547320, 2547321])}
proj_matrix was not set
total parameters: 50572564[w] b'GLX: No GLXFBConfigs returned'
[w] b'GLX: Failed to find a suitable GLXFBConfig'
[x] Window creation failed
[w] b'GLX: No GLXFBConfigs returned'
[w] b'GLX: Failed to find a suitable GLXFBConfig'
[x] Window creation failed
[w] b'GLX: No GLXFBConfigs returned'
[w] b'GLX: Failed to find a suitable GLXFBConfig'
[x] Window creation failed
Traceback (most recent call last):
File "/home/wenbo/anaconda3/envs/READ/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 990, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "/home/wenbo/anaconda3/envs/READ/lib/python3.9/multiprocessing/queues.py", line 114, in get
raise Empty
_queue.Empty
The above exception was the direct cause of the following exception:Traceback (most recent call last):
File "/home/wenbo/code_cy/READ-main/train.py", line 546, in <module>
train_loss = run_train(epoch, pipeline, args, iter_cb)
File "/home/wenbo/code_cy/READ-main/train.py", line 281, in run_train
return run_epoch(pipeline, 'train', epoch, args, iter_cb=iter_cb)
File "/home/wenbo/code_cy/READ-main/train.py", line 256, in run_epoch
run_sub(dl, extra_optimizer)
File "/home/wenbo/code_cy/READ-main/train.py", line 146, in run_sub
for it, data in enumerate(dl):
File "/home/wenbo/anaconda3/envs/READ/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/home/wenbo/anaconda3/envs/READ/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1186, in _next_data
idx, data = self._get_data()
File "/home/wenbo/anaconda3/envs/READ/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1152, in _get_data
success, data = self._try_get_data()
File "/home/wenbo/anaconda3/envs/READ/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1003, in _try_get_data
raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
RuntimeError: DataLoader worker (pid(s) 8638, 8658, 8678) exited unexpectedly
deleting buffers...
Process finished with exit code 1 @JOP-Lee Looking forward to your reply!
The text was updated successfully, but these errors were encountered: