Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test-conv1d and test-conv2d failed on GPUs with computation capability <= 6.1 #668

Closed
bssrdf opened this issue Dec 28, 2023 · 0 comments · Fixed by #669
Closed

test-conv1d and test-conv2d failed on GPUs with computation capability <= 6.1 #668

bssrdf opened this issue Dec 28, 2023 · 0 comments · Fixed by #669

Comments

@bssrdf
Copy link
Contributor

bssrdf commented Dec 28, 2023

Just to report right now two test cases test-conv1d and test-conv2d cannot pass if running on GPU backend and its computation capability <= 6.1. On a GTX 1070 with CUDA version 12.1, it gave
ggml_im2col (4320): PASSED
ggml_conv2d (480): FAILED

This might be related to this section of the code

ggml/src/ggml-cuda.cu

Lines 7618 to 7639 in 3d57e76

else {
cuda_pool_alloc<float> src0_ddq_as_f32;
if (src0->type != GGML_TYPE_F32) {
const to_fp32_cuda_t to_fp32_cuda = ggml_get_to_fp32_cuda(src0->type);
GGML_ASSERT(to_fp32_cuda != nullptr);
src0_ddq_as_f32.alloc(row_diff*ne00);
to_fp32_cuda(src0_dd_i, src0_ddq_as_f32.get(), row_diff*ne00, stream);
}
const float * src0_ddf_i = src0->type == GGML_TYPE_F32 ? (const float *) src0_dd_i : src0_ddq_as_f32.get();
const float alpha = 1.0f;
const float beta = 0.0f;
CUBLAS_CHECK(cublasSetStream(g_cublas_handles[id], stream));
CUBLAS_CHECK(
cublasSgemm(g_cublas_handles[id], CUBLAS_OP_T, CUBLAS_OP_N,
row_diff, src1_ncols, ne10,
&alpha, src0_ddf_i, ne00,
src1_ddf_i, ne10,
&beta, dst_dd_i, ldc));
}

Here src1 is not converted to FP32 while src0 is.

@bssrdf bssrdf changed the title test-conv2d failed on GPUs with computation capability <= 6.1 test-conv1d and test-conv2d failed on GPUs with computation capability <= 6.1 Dec 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant