Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ggml : make ggml_fp16_t private #720

Closed
ggerganov opened this issue Jan 31, 2024 · 2 comments
Closed

ggml : make ggml_fp16_t private #720

ggerganov opened this issue Jan 31, 2024 · 2 comments
Assignees
Labels
refactoring Refactoring

Comments

@ggerganov
Copy link
Owner

Currently the ggml_fp16_t typedef is exposed in the public API:

ggml/include/ggml/ggml.h

Lines 317 to 332 in 6b14d73

#if defined(__ARM_NEON) && defined(__CUDACC__)
typedef half ggml_fp16_t;
#elif defined(__ARM_NEON) && !defined(_MSC_VER)
typedef __fp16 ggml_fp16_t;
#else
typedef uint16_t ggml_fp16_t;
#endif
// convert FP16 <-> FP32
GGML_API float ggml_fp16_to_fp32(ggml_fp16_t x);
GGML_API ggml_fp16_t ggml_fp32_to_fp16(float x);
GGML_API void ggml_fp16_to_fp32_row(const ggml_fp16_t * x, float * y, int n);
GGML_API void ggml_fp32_to_fp16_row(const float * x, ggml_fp16_t * y, int n);

Since this type is platform specific, it would make sense to hide it by moving it in ggml-impl.h.
We will still expose an API for F16 <-> F32 conversions, but it sill operate on void * instead of ggml_fp16_t

@ggerganov ggerganov added the refactoring Refactoring label Jan 31, 2024
@slaren
Copy link
Collaborator

slaren commented Jan 31, 2024

We could probably define it to uint16_t always, and only cast it to __fp16 in the ARM code that can take advantage of that. The CUDA one is weird, it should only be used when compiling ggml-cuda.cu, which never uses this type anyway.

@ggerganov
Copy link
Owner Author

Ah yes, doing so would make this change much simpler

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
refactoring Refactoring
Projects
Status: Done
Development

No branches or pull requests

2 participants