GGML_ASSERT: /Users/liu/ggml/src/ggml.c:10798: ne02 == ne12 zsh: abort ./bin/sam -t 16 -i ../image.jpeg -m ../examples/sam/ggml-model-f16.bin

[bccw2021]

(base) liu@liuMacBook-Pro build % ./bin/sam -t 16 -i ../image.jpeg -m ../examples/sam/ggml-model-f16.bin
main: seed = 1692426857
main: loaded image '../image.jpeg' (680 x 453)
sam_image_preprocess: scale = 0.664062
main: preprocessed image (1024 x 1024)
sam_model_load: loading model from '../examples/sam/ggml-model-f16.bin' - please wait ...
sam_model_load: n_enc_state = 768
sam_model_load: n_enc_layer = 12
sam_model_load: n_enc_head = 12
sam_model_load: n_enc_out_chans = 256
sam_model_load: n_pt_embd = 4
sam_model_load: ftype = 1
sam_model_load: qntvr = 0
operator(): ggml ctx size = 202.32 MB
sam_model_load: ...................................... done
sam_model_load: model size = 185.05 MB / num tensors = 304
GGML_ASSERT: /Users/liu/ggml/src/ggml.c:10798: ne02 == ne12
zsh: abort ./bin/sam -t 16 -i ../image.jpeg -m ../examples/sam/ggml-model-f16.bin
(base) liu@liuMacBook-Pro build %

[ggerganov]

On MacOS, BLAS is enabled by default and sam needs broadcast support in ggml_mul_mat.
This is currently not implemented:

ggml/src/ggml.c

Lines 10796 to 10799 in 08c57df

	// TODO: handle case when src0 is broadcast-able into src1 across 2nd,3rd dimension
	// ref: https://github.com/ggerganov/ggml/pull/224
	GGML_ASSERT(ne02 == ne12);
	GGML_ASSERT(ne03 == ne13);

A temporary workaround is to use cmake GGML_NO_ACCELERATE=ON to disable BLAS and it will work, but it will be slower when using F16 and F32 data.

[bccw2021]

Thank you!

[ggerganov]

Should be fixed now