Cannot load teknium/Replit-v2-CodeInstruct-3B #436
Comments
|
The model loads fine for me. The named tensor should be recognized and loaded. Did you get any compilation warnings? |
|
@klosax One thing might be that I had received this error when I ran So I rephrased the command like this: and it compiled successfully. Could this have anything to do with it? |
I guess not if the model file was converted successfully. Any compilation warnings when compling the inference binary? |
|
Nothing stood out to me in particular... cmake .. && make -j4 replit replit-quantize
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Linux detected
-- x86 detected
-- Linux detected
-- Configuring done (0.1s)
-- Generating done (0.3s)
-- Build files have been written to: ~/ggml/build
[ 25%] Building CXX object examples/CMakeFiles/common.dir/common.cpp.o
[ 25%] Building C object src/CMakeFiles/ggml.dir/ggml.c.o
In file included from /usr/include/string.h:535,
from ~/ggml/src/ggml.c:21:
In function ‘memcpy’,
inlined from ‘ggml_set_op_params’ at ~/ggml/src/ggml.c:4642:5,
inlined from ‘ggml_conv_1d’ at ~/ggml/src/ggml.c:6883:5:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10: warning: ‘__builtin_memcpy’ offset [0, 11] is out of the bounds [0, 0] [-Warray-bounds]
29 | return __builtin___memcpy_chk (__dest, __src, __len,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
30 | __glibc_objsize0 (__dest));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘memcpy’,
inlined from ‘ggml_set_op_params’ at ~/ggml/src/ggml.c:4642:5,
inlined from ‘ggml_conv_2d’ at ~/ggml/src/ggml.c:6923:5:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10: warning: ‘__builtin_memcpy’ offset [0, 23] is out of the bounds [0, 0] [-Warray-bounds]
29 | return __builtin___memcpy_chk (__dest, __src, __len,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
30 | __glibc_objsize0 (__dest));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘memcpy’,
inlined from ‘ggml_set_op_params’ at ~/ggml/src/ggml.c:4642:5,
inlined from ‘ggml_conv_1d’ at ~/ggml/src/ggml.c:6883:5,
inlined from ‘ggml_conv_1d_ph’ at ~/ggml/src/ggml.c:6942:12:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10: warning: ‘__builtin_memcpy’ offset [0, 11] is out of the bounds [0, 0] [-Warray-bounds]
29 | return __builtin___memcpy_chk (__dest, __src, __len,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
30 | __glibc_objsize0 (__dest));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘memcpy’,
inlined from ‘ggml_set_op_params’ at ~/ggml/src/ggml.c:4642:5,
inlined from ‘ggml_pool_2d’ at ~/ggml/src/ggml.c:7015:5:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10: warning: ‘__builtin_memcpy’ offset [0, 27] is out of the bounds [0, 0] [-Warray-bounds]
29 | return __builtin___memcpy_chk (__dest, __src, __len,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
30 | __glibc_objsize0 (__dest));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
In function ‘memcpy’,
inlined from ‘ggml_set_op_params’ at ~/ggml/src/ggml.c:4642:5,
inlined from ‘ggml_win_part’ at ~/ggml/src/ggml.c:7183:5:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10: warning: ‘__builtin_memcpy’ offset [0, 11] is out of the bounds [0, 0] [-Warray-bounds]
29 | return __builtin___memcpy_chk (__dest, __src, __len,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
30 | __glibc_objsize0 (__dest));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
[ 37%] Linking CXX static library libcommon.a
[ 37%] Built target common
[ 50%] Linking C static library libggml.a
[ 50%] Built target ggml
[ 62%] Building CXX object examples/CMakeFiles/common-ggml.dir/common-ggml.cpp.o
[ 75%] Linking CXX static library libcommon-ggml.a
[ 75%] Built target common-ggml
[ 87%] Building CXX object examples/replit/CMakeFiles/replit.dir/main.cpp.o
[100%] Linking CXX executable ../../bin/replit
[100%] Built target replit
[ 37%] Built target common
[ 50%] Built target ggml
[ 75%] Built target common-ggml
[ 87%] Building CXX object examples/replit/CMakeFiles/replit-quantize.dir/quantize.cpp.o
[100%] Linking CXX executable ../../bin/replit-quantize
[100%] Built target replit-quantize |
The model file seems to be fine since the tensor Try updating or reinstalling the compiler. |
|
This is my version. Should it be upgraded? |
|
I think it should work with your compiler. But you could try change this line Line 379 in 244776a to
and compile again. |
|
This worked! @klosax , thanks for your time and for the help. I was lost without you 🙏 Would this change be useful to others as well? Should I commit and raise a PR? |
|
Great! Then all references of
It looks like this error can also be found in other examples and all of them should be fixed. |
|
Wouldn't that be breaking compilation of other models as well? Would you like me to try and reproduce for other classes of models before making a fix? Because if what you say is true, then wouldn't this be a huge change? 🤔 |
|
All examples compile and works fine for me using gcc 9, so my guess is that your gcc 11 is handling this different than the older compilers, and that is the reason it wont work for you. |
|
Understood. So if I'm understanding correctly, even if Apologies for asking what might be basic questions. My C++ is rusty, so I don't want to be creating a regression and getting angry emails 😅 |
|
Yes the changes wont break anything for older compilers. I will make a PR for this to change all examples. |
If you like you could test one other example to see if the same error is there and if it is fixed by this change. |
This issue is on similar lines as #248 , but is regarding replit-v2 models, not replit-v1
I am using ggml@ a301077 and am having trouble loading
teknium/Replit-v2-CodeInstruct-3BI used
examples/replit/convert-h5-to-ggml.pyto convert to ggmlf32.Also created both a
q4_1as well asq8_0quantized versions usingreplit-quantize.However, when trying to load either f32, q4_1 or q8_0 versions of the models with replit
(e.g.,
./bin/replit -m Replit-v2-CodeInstruct-3B-f32.bin -p "def hello_world():")I get:
Any ideas?
The text was updated successfully, but these errors were encountered: