Embed yolo files #831

katsu560 · 2024-05-19T11:32:36Z

Some app like yolov3-tiny needs additional files to execute such as label(coco.names) and alphabet labels(100_0.png, ...) files.
If these files are embedded to a model(gguf) file and the app read them from the model file, the app is more portable.

I added below

added new GGUF_TYPE_NAMEDOBJECT with name(file path) and value(file body) for adding files to gguf
expanded gguf-py to support NAMEDOBJECT, constants.py, gguf_reader.py, gguf_writer.py
- please see pull request to llama.cpp
added gguf-addfile.py script to add files to gguf file
- add files as NAMEDOBJECT (general.namedobject.N) or add files as NAMEDOBJECT array (general.namedobject[N] with --array option)
expanded ggml to support NAMEDOBJECT, ggml.h ggml.c
expanded yolov3-tiny to read coco.names and alphabet labels from gguf file,
- at first read from gguf, then read from file if failed from gguf

NAMEDOBJECT constructed from name(file path) and value(file body)

    struct gguf_nobj {
        uint64_t nname;  // length of name
        char   * name;   // name in utf8
        uint64_t n;      // length of data in bytes
        char   * data;   // data body (file body)
    };

function usage:

struct gguf_nobj gguf_find_name_nobj(const struct gguf_context * ctx, const char * name)

call gguf_find_name_nobj() with const struct gguf_context *ctx and const char *name.
ctx is gguf_context pointer. name is string encoded UTF8 like filename.
search 'name' NAMEDOBJECT and return struct nobj.
if not found, return struct nobj(0, NULL, 0, NULL). so if nobj.n == 0 means 'not found'.
if found, return nobj with nobj.name has name, nobj.n has length of nobj.data, nobj.data has byte stream of data.

    struct gguf_nobj nobj = gguf_find_name_nobj(ctx, filename);
    if (nobj.n == 0) {
        return false;
    }
    membuf buf(nobj.data, nobj.data + nobj.n);
    std::istream file_in(&buf);
    if (!file_in) {
        return false;
    }
    std::string line;
    while (std::getline(file_in, line)) {
        labels.push_back(line);
    }

script usage:

python3 gguf-addfile.py [--array] input-gguf-file output-gguf-file files ...

add files as NAMEDOBJECT (general.namedobject.N)
add files as NAMEDOBJECT array (general.namedobject[N]) with --array option

slaren · 2024-05-19T15:53:12Z

Is it really necessary to a new type of object to the GGUF format to do this? The file data could be stored either as an array metadata or as a tensor.

ggerganov · 2024-05-19T15:56:49Z

I agree with @slaren - don't think it's necessary to introduce named object. But the rest of the idea to embed the data in the GGUF file is nice

katsu560 · 2024-05-19T17:23:18Z

Thanks for prompt checking, @slaren and @ggerganov .
If current data structure meet embedding files, I agreed no adding NAMEDOBJECT.

But, I think embedding files need 3 elements, such as path name string(GGUF string 2 part as length and string byte stream), length of data, data stream.
I think key string and GGUF_TYPE_STRING has string, length and bytes stream.
If we can use key string as path name, we can't embed same name file as existing key names, such as general.name, general.version, tokenizer.chat_template, etc.
And someone expects string body has no NULL byte in the way, but as you can see, file body has NULL byte(\0).
So, I added new type as NAMEDOBJECT.

CISC · 2024-05-19T18:33:43Z

You can store the data in an UINT8 array, if you need to store path too you can store it as an array of arrays, ie: [[path, data], [path, data]], though I'm unsure if you would then have to store path as UINT8 too, or if it's allowed to have mixed data? Probably best to store path and data in separate entries.

ggerganov · 2024-05-20T06:21:46Z

You can for example store a KV string array with filenames and for each filename have a U8 tensor for each file containing the binary data:

"embedded_files": ["my-file.dat", "another-file.bin"]
tensors:
- "my-file.dat"
- "another-file.bin"
- ...

katsu560 · 2024-05-23T23:07:55Z

Thanks comments, and sorry for my late response because of my hard working days.
I seek another way in this weekend, such as array of array, using tensor data.

katsu560 · 2024-05-31T21:02:44Z

finally, I added file data as follows;

store the file path as key with starting '/' to avoid from conflicts to other key names.
ex. storing file 'data/coco.names' as '/data/coco.names'
if storing absolute file path '/a/b/c' as '//a/b/c'
store the file contents as GGUF_TYPE_STRING's value.

So, I deleted all NAMEDOBJECT part.

katsu560 · 2024-05-31T21:24:20Z

I also removed dump code from gguf-addfile.py script.

this script usage example:
python3 gguf-addfile.py path/to/yolov3-tiny.gguf yolov3-tiny-addfiles.gguf data/coco.names data/labels/*

ggerganov

Can you try to update ci/run.sh to use the new script in the tests?

Adding the files as tensors instead of KV can have some advantages, because the GGUF header remains small. For example, I'm not sure how the GGUF viewer on HuggingFace would handle big data in the header. So it's an option that might be worth exploring

ggerganov · 2024-06-05T08:49:01Z

examples/yolo/yolov3-tiny.cpp

@@ -30,6 +30,7 @@ struct yolo_model {
 int height = 416;
 std::vector<conv2d_layer> conv2d_layers;
 struct ggml_context * ctx;
+ struct gguf_context * ggufctx;


Suggested change

struct gguf_context * ggufctx;

struct gguf_context * ctx_gguf;

ggerganov · 2024-06-05T08:50:38Z

include/ggml/ggml.h

+ GGML_API uint8_t gguf_get_val_u8 (const struct gguf_context * ctx, int key_id);
+ GGML_API int8_t gguf_get_val_i8 (const struct gguf_context * ctx, int key_id);
+ GGML_API uint16_t gguf_get_val_u16 (const struct gguf_context * ctx, int key_id);
+ GGML_API int16_t gguf_get_val_i16 (const struct gguf_context * ctx, int key_id);
+ GGML_API uint32_t gguf_get_val_u32 (const struct gguf_context * ctx, int key_id);
+ GGML_API int32_t gguf_get_val_i32 (const struct gguf_context * ctx, int key_id);
+ GGML_API float gguf_get_val_f32 (const struct gguf_context * ctx, int key_id);
+ GGML_API uint64_t gguf_get_val_u64 (const struct gguf_context * ctx, int key_id);
+ GGML_API int64_t gguf_get_val_i64 (const struct gguf_context * ctx, int key_id);
+ GGML_API double gguf_get_val_f64 (const struct gguf_context * ctx, int key_id);
+ GGML_API bool gguf_get_val_bool (const struct gguf_context * ctx, int key_id);
+ GGML_API const char * gguf_get_val_str (const struct gguf_context * ctx, int key_id);
+ GGML_API uint64_t gguf_get_val_str_len(const struct gguf_context * ctx, int key_id);
+ GGML_API const void * gguf_get_val_data (const struct gguf_context * ctx, int key_id);
+ GGML_API int gguf_get_arr_n (const struct gguf_context * ctx, int key_id);
+ GGML_API const void * gguf_get_arr_data (const struct gguf_context * ctx, int key_id);
+ GGML_API const char * gguf_get_arr_str (const struct gguf_context * ctx, int key_id, int i);


Suggested change

GGML_API uint8_t gguf_get_val_u8 (const struct gguf_context * ctx, int key_id);

GGML_API int8_t gguf_get_val_i8 (const struct gguf_context * ctx, int key_id);

GGML_API uint16_t gguf_get_val_u16 (const struct gguf_context * ctx, int key_id);

GGML_API int16_t gguf_get_val_i16 (const struct gguf_context * ctx, int key_id);

GGML_API uint32_t gguf_get_val_u32 (const struct gguf_context * ctx, int key_id);

GGML_API int32_t gguf_get_val_i32 (const struct gguf_context * ctx, int key_id);

GGML_API float gguf_get_val_f32 (const struct gguf_context * ctx, int key_id);

GGML_API uint64_t gguf_get_val_u64 (const struct gguf_context * ctx, int key_id);

GGML_API int64_t gguf_get_val_i64 (const struct gguf_context * ctx, int key_id);

GGML_API double gguf_get_val_f64 (const struct gguf_context * ctx, int key_id);

GGML_API bool gguf_get_val_bool (const struct gguf_context * ctx, int key_id);

GGML_API const char * gguf_get_val_str (const struct gguf_context * ctx, int key_id);

GGML_API uint64_t gguf_get_val_str_len(const struct gguf_context * ctx, int key_id);

GGML_API const void * gguf_get_val_data (const struct gguf_context * ctx, int key_id);

GGML_API int gguf_get_arr_n (const struct gguf_context * ctx, int key_id);

GGML_API const void * gguf_get_arr_data (const struct gguf_context * ctx, int key_id);

GGML_API const char * gguf_get_arr_str (const struct gguf_context * ctx, int key_id, int i);

GGML_API uint8_t gguf_get_val_u8 (const struct gguf_context * ctx, int key_id);

GGML_API int8_t gguf_get_val_i8 (const struct gguf_context * ctx, int key_id);

GGML_API uint16_t gguf_get_val_u16 (const struct gguf_context * ctx, int key_id);

GGML_API int16_t gguf_get_val_i16 (const struct gguf_context * ctx, int key_id);

GGML_API uint32_t gguf_get_val_u32 (const struct gguf_context * ctx, int key_id);

GGML_API int32_t gguf_get_val_i32 (const struct gguf_context * ctx, int key_id);

GGML_API float gguf_get_val_f32 (const struct gguf_context * ctx, int key_id);

GGML_API uint64_t gguf_get_val_u64 (const struct gguf_context * ctx, int key_id);

GGML_API int64_t gguf_get_val_i64 (const struct gguf_context * ctx, int key_id);

GGML_API double gguf_get_val_f64 (const struct gguf_context * ctx, int key_id);

GGML_API bool gguf_get_val_bool (const struct gguf_context * ctx, int key_id);

GGML_API const char * gguf_get_val_str (const struct gguf_context * ctx, int key_id);

GGML_API uint64_t gguf_get_val_str_len(const struct gguf_context * ctx, int key_id);

GGML_API const void * gguf_get_val_data (const struct gguf_context * ctx, int key_id);

GGML_API int gguf_get_arr_n (const struct gguf_context * ctx, int key_id);

GGML_API const void * gguf_get_arr_data (const struct gguf_context * ctx, int key_id);

GGML_API const char * gguf_get_arr_str (const struct gguf_context * ctx, int key_id, int i);

ggerganov · 2024-06-05T08:51:53Z

src/ggml.c

+ case GGUF_TYPE_UINT8:  ok = ok && gguf_fread_el  (file, &kv->value.uint8, sizeof(kv->value.uint8), &offset); break;
+ case GGUF_TYPE_INT8:  ok = ok && gguf_fread_el  (file, &kv->value.int8, sizeof(kv->value.int8), &offset); break;
+ case GGUF_TYPE_UINT16:  ok = ok && gguf_fread_el  (file, &kv->value.uint16, sizeof(kv->value.uint16), &offset); break;
+ case GGUF_TYPE_INT16:  ok = ok && gguf_fread_el  (file, &kv->value.int16, sizeof(kv->value.int16), &offset); break;
+ case GGUF_TYPE_UINT32:  ok = ok && gguf_fread_el  (file, &kv->value.uint32, sizeof(kv->value.uint32), &offset); break;
+ case GGUF_TYPE_INT32:  ok = ok && gguf_fread_el  (file, &kv->value.int32, sizeof(kv->value.int32), &offset); break;
+ case GGUF_TYPE_FLOAT32:  ok = ok && gguf_fread_el  (file, &kv->value.float32, sizeof(kv->value.float32), &offset); break;
+ case GGUF_TYPE_UINT64:  ok = ok && gguf_fread_el  (file, &kv->value.uint64, sizeof(kv->value.uint64), &offset); break;
+ case GGUF_TYPE_INT64:  ok = ok && gguf_fread_el  (file, &kv->value.int64, sizeof(kv->value.int64), &offset); break;
+ case GGUF_TYPE_FLOAT64:  ok = ok && gguf_fread_el  (file, &kv->value.float64, sizeof(kv->value.float64), &offset); break;
+ case GGUF_TYPE_BOOL:  ok = ok && gguf_fread_el  (file, &kv->value.bool_, sizeof(kv->value.bool_), &offset); break;
+ case GGUF_TYPE_STRING:  ok = ok && gguf_fread_str (file, &kv->value.str, &offset); break;
 case GGUF_TYPE_ARRAY:


Suggested change

case GGUF_TYPE_UINT8: ok = ok && gguf_fread_el (file, &kv->value.uint8, sizeof(kv->value.uint8), &offset); break;

case GGUF_TYPE_INT8: ok = ok && gguf_fread_el (file, &kv->value.int8, sizeof(kv->value.int8), &offset); break;

case GGUF_TYPE_UINT16: ok = ok && gguf_fread_el (file, &kv->value.uint16, sizeof(kv->value.uint16), &offset); break;

case GGUF_TYPE_INT16: ok = ok && gguf_fread_el (file, &kv->value.int16, sizeof(kv->value.int16), &offset); break;

case GGUF_TYPE_UINT32: ok = ok && gguf_fread_el (file, &kv->value.uint32, sizeof(kv->value.uint32), &offset); break;

case GGUF_TYPE_INT32: ok = ok && gguf_fread_el (file, &kv->value.int32, sizeof(kv->value.int32), &offset); break;

case GGUF_TYPE_FLOAT32: ok = ok && gguf_fread_el (file, &kv->value.float32, sizeof(kv->value.float32), &offset); break;

case GGUF_TYPE_UINT64: ok = ok && gguf_fread_el (file, &kv->value.uint64, sizeof(kv->value.uint64), &offset); break;

case GGUF_TYPE_INT64: ok = ok && gguf_fread_el (file, &kv->value.int64, sizeof(kv->value.int64), &offset); break;

case GGUF_TYPE_FLOAT64: ok = ok && gguf_fread_el (file, &kv->value.float64, sizeof(kv->value.float64), &offset); break;

case GGUF_TYPE_BOOL: ok = ok && gguf_fread_el (file, &kv->value.bool_, sizeof(kv->value.bool_), &offset); break;

case GGUF_TYPE_STRING: ok = ok && gguf_fread_str (file, &kv->value.str, &offset); break;

case GGUF_TYPE_ARRAY:

case GGUF_TYPE_UINT8: ok = ok && gguf_fread_el (file, &kv->value.uint8, sizeof(kv->value.uint8), &offset); break;

case GGUF_TYPE_INT8: ok = ok && gguf_fread_el (file, &kv->value.int8, sizeof(kv->value.int8), &offset); break;

case GGUF_TYPE_UINT16: ok = ok && gguf_fread_el (file, &kv->value.uint16, sizeof(kv->value.uint16), &offset); break;

case GGUF_TYPE_INT16: ok = ok && gguf_fread_el (file, &kv->value.int16, sizeof(kv->value.int16), &offset); break;

case GGUF_TYPE_UINT32: ok = ok && gguf_fread_el (file, &kv->value.uint32, sizeof(kv->value.uint32), &offset); break;

case GGUF_TYPE_INT32: ok = ok && gguf_fread_el (file, &kv->value.int32, sizeof(kv->value.int32), &offset); break;

case GGUF_TYPE_FLOAT32: ok = ok && gguf_fread_el file, &kv->value.float32, sizeof(kv->value.float32), &offset); break;

case GGUF_TYPE_UINT64: ok = ok && gguf_fread_el (file, &kv->value.uint64, sizeof(kv->value.uint64), &offset); break;

case GGUF_TYPE_INT64: ok = ok && gguf_fread_el (file, &kv->value.int64, sizeof(kv->value.int64), &offset); break;

case GGUF_TYPE_FLOAT64: ok = ok && gguf_fread_el (file, &kv->value.float64, sizeof(kv->value.float64), &offset); break;

case GGUF_TYPE_BOOL: ok = ok && gguf_fread_el (file, &kv->value.bool_, sizeof(kv->value.bool_), &offset); break;

case GGUF_TYPE_STRING: ok = ok && gguf_fread_str(file, &kv->value.str, &offset); break;

case GGUF_TYPE_ARRAY:

ggerganov · 2024-06-05T08:52:21Z

src/ggml.c

+ case GGUF_TYPE_UINT8: gguf_bwrite_el (buf, &kv->value.uint8, sizeof(kv->value.uint8) ); break;
+ case GGUF_TYPE_INT8: gguf_bwrite_el  (buf, &kv->value.int8, sizeof(kv->value.int8) ); break;
+ case GGUF_TYPE_UINT16: gguf_bwrite_el  (buf, &kv->value.uint16, sizeof(kv->value.uint16) ); break;
+ case GGUF_TYPE_INT16: gguf_bwrite_el  (buf, &kv->value.int16, sizeof(kv->value.int16) ); break;
+ case GGUF_TYPE_UINT32: gguf_bwrite_el  (buf, &kv->value.uint32, sizeof(kv->value.uint32) ); break;
+ case GGUF_TYPE_INT32: gguf_bwrite_el  (buf, &kv->value.int32, sizeof(kv->value.int32) ); break;
+ case GGUF_TYPE_FLOAT32: gguf_bwrite_el  (buf, &kv->value.float32, sizeof(kv->value.float32)); break;
+ case GGUF_TYPE_UINT64: gguf_bwrite_el  (buf, &kv->value.uint64, sizeof(kv->value.uint64) ); break;
+ case GGUF_TYPE_INT64: gguf_bwrite_el  (buf, &kv->value.int64, sizeof(kv->value.int64) ); break;
+ case GGUF_TYPE_FLOAT64: gguf_bwrite_el  (buf, &kv->value.float64, sizeof(kv->value.float64)); break;
+ case GGUF_TYPE_BOOL: gguf_bwrite_el  (buf, &kv->value.bool_, sizeof(kv->value.bool_) ); break;
+ case GGUF_TYPE_STRING: gguf_bwrite_str (buf, &kv->value.str ); break;


Suggested change

case GGUF_TYPE_UINT8: gguf_bwrite_el (buf, &kv->value.uint8, sizeof(kv->value.uint8) ); break;

case GGUF_TYPE_INT8: gguf_bwrite_el (buf, &kv->value.int8, sizeof(kv->value.int8) ); break;

case GGUF_TYPE_UINT16: gguf_bwrite_el (buf, &kv->value.uint16, sizeof(kv->value.uint16) ); break;

case GGUF_TYPE_INT16: gguf_bwrite_el (buf, &kv->value.int16, sizeof(kv->value.int16) ); break;

case GGUF_TYPE_UINT32: gguf_bwrite_el (buf, &kv->value.uint32, sizeof(kv->value.uint32) ); break;

case GGUF_TYPE_INT32: gguf_bwrite_el (buf, &kv->value.int32, sizeof(kv->value.int32) ); break;

case GGUF_TYPE_FLOAT32: gguf_bwrite_el (buf, &kv->value.float32, sizeof(kv->value.float32)); break;

case GGUF_TYPE_UINT64: gguf_bwrite_el (buf, &kv->value.uint64, sizeof(kv->value.uint64) ); break;

case GGUF_TYPE_INT64: gguf_bwrite_el (buf, &kv->value.int64, sizeof(kv->value.int64) ); break;

case GGUF_TYPE_FLOAT64: gguf_bwrite_el (buf, &kv->value.float64, sizeof(kv->value.float64)); break;

case GGUF_TYPE_BOOL: gguf_bwrite_el (buf, &kv->value.bool_, sizeof(kv->value.bool_) ); break;

case GGUF_TYPE_STRING: gguf_bwrite_str (buf, &kv->value.str ); break;

case GGUF_TYPE_UINT8: gguf_bwrite_el (buf, &kv->value.uint8, sizeof(kv->value.uint8) ); break;

case GGUF_TYPE_INT8: gguf_bwrite_el (buf, &kv->value.int8, sizeof(kv->value.int8) ); break;

case GGUF_TYPE_UINT16: gguf_bwrite_el (buf, &kv->value.uint16, sizeof(kv->value.uint16) ); break;

case GGUF_TYPE_INT16: gguf_bwrite_el (buf, &kv->value.int16, sizeof(kv->value.int16) ); break;

case GGUF_TYPE_UINT32: gguf_bwrite_el (buf, &kv->value.uint32, sizeof(kv->value.uint32) ); break;

case GGUF_TYPE_INT32: gguf_bwrite_el (buf, &kv->value.int32, sizeof(kv->value.int32) ); break;

case GGUF_TYPE_FLOAT32: gguf_bwrite_el (buf, &kv->value.float32, sizeof(kv->value.float32)); break;

case GGUF_TYPE_UINT64: gguf_bwrite_el (buf, &kv->value.uint64, sizeof(kv->value.uint64) ); break;

case GGUF_TYPE_INT64: gguf_bwrite_el (buf, &kv->value.int64, sizeof(kv->value.int64) ); break;

case GGUF_TYPE_FLOAT64: gguf_bwrite_el (buf, &kv->value.float64, sizeof(kv->value.float64)); break;

case GGUF_TYPE_BOOL: gguf_bwrite_el (buf, &kv->value.bool_, sizeof(kv->value.bool_) ); break;

case GGUF_TYPE_STRING: gguf_bwrite_str(buf, &kv->value.str ); break;

katsu560 · 2024-06-15T16:05:36Z

I revised code as to add files to tensor data.
I also applied your suggestions.

I try to update ci/run.sh later.

ggerganov · 2024-06-16T09:10:26Z

include/ggml/ggml.h

+ GGML_API char * gguf_get_tensor_name (const struct gguf_context * ctx, int i);
+ GGML_API enum ggml_type gguf_get_tensor_type (const struct gguf_context * ctx, int i);
+ GGML_API size_t gguf_get_tensor_size (const struct gguf_context * ctx, int i);
+ GGML_API int gguf_find_and_get_tensor(const struct gguf_context * ctx, const char * name, char ** data, size_t * size);


This is looking better, but still needs some work. Neither of the changes to gguf are needed, so try to avoid them and do the same using the existing API. The final PR should not contain any modifications to ggml.h. The only one that can remain is the gguf_get_tensor_size() helper function

katsu560 · 2024-06-22T04:21:51Z

I added two functions to ggml.c, gguf_get_tensor_size and gguf_find_key_array.
I think it is minimum adding.

katsu560 · 2024-06-22T19:57:27Z

I also revised ci/run.sh.
I added test code to create gguf file and test by yolov3-tiny for reading files from gguf file.

katsu560 · 2024-06-23T14:58:49Z

I fixed script gguf-addfile.py

fix copying key value other than embedded_files
refactor code
remove unused code
check overwriting output file
add --force option

ggerganov · 2024-06-25T13:05:26Z

include/ggml/ggml.h

@@ -2305,6 +2305,7 @@ extern "C" {

 GGML_API int gguf_get_n_kv(const struct gguf_context * ctx);
 GGML_API int gguf_find_key(const struct gguf_context * ctx, const char * key);
+ GGML_API int gguf_find_key_array(const struct gguf_context * ctx, const char * key, const char * val);


No need to add gguf_find_key_array() - it's result is never used for anything, so we can simply remove it

katsu560 · 2024-06-25T18:39:57Z

deleted gguf_find_key_array() and related code from examples/yolo/yolov3-tiny.cpp.
please confirm.

ggerganov · 2024-06-26T17:28:54Z

examples/yolo/yolov3-tiny.cpp

+ return false;
+ }
+ const size_t offset = gguf_get_tensor_offset(ctx, tensor);
+ const size_t len = gguf_get_tensor_size(ctx, tensor);


Somehow I didn't notice this before: gguf_get_tensor_size() is not needed too. You can instead use:

Suggested change

const size_t len = gguf_get_tensor_size(ctx, tensor);

const size_t len = ggml_nelements(tensor);

So remove gguf_get_tensor_size all together

okay, i removed gguf_get_tensor_size from ggml.h, ggml.c, yolov3-tiny.cpp.

katsu560 added 3 commits May 19, 2024 19:58

ggml : add namedobject to GGUF_TYPE for adding files to model file

6af7435

yolo : add reading labels and alphabet labels from model file

661588c

yolo : add files to gguf file script

ecf8043

katsu560 mentioned this pull request May 19, 2024

gguf : embed files to gguf model file ggerganov/llama.cpp#7392

Closed

katsu560 added 2 commits June 1, 2024 05:09

read data from kv string

33cf5b3

remove NAMEDOBJECT, use key and STRING value

73a168b

remove dump code

3234fa1

ggerganov reviewed Jun 5, 2024

View reviewed changes

katsu560 added 2 commits June 16, 2024 00:52

read file data from tensor

aaa93bc

add files to kv and tensor data

bcf4ec8

Merge branch 'ggerganov:master' into embed_yolo_files

8d6feac

ggerganov reviewed Jun 16, 2024

View reviewed changes

katsu560 and others added 5 commits June 17, 2024 23:03

Merge branch 'ggerganov:master' into embed_yolo_files

d13e8ba

Merge branch 'ggerganov:master' into embed_yolo_files

2210bb0

load files from model

2c3603e

load files from model

50d5683

load files from model

9f70ebf

add yolo test, making gguf and reading files from gguf

e8720f6

remove debug code, unused code

695fbaf

refactor code, fix copying key value, add --force

e18593c

ggerganov reviewed Jun 25, 2024

View reviewed changes

delete gguf_find_key_array()

3f06cef

katsu560 mentioned this pull request Jun 25, 2024

Embed files ggerganov/llama.cpp#8121

Open

4 tasks

ggerganov reviewed Jun 26, 2024

View reviewed changes

remove gguf_get_tensor_size

7d59c7a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Embed yolo files #831

Embed yolo files #831

katsu560 commented May 19, 2024

slaren commented May 19, 2024

ggerganov commented May 19, 2024

katsu560 commented May 19, 2024 •

edited

Loading

CISC commented May 19, 2024 •

edited

Loading

ggerganov commented May 20, 2024

katsu560 commented May 23, 2024

katsu560 commented May 31, 2024

katsu560 commented May 31, 2024 •

edited

Loading

ggerganov left a comment

ggerganov Jun 5, 2024

katsu560 Jun 15, 2024

ggerganov Jun 5, 2024

ggerganov Jun 5, 2024

ggerganov Jun 5, 2024

katsu560 commented Jun 15, 2024

ggerganov Jun 16, 2024

katsu560 commented Jun 22, 2024

katsu560 commented Jun 22, 2024

katsu560 commented Jun 23, 2024

ggerganov Jun 25, 2024

katsu560 commented Jun 25, 2024

ggerganov Jun 26, 2024 •

edited

Loading

katsu560 Jun 29, 2024

	struct gguf_context * ggufctx;
	struct gguf_context * ctx_gguf;

	const size_t len = gguf_get_tensor_size(ctx, tensor);
	const size_t len = ggml_nelements(tensor);

Embed yolo files #831

Are you sure you want to change the base?

Embed yolo files #831

Conversation

katsu560 commented May 19, 2024

slaren commented May 19, 2024

ggerganov commented May 19, 2024

katsu560 commented May 19, 2024 • edited Loading

CISC commented May 19, 2024 • edited Loading

ggerganov commented May 20, 2024

katsu560 commented May 23, 2024

katsu560 commented May 31, 2024

katsu560 commented May 31, 2024 • edited Loading

ggerganov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

katsu560 commented Jun 15, 2024

Choose a reason for hiding this comment

katsu560 commented Jun 22, 2024

katsu560 commented Jun 22, 2024

katsu560 commented Jun 23, 2024

Choose a reason for hiding this comment

katsu560 commented Jun 25, 2024

ggerganov Jun 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

katsu560 commented May 19, 2024 •

edited

Loading

CISC commented May 19, 2024 •

edited

Loading

katsu560 commented May 31, 2024 •

edited

Loading

ggerganov Jun 26, 2024 •

edited

Loading