weird "Segmentation fault" with FFI

[foldl]

What version of Bun is running?

1.1.3

What platform is your computer?

WSL1: Linux 4.4.0-22621-Microsoft x86_64 x86_64

What steps can reproduce the bug?

1. Build libchatllm
2. Prepare a LLM:
   • either download a supported LLM and convert it,
   • or, download a quantized one.

Now, let's assume the quantized model is saved as /path/to/qwen1.5-1.8b.bin.
Run this command, and "Segmentation fault" occurs:

$ bun run bugs.ts -i -m /path/to/qwen1.5-1.8b.bin 
[ "-i", "-m", "/path/to/qwen1.5-1.8b.bin" ]
    ________          __  __    __    __  ___ (通义千问)
   / ____/ /_  ____ _/ /_/ /   / /   /  |/  /_________  ____
  / /   / __ \/ __ `/ __/ /   / /   / /|_/ // ___/ __ \/ __ \
 / /___/ / / / /_/ / /_/ /___/ /___/ /  / // /__/ /_/ / /_/ /
 \____/_/ /_/\__,_/\__/_____/_____/_/  /_(_)___/ .___/ .___/
You are served by QWen2,                      /_/   /_/
with 1836828672 (1.8B) parameters.
Segmentation fault (core dumped)

---

Content of bugs.ts:

import { dlopen, FFIType, suffix, JSCallback, ptr, CString } from "bun:ffi";

const path = `libchatllm.${suffix}`;

const {
    symbols: {
        chatllm_create,
        chatllm_append_param,
        chatllm_set_print_reference,
        chatllm_set_print_rewritten_query,
        chatllm_start,
        chatllm_user_input,
        chatllm_abort_generation,
    },
} = dlopen(
    path,
    {
        chatllm_create: {
            args: [],
            returns: FFIType.ptr,
        },
        chatllm_append_param: {
            args: [FFIType.ptr, FFIType.cstring],
        },
        chatllm_set_print_reference: {
            args: [FFIType.ptr, FFIType.function],
            returns: FFIType.i32
        },
        chatllm_set_print_rewritten_query: {
            args: [FFIType.ptr, FFIType.function],
            returns: FFIType.i32
        },
        chatllm_start: {
            args: [FFIType.ptr, FFIType.function, FFIType.function, FFIType.ptr],
            returns: FFIType.i32
        },
        chatllm_user_input: {
            args: [FFIType.ptr, FFIType.cstring],
            returns: FFIType.i32
        },
        chatllm_abort_generation: {
            args: [FFIType.ptr]
        },
    },
);

class ChatLLMHandler {
    references: string[]

    constructor() {
        this.references = []
    }

    print(s: string) {
       // process.stdout.write(s);
    }

    print_reference(s: string) {
        this.references.push(s);
    }

    print_rewritten_query(s: string) { }

    start () {
        this.references = [];
    }

    end() { }
};

class ChatLLM {
    references: string[];
    obj: any;
    callback_print: JSCallback
    callback_print_reference: JSCallback
    callback_print_rewritten_query: JSCallback
    callback_end: JSCallback
    handler: ChatLLMHandler

    constructor(params: string[], handler: ChatLLMHandler) {
        this.handler = handler
        this.obj = chatllm_create();
        this.callback_print = new JSCallback(
            (p_obj, ptr) => process.stdout.write(new CString(ptr)), //this.handler.print(new CString(ptr)),
            {
                args: ["ptr", "ptr"],
            },
        );
        this.callback_print_reference = new JSCallback(
            (p_obj, ptr) => 0,// this.handler.print_reference(new CString(ptr)),
            {
                args: ["ptr", "ptr"],
            },
        );
        this.callback_print_rewritten_query = new JSCallback(
            (p_obj, ptr) => 0, //this.handler.print_rewritten_query(new CString(ptr)),
            {
                args: ["ptr", "ptr"],
            },
        );
        this.callback_end = new JSCallback(
            (p_obj) => 0, // this.handler.end(),
            {
                args: ["ptr"],
            },
        );
        if (params.length > 0) {
            console.log(params);
            for (let param of params)
                this.append_param(param);
            this.start();
        }
    }

    append_param(s) {
        let str = Buffer.from(s + '\0', "utf8")
        chatllm_append_param(this.obj, ptr(str));
    }

    start() {
        //chatllm_set_print_reference(this.obj, this.callback_print_reference)
        //chatllm_set_print_rewritten_query(this.obj, this.callback_print_rewritten_query)
        let r = chatllm_start(this.obj, this.callback_print, this.callback_end, 0);
        if (r != 0) {
            throw `ChatLLM::start error code ${r}`;
        }
    }

    chat(s) {
        //this.handler.start()
        let str = Buffer.from(s + '\0', "utf8")
        let r = chatllm_user_input(this.obj, ptr(str));
        if (r != 0) {
            throw `ChatLLM::chat error code ${r}`;
        }
    }

    abort() {
        chatllm_abort_generation(this.obj);
    }
};



let llm = new ChatLLM(Bun.argv.slice(2), new ChatLLMHandler());

const prompt = 'You  > ';
const AI     = 'A.I. > ';

process.stdout.write(prompt);
for await (const line of console) {
    process.stdout.write(AI);
    llm.chat(line);
    process.stdout.write(prompt);
}

class StdIOHandler {//} extends ChatLLMHandler {

    print(s: string) {
        process.stdout.write(s);
    }

    print_rewritten_query(s: string) {
        console.log(`Searching ${s} ...`)
    }

    end() {
       // var a =100;
       // process.stdout.write('end');
       console.log('end');
       // console.log(this);
       // if (this.references.length < 1) return;

        console.log("References:")
        for (let s of this.references)
            console.log(s)
    }

};

What is the expected behavior?

We should have a command line based chatbot. Such as,

[ "-i", "-m", "/mnt/c/projects/chatllm_quantized_models/qwen1.5-1.8b.bin" ]
    ________          __  __    __    __  ___ (通义千问)
   / ____/ /_  ____ _/ /_/ /   / /   /  |/  /_________  ____  
  / /   / __ \/ __ `/ __/ /   / /   / /|_/ // ___/ __ \/ __ \ 
 / /___/ / / / /_/ / /_/ /___/ /___/ /  / // /__/ /_/ / /_/ / 
 \____/_/ /_/\__,_/\__/_____/_____/_/  /_(_)___/ .___/ .___/  
You are served by QWen2,                      /_/   /_/       
with 1836828672 (1.8B) parameters.

You  > hello
A.I. > Hello! How can I help you today? Is there something you would like to know, talk about, or ask? I'm here to provide information and answer any questions you may have to the best of my ability. Whether it's a specific topic, a question about a particular subject, or just a casual conversation, feel free to ask me anything and I'll do my best to assist you.

What do you see instead?

Segmentation fault (core dumped)

Additional information

The senselessness of bugs.ts shows how I had tried to fix it. If line 170 (console.log('end')) is commented out,
"Segmentation fault" is gone.

BTW, I have also tested bugs.ts on Windows (Microsoft Windows NT 10.0.22635.0 x64) with 1.1.3. It seems OK.

[foldl]

I closed this by mistake.