How to create chat endpoint with express?

[spy16]

I am trying to setup Vercel AI SDK with an express endpoint which I will then use with useChat() hook.

After all my custom logic I have a StreamingTextResponse type value. How do I stream this to the client?

This just has a pipeTo method but I am not able to find a way to get a Writable from the express-js Response type.

[MaxLeiter]

You can use streamToResponse: https://sdk.vercel.ai/docs/api-reference/stream-to-response#example-nodejs-http-server

[spy16]

Oh i just found an open issue on this

#199

[MaxLeiter]

Ah apologies, I didn't know you were using remix. can we close this and track there / upstream?

[spy16]

Yea, i didn't think the remix part was necessary since i was making an express handler, so didn't mention that initially. It's clear now that it is related to remix .

I think this is already marked as closed. I will follow the already open issues. Thanks! 🙂

[spy16]

I just figured this out. Just leaving the idea here for anyone else stumbling on this discussion looking for answers:

1. Copy-paste the file from this comment: StreamingTextResponse not compatible with Remix action function #199 (comment)

2. Take the response from openai.createChatCompletion() and re-create it as:

    new Response(
      StreamCompat.toReadable(resp.body),
      {
          status: resp.status,
          headers: resp.headers,
          statusText: resp.statusText,
      }
    )

3. Then use the new StreamingTextResponse(OpenAiStream(resp)) to create the stream that can be returned from remix-run action.

[lgrammel]

Not sure if this helps, but the AI SDK uses the Node standard fetch.

Have you tried setting the following in vite.config.js?

installGlobals({ nativeFetch: true })

More info: https://remix.run/docs/en/main/guides/single-fetch