Streaming not working.

[tetaplaun]

Hi,

using Nextjs 14.0.2 with the latest version of the Vercel AI SDK. Streaming is not working, expect locally.

My code is basically the same as the official example. On my local system, everything works fined, in development and production mode.

If I access the site from a different server on the same network, streaming doesn't work. In the Chrome Dev Tools, Network tab, the requests to the API endpoint /api/chat stay on "Pending", until the complete response is ready, then is updated on the website.

I tried adding

export const maxDuration = 30
export const dynamic = "force-dynamic"

almost everywhere (layout.tsx, page.tsx, route.tsx), but this doesn't change anything.

Any further suggestions I could try?

Greetings,
Chris

[lgrammel]

please check out https://sdk.vercel.ai/docs/troubleshooting/common-issues/streaming-not-working-on-vercel

[tetaplaun]

After coding the openAI and Gemini hooks, I decided to also make a hook for Claude. Important note: The openAI and Gemini hook speak to the API directly. For the Claude hook I had to use an API route like Vercel AI SDK, because of their wired CORS politics. And guess what? Same problem as with the Vercel AI SDK. Locally the streams works, when accessing the page from a client computer, streaming breaks and only the final result gets submitted.
I have to go back on step, the Vercel AI SDK is NOT the issue, but Next.js. Of course, I added the magic export const dynamic = "force-dynamic" line to the route.
I use a pretty basic Next.js setup, didn't change anything fundamental, next.config.js is empty.
If anyone more involved in Nextjs is read this, who can I address this issue/question to? @lgrammel
BTW: Happy to share the code, for anyone interested.

[lgrammel]

@tetaplaun thanks for investigating! It would be super helpful if you could share your code so I can try it out myself

[tetaplaun]

Hook
Api Route

Just as a note again, I'm running this project self-hosted on node 18.17.0

[tetaplaun]

After further testing, I have determined that the streaming issue is likely due to a problem with the client’s Windows configuration, possibly related to CORS. I will need to consult with our server department to investigate this further. Using a different client, such as an iPad or iPhone on our local network, the streaming works fine. Initially, I ruled out this possibility, which was my mistake. I am closing this issue for now. If I discover the cause, I will update this post.

[lgrammel]

Thanks for further investigating. It's good to know that the problem is in the network setup and not in the libraries.

[tetaplaun]

And one final note: I build my custom hooks for openai and gemini, which stream just perfect. There must be some issue with vercels ai sdk, that's causing the problem. Also, I'm not the only one, as a quick google search reveals.
I hope you can find and fix the bug one day. The vercel ai sdk is, if it works, an easy and welcome solution!