Add support for HF summarization endpoint in the websearch

[nsarrazin]

If the user has set an HF_ACCESS_TOKEN we use it to call up an inference endpoint trained for summarization. If the user didn't set their token, we use their LLM endpoint (could be self-hosted w/ no HF token) to try and make a summary the old way.

In my local testing this returns more accurate & faster summaries and it would also help reduce the load on the LLM endpoint. (summarization takes a huge context window which is much larger than most conversations)

I'm not sure if the model I chose is optimal as there are multiple models that support summarization. I'm also not sure if we could get rate limited by the API since the calls for all users would be coming from one server using the prod HF token.

[gary149]

Cool, just 2 things:

• Is this only in English? I'd rather do something that supports multiple languages - using a different model, but I'll let you benchmark.
• Would it be better to fall back on the actual method if there's an error on inference?

[nsarrazin]

So I tested the multi-language stuff, and even if you ask a question in another language, the LLM often generates a search query in English, and the serpapi settings are configured to fetch results from the US so it's unlikely we'll fetch results that are not in English atm. Still, I tried the model and while it works sometimes, most of the time it generates one-sentence summary that are too short to be useful imo

But good catch on the fallback, I added the feature!

[gary149]

Ok feel free to merge then :)