bug: natural language response hallucination when question returns 0 rows

[mhordynski]

What happened?

When a query returns zero rows, the prompt to NLResponder is formatted incorrectly. As a result, the LLM lacks information about the empty result and tends to hallucinate a response.

How can we reproduce it?

import asyncio
import dbally
import pandas as pd
from dbally import DataFrameBaseView
from dbally.audit import CLIEventHandler
from dbally.views import decorators
from dbally.llms import LiteLLM


class MyView(DataFrameBaseView):

    @decorators.view_filter()
    def filter_by_city(self, city: str):
        return self.df['city'] == city


async def main():
    df = pd.DataFrame({
        "name": ["Alice", "Bob", "Charlie", "David", "Eve"],
        "city": ["New York", "Los Angeles", "Chicago", "Houston", "Phoenix"],
    })

    collection = dbally.create_collection(
        "clients",
        llm=LiteLLM(model_name="gpt-4o"),
        event_handlers=[CLIEventHandler()]
    )
    collection.add(MyView, lambda: MyView(df))

    result = await collection.ask("What clients are from Radom?", return_natural_response=True)

    print(result)


if __name__ == '__main__':
    asyncio.run(main())

Relevant log output

Request starts...
MESSAGE: find all people from NY

=======================================
=======================================

LLM event starts...
LLM EVENT PROMPT TYPE: IQLPromptTemplate
[{'role': 'system', 'content': 'You have access to API that lets you query a database:\n\nfilter_by_city(city: str)\nPlease suggest which one(s) to call and how they
should be joined with logic operators (AND, OR, NOT).\nRemember! Don\'t give any comments, just the function calls.\nThe output will look like this:\nfilter1("arg1") AND
(NOT filter2(120) OR filter3(True))\nDO NOT INCLUDE arguments names in your response. Only the values.\nYou MUST use only these methods:\n\nfilter_by_city(city: str)\nIt
is VERY IMPORTANT not to use methods other than those listed above.If you DON\'T KNOW HOW TO ANSWER DON\'T SAY "", SAY: `UNSUPPORTED QUERY` INSTEAD! This is CRUCIAL,
otherwise the system will crash. '}, {'role': 'user', 'content': 'find all people from NY'}]

RESPONSE: filter_by_city("NY")

=======================================
=======================================

LLM event starts...
LLM EVENT PROMPT TYPE: NLResponderPromptTemplate
[{'role': 'system', 'content': "You are a helpful assistant that helps answer the user's questions based on the table provided. You MUST use the table to answer the
question. You are very intelligent and obedient.\nThe table ALWAYS contains full answer to a question.\nAnswer the question in a way that is easy to understand and
informative.\nDON'T MENTION using a table in your answer."}, {'role': 'user', 'content': 'The table below represents the answer to a question: find all people from
NY.\n\nAnswer the question: find all people from NY.'}]

RESPONSE: The people from NY are:

1. Sarah, Age 28
2.  John, Age 45

=======================================
=======================================

REQUEST OUTPUT:
Number of rows: 0
SELECT clients.id, clients.name, clients.city, clients.date_joined, clients.total_orders
FROM clients
WHERE clients.city = 'NY'