how to train data once there is some errors

[njalan]

Can we have a chance to set answer is wrong for training data once there are some errors in the answer.

[zainhoda]

Yes, we should add manual and automated (LLM-based) error correction

[andreped]

Yes, we should add manual and automated (LLM-based) error correction

I think simply passing the error message that is produced when attempting to extract the records to the LLM for a second iteration is a simple solution that should work (together with the rest of the context, current user question and previous sql query as well, of course).

This can be done several times, if of interest, to iteratively converge to the right SQL query. The number of retries could be a parameter that is set through the config.

@zainhoda I can make a PR for this, if the solution sounds sufficient :]

[zainhoda]

Thanks for the offer @andreped! I actually have both of these (manual and automated correction) mostly completed in a local branch and it works quite well. I'm just adding a bunch of options to the built-in UI and I'll include it in the next release

[andreped]

Thanks for the offer @andreped! I actually have both of these (manual and automated correction) mostly completed in a local branch and it works quite well. I'm just adding a bunch of options to the built-in UI and I'll include it in the next release

Beat me to it then 😅 That's great! Looking forward to testing it with the upcoming release!

[njalan]

@zainhoda @andreped Thanks your help. May I know when will be the next release?

[andreped]

@zainhoda Are you able to make this release today? We wanted this feature included before upgrading our Vanna version :]

[andreped]

@zainhoda Where do we allow generate_sql to retry if it fails by passing the error message?

I fail to see where in the code this logic happens. I saw the PR mentioned above and I can see that there is some retry logic for the VannaFlaskApp, but how does this affect me when I am just using Vanna as a library?

I am using vanna==0.3.4 and Azure OpenAI and Chroma clients.

[zainhoda]

@andreped you’re right — right now it’s in the flask app only. Do you have a proposal for where and how this would fit in the base package?