I improved the performance of my data visualization Agent using DSPy

[ArslanS1997]

Here is the link to the full post: https://medium.com/firebird-technologies/improving-performance-for-data-visualization-ai-agent-d677ccb71e81

Here is a brief summary (used an LLM to summarize)

Here's a more detailed summary of the article:

Introduction

The article discusses the development of an AI agent for data visualization using Plotly. The agent is designed to take in a user query and generate a Plotly code that produces a visualization of the data.

Agent Development

The agent is developed using a dataset of queries and expected outputs. The dataset is used to train the agent to generate Plotly code that matches the expected output. The agent is evaluated using a scoring metric that checks for the presence of certain attributes in the output code.

Optimization

The agent is optimized using two techniques: COPRO and BootstrapFewShotWithRandomSearch.

• COPRO is a technique that refines the prompt instructions and prefixes to improve the agent's performance. It generates new sets of instructions and improves them step-by-step using a metric function and a training dataset.
• BootstrapFewShotWithRandomSearch is a technique that generates new examples and refines the agent's performance. It uses the LLM to generate examples and evaluates them using the scoring metric.

Results

The optimized agent performs better than the original agent on the evaluation set. The results show that the optimized agent achieves a score of 71% on the evaluation set, while the original agent achieves a score of 60%.

Future Improvements

The article concludes by discussing plans for future improvements to the agent. The author plans to add additional functionality to the agent, such as basic EDA and statistical modeling.

Key Takeaways

• The agent is developed using a dataset of queries and expected outputs.
• The agent is optimized using COPRO and BootstrapFewShotWithRandomSearch.
• The optimized agent performs better than the original agent on the evaluation set.
• The article discusses plans for future improvements to the agent.

Thanks

[tom-doerr]

Maybe it makes sense to add this article to the example section https://github.com/stanfordnlp/dspy?tab=readme-ov-file#c-examples