Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report dollars spent on OpenAI #52

Merged
merged 3 commits into from
Oct 9, 2023

Conversation

kierangilliam
Copy link
Contributor

The user does not have visibility into the cost of each session. This PR adds a report_metrics callback for environments to display useful metrics to the user.

This has been implemented for the TaskSolving environment. Try "brainstorming" to see the result.

$ python main_tasksolving_cli.py --task tasksolving/brainstorming
...
Agent spend:  
Agent (Role: AGENT_TYPES.ROLE_ASSIGNMENT) role assigner: $0.001631  
Agent (Role: AGENT_TYPES.SOLVER) A mechanical engineer specialized in hydrogen storage technologies: $0.01  
Agent (Role: AGENT_TYPES.CRITIC) An electrical engineer with expertise in renewable energy integration: $0.002690  
Agent (Role: AGENT_TYPES.CRITIC) A regulatory specialist familiar with hydrogen storage regulations: $0.002037  
Agent (Role: AGENT_TYPES.CRITIC) An infrastructure planner with experience in large-scale projects: $0.002518  
Agent (Role: AGENT_TYPES.EXECUTION) Dummy Executor: $0.000000  
Agent (Role: AGENT_TYPES.EVALUATION) Evaluator: $0.003592  
Total spent: $0.018993 

@kierangilliam kierangilliam marked this pull request as draft October 8, 2023 16:19
@kierangilliam kierangilliam marked this pull request as ready for review October 8, 2023 16:23
@chenweize1998
Copy link
Collaborator

Awesome! Thanks for your help

@chenweize1998 chenweize1998 merged commit abd444e into OpenBMB:main Oct 9, 2023
0 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants