Skip to content

Issues: promptfoo/promptfoo

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

75 Open 386 Closed
75 Open 386 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[Question]Does function have specified names for custom python scripts
#1050 opened Jun 30, 2024 by landhu
1
Allow filtering pre-defined providers (like --filter-pattern but for providers)
#1048 opened Jun 30, 2024 by aantn
Is there a way to run asserts/evaluations dynamically based on an input prompt ? enhancement New feature or request question Further information is requested
#1039 opened Jun 28, 2024 by SysOverdrive
Provider errors from custom python script not displayed in the webui bug Something isn't working
#1037 opened Jun 28, 2024 by spiderbites
@typpo
Allow configuring which UI columns to show from cli enhancement New feature or request
#1032 opened Jun 28, 2024 by aantn
Allow favoriting evals enhancement New feature or request
#1031 opened Jun 28, 2024 by aantn
eval failing with RangeError: Invalid count value: -10 bug Something isn't working
#1028 opened Jun 27, 2024 by bellini666 next
1
How to compare two separate eval runs in the Web UI enhancement New feature or request
#1025 opened Jun 27, 2024 by efung
1
Feature: typescript variables enhancement New feature or request
#1023 opened Jun 27, 2024 by fvdnabee
Support setup/teardown commands for tests enhancement New feature or request
#1022 opened Jun 27, 2024 by aantn
3
Single test suite with multiple conversation test cases enhancement New feature or request
#1020 opened Jun 26, 2024 by tenhsor1
http provider do evaluation, response no output! bug Something isn't working
#1017 opened Jun 26, 2024 by ReverseFelicity next
1
Expose runAssertion for advanced integration enhancement New feature or request
#1014 opened Jun 26, 2024 by mihalik next
4
WebUI Concurrent Editing Issue: Data Overwrites bug Something isn't working
#1013 opened Jun 26, 2024 by pchuri
Proxy Support for Azure with Client credentials enhancement New feature or request
#1012 opened Jun 25, 2024 by chittamitgit
@typpo
1
Feature request: integrate file_ids for evaluating assistants dependencies Pull requests that update a dependency file enhancement New feature or request
#1007 opened Jun 24, 2024 by gonzab
[Web UI] Increase robustness of markdown rendering for tables enhancement New feature or request
#1003 opened Jun 24, 2024 by mannharleen
Enhance UI to group --repeat tests into collapsible sections enhancement New feature or request
#992 opened Jun 22, 2024 by aantn
1
How to calculate something like F1 score? question Further information is requested
#979 opened Jun 20, 2024 by manojlds
1
How to run a test_use_cases.yaml on all the prompts from an input CSV. question Further information is requested
#974 opened Jun 20, 2024 by SysOverdrive
[WebUI] Do not re-render markdown for images in result cells bug Something isn't working
#969 opened Jun 19, 2024 by chrismaltais
1
punycode module is deprecated bug Something isn't working
#961 opened Jun 18, 2024 by chenrui333
1
Is there a way to attach a Debugger to the call_api Python External Provider enhancement New feature or request
#955 opened Jun 18, 2024 by SysOverdrive 0.65.0
2
Avg Latency is not cached enhancement New feature or request
#945 opened Jun 17, 2024 by jay0129
[Feature suggestion]: Exposing model costs for usage with Custom Providers enhancement New feature or request
#940 opened Jun 16, 2024 by Codeshark-NET
3
ProTip! Add no:assignee to see everything that’s not assigned.