Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion: Add training_data metrics in cv return #1105

Closed
CliveHugo opened this issue Dec 6, 2017 · 3 comments
Closed

Suggestion: Add training_data metrics in cv return #1105

CliveHugo opened this issue Dec 6, 2017 · 3 comments

Comments

@CliveHugo
Copy link

CliveHugo commented Dec 6, 2017

Environment info

Operating System: WIN10
CPU:
C++/Python/R version: python3.6

a suggestion :
Add training_data performance(metrics) in cv return for drawing validation curve (just as Xgboost.cv's return) since skleran.validation_curve is very inefficient (run n time , n =number of iteration). Don't know if anyone else has this need though.


Example from return of Xgboost.cv (after` pandas.from_dict):

Out[23]:
test-auc-mean test-auc-std train-auc-mean train-auc-std
0 0.847553 0.011909 0.849978 0.000601
1 0.852323 0.012572 0.856189 0.000791
2 0.856945 0.011464 0.863983 0.000770
3 0.857423 0.011400 0.869057 0.000768
4 0.857955 0.010694 0.872355 0.000579
5 0.858113 0.011444 0.874590 0.000896
6 0.857701 0.011529 0.876404 0.000870
`

@jcrosskey
Copy link

This would be a nice feature to have, it'll help monitor whether overfit occurred judging from cv.

@wjmj
Copy link

wjmj commented Apr 1, 2018

It will be a very useful feature, I hope it will be implemented as soon as possible.

@StrikerRUS
Copy link
Collaborator

Closed via #2089.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants