Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eval dataset should not be balanced #429

Merged
merged 14 commits into from
Oct 21, 2019

Conversation

AdamChit
Copy link
Collaborator

Related issues
validationPrepare was being called on the validation datasets which will balance the dataset. This should not be done because the validation set should represent the real distribution.

Describe the proposed solution
remove the call to validationPrepare

@AdamChit AdamChit changed the title Ac/unblanace test set eval dataset should not be balanced Oct 18, 2019
@AdamChit AdamChit changed the title eval dataset should not be balanced Eval dataset should not be balanced Oct 18, 2019
@codecov
Copy link

codecov bot commented Oct 18, 2019

Codecov Report

Merging #429 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #429      +/-   ##
==========================================
- Coverage   86.97%   86.96%   -0.02%     
==========================================
  Files         337      337              
  Lines       11082    11082              
  Branches      355      588     +233     
==========================================
- Hits         9639     9637       -2     
- Misses       1443     1445       +2
Impacted Files Coverage Δ
...op/stages/impl/tuning/OpTrainValidationSplit.scala 100% <ø> (ø) ⬆️
...orce/op/stages/impl/tuning/OpCrossValidation.scala 97.95% <ø> (ø) ⬆️
...es/src/main/scala/com/salesforce/op/OpParams.scala 85.71% <0%> (-4.09%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8624922...f52c945. Read the comment docs.

@tovbinm
Copy link
Collaborator

tovbinm commented Oct 19, 2019

@leahmcguire @AdamChit didn't we just change it in this PR - #424 ?

@leahmcguire
Copy link
Collaborator

Yes @tovbinm I was too quick to make the change :-)

@tovbinm tovbinm merged commit 7ff9b5d into salesforce:master Oct 21, 2019
@gerashegalov
Copy link
Contributor

@AdamChit @leahmcguire Can we capture the expected behavior in a test to reduce the risk of merging "too quick"?

@nicodv nicodv mentioned this pull request Jun 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants