Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding binaryclassification bin score evaluator #119

Merged
merged 20 commits into from
Sep 14, 2018
Merged

Adding binaryclassification bin score evaluator #119

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
2d9cb1d
creating binaryclassification bin score evaluator
gokulsfdc Sep 7, 2018
9658eb4
minor changes
gokulsfdc Sep 7, 2018
56d6280
Doc Changes
gokulsfdc Sep 7, 2018
0eebe3d
Fixing code factor issue
gokulsfdc Sep 7, 2018
41123d5
Incorporating code review comments
gokulsfdc Sep 8, 2018
a16fc86
code review comments
gokulsfdc Sep 9, 2018
ee8924f
removing partitioner
gokulsfdc Sep 9, 2018
919b933
code review comments
gokulsfdc Sep 10, 2018
b421d33
fixing testbuilder for tests
gokulsfdc Sep 10, 2018
b5a70ee
Merge branch 'master' into op/bin_evaluator
leahmcguire Sep 11, 2018
9e613ed
Merge branch 'master' into op/bin_evaluator
tovbinm Sep 12, 2018
728aa99
computing stats efficiently
gokulsfdc Sep 11, 2018
db653bf
unfolding the tuples
gokulsfdc Sep 12, 2018
6ee1a82
code review comments
gokulsfdc Sep 12, 2018
4b32250
Fixing the stats when score goes out of bound
gokulsfdc Sep 12, 2018
483c937
Merge branch 'master' of github.com:salesforce/TransmogrifAI into op/…
gokulsfdc Sep 13, 2018
d928ba7
fixing an issue where prediction column is missing
gokulsfdc Sep 13, 2018
01c89eb
Merge branch 'master' into op/bin_evaluator
tovbinm Sep 13, 2018
b36ec1d
finding max and min using foldleft
gokulsfdc Sep 14, 2018
01250d2
Finding min max using map-fold
gokulsfdc Sep 14, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
code review comments
  • Loading branch information
@gokulsfdc
gokulsfdc committed Sep 12, 2018
commit 6ee1a82a8d97ad41e182b07ee72c4de83a165d20
10 changes: 4 additions & 6 deletions core/src/test/scala/com/salesforce/op/evaluators/OpBinScoreEvaluatorTest.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ class OpBinScoreEvaluatorTest extends FlatSpec with TestSparkContext {
)
)

val (dataset_skewed, prediction_skewed, label_skewed) = TestFeatureBuilder(
val (dataSkewed, predictionSkewedData, labelSkewedData) = TestFeatureBuilder(
Seq (
Prediction(1.0, Array(10.0, 10.0), Array(0.0001, 0.99999)) -> 1.0.toRealNN,
Prediction(1.0, Array(10.0, 10.0), Array(0.0001, 0.99999)) -> 1.0.toRealNN,
Expand All @@ -59,9 +59,7 @@ class OpBinScoreEvaluatorTest extends FlatSpec with TestSparkContext {
)
)

val (emptyDataSet, prediction_emptydataset, label_empty_dataset) = TestFeatureBuilder[Prediction, RealNN](
Seq()
)
val (emptyData, predictionEmptyData, labelEmptyData) = TestFeatureBuilder[Prediction, RealNN](Seq())

Spec[OpBinScoreEvaluator] should "return the bin metrics" in {
val metrics = new OpBinScoreEvaluator(numBins = 4)
Expand All @@ -84,14 +82,14 @@ class OpBinScoreEvaluatorTest extends FlatSpec with TestSparkContext {

it should "evaluate the empty data" in {
val metrics = new OpBinScoreEvaluator(numBins = 10)
.setLabelCol(label_empty_dataset.name).setPredictionCol(prediction_emptydataset.name).evaluateAll(emptyDataSet)
.setLabelCol(labelEmptyData.name).setPredictionCol(predictionEmptyData.name).evaluateAll(emptyData)

metrics shouldBe BinaryClassificationBinMetrics(0.0, Seq(), Seq(), Seq(), Seq())
}

it should "evaluate bin metrics for skewed data" in {
val metrics = new OpBinScoreEvaluator(numBins = 5)
.setLabelCol(label_skewed.name).setPredictionCol(prediction_skewed.name).evaluateAll(dataset_skewed)
.setLabelCol(labelSkewedData.name).setPredictionCol(predictionSkewedData.name).evaluateAll(dataSkewed)

metrics shouldBe BinaryClassificationBinMetrics(
7.294225500000013E-4,
Expand Down