Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Renaming JaccardDistance to JaccardSimilarity #80

Merged
merged 4 commits into from
Aug 22, 2018
Merged

Renaming JaccardDistance to JaccardSimilarity #80

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9fdc904
Renaming JaccardDistance to JaccardSimilarity
tillbe Aug 22, 2018
66a957e
Finish refactoring
tillbe Aug 22, 2018
6a9782c
avoiding name conflicts
tillbe Aug 22, 2018
47f097a
Merge branch 'master' into tb/naming-fix
tovbinm Aug 22, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions core/src/main/scala/com/salesforce/op/stages/impl/feature/JaccardSimilarity.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ package com.salesforce.op.stages.impl.feature
import com.salesforce.op.UID
import com.salesforce.op.features.types._
import com.salesforce.op.stages.base.binary.BinaryTransformer
import com.salesforce.op.utils.stats.JaccardDistance
import com.salesforce.op.utils.stats.JaccardSim

/**
* Calculates the Jaccard Similarity between two sets.
Expand All @@ -42,5 +42,5 @@ import com.salesforce.op.utils.stats.JaccardDistance
class JaccardSimilarity(uid: String = UID[JaccardSimilarity])
extends BinaryTransformer[MultiPickList, MultiPickList, RealNN](operationName = "jacSim", uid = uid) {

def transformFn: (MultiPickList, MultiPickList) => RealNN = (x, y) => JaccardDistance(x.value, y.value).toRealNN
def transformFn: (MultiPickList, MultiPickList) => RealNN = (x, y) => JaccardSim(x.value, y.value).toRealNN
}
4 changes: 2 additions & 2 deletions core/src/main/scala/com/salesforce/op/stages/impl/feature/PhoneNumberParser.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ import com.salesforce.op.UID
import com.salesforce.op.features.types._
import com.salesforce.op.stages.base.binary.BinaryTransformer
import com.salesforce.op.stages.base.unary.UnaryTransformer
import com.salesforce.op.utils.stats.JaccardDistance
import com.salesforce.op.utils.stats.JaccardSim
import org.apache.spark.ml.param.{BooleanParam, Param, Params, StringArrayParam}

import scala.util.Try
Expand Down Expand Up @@ -298,7 +298,7 @@ case object PhoneNumberParser {
val rcBi = rc.trim.sliding(2).toSet
regionCodes.zip(countryNames).flatMap {
case (regCode, country) => country.split(",").map{ // Can have multiple versions of country name
c => regCode -> JaccardDistance(rcBi, c.trim.sliding(2).toSet)
c => regCode -> JaccardSim(rcBi, c.trim.sliding(2).toSet)
}}.maxBy(_._2)._1
case _ => defaultRegionCode
}
Expand Down
12 changes: 6 additions & 6 deletions ...orce/op/utils/stats/JaccardDistance.scala → ...alesforce/op/utils/stats/JaccardSim.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,19 @@
package com.salesforce.op.utils.stats

/**
* Calculates the Jaccard Distance between two sets.
* If both inputs are empty, Jaccard Distance is defined as 1.0
* Calculates the Jaccard Similarity between two sets.
* If both inputs are empty, Jaccard Similarity is defined as 1.0
*/
object JaccardDistance {
object JaccardSim {

/**
* Calculates the Jaccard Distance between two sets.
* If both inputs are empty, Jaccard Distance is defined as 1.0
* Calculates the Jaccard Similarity between two sets.
* If both inputs are empty, Jaccard Similarity is defined as 1.0
*
* @param s1 first set
* @param s2 second set
* @tparam A set type
* @return Jaccard Distance
* @return Jaccard Similarity
*/
def apply[A](s1: Set[A], s2: Set[A]): Double = {
val intersectSize = s1.intersect(s2).size
Expand Down