Skip to content

Commit

Permalink
[SPARK-35672][FOLLOWUP][TESTS] Add more exclusion rules to MimaExclud…
Browse files Browse the repository at this point in the history
…es.scala for Scala 2.13

### What changes were proposed in this pull request?

This PR adds more MiMa exclusion rules for Scala 2.13.
apache#34649 partially resolved the compatibility issue but additional 3 compatibility problems are raised.

```
$ build/sbt clean
$ dev/change-scala-version.sh 2.13
$ build/sbt -Pscala-2.13 clean
$ dev/mima

...
[error] spark-core: Failed binary compatibility check against org.apache.spark:spark-core_2.13:3.2.0! Found 3 potential problems (filtered 910)
[error]  * synthetic method copy$default$8()scala.collection.mutable.ListBuffer in class org.apache.spark.executor.CoarseGrainedExecutorBackend#Arguments has a different result type in current version, where it is scala.Option rather than scala.collection.mutable.ListBuffer
[error]    filter with: ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.executor.CoarseGrainedExecutorBackend#Arguments.copy$default$8")
[error]  * synthetic method copy$default$9()scala.Option in class org.apache.spark.executor.CoarseGrainedExecutorBackend#Arguments has a different result type in current version, where it is Int rather than scala.Option
[error]    filter with: ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.executor.CoarseGrainedExecutorBackend#Arguments.copy$default$9")
[error]  * the type hierarchy of object org.apache.spark.executor.CoarseGrainedExecutorBackend#Arguments is different in current version. Missing types {scala.runtime.AbstractFunction10}
[error]    filter with: ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.executor.CoarseGrainedExecutorBackend$Arguments$")
...
```

### Why are the changes needed?

To keep the build stable.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Confirmed MiMa passed.
```
$ build/sbt clean
$ dev/change-scala-version.sh 2.13
$ build/sbt -Pscala-2.13 clean
$ dev/mima

Closes apache#34664 from sarutak/followup-SPARK-35672-mima-take2.

Authored-by: Kousuke Saruta <[email protected]>
Signed-off-by: Kousuke Saruta <[email protected]>
  • Loading branch information
sarutak committed Nov 19, 2021
1 parent 227715e commit 4f20898
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion project/MimaExcludes.scala
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,10 @@ object MimaExcludes {
// Exclude rules for 3.3.x from 3.2.0
lazy val v33excludes = v32excludes ++ Seq(
// [SPARK-35672][CORE][YARN] Pass user classpath entries to executors using config instead of command line
// This is necessary for Scala 2.13.
// The followings are necessary for Scala 2.13.
ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.executor.CoarseGrainedExecutorBackend#Arguments.*"),
ProblemFilters.exclude[IncompatibleResultTypeProblem]("org.apache.spark.executor.CoarseGrainedExecutorBackend#Arguments.*"),
ProblemFilters.exclude[MissingTypesProblem]("org.apache.spark.executor.CoarseGrainedExecutorBackend$Arguments$")
)

// Exclude rules for 3.2.x from 3.1.1
Expand Down

0 comments on commit 4f20898

Please sign in to comment.