SPARK-2749 [BUILD]. Spark SQL Java tests aren't compiling in Jenkins'…

… Maven builds; missing junit:junit dep

The Maven-based builds in the build matrix have been failing for a few days:

https://amplab.cs.berkeley.edu/jenkins/view/Spark/

On inspection, it looks like the Spark SQL Java tests don't compile:

https://amplab.cs.berkeley.edu/jenkins/view/Spark/job/Spark-Master-Maven-pre-YARN/hadoop.version=1.0.4,label=centos/244/consoleFull

I confirmed it by repeating the command vs master:

`mvn -Dhadoop.version=1.0.4 -Dlabel=centos -DskipTests clean package`

The problem is that this module doesn't depend on JUnit. In fact, none of the modules do, but `com.novocode:junit-interface` (the SBT-JUnit bridge) pulls it in, in most places. However this module doesn't depend on `com.novocode:junit-interface`

Adding the `junit:junit` dependency fixes the compile problem. In fact, the other modules with Java tests should probably depend on it explicitly instead of happening to get it via `com.novocode:junit-interface`, since that is a bit SBT/Scala-specific (and I am not even sure it's needed).

Author: Sean Owen <[email protected]>

Closes apache#1660 from srowen/SPARK-2749 and squashes the following commits:

858ff7c [Sean Owen] Add explicit junit dep to other modules with Java tests for robustness
9636794 [Sean Owen] Add junit dep so that Spark SQL Java tests compile

core/pom.xml

@@ -262,6 +262,11 @@
  <artifactId>asm</artifactId>
  <scope>test</scope>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>

external/flume/pom.xml

@@ -72,6 +72,11 @@
  <artifactId>scalacheck_${scala.binary.version}</artifactId>
  <scope>test</scope>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>

external/kafka/pom.xml

@@ -80,6 +80,11 @@
  <artifactId>scalacheck_${scala.binary.version}</artifactId>
  <scope>test</scope>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>

external/mqtt/pom.xml

@@ -67,6 +67,11 @@
  <artifactId>scalacheck_${scala.binary.version}</artifactId>
  <scope>test</scope>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>

external/twitter/pom.xml

@@ -62,6 +62,11 @@
  <artifactId>scalacheck_${scala.binary.version}</artifactId>
  <scope>test</scope>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>

external/zeromq/pom.xml

@@ -62,6 +62,11 @@
  <artifactId>scalacheck_${scala.binary.version}</artifactId>
  <scope>test</scope>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>

extras/java8-tests/pom.xml

@@ -50,6 +50,11 @@
  <version>${project.version}</version>
  <type>test-jar</type>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>

mllib/pom.xml

@@ -72,6 +72,11 @@
  <artifactId>scalacheck_${scala.binary.version}</artifactId>
  <scope>test</scope>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>

sql/core/pom.xml

@@ -68,6 +68,11 @@
  <artifactId>jackson-databind</artifactId>
  <version>2.3.0</version>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>org.scalatest</groupId>
  <artifactId>scalatest_${scala.binary.version}</artifactId>

streaming/pom.xml

@@ -58,6 +58,11 @@
  <artifactId>scalacheck_${scala.binary.version}</artifactId>
  <scope>test</scope>
  </dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
  <dependency>
  <groupId>com.novocode</groupId>
  <artifactId>junit-interface</artifactId>