Skip to content

Commit d25f1fa

Browse files
PhilDakingerashegalov
PhilDakin
and
gerashegalov
authored
Bloop updates - require JDK11 in buildall + docs, build bloop for all targets. (NVIDIA#9615)
* Bloop updates - require JDK11 in buildall + docs, build bloop for all targets. Signed-off-by: Phil Dakin <[email protected]> * Address PR comments. Signed-off-by: Phil Dakin <[email protected]> * Add a bloopInstall profile (#1) * Add bloopInstall profile Signed-off-by: Gera Shegalov <[email protected]> * Add bloopInstall profile Fixes NVIDIA#9621 Signed-off-by: Gera Shegalov <[email protected]> * add comment for install in scala2.13 Signed-off-by: Gera Shegalov <[email protected]> * simplify Signed-off-by: Gera Shegalov <[email protected]> * undo unrelated Signed-off-by: Gera Shegalov <[email protected]> * Apply suggestions from code review Co-authored-by: PhilDakin <[email protected]> * review 1 Signed-off-by: Gera Shegalov <[email protected]> * Configurable bloopInstall phase Signed-off-by: Gera Shegalov <[email protected]> * move bloop config dir to execution conf Signed-off-by: Gera Shegalov <[email protected]> --------- Signed-off-by: Gera Shegalov <[email protected]> Co-authored-by: PhilDakin <[email protected]> * Propagate bloopInstall profile changes, make config directory overridable from command line. Signed-off-by: Phil Dakin <[email protected]> * Cleanup. Signed-off-by: Phil Dakin <[email protected]> --------- Signed-off-by: Phil Dakin <[email protected]> Signed-off-by: Gera Shegalov <[email protected]> Co-authored-by: Gera Shegalov <[email protected]>
1 parent d6d9d5a commit d25f1fa

File tree

4 files changed

+156
-26
lines changed

4 files changed

+156
-26
lines changed

‎CONTRIBUTING.md‎

Lines changed: 14 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -353,13 +353,10 @@ interested in. For example, to generate the Bloop projects for the Spark 3.2.0 d
353353
just for the production code run:
354354
355355
```shell script
356-
mvn install ch.epfl.scala:bloop-maven-plugin:bloopInstall -pl aggregator -am \
357-
-DdownloadSources=true \
358-
-Dbuildver=320 \
359-
-DskipTests \
360-
-Dskip \
361-
-Dmaven.scalastyle.skip=true \
362-
-Dmaven.updateconfig.skip=true
356+
mvn -B clean install \
357+
-DbloopInstall \
358+
-DdownloadSources=true \
359+
-Dbuildver=320
363360
```
364361
365362
With `--generate-bloop` we integrated Bloop project generation into `buildall`. It makes it easier
@@ -384,6 +381,11 @@ You can now open the spark-rapids as a
384381
385382
Read on for VS Code Scala Metals instructions.
386383
384+
##### JDK 11 Requirement
385+
386+
It is known that Bloop's SemanticDB generation with JDK 8 is broken for spark-rapids. Please use JDK
387+
11 or later for Bloop builds.
388+
387389
#### Bloop, Scala Metals, and Visual Studio Code
388390
389391
_Last tested with 1.63.0-insider (Universal) Commit: bedf867b5b02c1c800fbaf4d6ce09cefba_
@@ -431,18 +433,11 @@ jps -l
431433
###### java.lang.RuntimeException: boom
432434
433435
Metals background compilation process status appears to be resetting to 0% after reaching 99%
434-
and you see a peculiar error message [`java.lang.RuntimeException: boom`][1]. You can work around
435-
it by making sure Metals Server (Bloop client) and Bloop Server are both running on Java 11+.
436-
437-
1. To this end make sure that Bloop projects are generated using Java 11+
438-
439-
```bash
440-
JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64 \
441-
mvn install ch.epfl.scala:bloop-maven-plugin:bloopInstall \
442-
-DdownloadSources=true \
443-
-Dbuildver=331 \
444-
-Dskip -DskipTests
445-
```
436+
and you see a peculiar error message [`java.lang.RuntimeException: boom`][1]. This is a known issue
437+
when running Metals/Bloop on Java 8. To work around it, ensure Metals and Bloop are both running on
438+
Java 11+.
439+
440+
1. The `-DbloopInstall` profile will enforce Java 11+ compliance.
446441
447442
1. Add [`metals.javaHome`][2] to VSCode preferences to point to Java 11+.
448443

‎build/buildall‎

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -65,15 +65,10 @@ function bloopInstall() {
6565
bloopTmpConfigDir="$bloopTmpDir/.bloop$bv"
6666
mkdir -p $bloopTmpConfigDir
6767
$MVN -B clean install \
68-
ch.epfl.scala:bloop-maven-plugin:bloopInstall \
69-
-pl aggregator -am \
68+
-DbloopInstall \
7069
-Dbloop.configDirectory="$bloopTmpConfigDir" \
7170
-DdownloadSources=true \
72-
-Dbuildver="$bv" \
73-
-DskipTests \
74-
-Dskip \
75-
-Dmaven.scalastyle.skip=true \
76-
-Dmaven.updateconfig.skip=true
71+
-Dbuildver="$bv"
7772

7873
specifier="spark$bv"
7974
bloopDir=$PWD/.bloop-$specifier

‎pom.xml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -569,6 +569,57 @@
569569
<jni.classifier>${cuda.version}-arm64</jni.classifier>
570570
</properties>
571571
</profile>
572+
<profile>
573+
<id>bloopInstall</id>
574+
<activation>
575+
<property>
576+
<name>bloopInstall</name>
577+
<value>true</value>
578+
</property>
579+
</activation>
580+
<properties>
581+
<skipTests>true</skipTests>
582+
<maven.scaladoc.skip>true</maven.scaladoc.skip>
583+
<maven.scalastyle.skip>true</maven.scalastyle.skip>
584+
</properties>
585+
<build>
586+
<plugins>
587+
<plugin>
588+
<groupId>org.apache.maven.plugins</groupId>
589+
<artifactId>maven-enforcer-plugin</artifactId>
590+
<executions>
591+
<execution>
592+
<id>enforce-bloop-rules</id>
593+
<goals><goal>enforce</goal></goals>
594+
<configuration>
595+
<rules>
596+
<requireJavaVersion>
597+
<message>Metals semantic database requires JAVA_HOME pointing to JDK 11+, actual Java version: ${java.version}</message>
598+
<version>[11,)</version>
599+
</requireJavaVersion>
600+
</rules>
601+
</configuration>
602+
</execution>
603+
</executions>
604+
</plugin>
605+
<plugin>
606+
<groupId>ch.epfl.scala</groupId>
607+
<artifactId>bloop-maven-plugin</artifactId>
608+
<executions>
609+
<execution>
610+
<id>generate-bloop-projects</id>
611+
<goals><goal>bloopInstall</goal></goals>
612+
<phase>${bloop.installPhase}</phase>
613+
<configuration>
614+
<!-- Metals looks at the repo root. so to accomodate scala 2.13 define -->
615+
<bloopConfigDir>${bloop.configDirectory}</bloopConfigDir>
616+
</configuration>
617+
</execution>
618+
</executions>
619+
</plugin>
620+
</plugins>
621+
</build>
622+
</profile>
572623
</profiles>
573624

574625
<properties>
@@ -655,6 +706,7 @@
655706
<scala.local-lib.path>org/scala-lang/scala-library/${scala.version}/scala-library-${scala.version}.jar</scala.local-lib.path>
656707
<target.classifier>${spark.version.classifier}</target.classifier>
657708
<maven.clean.plugin.version>3.1.0</maven.clean.plugin.version>
709+
<maven.scaladoc.skip>false</maven.scaladoc.skip>
658710
<maven.scalastyle.skip>false</maven.scalastyle.skip>
659711
<dist.jar.compress>true</dist.jar.compress>
660712
<spark330.iceberg.version>0.14.1</spark330.iceberg.version>
@@ -765,6 +817,8 @@
765817
-Djdk.reflect.useDirectMethodHandle=false
766818
</extraJavaTestArgs>
767819
<cloudera.repo.enabled>false</cloudera.repo.enabled>
820+
<bloop.installPhase>install</bloop.installPhase>
821+
<bloop.configDirectory>${spark.rapids.source.basedir}/.bloop</bloop.configDirectory>
768822
</properties>
769823

770824
<dependencyManagement>
@@ -1043,6 +1097,7 @@
10431097
<arg>-doc-external-doc:${settings.localRepository}/${scala.local-lib.path}#https://scala-lang.org/api/${scala.version}/</arg>
10441098
<arg>-doc-external-doc:${settings.localRepository}/org/apache/spark/spark-sql_${scala.binary.version}/${spark.version}/spark-sql_${scala.binary.version}-${spark.version}.jar#https://spark.apache.org/docs/${spark.version}/api/scala/index.html</arg>
10451099
</args>
1100+
<skip>${maven.scaladoc.skip}</skip>
10461101
</configuration>
10471102
</execution>
10481103
</executions>
@@ -1212,6 +1267,21 @@
12121267
</pluginManagement>
12131268

12141269
<plugins>
1270+
<plugin>
1271+
<groupId>ch.epfl.scala</groupId>
1272+
<artifactId>bloop-maven-plugin</artifactId>
1273+
<version>2.0.0</version>
1274+
<executions>
1275+
<execution>
1276+
<id>default-cli</id>
1277+
<configuration>
1278+
<skip>true</skip>
1279+
<!-- workaround: skip is not skipping -->
1280+
<bloopConfigDir>/dev/null/ERROR: Do not specify the bloop-maven-plugin on the command line. Instead invoke `mvn install -DbloopInstall ...`</bloopConfigDir>
1281+
</configuration>
1282+
</execution>
1283+
</executions>
1284+
</plugin>
12151285
<plugin>
12161286
<groupId>org.apache.maven.plugins</groupId>
12171287
<artifactId>maven-enforcer-plugin</artifactId>

‎scala2.13/pom.xml‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -569,6 +569,57 @@
569569
<jni.classifier>${cuda.version}-arm64</jni.classifier>
570570
</properties>
571571
</profile>
572+
<profile>
573+
<id>bloopInstall</id>
574+
<activation>
575+
<property>
576+
<name>bloopInstall</name>
577+
<value>true</value>
578+
</property>
579+
</activation>
580+
<properties>
581+
<skipTests>true</skipTests>
582+
<maven.scaladoc.skip>true</maven.scaladoc.skip>
583+
<maven.scalastyle.skip>true</maven.scalastyle.skip>
584+
</properties>
585+
<build>
586+
<plugins>
587+
<plugin>
588+
<groupId>org.apache.maven.plugins</groupId>
589+
<artifactId>maven-enforcer-plugin</artifactId>
590+
<executions>
591+
<execution>
592+
<id>enforce-bloop-rules</id>
593+
<goals><goal>enforce</goal></goals>
594+
<configuration>
595+
<rules>
596+
<requireJavaVersion>
597+
<message>Metals semantic database requires JAVA_HOME pointing to JDK 11+, actual Java version: ${java.version}</message>
598+
<version>[11,)</version>
599+
</requireJavaVersion>
600+
</rules>
601+
</configuration>
602+
</execution>
603+
</executions>
604+
</plugin>
605+
<plugin>
606+
<groupId>ch.epfl.scala</groupId>
607+
<artifactId>bloop-maven-plugin</artifactId>
608+
<executions>
609+
<execution>
610+
<id>generate-bloop-projects</id>
611+
<goals><goal>bloopInstall</goal></goals>
612+
<phase>${bloop.installPhase}</phase>
613+
<configuration>
614+
<!-- Metals looks at the repo root. so to accomodate scala 2.13 define -->
615+
<bloopConfigDir>${bloop.configDirectory}</bloopConfigDir>
616+
</configuration>
617+
</execution>
618+
</executions>
619+
</plugin>
620+
</plugins>
621+
</build>
622+
</profile>
572623
</profiles>
573624

574625
<properties>
@@ -655,6 +706,7 @@
655706
<scala.local-lib.path>org/scala-lang/scala-library/${scala.version}/scala-library-${scala.version}.jar</scala.local-lib.path>
656707
<target.classifier>${spark.version.classifier}</target.classifier>
657708
<maven.clean.plugin.version>3.1.0</maven.clean.plugin.version>
709+
<maven.scaladoc.skip>false</maven.scaladoc.skip>
658710
<maven.scalastyle.skip>false</maven.scalastyle.skip>
659711
<dist.jar.compress>true</dist.jar.compress>
660712
<spark330.iceberg.version>0.14.1</spark330.iceberg.version>
@@ -765,6 +817,8 @@
765817
-Djdk.reflect.useDirectMethodHandle=false
766818
</extraJavaTestArgs>
767819
<cloudera.repo.enabled>false</cloudera.repo.enabled>
820+
<bloop.installPhase>install</bloop.installPhase>
821+
<bloop.configDirectory>${spark.rapids.source.basedir}/.bloop</bloop.configDirectory>
768822
</properties>
769823

770824
<dependencyManagement>
@@ -1043,6 +1097,7 @@
10431097
<arg>-doc-external-doc:${settings.localRepository}/${scala.local-lib.path}#https://scala-lang.org/api/${scala.version}/</arg>
10441098
<arg>-doc-external-doc:${settings.localRepository}/org/apache/spark/spark-sql_${scala.binary.version}/${spark.version}/spark-sql_${scala.binary.version}-${spark.version}.jar#https://spark.apache.org/docs/${spark.version}/api/scala/index.html</arg>
10451099
</args>
1100+
<skip>${maven.scaladoc.skip}</skip>
10461101
</configuration>
10471102
</execution>
10481103
</executions>
@@ -1212,6 +1267,21 @@
12121267
</pluginManagement>
12131268

12141269
<plugins>
1270+
<plugin>
1271+
<groupId>ch.epfl.scala</groupId>
1272+
<artifactId>bloop-maven-plugin</artifactId>
1273+
<version>2.0.0</version>
1274+
<executions>
1275+
<execution>
1276+
<id>default-cli</id>
1277+
<configuration>
1278+
<skip>true</skip>
1279+
<!-- workaround: skip is not skipping -->
1280+
<bloopConfigDir>/dev/null/ERROR: Do not specify the bloop-maven-plugin on the command line. Instead invoke `mvn install -DbloopInstall ...`</bloopConfigDir>
1281+
</configuration>
1282+
</execution>
1283+
</executions>
1284+
</plugin>
12151285
<plugin>
12161286
<groupId>org.apache.maven.plugins</groupId>
12171287
<artifactId>maven-enforcer-plugin</artifactId>

0 commit comments

Comments
 (0)