Skip to content

Commit 129b550

Browse files
allisonport-dbTimothyW553
authored andcommitted
[Spark][Build] Fix filtering of internal deps from final spark published jar to account for Spark-versioned builds (delta-io#5542)
<!-- Thanks for sending a pull request! Here are some tips for you: 1. If this is your first time, please read our contributor guidelines: https://github.com/delta-io/delta/blob/master/CONTRIBUTING.md 2. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP] Your PR title ...'. 3. Be sure to keep the PR description updated to reflect all changes. 4. Please write your PR title to summarize what this PR proposes. 5. If possible, provide a concise example to reproduce the issue for a faster review. 6. If applicable, include the corresponding issue number in the PR title and link it in the body. --> #### Which Delta project/connector is this regarding? <!-- Please add the component selected below to the beginning of the pull request title For example: [Spark] Title of my pull request --> - [X] Spark - [ ] Standalone - [ ] Flink - [ ] Kernel - [ ] Other (fill in here) ## Description When trying to compile examples for delta-io#5530 I ran into this issue since we changed the artifact naming for non-default Spark versions. It's hard to test this without updating `examples/build.sbt` which we should do eventually, but for now just make this fix. ## How was this patch tested? Compiled examples locally for delta-io#5530 ## Does this PR introduce _any_ user-facing changes? No
1 parent 3b4cd31 commit 129b550

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

build.sbt

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -586,7 +586,10 @@ lazy val spark = (project in file("spark-unified"))
586586
override def transform(n: Node): Seq[Node] = n match {
587587
case e: Elem if e.label == "dependency" =>
588588
val artifactId = (e \ "artifactId").text
589-
if (internalModules.contains(artifactId)) Seq.empty else Seq(n)
589+
// Check if artifactId starts with any internal module name
590+
// (e.g., "delta-spark-v1_4.1_2.13" starts with "delta-spark-v1")
591+
val isInternal = internalModules.exists(module => artifactId.startsWith(module))
592+
if (isInternal) Seq.empty else Seq(n)
590593
case _ => Seq(n)
591594
}
592595
}).transform(node).head

0 commit comments

Comments
 (0)