Skip to content

Commit 3329cf6

Browse files
committed
[SPARK-52339][SQL][FOLLOWUP] Sort paths in InMemoryFileIndex#equal only when size matches
### What changes were proposed in this pull request? A follow-up for #51043 that sorts paths in InMemoryFileIndex#equal only when size matches ### Why are the changes needed? Avoid potential perf regression. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing test from #51043 ### Was this patch authored or co-authored using generative AI tooling? No Closes #51263 from yaooqinn/SPARK-52339. Authored-by: Kent Yao <[email protected]> Signed-off-by: Kent Yao <[email protected]> (cherry picked from commit 1cfe07c) Signed-off-by: Kent Yao <[email protected]>
1 parent 6bdef09 commit 3329cf6

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InMemoryFileIndex.scala

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,8 @@ class InMemoryFileIndex(
103103
}
104104

105105
override def equals(other: Any): Boolean = other match {
106-
case hdfs: InMemoryFileIndex => rootPaths.sorted == hdfs.rootPaths.sorted
106+
case hdfs: InMemoryFileIndex if rootPaths.size == hdfs.rootPaths.size =>
107+
rootPaths.sorted == hdfs.rootPaths.sorted
107108
case _ => false
108109
}
109110

0 commit comments

Comments
 (0)