Skip to content

Commit 5dfb1cd

Browse files
zml1206IvanK-db
authored andcommitted
[SPARK-46632][SQL] Fix subexpression elimination when equivalent ternary expressions have different children
### What changes were proposed in this pull request? Remove unexpected exception thrown in `EquivalentExpressions.updateExprInMap()`. Equivalent expressions may contain different children, it should happen expression not in map and `useCount` is -1. For example, before this PR will throw IllegalStateException ``` Seq((1, 2, 3), (2, 3, 4)).toDF("a", "b", "c") .selectExpr("case when a + b + c>3 then 1 when c + a + b>0 then 2 else 0 end as d").show() ``` ### Why are the changes needed? Bug fix. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? New unit test, before this PR will throw IllegalStateException: *** with use count: -1 ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#46135 from zml1206/SPARK-46632. Authored-by: zml1206 <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent a3e307d commit 5dfb1cd

File tree

2 files changed

+12
-4
lines changed

2 files changed

+12
-4
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -80,10 +80,6 @@ class EquivalentExpressions(
8080
case _ =>
8181
if (useCount > 0) {
8282
map.put(wrapper, ExpressionStats(expr)(useCount))
83-
} else {
84-
// Should not happen
85-
throw SparkException.internalError(
86-
s"Cannot update expression: $expr in map: $map with use count: $useCount")
8783
}
8884
false
8985
}

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/SubexpressionEliminationSuite.scala

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -494,6 +494,18 @@ class SubexpressionEliminationSuite extends SparkFunSuite with ExpressionEvalHel
494494
checkShortcut(Or(equal, Literal(true)), 1)
495495
checkShortcut(Not(And(equal, Literal(false))), 1)
496496
}
497+
498+
test("Equivalent ternary expressions have different children") {
499+
val add1 = Add(Add(Literal(1), Literal(2)), Literal(3))
500+
val add2 = Add(Add(Literal(3), Literal(1)), Literal(2))
501+
val conditions1 = (GreaterThan(add1, Literal(3)), Literal(1)) ::
502+
(GreaterThan(add2, Literal(0)), Literal(2)) :: Nil
503+
504+
val caseWhenExpr1 = CaseWhen(conditions1, Literal(0))
505+
val equivalence1 = new EquivalentExpressions
506+
equivalence1.addExprTree(caseWhenExpr1)
507+
assert(equivalence1.getCommonSubexpressions.size == 1)
508+
}
497509
}
498510

499511
case class CodegenFallbackExpression(child: Expression)

0 commit comments

Comments
 (0)