[SPARK-52575][SQL] Introduce contextIndependentFoldable attribute for Expressions

[gengliangwang]

What changes were proposed in this pull request?

Introduces a method to determine whether an expression can be folded without relying on any external context (e.g., time zone, session configurations, or catalogs). If an expression is context-independent foldable, it can be safely evaluated during DDL operations such as creating tables, views, or constraints. This enables systems to store the computed value rather than the original expression, simplifying implementation and improving performance. By default, expressions are not considered context-independent foldable to ensure explicit annotation of context independence.

Examples of context-independent foldable:

• 1 + 1: Always produces 2, regardless of context.
• 1 > 0: Always evaluates to true.
• CAST('2025-08-01' AS DATE): Always gives the same date value.

Examples of not context-independent foldable:

• CURRENT_DATE: Result depends on the current date and time zone.
• CURRENT_USER: Result depends on the session/user context.
• CAST('2025-08-01 00:00:00' AS TIMESTAMP): Result can vary depending on the session time zone setting

Why are the changes needed?

This allows catalogs and connectors to store the computed value rather than the expression itself, improving both simplicity and performance.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Unit tests

Was this patch authored or co-authored using generative AI tooling?

No

[gengliangwang]

I will create a follow-up to use this new method in the V2 expression conversion

 class V2ExpressionBuilder(e: Expression, isPredicate: Boolean = false) extends Logging  {
 
-  def build(): Option[V2Expression] = generateExpression(e, isPredicate)
+  def build(): Option[V2Expression] = {
+    val exprToBuild = if (e.foldable && e.contextIndependentFoldable) {
+      ConstantFolding.constantFolding(e)
+    } else {
+      e
+    }
+    generateExpression(exprToBuild, isPredicate)
+  }

cc @cloud-fan @beliefer @aokolnychyi

[aokolnychyi]

This would be super helpful to simplify expressions before converting them to DSv2. I only have a question about marking expressions context-free by default for all unary and binary expressions.

[aokolnychyi]

Technically, we could consider splitting contextIndependent and foldable into separate methods. Are there use cases where we want to know if the expression is context independent without checking it is foldable?

[gengliangwang]

I can't think of such special cases. I think we can always check foldable before checking contextIndependentFoldable, which seems very safe to me. The syntax for this method is when it is foldable, does it depend on context?

[beliefer]

I guess some unfoldable expression could be context independent. Such as: 1 + a', the type of attribute a' is int.
But I don't know should we consider these cases.
I'm +1 for @aokolnychyi if we must consider the case I mentioned.
I'm +1 for @gengliangwang if we not.

[gengliangwang]

Such as: 1 + a', the type of attribute a' is int

@beliefer in such a case, column a' is not contextIndependent

[beliefer]

I means the context that is related to the Spark context or Session context. The value of attribute is not.

[beliefer]

Or the time zone etc.

[beliefer]

The key is how to define the context ?

[gengliangwang]

Context-independent means that the expression is independent of contexts(current user, current database, current time, etc) when it is foldable.
For such expressions, we can evaluate them before storing in column default values, table constraints, etc.

[cloud-fan]

This is confusing. How can a data type have context dependency? It's a data type, not an operation.

[gengliangwang]

ok, I renamed the method to containsTimestampOrUDT