Skip to content

Commit 7a16fea

Browse files
Spark Declarative Pipelines moved to its own place
1 parent 8297b3d commit 7a16fea

81 files changed

Lines changed: 6 additions & 4154 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/.nav.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,6 @@ nav:
3030
- Parameterized Queries: parameterized-queries
3131
- Partition File Metadata Caching: partition-file-metadata-caching
3232
- Runtime Filtering: runtime-filtering
33-
- Spark Declarative Pipelines: declarative-pipelines
3433
- Spark Thrift Server: thrift-server
3534
- new-and-noteworthy/statistics.md
3635
- Storage-Partitioned Joins: storage-partitioned-joins

docs/SQLConf.md

Lines changed: 0 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1120,33 +1120,3 @@ Used when:
11201120
Used when:
11211121

11221122
* [WindowExec](physical-operators/WindowExec.md) unary physical operator is executed
1123-
1124-
## spark.sql.pipelines
1125-
1126-
### <span id="PIPELINES_STREAM_STATE_POLLING_INTERVAL"> streamStatePollingInterval { #streamStatePollingInterval }
1127-
1128-
[spark.sql.pipelines.execution.streamstate.pollingInterval](./declarative-pipelines/configuration-properties.md#spark.sql.pipelines.execution.streamstate.pollingInterval)
1129-
1130-
### <span id="PIPELINES_WATCHDOG_MIN_RETRY_TIME_IN_SECONDS"> watchdogMinRetryTimeInSeconds { #watchdogMinRetryTimeInSeconds }
1131-
1132-
[spark.sql.pipelines.execution.watchdog.minRetryTime](./declarative-pipelines/configuration-properties.md#spark.sql.pipelines.execution.watchdog.minRetryTime)
1133-
1134-
### <span id="PIPELINES_WATCHDOG_MAX_RETRY_TIME_IN_SECONDS"> watchdogMaxRetryTimeInSeconds { #watchdogMaxRetryTimeInSeconds }
1135-
1136-
[spark.sql.pipelines.execution.watchdog.maxRetryTime](./declarative-pipelines/configuration-properties.md#spark.sql.pipelines.execution.watchdog.maxRetryTime)
1137-
1138-
### <span id="PIPELINES_MAX_CONCURRENT_FLOWS"> maxConcurrentFlows { #maxConcurrentFlows }
1139-
1140-
[spark.sql.pipelines.execution.maxConcurrentFlows](./declarative-pipelines/configuration-properties.md#spark.sql.pipelines.execution.maxConcurrentFlows)
1141-
1142-
### <span id="PIPELINES_TIMEOUT_MS_FOR_TERMINATION_JOIN_AND_LOCK"> timeoutMsForTerminationJoinAndLock { #timeoutMsForTerminationJoinAndLock }
1143-
1144-
[spark.sql.pipelines.timeoutMsForTerminationJoinAndLock](./declarative-pipelines/configuration-properties.md#spark.sql.pipelines.timeoutMsForTerminationJoinAndLock)
1145-
1146-
### <span id="PIPELINES_MAX_FLOW_RETRY_ATTEMPTS"> maxFlowRetryAttempts { #maxFlowRetryAttempts }
1147-
1148-
[spark.sql.pipelines.maxFlowRetryAttempts](./declarative-pipelines/configuration-properties.md#spark.sql.pipelines.maxFlowRetryAttempts)
1149-
1150-
### <span id="PIPELINES_EVENT_QUEUE_CAPACITY"> queueCapacity { #queueCapacity }
1151-
1152-
[spark.sql.pipelines.event.queue.capacity](./declarative-pipelines/configuration-properties.md#spark.sql.pipelines.event.queue.capacity)

docs/configuration-properties/spark.sql.pipelines.md

Lines changed: 0 additions & 7 deletions
This file was deleted.

docs/declarative-pipelines/.nav.yml

Lines changed: 0 additions & 5 deletions
This file was deleted.

docs/declarative-pipelines/AppendOnceFlow.md

Lines changed: 0 additions & 30 deletions
This file was deleted.

docs/declarative-pipelines/BatchTableWrite.md

Lines changed: 0 additions & 56 deletions
This file was deleted.

docs/declarative-pipelines/CompleteFlow.md

Lines changed: 0 additions & 17 deletions
This file was deleted.

docs/declarative-pipelines/CoreDataflowNodeProcessor.md

Lines changed: 0 additions & 42 deletions
This file was deleted.

docs/declarative-pipelines/DataflowGraph.md

Lines changed: 0 additions & 90 deletions
This file was deleted.

docs/declarative-pipelines/DataflowGraphRegistry.md

Lines changed: 0 additions & 97 deletions
This file was deleted.

0 commit comments

Comments
 (0)