SupportsTriggerAvailableNow + simpler navigation

jaceklaskowski · jaceklaskowski · commit eac469ce45eb · 2026-01-24T16:41:41.000+01:00
diff --git a/docs/.nav.yml b/docs/.nav.yml
@@ -146,7 +146,6 @@ nav:
       - ContinuousWriteRDD: ContinuousWriteRDD.md
       - ContinuousDataSourceRDD: ContinuousDataSourceRDD.md
     - testing.md
-    - Web UI: webui/index.md
   - High-Level APIs:
     - AcceptsLatestSeenOffset: AcceptsLatestSeenOffset.md
     - ContinuousStream: ContinuousStream.md
@@ -167,94 +166,27 @@ nav:
     - SupportsAdmissionControl: SupportsAdmissionControl.md
     - SupportsTriggerAvailableNow: SupportsTriggerAvailableNow.md
     - Trigger: Trigger.md
-  - Streaming Operators:
-    - operators/index.md
-    - crossJoin: operators/crossJoin.md
-    - dropDuplicates: operators/dropDuplicates.md
-    - explain: operators/explain.md
-    - flatMapGroupsWithState: operators/flatMapGroupsWithState.md
-    - groupBy: operators/groupBy.md
-    - groupByKey: operators/groupByKey.md
-    - join: operators/join.md
-    - joinWith: operators/joinWith.md
-    - mapGroupsWithState: operators/mapGroupsWithState.md
-    - window: operators/window.md
-    - withWatermark: operators/withWatermark.md
-    - writeStream: operators/writeStream.md
+  - Streaming Operators: operators
   - Data Sources:
     - datasources/index.md
-    - Console:
-      - datasources/console/index.md
-      - ConsoleSinkProvider: datasources/console/ConsoleSinkProvider.md
-      - ConsoleWriter: datasources/console/ConsoleWriter.md
-    - File:
-      - datasources/file/index.md
-      - FileStreamSource: datasources/file/FileStreamSource.md
-      - FileStreamSink: datasources/file/FileStreamSink.md
-      - CompactibleFileStreamLog: datasources/file/CompactibleFileStreamLog.md
-      - SinkFileStatus: datasources/file/SinkFileStatus.md
-      - ManifestFileCommitProtocol: datasources/file/ManifestFileCommitProtocol.md
-      - MetadataLogFileIndex: datasources/file/MetadataLogFileIndex.md
-      - FileStreamSourceCleaner: datasources/file/FileStreamSourceCleaner.md
-      - FileStreamSourceLog: datasources/file/FileStreamSourceLog.md
-      - FileStreamSinkLog: datasources/file/FileStreamSinkLog.md
-    - Foreach:
-      - datasources/foreach/index.md
-      - ForeachSink: datasources/foreach/ForeachSink.md
-      - ForeachWriter: datasources/foreach/ForeachWriter.md
-      - ForeachWriterProvider: datasources/foreach/ForeachWriterProvider.md
-      - ForeachWriterTable: datasources/foreach/ForeachWriterTable.md
+    - datasources/console
+    - datasources/file
+    - datasources/foreach
     - ForeachBatch:
       - ForeachBatchSink: datasources/ForeachBatchSink.md
     - Kafka:
       - kafka/index.md
-      - CachedKafkaConsumer: kafka/CachedKafkaConsumer.md
+      - Options: kafka/options.md
       - Consumer Strategies:
-        - ConsumerStrategy: kafka/ConsumerStrategy.md
-        - SubscribeStrategy: kafka/SubscribeStrategy.md
-      - KafkaBatch: kafka/KafkaBatch.md
-      - KafkaBatchInputPartition: kafka/KafkaBatchInputPartition.md
-      - KafkaContinuousInputPartition: kafka/KafkaContinuousInputPartition.md
-      - KafkaContinuousStream: kafka/KafkaContinuousStream.md
-      - KafkaDataConsumer: kafka/KafkaDataConsumer.md
-      - KafkaMicroBatchInputPartition: kafka/KafkaMicroBatchInputPartition.md
-      - KafkaMicroBatchInputPartitionReader: kafka/KafkaMicroBatchInputPartitionReader.md
-      - KafkaMicroBatchStream: kafka/KafkaMicroBatchStream.md
-      - KafkaOffsetRangeCalculator: kafka/KafkaOffsetRangeCalculator.md
-      - KafkaOffsetRangeLimit: kafka/KafkaOffsetRangeLimit.md
+        - "kafka/*Strategy.md"
       - KafkaOffsetReaders:
-        - KafkaOffsetReader: kafka/KafkaOffsetReader.md
-        - KafkaOffsetReaderAdmin: kafka/KafkaOffsetReaderAdmin.md
-        - KafkaOffsetReaderConsumer: kafka/KafkaOffsetReaderConsumer.md
-      - KafkaRelation: kafka/KafkaRelation.md
-      - KafkaScan: kafka/KafkaScan.md
-      - KafkaSink: kafka/KafkaSink.md
-      - KafkaSource: kafka/KafkaSource.md
-      - KafkaSourceInitialOffsetWriter: kafka/KafkaSourceInitialOffsetWriter.md
-      - KafkaSourceOffset: kafka/KafkaSourceOffset.md
-      - KafkaSourceProvider: kafka/KafkaSourceProvider.md
-      - KafkaSourceRDD: kafka/KafkaSourceRDD.md
-      - KafkaStreamingWrite: kafka/KafkaStreamingWrite.md
-      - KafkaStreamWriterFactory: kafka/KafkaStreamWriterFactory.md
-      - KafkaTable: kafka/KafkaTable.md
-      - Options: kafka/options.md
+        - "kafka/KafkaOffsetReader*.md"
+      - "kafka/*"
     - datasources/memory
-    - Rate:
-      - datasources/rate/index.md
-      - RateStreamProvider: datasources/rate/RateStreamProvider.md
-      - RateStreamContinuousReader: datasources/rate/RateStreamContinuousReader.md
-    - Rate Per Micro-Batch:
-      - datasources/rate-micro-batch/index.md
-      - Options: datasources/rate-micro-batch/options.md
-      - RatePerMicroBatchProvider: datasources/rate-micro-batch/RatePerMicroBatchProvider.md
-      - RatePerMicroBatchStream: datasources/rate-micro-batch/RatePerMicroBatchStream.md
-      - RatePerMicroBatchTable: datasources/rate-micro-batch/RatePerMicroBatchTable.md
-    - State:
-      - datasources/state/index.md
-    - Text Socket:
-      - datasources/socket/index.md
-      - TextSocketSourceProvider: datasources/socket/TextSocketSourceProvider.md
-      - TextSocketSource: datasources/socket/TextSocketSource.md
+    - datasources/rate
+    - Rate Per Micro-Batch: datasources/rate-micro-batch
+    - datasources/state
+    - datasources/socket
   - Monitoring:
     - StreamingQueryListener: monitoring/StreamingQueryListener.md
     - StreamingQueryProgress:
@@ -266,13 +198,7 @@ nav:
     - StreamingQueryStatus: monitoring/StreamingQueryStatus.md
     - MetricsReporter: monitoring/MetricsReporter.md
     - Logging: spark-logging.md
-  - Web UI:
-    - webui/index.md
-    - StreamingQueryPage: webui/StreamingQueryPage.md
-    - StreamingQueryStatisticsPage: webui/StreamingQueryStatisticsPage.md
-    - StreamingQueryStatusListener: webui/StreamingQueryStatusListener.md
-    - StreamingQueryStatusStore: webui/StreamingQueryStatusStore.md
-    - StreamingQueryTab: webui/StreamingQueryTab.md
+  - Web UI: webui
   - Demo:
     - demo/index.md
     - Arbitrary Stateful Streaming Aggregation with KeyValueGroupedDataset.flatMapGroupsWithState Operator: demo/arbitrary-stateful-streaming-aggregation-flatMapGroupsWithState.md
diff --git a/docs/SupportsTriggerAvailableNow.md b/docs/SupportsTriggerAvailableNow.md
@@ -1,17 +1,26 @@
 # SupportsTriggerAvailableNow
 
-`SupportsTriggerAvailableNow` is an [extension](#contract) of the [SupportsAdmissionControl](SupportsAdmissionControl.md) abstraction for [streaming sources](#implementations) to support [Trigger.AvailableNow](Trigger.md#AvailableNow) mode.
+`SupportsTriggerAvailableNow` is an [extension](#contract) of the [SupportsAdmissionControl](SupportsAdmissionControl.md) abstraction for [streaming sources](#implementations) that support [Trigger.AvailableNow](Trigger.md#AvailableNow) mode.
 
 ## Contract
 
-### <span id="prepareForTriggerAvailableNow"> prepareForTriggerAvailableNow
+### prepareForTriggerAvailableNow { #prepareForTriggerAvailableNow }
 
 ```java
 void prepareForTriggerAvailableNow()
 ```
 
 Lets a streaming source to prepare for the [default ReadLimit](SupportsAdmissionControl.md#getDefaultReadLimit) (in [Trigger.AvailableNow](Trigger.md#AvailableNow) mode)
 
+See:
+
+* [AvailableNowDataStreamWrapper](AvailableNowDataStreamWrapper.md#prepareForTriggerAvailableNow)
+* [FileStreamSource](datasources/file/FileStreamSource.md#prepareForTriggerAvailableNow)
+* [KafkaMicroBatchStream](kafka/KafkaMicroBatchStream.md#prepareForTriggerAvailableNow)
+* [KafkaSource](kafka/KafkaSource.md#prepareForTriggerAvailableNow)
+* [MemoryStreamBaseClass](datasources/memory/MemoryStreamBaseClass.md#prepareForTriggerAvailableNow)
+* [RatePerMicroBatchStream](datasources/rate-micro-batch/RatePerMicroBatchStream.md#prepareForTriggerAvailableNow)
+
 Used when:
 
 * `MicroBatchExecution` is requested for the [logicalPlan](micro-batch-execution/MicroBatchExecution.md#logicalPlan) (and the [uniqueSources](micro-batch-execution/MicroBatchExecution.md#uniqueSources) for [MultiBatchExecutor](TriggerExecutor.md#MultiBatchExecutor))
@@ -22,4 +31,5 @@ Used when:
 * [FileStreamSource](datasources/file/FileStreamSource.md)
 * [KafkaMicroBatchStream](kafka/KafkaMicroBatchStream.md)
 * [KafkaSource](kafka/KafkaSource.md)
+* [MemoryStreamBaseClass](datasources/memory/MemoryStreamBaseClass.md)
 * [RatePerMicroBatchStream](datasources/rate-micro-batch/RatePerMicroBatchStream.md)
diff --git a/docs/datasources/index.md b/docs/datasources/index.md
@@ -1,3 +1,3 @@
-# Data Sources
+# Streaming Data Sources
 
 Spark Structured Streaming comes with a variety of built-in streaming data sources.
diff --git a/docs/datasources/memory/MemoryStreamBaseClass.md b/docs/datasources/memory/MemoryStreamBaseClass.md
@@ -0,0 +1,3 @@
+# MemoryStreamBaseClass
+
+`MemoryStreamBaseClass` is...FIXME
diff --git a/docs/kafka/KafkaOffsetRangeLimit.md b/docs/kafka/KafkaOffsetRangeLimit.md
@@ -1,3 +1,7 @@
+---
+title: KafkaOffsetRangeLimit
+---
+
 # KafkaOffsetRangeLimit &mdash; Desired Offset Range Limits
 
 `KafkaOffsetRangeLimit` represents the desired offset range limits for starting, ending, and specific offsets in [Kafka Data Source](index.md).
diff --git a/docs/operators/crossJoin.md b/docs/operators/crossJoin.md
@@ -1,3 +1,7 @@
+---
+title: crossJoin
+---
+
 # crossJoin Operator &mdash; Streaming Join
 
 ```scala
diff --git a/docs/operators/dropDuplicates.md b/docs/operators/dropDuplicates.md
@@ -1,3 +1,7 @@
+---
+title: dropDuplicates
+---
+
 # dropDuplicates Operator &mdash; Streaming Deduplication
 
 ```scala
diff --git a/docs/operators/explain.md b/docs/operators/explain.md
@@ -1,3 +1,7 @@
+---
+title: explain
+---
+
 # Dataset.explain Operator &mdash; Explaining Streaming Queries
 
 ```scala
diff --git a/docs/operators/flatMapGroupsWithState.md b/docs/operators/flatMapGroupsWithState.md
@@ -1,3 +1,7 @@
+---
+title: flatMapGroupsWithState
+---
+
 # flatMapGroupsWithState Operator
 
 `flatMapGroupsWithState` is part of `KeyValueGroupedDataset` ([Spark SQL]({{ book.spark_sql }}/basic-aggregation/KeyValueGroupedDataset)) API for [Arbitrary Stateful Streaming Aggregation](../arbitrary-stateful-streaming-aggregation/index.md) with an explicit state logic.
diff --git a/docs/operators/groupBy.md b/docs/operators/groupBy.md
@@ -1,3 +1,7 @@
+---
+title: groupBy
+---
+
 # groupBy Operator &mdash; Streaming Aggregation
 
 ```scala
diff --git a/docs/operators/groupByKey.md b/docs/operators/groupByKey.md
@@ -1,3 +1,7 @@
+---
+title: groupByKey
+---
+
 # groupByKey Operator &mdash; Streaming Aggregation
 
 ```scala
diff --git a/docs/operators/join.md b/docs/operators/join.md
@@ -1,3 +1,7 @@
+---
+title: join
+---
+
 # join Operator
 
 `join` is part of `Dataset` API for [Streaming Join](../join/index.md).
diff --git a/docs/operators/joinWith.md b/docs/operators/joinWith.md
@@ -1,3 +1,7 @@
+---
+title: joinWith
+---
+
 # joinWith Operator &mdash; Streaming Join
 
 ```scala
diff --git a/docs/operators/mapGroupsWithState.md b/docs/operators/mapGroupsWithState.md
@@ -1,3 +1,7 @@
+---
+title: mapGroupsWithState
+---
+
 # mapGroupsWithState Operator
 
 `flatMapGroupsWithState` is part of `KeyValueGroupedDataset` ([Spark SQL]({{ book.spark_sql }}/basic-aggregation/KeyValueGroupedDataset)) API for [stateful streaming aggregation](../streaming-aggregation/index.md) with an explicit state logic.
diff --git a/docs/operators/window.md b/docs/operators/window.md
@@ -1,32 +1,34 @@
-== [[window]] window Function -- Stream Time Windows
+---
+title: window
+---
 
-`window` is a standard function that generates *tumbling*, *sliding* or *delayed* stream time window ranges (on a timestamp column).
+# window Function &mdash; Stream Time Windows
 
-[source, scala]
-----
+`window` is a standard function that generates **tumbling**, **sliding** or **delayed** stream time window ranges (on a timestamp column).
+
+```scala
 window(
   timeColumn: Column,
-  windowDuration: String): Column  // <1>
+  windowDuration: String): Column  // (1)!
 window(
   timeColumn: Column,
   windowDuration: String,
-  slideDuration: String): Column   // <2>
+  slideDuration: String): Column   // (2)!
 window(
   timeColumn: Column,
   windowDuration: String,
   slideDuration: String,
-  startTime: String): Column       // <3>
-----
-<1> Creates a tumbling time window with `slideDuration` as `windowDuration` and `0 second` for `startTime`
-<2> Creates a sliding time window with `0 second` for `startTime`
-<3> Creates a delayed time window
+  startTime: String): Column       // (3)!
+```
 
-[NOTE]
-====
-From https://msdn.microsoft.com/en-us/library/azure/dn835055.aspx[Tumbling Window (Azure Stream Analytics)]:
+1. Creates a tumbling time window with `slideDuration` as `windowDuration` and `0 second` for `startTime`
+2. Creates a sliding time window with `0 second` for `startTime`
+3. Creates a delayed time window
 
-> *Tumbling windows* are a series of fixed-sized, non-overlapping and contiguous time intervals.
-====
+!!! note "Tumbling Window"
+    From [Tumbling Window](https://learn.microsoft.com/en-gb/stream-analytics-query/tumbling-window-azure-stream-analytics):
+
+    > **Tumbling windows** are a series of fixed-sized, non-overlapping and contiguous time intervals.
 
 [NOTE]
 ====
diff --git a/docs/operators/withWatermark.md b/docs/operators/withWatermark.md
@@ -1,3 +1,7 @@
+---
+title: withWatermark
+---
+
 # withWatermark Operator &mdash; Event-Time Watermark
 
 ```scala
diff --git a/docs/operators/writeStream.md b/docs/operators/writeStream.md
@@ -1,7 +1,11 @@
+---
+title: writeStream
+---
+
 # writeStream Operator
 
 ```scala
 writeStream: DataStreamWriter[T]
 ```
 
-`writeStream` creates a [DataStreamWriter](../DataStreamWriter.md) for persisting the result of a streaming query to an external data system
+`writeStream` creates a [DataStreamWriter](../DataStreamWriter.md) for persisting (_saving_) the result of a streaming query to an external data system.
diff --git a/mkdocs.yml b/mkdocs.yml
@@ -146,7 +146,7 @@ extra:
     version: 4.1.1
     commit: https://github.com/apache/spark/commit
     docs: https://spark.apache.org/docs/4.1.1
-    github: https://github.com/apache/spark/tree/v4.1.1
+    github: https://github.com/apache/spark/blob/v4.1.1
     jira: https://issues.apache.org/jira/browse
 
 validation:

Original file line number	Diff line number	Diff line change
`@@ -1,3 +1,3 @@`
`1`		`-# Data Sources`
	`1`	`+# Streaming Data Sources`
`2`	`2`
`3`	`3`	`Spark Structured Streaming comes with a variety of built-in streaming data sources.`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+# MemoryStreamBaseClass`
	`2`	`+`
	`3`	+`MemoryStreamBaseClass` is...FIXME