File tree Expand file tree Collapse file tree 1 file changed +24
-0
lines changed
docs/source/user-guide/dataframe Expand file tree Collapse file tree 1 file changed +24
-0
lines changed Original file line number Diff line number Diff line change @@ -203,6 +203,30 @@ To work with the stream directly, use
203203 for batch in stream:
204204 ...
205205
206+ Execute as Stream
207+ ^^^^^^^^^^^^^^^^^
208+
209+ For finer control over streaming execution, use
210+ :py:meth: `~datafusion.DataFrame.execute_stream ` to obtain a
211+ :py:class: `pyarrow.RecordBatchReader `:
212+
213+ .. code-block :: python
214+
215+ reader = df.execute_stream()
216+ for batch in reader:
217+ ... # process each batch as it is produced
218+
219+ When partition boundaries are important,
220+ :py:meth: `~datafusion.DataFrame.execute_stream_partitioned `
221+ returns an iterable of :py:class: `pyarrow.RecordBatchReader ` objects, one per
222+ partition:
223+
224+ .. code-block :: python
225+
226+ for stream in df.execute_stream_partitioned():
227+ for batch in stream:
228+ ... # each stream yields RecordBatches
229+
206230 See :doc: `../io/arrow ` for additional details on the Arrow interface.
207231
208232HTML Rendering
You can’t perform that action at this time.
0 commit comments