to_pyarrow_batches

xorq.api.to_pyarrow_batches(expr, *, chunk_size=1000000, **kwargs)

Execute expression and return a RecordBatchReader.

This method is eager and will execute the associated expression immediately.

The returned reader must be fully consumed: drain threads are joined and temp tables are dropped only after the last batch is read. Consuming partially (an early break) or discarding the reader leaks those resources. Drain failures are surfaced when the reader is exhausted.

Parameters

Name Type Description Default
chunk_size int Maximum number of rows in each returned record batch. 1000000
kwargs Any Keyword arguments {}

Returns

Name Type Description
results RecordBatchReader