catalog run

Compose and execute catalog entries, writing data to disk or stdout.

A single entry runs directly; multiple entries compose as source + transforms. Unbound entries can have their input streamed in over Arrow IPC. To persist a composed expression back to the catalog instead of executing it, use xorq catalog compose.

Usage

xorq catalog run [ENTRIES]... [OPTIONS]

Arguments

  • ENTRIES—One or more entry names or aliases; multiple entries compose as source + transforms.

Options

Option Default Description
-c, --code none Inline Ibis code expression applied to source.
-o, --output-path /dev/null (discard) Path to write output. Use ‘-’ for stdout.
-f, --format parquet Output format.
--limit unlimited Maximum number of rows to output.
-i, --instream stdin Stream to read Arrow IPC record batches from.
--fuse / --no-fuse --fuse Enable catalog source fusion.
--rename-params none Rename a parameter on a specific entry: entry,old_name,new_name (repeatable).
-p, --params none Override an expression parameter as key=value (repeatable, for example –params threshold=0.5).
--cache-dir $XORQ_CACHE_DIR or ~/.cache/xorq Directory for parquet cache files.
--use-this-venv / --no-use-this-venv --no-use-this-venv Execute in the current Python environment instead of spawning uv tool run on the entry’s pinned env. Faster (no subprocess + uv venv lookup) but only correct when the calling venv already has every package the expression needs (Xorq itself plus any UDFs from the entries’ wheels). Default is the isolated uv tool run path.

Examples

# Run a single entry directly
xorq catalog run src -o - -f csv
# Compose source + transform and execute
xorq catalog run src trn -o - -f csv
# Add inline transformation code
xorq catalog run src trn -c "source.filter(source.amount > 100)" -o - -f csv
# Pipe Arrow data into an unbound entry
xorq catalog run src -o - -f arrow | xorq catalog run trn -o - -f csv
# Override an expression parameter
xorq catalog run pipeline -p threshold=0.5 -o results.parquet

See also