9.2 Pipeline, Push Execution, and Query Code Generation

JIT compilation still preserves operator boundaries. A more aggressive query compilation approach generates a pipeline loop via produce/consume, where each operator executes directly: the scan operator produces tuples or batches, the filter applies immediate evaluation, the projection performs direct computation, and the sink writes directly into a hash table or final result.

Pull, push, and compiled pipeline

Pull iterator

The parent requests the next row or batch from the child:

root.next()
  -> filter.next()
     -> scan.next()

Push execution

The source actively pushes a batch to downstream consumers:

scan -> filter -> projection -> sink

Push execution does not automatically imply native code generation. DuckDB's current documentation describes a push-based vectorized execution model, where DataChunks flow between physical operators. This is distinct from the concept of "compiling an entire query into LLVM machine code."

Compiled produce/consume

Operators participate in generating a shared loop:

for (const auto& row : scan(table)) {
    if (row.level >= 30) {
        hash_table.insert(row.adventurer_id, row.name);
    }
}

The above is conceptual code for result generation, it is not a specific database source API.

Pipeline breaker

A pipeline can stream between operators without requiring the full input to be processed. The following states typically form boundaries:

• hash join build: a probe table must be constructed prior to probing;
• full sort: sorted runs must be formed or merged before the first output row is produced;
• hash aggregate: final groupings require complete or partitioned input;
• materialize/spool;
• certain frames or orders in window functions;
• exchange/shuffle;
• blocking UDFs or external calls.

Example:

Pipeline A: Scan(adventurers) -> Filter -> HashBuild
                                      [breaker: hash table]
Pipeline B: Scan(quests) -> HashProbe -> AggregateBuild
                                      [breaker: group states]
Pipeline C: AggregateScan -> Sort -> Result
                              [breaker: sorted runs]

A breaker does not necessarily write data into generic rows; intermediate states may instead be hash tables, run files, compressed vectors, or partition buffers.

Benefits of Fusing

• Reduces materialized intermediate tuples;
• Eliminates operator dispatch;
• Keeps column values in registers or local variables;
• Decodes expensive output columns only after filtering;
• Enables specialization of constants, types, and NULLability;
• Allows the compiler to perform common-subexpression elimination across expressions.

Risks of Fusing

• Giant functions increase compile time and code size;
• Increased pressure on instruction cache;
• Register pressure leading to spills;
• Rare branches or UDFs make the code more complex;
• Skew or runtime cardinality diverging from compile-time assumptions;
• Difficulties in debugging, profiling, and mapping to generated code.

The optimal pipeline is not "fuse as much as possible." The engine can split at operator or fragment level, helping maintain code size and compile latency.

HyPer's Research Path

HyPer's produce/consume code generation compiles relational operators into data-centric pipelines, with the core goal of keeping data in CPU registers and cache while avoiding iterator overhead. It serves as a key representative in the study of query compilation.

Reading entry: Thomas Neumann, Efficiently Compiling Efficient Query Plans for Modern Hardware (VLDB 2011). The paper's results are based on a specific prototype, workload, and hardware setup and cannot be generalized to claim that any TPC-H query runs 10–100 times faster than PostgreSQL.

DuckDB Should Not Be Classified as "Full Query JIT"

DuckDB employs vectorized execution, with Vector/DataChunk serving as its primary data transfer format. The current internals documentation describes a push-based, vectorized physical execution model. This approach achieves low overhead through typed/vector kernels, constant/dictionary vectors, and operator pipelines, but it is not LLVM-based query JIT.

Official references:

• DuckDB Internals Overview
• DuckDB Execution Format

The fact that "C++ templates are expanded during the compilation of the DuckDB binary" does not mean that each SQL statement is compiled at runtime into a dedicated native function.

Adaptive/tiered compilation

The system can initially execute code using an interpreter or vector engine, while simultaneously compiling a hot pipeline in the background. Once certain thresholds are reached, the execution switches to the compiled version:

start quickly with generic code
  -> collect rows/types/selectivity/skew
  -> compile hot path
  -> switch at safe boundary

This approach draws inspiration from JVM tiered compilation to reduce cold-query latency. However, the implementation must ensure:

• state remains compatible between generic and compiled code;
• snapshot and exception semantics are preserved;
• compiled assumptions can be guarded or deoptimized;
• the code cache includes eviction policies;
• concurrent compilation is bounded by resource limits.

Compilation cache key

Reusable generated code depends at least on:

• the logical or physical plan shape;
• schema, type, and nullability definitions;
• available functions, operators, and collation rules;
• whether constants are baked into the code;
• CPU instruction set architecture and features;
• security and tenant context;
• engine version and configuration.

A cache key that is too specific results in low hit rates, while one that is too broad risks incorrect code reuse. Schema migrations, extension or UDF replacements, and heterogeneous CPU cluster environments all require cache invalidation or dynamic dispatch.

Parameter specialization

Embedding constant parameters directly into code can eliminate branches but generates separate code for each value; generic compiled functions offer better code reuse, though with limited optimization opportunities.

Options include:

• Generic parameters
• Specialize hot values or shapes
• Runtime guard with fallback
• Code cache size or time-to-live (TTL)
• Profile-guided recompilation

This relates directly to the optimizer's generic versus custom plan dilemma: first selecting a plan, then deciding on code specialization, both layers are susceptible to parameter skew.

Compilation Does Not Alter SQL Semantics

The generated code must preserve:

• three-valued NULL logic;
• overflow and decimal precision behavior;
• collation and timezone settings;
• floating-point NaN handling and ordering;
• constraints on the number of calls and invocation order of volatile functions;
• allowable boundaries for exception and error timing;
• transaction snapshot behavior and cancellation;
• memory and resource accounting.

The compiler’s constant folding and reordering operations must not substitute database type semantics with ordinary arithmetic semantics from C/C++.

Benchmark Framework

Separate reporting:

parse/bind/optimize time
code generation/optimization/emission time
first-row latency
steady-state execution CPU
total wall time
code cache hit/miss and bytes
rows/batches, branch/cache counters
peak memory and spill
result correctness

Must include at least: short OLTP, medium repeated query, long CPU scan, I/O-bound query, complex UDF, and parameter skew. Testing only long scans would overstate compilation benefits.

Acceptance Checklist

• [ ] Separate documentation for push/vectorized and JIT/codegen;
• [ ] Pipeline breakers identified based on state dependencies;
• [ ] Compile time included in end-to-end latency;
• [ ] Code size, cache behavior, and invalidation have clear boundaries;
• [ ] SQL NULL, type, and collation semantics are preserved;
• [ ] First-execution and reused-execution scenarios are measured separately;
• [ ] DuckDB is no longer referred to as a full-query LLVM compiler.

Chapter Summary

Query compilation evolves from expression JIT to pipeline code generation, reducing dispatch and materialization overhead while introducing compile latency, code-cache pressure, and specialization risks. The next chapter returns to another CPU optimization path: instead of generating dedicated control flow for every line of code, it leverages vector batches and SIMD to process data of the same type in bulk.