[ErikBPF]

Which issue does this PR close?

Refs #7573. This draft does not close the issue.

Rationale for this change

Open a collaboration branch following @dwsmith1983's request. The initial commit supplies an independently runnable Delta reference baseline. @dwsmith1983 plans to contribute his Rust ordering code and establish a working base before handing it back for extension and cleanup.

The accepted issue scope remains Delta-compatible InterleaveBits for one or two INT inputs. Hilbert ordering and broader clustering operations require separate scope; this draft does not claim support for them.

What changes are included in this PR?

• A manual Scala probe covering exact bytes, null-as-zero, signed boundaries, deterministic integer pairs, mixed literals/columns and controlled constant folding against both Delta interleaving paths.
• Launch instructions and explicit limitations. No native implementation, expression admission changes or CI integration are included yet.

Based on main at 2f9552d022eb84a79b31b3870c70be9725fdb41f. The initial files do not overlap our six other open upstream PRs, including #7593; this baseline does not depend on that pairing PR.

How are these changes tested?

Apollo: fresh compilation and execution on Spark 4.2.0 / Java 17, Delta 4.4.0 and 4.4.1, stock Spark and released Comet 1.2.0, local[2], 2 GiB driver. Each lane requires 15 named checks; row/schema parity is compared between stock and Comet. An accelerated control verifies Comet is active, while the requested expression can still fall back to Spark.

These are manual reference probes, not reactor tests or native acceptance. Current-main runtime, the future Rust implementation and performance remain unverified. Before readiness, add native plan assertions and CI tests, preserve fallback for unsupported types/arities, and validate the native integration against Delta's implementation and tests.

[dwsmith1983]

@ErikBPF thanks for opening this. Could you add me as a collaborator on ErikBPF/datafusion-comet so I can push to feat/7573-interleave-handoff? "Allow edits by maintainers" doesn't cover me, since I don't have write access here. I'll bring over only the Morton interleave for InterleaveBits; Hilbert will get its own issue and PR.

[dwsmith1983]

Here's how I'd split the work so each piece stays reviewable:

1. A core PR adding a CometExpressionContrib hook, so the Delta contrib can register a serde for InterleaveBits. It waits on the maintainers' answer in Support Delta InterleaveBits for one or two INT inputs #7573.
2. A Rust-only PR with the delta_interleave_bits kernel, tested against vectors generated from both of Delta's code paths.
3. This PR on top of both, with the Delta serde and an expression suite that takes over the checks from your probe.

I'll open the first two from my fork, link them here, and add you as a collaborator on it so you can push to those branches. Once you've added me on yours, I'll move the serde and suite into this PR and hand it back to you.

[dwsmith1983]

The native kernel is up as #7600 (draft). The core hook PR follows once the design question in #7573 is settled.

[ErikBPF]

@dwsmith1983 Thanks. I sent a collaborator invitation for ErikBPF/datafusion-comet with push access; accept it and you can push directly to feat/7573-interleave-handoff. No separate branch is needed.

The proposed split works for us:

1. Core CometExpressionContrib hook once Support Delta InterleaveBits for one or two INT inputs #7573 gets a maintainer answer.
2. Rust-only delta_interleave_bits kernel checked against both Delta reference paths.
3. This PR on top with the Delta serde and the expression suite.

I will keep the manual probe in this PR as the reference baseline until the suite supersedes it. The only open dependency is the expression-admission design in #7573; we will not add core Delta class matching on our side unless that is the maintainers' preference.