Skip to content

Unify typed pipelines around ordinary Python callables - #43

Draft
ericklt wants to merge 4 commits into
mainfrom
codex/unified-pipelines
Draft

ericklt wants to merge 4 commits into
mainfrom
codex/unified-pipelines

Conversation

@ericklt

@ericklt ericklt commented Sep 5, 2026 •

Copy link
Copy Markdown

Gloe currently requires separate transformer variants and repeated decorators to express sync, async, multiargument and configured stages. This breaking 0.8 proposal replaces them with one immutable Transformer[P, R] and ordinary typed callables, with a bundled mypy plugin checking the actual composition boundaries.

import gloe

@gloe.transformer
def add(value: int, *, amount: int) -> int:
    return value + amount

def multiply(value: int) -> int:
    return value * 2

def divide(value: int) -> float:
    return value / 2

pipe = gloe.input() >> add(amount=2) >> (multiply, divide) >> sum >> print
pipe.sync(2)  # prints 10.0
# From async code: await pipe(2)

Public behavior

  • Infer concrete input types from the first stage; retain input[T]() for generic or ambiguous entries. Accept typed functions, configured callables, partials, bound methods, and callable objects.
  • Await pipeline calls consistently, resolving sync and async stages through one execution path. .sync(...) is the blocking entry and rejects a running event loop. Synchronous work stays on the caller's thread.
  • Use positional callbacks for gloe.map(fn), gloe.filter(predicate), gloe.parallel(branches), gloe.sequential(branches), and gloe.condition(predicate, then=..., otherwise=...).
  • Preserve keyword-only configuration through @gloe.transformer, decorator-based ensure, and explicit tuple spreading through gloe.unpack.
  • Keep early values with prefix >> (gloe.input(), work) >> gloe.unpack(finish). Branches retain references; there is no hidden context store or cache. Composing complete pipelines repeats their stages; prefix >> (left, right) shares the prefix explicitly.
  • Cancel and await sibling branches on failure or cancellation. Graph projection is side-effect free and optional Graphviz export remains available.

Migration and guardrails

Support Python 3.12–3.14 and pin the tested mypy 2.3.1 release in gloe[types]. Users enable gloe.mypy_plugin explicitly in checker configuration; installation does not modify their files. The composition contract fails closed without the plugin and does not claim equivalent support from other checkers.

Remove the old async/multiargument transformer hierarchy, helper class variants and bridge API. Document the migration, limitations, architecture and agent-oriented guidance. Examples include actual calls and expected results; the runnable cookbook is checked by both runtime tests and mypy. Workflow tests cover reporting, reconciliation, monetary conservation, route preparation, indicators, transactional imports, request handling, and event ingestion with in-memory fixtures. tests/contract-matrix.md accounts for the removed 0.7 test groups, supported forms, rejected forms and adapter boundaries.

Validation

  • 84 unittest checks pass on Python 3.12, 3.13 and 3.14, including the typing harness; the optional image test also passes in the isolated Graphviz environment.
  • Public typing fixtures cover exact inferred results, generic factories, overloads, partials, callable objects, recursive aliases, async normalization, narrowing and ensurers. Representative programs are both type-checked and executed. There are 94 exact type assertions and 166 rejection cases (including missing plugin setup), with expected messages and error codes. The harness also checks cold/warm caches and invalidation after a dependency changes.
  • Strict source mypy, Black, Flake8 and Sphinx warnings-as-errors pass.
  • Built wheel and source distribution; verified the installed wheel in a clean Python 3.12 environment and the extracted source distribution. CI runs the supported Python matrix, type contract, graph export, documentation build and packaging checks.

This is a source-breaking change targeting main.

@codecov

codecov Bot commented Sep 5, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (db8da8c) to head (fa6f3a1).

Additional details and impacted files
@@             Coverage Diff             @@
##              main       #43     +/-   ##
===========================================
  Coverage   100.00%   100.00%             
===========================================
  Files           37         5     -32     
  Lines         1496       324   -1172     
  Branches         0        37     +37     
===========================================
- Hits          1496       324   -1172     
Flag Coverage Δ
python-3.12 99.07% <99.07%> (?)
python-3.13 99.07% <99.07%> (?)
python-3.14 99.38% <99.38%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ericklt
ericklt marked this pull request as draft September 5, 2026 01:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant