Can Workflow Search Be Amortized?
Demonstration-Guided Synthesis of LLM Agent Workflows
📄 Paper · 🔎 Overview · 🏆 Results · 🚀 Getting Started · 📚 Resources · 📌 Roadmap · 📝 Citation
Reuse prior workflow search. Synthesize a workflow for a new task in one generation pass.
- 🎉 SWIFT has been accepted to the EMNLP 2026 Main Conference!
- [2026-04-27] The SWIFT preprint is available on arXiv.
- The idea: reuse workflow structures discovered on source tasks to synthesize a workflow for a new task in one generation pass.
- The ingredients: executable workflow demonstrations, distilled design heuristics, and output-interface contracts.
- The result: 85.34% average accuracy on five main benchmarks with GPT-4o-mini, with substantially lower marginal optimization cost given an existing source bank.
🚧 Code release in preparation. The paper overview and results are available now. Implementation, workflow assets, and runnable examples will follow.
📑 Table of Contents
Designing an LLM agent workflow often means searching over many combinations of reasoning, verification, ensembling, and code execution for every new task. SWIFT reuses what previous searches have already discovered.
Given source-task search trajectories and workflow demonstrations, SWIFT distills reusable design heuristics and output-interface contracts. A single LLM generation call then combines these priors with a new task description to synthesize an executable workflow, without iterative target-task optimization.
Method overview from Figure 2 of the camera-ready paper.
Workflow synthesis produces the program; workflow execution runs that program and may use multiple operator and LLM calls. The reported cost advantage concerns optimization with an existing source bank.
Compare successful and unsuccessful source-task workflows and their execution or parsing failures to extract:
- Compositional heuristics: design rules for choosing and connecting operators.
- Output contracts: intermediate and final output-format constraints that help operators exchange compatible results.
- Workflow demonstrations: source-task descriptions, executable workflow code, and design rationales.
Build a meta-prompt containing the operator library, distilled heuristics, output contracts, source demonstrations, and the target-task description. The generator produces one executable candidate workflow in a single pass.
Run the generated program on task inputs using the execution model. Workflow synthesis and workflow execution are separate stages: the synthesized structure can coordinate reasoning, verification, ensembling, or execution-feedback repair.
Evaluation protocol. Source-pool tasks use task-level leave-one-out: the target task's examples, search trajectories, error logs, and optimized workflows are excluded from the synthesis inputs. Held-out benchmarks are absent from both trajectory distillation and demonstration construction.
Start with the paper and method overview. This repository does not yet include an installable package or runnable implementation.
| Resource | Availability |
|---|---|
| Paper | Read on arXiv |
| Method overview & reported results | Available in this README |
| Synthesis & execution implementation | Planned for the code release |
| Demonstrations, heuristics & output contracts | Planned for the code release |
| Installation & minimal example | Planned alongside the implementation |
| Custom-task guide & reproduction configs | Planned after the end-to-end example |
The camera-ready evaluation covers math reasoning, code generation, and question answering. Inclusion in this table describes the paper's experiments, not currently available dataset loaders.
| Evaluation group | Benchmarks | Role |
|---|---|---|
| Source pool | GSM8K, MATH, HumanEval, MBPP | Task-level leave-one-out synthesis |
| Held-out transfer | MultiArith, SVAMP, AIME 2024–2025, AQuA, BigCodeBench | Never included in the source demonstration pool |
| Execution-backbone portability | GSM8K, MATH, HumanEval, MBPP, DROP, HotpotQA | Fixed workflows executed with Grok, Qwen, and Gemma |
Results below are reported in the camera-ready paper; they have not yet been reproduced from this repository.
Test accuracy (%) with GPT-4o-mini. Selected comparisons from Table 1; the paper includes the full baseline set.
| Method | GSM8K | MATH | HumanEval | MBPP | MultiArith¹ | Average |
|---|---|---|---|---|---|---|
| Vanilla | 87.45 | 46.29 | 87.08 | 71.83 | 96.85 | 77.90 |
| AFlow | 91.16 | 51.28 | 90.93 | 81.67 | 96.22 | 82.25 |
| MaAS | 92.30 | 51.82 | 92.85 | 82.17 | 98.80 | 83.59 |
| SWIFT | 94.31 | 55.14 | 95.42 | 83.28 | 98.54 | 85.34 |
¹ MultiArith is not part of the source demonstration pool. SWIFT improves over AFlow on all five main benchmarks and has the highest average accuracy among the methods compared in the paper.
🌍 Transfer to additional held-out benchmarks
Accuracy (%) from Table 2. None of these benchmarks is included in the source demonstration pool.
| Benchmark | Vanilla | SWIFT | Gain (percentage points) |
|---|---|---|---|
| SVAMP | 81.0 | 95.8 | +14.8 |
| AIME 2024–2025 | 8.3 | 14.6 | +6.3 |
| AQuA | 52.9 | 79.9 | +27.0 |
| BigCodeBench | 31.6 | 34.3 | +2.7 |
⏱️ Optimization efficiency and source-bank accounting
MATH with GPT-4o-mini, from Table 3. These columns describe workflow optimization, excluding execution on the test set.
| Method | Optimization cost (USD) | Optimization time | Accuracy (%) |
|---|---|---|---|
| AFlow | $22.50 | 184 min | 51.28 |
| MaAS | $3.38 | 53 min | 51.82 |
| SWIFT | $0.004 | 0.13 min | 55.14 |
SWIFT's reported optimization cost includes contrastive distillation over available source trajectories and single-pass synthesis. It excludes the search needed to create those source trajectories. Under the paper's cold-start accounting, the upstream source bank costs approximately $112.50, with a break-even point of about five downstream tasks relative to per-task search (Section 4.2 and Appendix G). These are experimental measurements, not current API price estimates or guarantees for arbitrary tasks.
🔄 Execution-model portability
Holding the synthesized workflows and operator instructions fixed, the paper evaluates Grok-4.1-fast, Qwen3-8B, and Gemma-3-12B-IT across six benchmarks. SWIFT improves accuracy on 16 of 18 benchmark–model pairs (Table 5). Gains vary by model and task; for example, Gemma's MATH accuracy decreases, illustrating that transferred workflows depend on the executor's instruction-following ability.
SWIFT is most relevant when source workflows or search traces already exist, or when their construction cost can be shared across multiple related tasks. The paper primarily evaluates static reasoning and code-generation benchmarks with automatic correctness signals.
Transfer is sensitive to task-family alignment, output-interface compatibility, execution dependencies, and the capabilities of the execution model. The reported results do not establish effectiveness in general stochastic, interactive, or irreversible environments.
Explore the research resource guide for an annotated reading path, official implementations, benchmark tools, and suggestions for designing informative comparisons.
| Direction | Starting points |
|---|---|
| Workflow & architecture search | AFlow, ADAS, MaAS, AgentSquare |
| Workflow & experience reuse | Agent Workflow Memory, SWIFT |
| LM program optimization | DSPy, TextGrad |
| Benchmarks & evaluation | GSM8K, MATH, EvalPlus, BigCodeBench, WebArena |
Each entry explains its research relevance and links to primary sources. Contributions of relevant papers, tools, and corrected links are welcome.
- Paper overview, method figure, and main results.
- arXiv citation and paper link.
- Release workflow synthesis and execution code.
- Release source demonstrations, distilled heuristics, and output contracts.
- Add verified installation instructions and a minimal end-to-end example.
- Document how to adapt SWIFT to a custom task.
- Add evaluation configurations and reproduction instructions.
- Establish automated checks and document tested model configurations.
We welcome feedback on the paper, documentation improvements, and suggestions for tasks where workflow reuse would be useful.
- Questions or ideas: open an issue.
- Documentation fixes: submit a focused pull request describing the change.
- Future code contributions: task adapters, workflow demonstrations, model integrations, and reproducibility improvements are useful directions once the implementation is available.
Development and testing instructions will be added with the code release.
If you find SWIFT helpful for your research, please consider citing our paper:
@misc{du2026swift,
title = {Why Search When You Can Transfer? Amortized Agentic Workflow Design from Structural Priors},
author = {Du, Shiyi and Liu, Jiayuan and Du, Weihua and Huang, Yue and Li, Jiayi and Luo, Yingtao and Zhang, Xiangliang and Conitzer, Vincent and Kingsford, Carl},
year = {2026},
eprint = {2604.25012},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
doi = {10.48550/arXiv.2604.25012},
url = {https://arxiv.org/abs/2604.25012}
}