Skip to content

Eliminate concat after reshapes - #5137

Open
pfultz2 wants to merge 12 commits into
developfrom
eliminate-concat-after-reshapes
Open

Eliminate concat after reshapes#5137
pfultz2 wants to merge 12 commits into
developfrom
eliminate-concat-after-reshapes

Conversation

@pfultz2

@pfultz2 pfultz2 commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Technical Details

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

    • Added: New functionality.
    • Changed: Changes to existing functionality.
    • Removed: Functionality or support that has been removed. (Compared to a previous release)
    • Optimized: Component performance that has been optimized or improved.
    • Resolved Issues: Known issues from a previous version that have been resolved.
    • Not Applicable: This PR is not to be included in the changelog.

Follow the LLVM AI Tool Use Policy for contributions using AI.

@pfultz2 pfultz2 mentioned this pull request Aug 19, 2026
Draft
@gh-app-migraphx-bot-pr-write

gh-app-migraphx-bot-pr-write Bot commented Aug 22, 2026

Copy link
Copy Markdown
Test Batch New Rate (aeb582) Old Rate (3a503c)* Diff Status
torchvision-resnet50 64 3,327.93 3,264.92 1.93% ✅
torchvision-resnet50_fp16 64 8,025.50 7,548.67 6.32% 🔆
torchvision-densenet121 32 nan 2,483.99 nan ❌
torchvision-densenet121_fp16 32 nan 5,004.24 nan ❌
torchvision-inceptionv3 32 2,070.50 2,058.51 0.58% ✅
torchvision-inceptionv3_fp16 32 4,445.67 4,416.99 0.65% ✅
cadene-inceptionv4 16 819.20 820.61 -0.17% ✅
cadene-resnext64x4 16 782.94 782.78 0.02% ✅
slim-mobilenet 64 8,385.26 8,386.36 -0.01% ✅
slim-nasnetalarge 64 nan 228.86 nan ❌
slim-resnet50v2 64 3,238.87 3,180.91 1.82% ✅
bert-mrpc-onnx 8 1,168.77 1,168.84 -0.01% ✅
bert-mrpc-tf 1 491.79 498.63 -1.37% ✅
pytorch-examples-wlang-gru 1 483.39 473.35 2.12% ✅
pytorch-examples-wlang-lstm 1 405.74 384.83 5.43% 🔆
torchvision-resnet50_1 1 1,054.88 1,046.63 0.79% ✅
cadene-dpn92_1 1 446.62 437.32 2.13% ✅
cadene-resnext101_1 1 366.14 365.89 0.07% ✅
onnx-taau-downsample 1 846.81 844.09 0.32% ✅
dlrm-criteoterabyte 1 nan 32.42 nan ❌
dlrm-criteoterabyte_fp16 1 nan 51.80 nan ❌
agentmodel 1 14,861.80 9,209.12 61.38% 🔆
unet_fp16 2 58.33 58.80 -0.80% ✅
resnet50v1_fp16 1 1,450.29 1,366.11 6.16% 🔆
resnet50v1_int8 1 1,780.11 1,883.96 -5.51% 🔴
bert_base_cased_fp16 64 1,097.58 1,098.16 -0.05% ✅
bert_large_uncased_fp16 32 345.69 345.59 0.03% ✅
bert_large_fp16 1 206.71 206.59 0.06% ✅
distilgpt2_fp16 16 2,092.23 2,092.89 -0.03% ✅
yolov5s 1 570.79 558.33 2.23% ✅
tinyllama 1 45.80 45.83 -0.06% ✅
vicuna-fastchat 1 44.28 44.20 0.18% ✅
whisper-tiny-encoder 1 412.88 411.87 0.24% ✅
whisper-tiny-decoder 1 410.89 408.48 0.59% ✅
llama2_7b 1 20.87 20.84 0.14% ✅
qwen1.5-7b 1 23.65 23.58 0.30% ✅
phi3-3.8b 1 26.70 26.72 -0.07% ✅
llama3-8b 1 21.72 21.80 -0.38% ✅
whisper-large-encoder 1 10.17 10.18 -0.07% ✅
whisper-large-decoder 1 106.67 105.30 1.30% ✅
mistral-7b 1 23.77 23.78 -0.04% ✅
FLUX.1-schnell 1 775.72 755.22 2.72% ✅

Regressions detected 🔴

* No develop baseline was found for this PR's branch point; compared against the latest available develop run instead.

@gh-app-migraphx-bot-pr-write

gh-app-migraphx-bot-pr-write Bot commented Aug 22, 2026

Copy link
Copy Markdown
Test Status Result
bert-mrpc-onnx ✅ PASSED: MIGraphX meets tolerance
bert-mrpc-tf ❌ ERROR - check error output
traceback
Traceback (most recent call last):
  File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 377, in 
    main()
  File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 313, in main
    import tensorflow as tf
  File "/usr/local/lib/python3.12/dist-packages/tensorflow/init.py", line 40, in 
    from tensorflow.python import pywrap_tensorflow as _pywrap_tensorflow  # pylint: disable=unused-import
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 37, in 
    self_check.preload_check()
  File "/usr/local/lib/python3.12/dist-packages/tensorflow/python/platform/self_check.py", line 63, in preload_check
    from tensorflow.python.platform import _pywrap_cpu_feature_guard
ImportError: libnuma.so.1: cannot open shared object file: No such file or directory
pytorch-examples-wlang-gru ✅ PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-lstm ✅ PASSED: MIGraphX meets tolerance
dlrm-criteoterabyte ❌ ERROR - check error output
traceback
2026-08-22 13:56:25.821746 [WARN] [/data/src/adjust_allocation.cpp:67] output buffer doesn't match output for gpu::precompile_op[op=gpu::mlir_op[op=dot],additional_args=1,ignore_modules=0,output_shape=nullopt]
2026-08-22 13:56:25.821808 [WARN] [/data/src/adjust_allocation.cpp:67] output buffer doesn't match output for gpu::precompile_op[op=gather[axis=1],additional_args=1,ignore_modules=0,output_shape=nullopt]
2026-08-22 13:56:28.890738 [ERROR] [/data/src/pass_manager.cpp:176] Error gpu::compile_ops: /data/src/targets/gpu/code_object_op.cpp:50: compute_shape: Input shapes have changed: [float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 1728}, {1728, 1}] -> [float_type, {307200, 64}, {415, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 1728}, {1728, 1}]
2026-08-22 13:56:28.890797 [ERROR] [/data/src/pass_manager.cpp:186] Dump: "/tmp/migraphx/gpu::compile_ops437622016363177.mxr"
Traceback (most recent call last):
  File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 377, in 
    main()
  File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 246, in main
    model.compile(
RuntimeError: /data/src/targets/gpu/code_object_op.cpp:50: compute_shape: Input shapes have changed: [float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 1728}, {1728, 1}] -> [float_type, {307200, 64}, {415, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 64}, {64, 1}, float_type, {307200, 1728}, {1728, 1}]
agentmodel ✅ PASSED: MIGraphX meets tolerance
unet ✅ PASSED: MIGraphX meets tolerance
resnet50v1 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
bert_base_cased_fp16 ✅ PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
bert_large ✅ PASSED: MIGraphX meets tolerance
yolov5s ✅ PASSED: MIGraphX meets tolerance
tinyllama ✅ PASSED: MIGraphX meets tolerance
vicuna-fastchat ✅ PASSED: MIGraphX meets tolerance
whisper-tiny-encoder ✅ PASSED: MIGraphX meets tolerance
whisper-tiny-decoder ✅ PASSED: MIGraphX meets tolerance
distilgpt2_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
llama2_7b ✅ PASSED: MIGraphX meets tolerance
qwen1.5-7b ✅ PASSED: MIGraphX meets tolerance
phi3-3.8b ✅ PASSED: MIGraphX meets tolerance
llama3-8b ✅ PASSED: MIGraphX meets tolerance
whisper-large-encoder ✅ PASSED: MIGraphX meets tolerance
whisper-large-decoder ✅ PASSED: MIGraphX meets tolerance
mistral-7b ✅ PASSED: MIGraphX meets tolerance
FLUX.1-schnell ✅ PASSED: MIGraphX meets tolerance

@pfultz2
pfultz2 marked this pull request as ready for review August 31, 2026 18:24
@pfultz2
pfultz2 requested a review from causten as a code owner August 31, 2026 18:24
@pfultz2
pfultz2 requested review from TedThemistokleous, bdevorem, kahmed10 and shivadbhavsar and a lite review from Copilot August 31, 2026 18:24
Copilot AI reviewed Aug 31, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the eliminate_concat optimization to elide more concats when inputs are produced through invertible view chains (e.g., transpose/reshape), enabling producers to write directly into slices/views of a super-buffer. It also adjusts GPU precompile_op shape reporting to propagate the actual output-buffer layout (strides) so downstream shape/layout expectations match what kernels write.

Changes:

  • Teach eliminate_concat to walk/invert view chains and replace a producer’s output allocation with an inverse-view into the concat super-buffer when safe.
  • Update GPU precompile_op::compute_shape to return the provided output-buffer layout when it matches element type + lens but differs in layout.
  • Refresh and add tests covering concat elimination through view chains, including new test/verify cases for alias vs copy fallback.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/eliminate_concat.cpp Adds view-chain discovery/inversion and copy-elision planning for concat elimination.
src/targets/gpu/compile_ops.cpp Makes precompile_op report the output buffer’s layout when aliasing into larger buffers.
test/eliminate_concat_test.cpp Updates existing expected-module tests and adds new unit tests for reshape-chain and slice-input behavior.
test/verify/test_concat_view_chain_alias.cpp New verify test validating aliasing through transpose+reshape view chains.
test/verify/test_concat_view_chain_copy.cpp New verify test validating copy fallback when inverse view cannot alias safely.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/eliminate_concat.cpp
Comment on lines +165 to +180
auto [producer, chain] = get_view_chain(input);
if(not can_write_inplace(producer))
return std::nullopt;
if(not slice_shape.packed() and not opt->supports_non_packed_output(producer, axis))
return std::nullopt;
// other users just read the strided view; the caller checks that
// they support non-packed inputs
if(chain.empty())
return plan{producer};
if(producer->outputs().size() != 1)
return std::nullopt;
auto inverse = invert_view_chain(chain, producer->get_shape(), slice_shape);
if(not inverse.has_value())
return std::nullopt;
return plan{producer, *inverse};
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
@shivadbhavsar shivadbhavsar shivadbhavsar approved these changes
@causten causten Awaiting requested review from causten causten is a code owner
@TedThemistokleous TedThemistokleous Awaiting requested review from TedThemistokleous
@bdevorem bdevorem Awaiting requested review from bdevorem
@kahmed10 kahmed10 Awaiting requested review from kahmed10

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@pfultz2 @shivadbhavsar