Skip to content

gemm: asymmetric INT8xINT4 (4-bit weights) on AIE2P 4x16x16 mmul — bit-exact + W4A8 llama decode - #182

Open
bong-water-water-bong wants to merge 19 commits into
amd:develfrom
1bit-MONSTER:devel
Open

gemm: asymmetric INT8xINT4 (4-bit weights) on AIE2P 4x16x16 mmul — bit-exact + W4A8 llama decode#182
bong-water-water-bong wants to merge 19 commits into
amd:develfrom
1bit-MONSTER:devel

llama: W4A8_ATTN_I8 knob — q,k,v want i8, only o_proj tolerates i4

fba7943
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs