Skip to content

[AUTOMATED] Keep the MSVC frame cookie out of a call's stack arguments - #1594

Open
borancar wants to merge 4 commits into
Noelo-Lab:mainfrom
borancar:gs-cookie
Open

borancar wants to merge 4 commits into
Noelo-Lab:mainfrom
borancar:gs-cookie

Conversation

@borancar

Copy link
Copy Markdown
Contributor

The /GS cookie an MSVC SEH prologue pushes becomes a stack argument of calls in the body:

$ kuna decompile decompiler/crates/kuna-analysis/tests/fixtures/stackcookie_pe_i386.exe 0x401000
  dat_40300c = (*dat_403004)(s_40301c,v1);
$ kuna decompile decompiler/crates/kuna-analysis/tests/fixtures/stackcookie_pe_i386.exe 0x401050
  sub_4010c0(a0 + 8,v1);

With this change:

  dat_40300c = (*dat_403004)(s_40301c);
  sub_4010c0(a0 + 8);

Fix

checkInputTrialUse marks a stack trial no-use at every call when its value is the frame cookie (p4_calls/kuna_stackcookie.rs). On every path, the value must be an INT_XOR of a value read from a fixed data address with the stack pointer, or an offset from it. The check follows the value back through copies, the INDIRECTs of earlier calls, and joins. That covers xor eax,esp; push eax and xor eax,ebp; mov [ebp-4],eax. Only the epilogue reads the slot back, in ECX. This is a correctness fix, so there is no option.

Tests

tests/stages/kuna-stackcookie.xml uses the new clean-room fixture stackcookie_pe_i386.exe:

  • an SEH frame around a call through a pointer;
  • the same frame around a this-only method;
  • an EBP frame storing the cookie at [ebp-4] and checking it.

On main, STACKCOOKIE #1 and #2 fail. STACKCOOKIE #3 and #4 pin the EBP frame, which already printed right, and the cookie still reaching the check in ECX.

verify_pebnames matched its SEH record by the name v2. On main the fixture's cookie was v1, as sub_401400's stray argument, so the record is now v1. The test now matches the record's shape, not its number.

python -m scripts.prepr.aritydiff against main:

  • Of the five public MSVC binaries, one call changes: WinHelloCPP.exe MoveToEx goes from 5 to 4 arguments, dropping the cookie.
  • A 2,198-function MSVC game is unchanged.
  • A 23,101-function VC8 game loses 2,653 arguments in 1,920 functions. Every one I checked is the cookie, or a frame slot the trial list only reached through it. Examples:
    • operator new(0x178) called as push 0x178; call; add esp,4, which went from 3 arguments to 1;
    • this-only calls with no push.
  • Three calls in that game gain arguments, all wrong.
  • decompile-all time is unchanged.

Fixes #1593

🤖 Generated with Claude Code

borancar and others added 4 commits November 15, 2026 10:45
A stack trial whose value is a global XORed with the stack or frame
pointer (/GS and EH: mov eax,[___security_cookie]; xor eax,esp; push eax)
is no-use at every call. Pushed in an SEH prologue, the cookie sat right
below the frame's slots and became the trailing argument of the first call
that took more than its own pushes.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
…iable number

The fixture's /GS cookie no longer becomes sub_401400's argument, so the
registration record is the first local and its name moves from v2 to v1.

Assertion-change: verify_pebnames: the record's name follows the dropped cookie variable

Co-Authored-By: Claude Opus 5.5 <[email protected]>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[AUTOMATED] x86-32: the MSVC /GS cookie pushed in an SEH prologue is printed as a call's stack argument

1 participant