Skip to content

Mac: the mouse as the left or right stick (DM_MOUSE_STICK) - #2392

Open
SanteeMarcel wants to merge 2 commits into
seedds:mainfrom
SanteeMarcel:mac-mouse-stick
Open

SanteeMarcel wants to merge 2 commits into
seedds:mainfrom
SanteeMarcel:mac-mouse-stick

Conversation

@SanteeMarcel

Copy link
Copy Markdown

What

On the Mac, the mouse can now drive a stick instead of being the touchscreen. Addresses #2346 (mouse as the right stick) and adds the left stick, which a reticle needs: Pokémon: Let's Go aims the Poké Ball with the left stick in handheld mode and throws with A.

  • DM_MOUSE_STICK=left / right (settings window: Controls › Mouse). Unset or off: the touchscreen, as before.
  • A click on the game picture captures the pointer (hidden, held in place); Escape or a focus loss gives it back. The capturing click reaches the game as nothing.
  • Movement adds to a stick position inside the unit circle that drifts back to the centre (half-life DM_MOUSE_RECENTER, 80 ms; 0 holds it). DM_MOUSE_SENS scales it (1.0 = 100 points for full deflection).
  • Mouse buttons: A / B for the left stick, ZR / ZL for the right one, or DM_MOUSE_BUTTONS=<left>,<right> by DM_KEYS name.
  • Each axis is merged with the keyboard's (the more deflected wins); buttons are ORed into the keyboard path.
  • The capture is let go wherever the touch is lifted today: focus loss, the Home quit prompt, the controller applet prompt, the software keyboard box. No capture starts while a prompt has the input.

Files

  • dm/host/mouse_stick.h (new, pure) and dm/host/tests/t_mouse_stick.c (new)
  • dm/host/front_mac.m: capture, event monitor, buttons, push() merge, releases
  • dm/games/settings.tsv: DM_MOUSE_STICK row (platform:mac)
  • docs/env_switches.tsv, docs/settings.md (Mouse), docs/playing.md
  • dm/host/Makefile, dm/host/tests/Makefile, dm/tools/tests/test_launcher_settings.py

Overlaps with #1490 (mouse camera on the Mac and iPad); happy to rebase onto it or fold this into it.

Checked

  • t_mouse_stick: 26 checks, also under ASan and UBSan
  • test_launcher_settings, test_settings, test_docs, test_env_switches

Not run

  • A build of front_mac.m and a game run: this was written without a Mac at hand. Needs a Mac build and a try before merge.
  • regress.py check: the change is front-end input only and reaches no game path.
  • lint_c.py, iOS (not touched).

@github-actions github-actions Bot added host The Horizon host: SVCs, services, threads, memory (dm/host) mac The Mac front end and app (dm/mac, front_mac.m) tools Tools and scripts (dm/tools, scripts) docs Documentation labels Nov 11, 2026
SanteeMarcel and others added 2 commits November 12, 2026 15:58
…settings window's Controls › Mouse), a click on the game picture captures the pointer and its movement pushes that stick, its buttons are Switch buttons, and Escape or a focus loss lets it go; unset or off, the mouse is the touchscreen as before

The mouse was only the touchscreen (front_mac.m: left button = finger 0), and the keyboard's right stick is Shift + the arrow keys, all or nothing (issue seedds#2346), so a reticle aimed with the left stick (Pokémon: Let's Go's throw in handheld mode) or a camera could not be moved by the mouse. Now mouse_stick.h (pure) keeps a stick position inside the unit circle: movement adds to it (DM_MOUSE_SENS, 1.0 = 100 points for a full deflection) and it goes back to the centre with a half-life of DM_MOUSE_RECENTER ms (80; 0 holds it in place), from a 120 Hz timer that runs only while captured, since a still mouse sends no events. front_mac.m captures with CGAssociateMouseAndMouseCursorPosition(false) and a hidden cursor, reads the deltas from a local event monitor (moved and dragged), merges each axis with the keyboard's (the more deflected wins, as the controller paths merge) and ORs the mouse buttons into the keyboard path: A / B for the left stick, ZR / ZL for the right one, or DM_MOUSE_BUTTONS by DM_KEYS name. The capture is let go wherever the touch is lifted: focus loss, the Home quit prompt, the controller applet's prompt, the software keyboard box; no capture starts while a prompt has the input. The capturing click reaches the game as nothing. New settings row DM_MOUSE_STICK (platform:mac), env_switches rows, settings.md (Mouse) and playing.md.

Front-ends: new: DM_MOUSE_STICK (off / left / right) with DM_MOUSE_SENS, DM_MOUSE_RECENTER and DM_MOUSE_BUTTONS, read by the Mac front end only; a Mac front end can offer it as a Controls setting

Checked: t_mouse_stick (new, 26 checks: the settings and their defaults, bad values, movement and the unit-circle clamp, the Y axis, recentering and its half-life, a held stick, button names, Switch units, merging), also under ASan and UBSan; test_launcher_settings (the Mac-only set gains DM_MOUSE_STICK), test_settings, test_docs, test_env_switches. Not run: a build of front_mac.m and any game (written without a Mac), regress.py (front end input only, no game path), lint_c.py, iOS (not touched)
The residency, upload-source and depth-zero tests passed nullable Metal
allocations or functions to APIs that require non-null arguments. These
three analyzer findings on main would keep lint-c failing after rebasing
the mouse-stick PR onto main.

Check buffer allocation in the residency helper, return a failed texel
comparison when the upload readback buffer cannot be allocated, and check
the depth counter's library and function before creating its pipeline.
The lint baseline is unchanged.

Checked: the three GPU tests on Apple M5 (408 checks, zero failures),
their builds, and clang-tidy on the affected files with LLVM 23.1.5
(no findings above or below the baseline). CI pins LLVM 23.1.4.
@github-actions github-actions Bot added the nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) label Nov 12, 2026
chgpalmer added a commit to chgpalmer/nxnative that referenced this pull request Nov 17, 2026
On the Mac the mouse was only the touch screen, so a game's camera and its
aiming could be turned only with the keyboard's all-or-nothing right stick:
Shift + arrows deflect the stick fully or not at all, so aim is corrected by
strafing, and a quick turn in a fight is not possible (seedds#2346, Red Dead
Redemption; here Ocarina of Time's camera, hookshot, bow and boomerang:
shooting the Poes from horseback in Hyrule Field is all but impossible on
the keys and works with the trackpad). Mouse
movement becomes a source of each analog output, as keys are of buttons: a
Mouse and trackpad section after the keys lists the right stick, the left
stick and the motion controls, each with its source.

- A source is None (the default: the touch screen, as before), Mouse, or a
  key + Mouse: the output takes the movement only while that key is held,
  the way the previous commit's chords make a key + a modifier a binding of
  its own. One output takes the movement at a time, a held key's first, and
  setting the plain Mouse or a key on one output takes it from another. This
  is Steam Input's mode shift (a key held switches the trackpad's mode),
  listed by the outputs it feeds, as the keys are; one section, since the
  outputs share one pointer and the section's note gives the rule.
- With a source set, a click on the picture captures the pointer as seedds#2346
  proposes (CGAssociateMouseAndMouseCursorPosition, the
  NSEvent deltas): hidden and held in place. Esc, a focus loss, the Keyboard
  Map panel, the controller applet's prompt and the text box let it go.
- A stick: a game turns its camera at a rate, so the movement's speed is the
  deflection, measured at 120 Hz, smoothed and lifted past a typical dead zone
  so slow movement still turns; the keys' stick wins where it is pushed
  further. The left stick is there for games that aim with it (seedds#2392).
- Motion controls: the movement turns player 1's motion sensor as a controller
  turned in the hands, a world-space turn as gyro aiming calls it (horizontal
  about the world's vertical, vertical a tilt within 70 degrees, the gravity
  following the tilt), so a game's own motion aiming gets fine analogue input.
  The rates are smoothed with a gain of 1, so the angle turned is the
  movement's. The controller rests as the console mode expects: upright in
  handheld mode (Ocarina of Time reads its horizontal aim about that axis and
  barely turned with a flat controller), flat when docked. It goes in through
  dm_motion_set_switch, the Switch-frame stream the iPhone's own sensor uses,
  which player 1 reads while no controller sends motion.
- Ocarina of Time's camera stick wins over its motion aiming, so there: the
  right stick's Mouse, and the motion controls' Space + Mouse (Space is ZR,
  the item). With the motion controls set the sensor keeps reporting while
  the movement goes to a stick, so the game never loses it.
- Each output keeps its own speed and Invert Y, shown while it has a source.
- Mouse buttons are keys of the map (click, rclick, mclick, mouse4, mouse5),
  bound as keys are in game keybinding screens: while a binding waits for
  its key, a mouse button pressed is the input (Add Key... puts one beside
  a key, ZR=space,click):
  while captured, or bound, they press what they are bound to.

All of it is part of DM_KEYMAP and launcher.DM_KEYMAP (MOUSERSTICK=mouse,
MOUSEMOTION=space+mouse, MOUSERSTICKSPEED ... MOUSEMOTIONINVERT) and of the
keymap files, and both the settings window and the in-game panel show it.

Tests: t_keymap (102 checks, ASan and UBSan: the sources, a key + Mouse held
and let go, one output each, either side of a modifier, the full form and
keymap files, mouse buttons as keys and in chords, the sticks merged);
test_nsp_mac_app, test_docs, test_env_switches, test_usage; lint_c.py tidy
with LLVM 23: no findings.

Played on a MacBook Pro (M3 Pro, 18 GB, macOS 27.2.1) with Ocarina of Time
(0400486024DEC000, v131072), handheld and docked, with the trackpad: the camera
on the right stick, and with Space held the boomerang aimed by motion, alike
in both modes; Esc gives the pointer back, Cmd-Shift-K opens the panel, the
chosen preset holds across starts, and no added input lag was noticed. The
host was built as for the commit before this one.

Front-ends: nothing needed; the mouse settings are part of DM_KEYMAP (docs/env_switches.tsv).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
chgpalmer added a commit to chgpalmer/nxnative that referenced this pull request Nov 18, 2026
On the Mac the mouse was only the touch screen, so a game's camera and its
aiming could be turned only with the keyboard's all-or-nothing right stick:
Shift + arrows deflect the stick fully or not at all, so aim is corrected by
strafing, and a quick turn in a fight is not possible (seedds#2346, Red Dead
Redemption; here Ocarina of Time's camera, hookshot, bow and boomerang:
shooting the Poes from horseback in Hyrule Field is all but impossible on
the keys and works with the trackpad). Mouse
movement becomes a source of each analog output, as keys are of buttons: a
Mouse and trackpad section after the keys lists the right stick, the left
stick and the motion controls, each with its source.

- A source is None (the default: the touch screen, as before), Mouse, or a
  key + Mouse: the output takes the movement only while that key is held,
  the way the previous commit's chords make a key + a modifier a binding of
  its own. One output takes the movement at a time, a held key's first, and
  setting the plain Mouse or a key on one output takes it from another. This
  is Steam Input's mode shift (a key held switches the trackpad's mode),
  listed by the outputs it feeds, as the keys are; one section, since the
  outputs share one pointer and the section's note gives the rule.
- With a source set, a click on the picture captures the pointer as seedds#2346
  proposes (CGAssociateMouseAndMouseCursorPosition, the
  NSEvent deltas): hidden and held in place. Esc, a focus loss, the Keyboard
  Map panel, the controller applet's prompt and the text box let it go.
- A stick: a game turns its camera at a rate, so the movement's speed is the
  deflection, measured at 120 Hz, smoothed and lifted past a typical dead zone
  so slow movement still turns; the keys' stick wins where it is pushed
  further. The left stick is there for games that aim with it (seedds#2392).
- Motion controls: the movement turns player 1's motion sensor as a controller
  turned in the hands, a world-space turn as gyro aiming calls it (horizontal
  about the world's vertical, vertical a tilt within 70 degrees, the gravity
  following the tilt), so a game's own motion aiming gets fine analogue input.
  The rates are smoothed with a gain of 1, so the angle turned is the
  movement's. The controller rests as the console mode expects: upright in
  handheld mode (Ocarina of Time reads its horizontal aim about that axis and
  barely turned with a flat controller), flat when docked. It goes in through
  dm_motion_set_switch, the Switch-frame stream the iPhone's own sensor uses,
  which player 1 reads while no controller sends motion.
- Ocarina of Time's camera stick wins over its motion aiming, so there: the
  right stick's Mouse, and the motion controls' Space + Mouse (Space is ZR,
  the item). With the motion controls set the sensor keeps reporting while
  the movement goes to a stick, so the game never loses it.
- Each output keeps its own speed and Invert Y, shown while it has a source.
- With the mouse section the map no longer fits one column, so both views
  lay it out in columns, as other emulators lay out their input pages:
  Buttons; the sticks and the D-pad; Motion and the mouse. The in-game panel
  opens at the size the three columns need, so the whole map shows beside a
  fullscreen game; the settings sheet puts each group in a card and packs the
  cards into as many columns as the settings window is wide (one to three).
  Both still scroll on a small screen.
- Mouse buttons are keys of the map (click, rclick, mclick, mouse4, mouse5),
  bound as keys are in game keybinding screens: while a binding waits for
  its key, a mouse button pressed is the input (Add Key... puts one beside
  a key, ZR=space,click):
  while captured, or bound, they press what they are bound to.

All of it is part of DM_KEYMAP and launcher.DM_KEYMAP (MOUSERSTICK=mouse,
MOUSEMOTION=space+mouse, MOUSERSTICKSPEED ... MOUSEMOTIONINVERT) and of the
keymap files, and both the settings window and the in-game panel show it.

Tests: t_keymap (102 checks, ASan and UBSan: the sources, a key + Mouse held
and let go, one output each, either side of a modifier, the full form and
keymap files, mouse buttons as keys and in chords, the sticks merged);
test_nsp_mac_app, test_docs, test_env_switches, test_usage; lint_c.py tidy
with LLVM 23: no findings.

Played on a MacBook Pro (M3 Pro, 18 GB, macOS 27.2.1) with Ocarina of Time
(0400486024DEC000, v131072), handheld and docked, with the trackpad: the camera
on the right stick, and with Space held the boomerang aimed by motion, alike
in both modes; Esc gives the pointer back, Cmd-Shift-K opens the panel, the
chosen preset holds across starts, and no added input lag was noticed. The
host was built as for the commit before this one.

Front-ends: nothing needed; the mouse settings are part of DM_KEYMAP (docs/env_switches.tsv).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
chgpalmer added a commit to chgpalmer/nxnative that referenced this pull request Nov 18, 2026
On the Mac the mouse was only the touch screen, so a game's camera and its
aiming could be turned only with the keyboard's all-or-nothing right stick:
Shift + arrows deflect the stick fully or not at all, so aim is corrected by
strafing, and a quick turn in a fight is not possible (seedds#2346, Red Dead
Redemption; here Ocarina of Time's camera, hookshot, bow and boomerang:
shooting the Poes from horseback in Hyrule Field is all but impossible on
the keys and works with the trackpad). Mouse
movement becomes a source of each analog output, as keys are of buttons: a
Mouse and trackpad section after the keys lists the right stick, the left
stick and the motion controls, each with its source.

- A source is None (the default: the touch screen, as before), Mouse, or a
  key + Mouse: the output takes the movement only while that key is held,
  the way the previous commit's chords make a key + a modifier a binding of
  its own. One output takes the movement at a time, a held key's first, and
  setting the plain Mouse or a key on one output takes it from another. This
  is Steam Input's mode shift (a key held switches the trackpad's mode),
  listed by the outputs it feeds, as the keys are; one section, since the
  outputs share one pointer and the section's note gives the rule.
- With a source set, a click on the picture captures the pointer as seedds#2346
  proposes (CGAssociateMouseAndMouseCursorPosition, the
  NSEvent deltas): hidden and held in place. Esc, a focus loss, the Keyboard
  Map panel, the controller applet's prompt and the text box let it go.
- A stick: a game turns its camera at a rate, so the movement's speed is the
  deflection, measured at 120 Hz, smoothed and lifted past a typical dead zone
  so slow movement still turns; the keys' stick wins where it is pushed
  further. The left stick is there for games that aim with it (seedds#2392).
- Motion controls: the movement turns player 1's motion sensor as a controller
  turned in the hands, a world-space turn as gyro aiming calls it (horizontal
  about the world's vertical, vertical a tilt within 70 degrees, the gravity
  following the tilt), so a game's own motion aiming gets fine analogue input.
  The rates are smoothed with a gain of 1, so the angle turned is the
  movement's. The controller rests as the console mode expects: upright in
  handheld mode (Ocarina of Time reads its horizontal aim about that axis and
  barely turned with a flat controller), flat when docked. It goes in through
  dm_motion_set_switch, the Switch-frame stream the iPhone's own sensor uses,
  which player 1 reads while no controller sends motion.
- Ocarina of Time's camera stick wins over its motion aiming, so there: the
  right stick's Mouse, and the motion controls' Space + Mouse (Space is ZR,
  the item). With the motion controls set the sensor keeps reporting while
  the movement goes to a stick, so the game never loses it.
- Each output keeps its own speed and Invert Y, shown while it has a source.
- With the mouse section the map no longer fits one column, so both views
  lay it out in columns, as other emulators lay out their input pages:
  Buttons; the sticks and the D-pad; Motion and the mouse. The in-game panel
  opens at the size the three columns need, so the whole map shows beside a
  fullscreen game; the settings sheet puts each group in a card and packs the
  cards into as many columns as the settings window is wide (one to three).
  Both still scroll on a small screen.
- Mouse buttons are keys of the map (click, rclick, mclick, mouse4, mouse5),
  bound as keys are in game keybinding screens: while a binding waits for
  its key, a mouse button pressed is the input (Add Key... puts one beside
  a key, ZR=space,click):
  while captured, or bound, they press what they are bound to.

All of it is part of DM_KEYMAP and launcher.DM_KEYMAP (MOUSERSTICK=mouse,
MOUSEMOTION=space+mouse, MOUSERSTICKSPEED ... MOUSEMOTIONINVERT) and of the
keymap files, and both the settings window and the in-game panel show it.

Tests: t_keymap (102 checks, ASan and UBSan: the sources, a key + Mouse held
and let go, one output each, either side of a modifier, the full form and
keymap files, mouse buttons as keys and in chords, the sticks merged);
test_nsp_mac_app, test_docs, test_env_switches, test_usage; lint_c.py tidy
with LLVM 23: no findings.

Played on a MacBook Pro (M3 Pro, 18 GB, macOS 27.2.1) with Ocarina of Time
(0400486024DEC000, v131072), handheld and docked, with the trackpad: the camera
on the right stick, and with Space held the boomerang aimed by motion, alike
in both modes; Esc gives the pointer back, Cmd-Shift-K opens the panel, the
chosen preset holds across starts, and no added input lag was noticed. The
host was built as for the commit before this one.

Front-ends: nothing needed; the mouse settings are part of DM_KEYMAP (docs/env_switches.tsv).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation host The Horizon host: SVCs, services, threads, memory (dm/host) mac The Mac front end and app (dm/mac, front_mac.m) nvn NVN on Metal and its tests (dm/gfx/nvn, dm/gfx/tests) tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants