Skip to content

ios/gamepad: optional swipe camera, floating left stick, touch settings - #258

Open
harshulkuhar wants to merge 3 commits into
seedds:mainfrom
harshulkuhar:ios-touch-control-settings
Open

harshulkuhar wants to merge 3 commits into
seedds:mainfrom
harshulkuhar:ios-touch-control-settings

Conversation

@harshulkuhar

@harshulkuhar harshulkuhar commented Oct 26, 2026 •

Copy link
Copy Markdown

What and why

In the 3D touch layout the camera is a floating right stick: a finger held away from where it landed keeps the camera turning, and short drags give little input, unlike a mobile swipe camera. Starting the left thumb near the stick's centre also walks the character through unintended directions. This adds five iOS rows to the launcher's Controls section, all in settings.tsv, stored and overridden like every other choice (#388). Every default keeps the current behaviour.

Setting Variable Default
Swipe camera: finger velocity instead of the floating right stick DM_TOUCH_SWIPE off
Swipe sensitivity DM_TOUCH_SWIPE_GAIN 100 % (25–400 %)
Camera touch area: where a camera touch may start, in either mode DM_TOUCH_RIGHT_AREA right third (or right half)
Left-stick dead zone DM_TOUCH_LEFT_DEADZONE 10 % (0–40 %)
Floating left stick DM_TOUCH_LEFT_FLOAT off
  • Swipe camera (DmSwipeLook.h): timestamped finger velocity is resampled at each display callback instead of being cleared every frame, which alternated tilt and neutral when touch, display and guest rates differed. Input expires 40 ms after the last movement sample, since UIKit sends none while a finger rests; release, cancellation and reset clear it at once. The gain is applied after the motion is normalized and clamped, so a fast swipe gives a quarter tilt at 25 % instead of saturating at every gain.
  • Floating left stick, as proposed earlier in this PR's conversation: a touch-down within 1.65 ring radii of the stick's home (1.4 for the fixed stick) becomes its centre, and the centre follows the thumb past the ring with the rule the camera stick already uses (dm_touch_follow, now shared by both). The ring is drawn at the centre while held and returns home on release. The diamond and D-pad keep their ownership rule (smallest distance over grab radius).
  • DmTouchControls.h holds the parsing, bounds and stick math without UIKit, so they are tested natively. Physical controllers and game code are unchanged.

The reference entry is Relative swipe camera input on iOS, with its line in the symptom index and in Breath of the Wild's note. docs/settings.md, docs/playing.md and docs/env_switches.tsv describe the five settings.

Checked

On ba9ce212, Xcode 27.2, Metal 32023.922, macOS on Apple silicon:

  • Tests: test_touch_controls (the native C test: mixed 30/60/120 Hz sampling and phases, expiry, reversal, gain bounds, parsing of invalid values, the unchanged radial response, the floating centre staying inside the ring, following past it and responding to a reversal at once), test_launcher_settings, test_launcher_storage (all five choices load, persist, survive relaunch, reset and env-file locking), test_settings, test_docs, test_pr_areas; docs_symptoms.py is current; git diff --check.
  • test_env_switches fails on main already (DM_NVN_* rows read nowhere, DM_GL_RENDERPROGS without a class); this change adds its row and does not touch the rest.
  • The Sanitizers job fails on main already: ASan reports a stack-use-after-return in svc_thread_impl (hle_thread.c:1081, reading g_unlock's local d) during t_condvar, reproduced locally on 87422462 with make -C dm/host/tests asan. This change touches no host code.
  • dm/ios/tools/compile_check.py: 264 files, 0 failed. lint_python.py: 0 new findings.
  • regress.py impact main..HEAD: no NVN or HLE entry point is reached; the four games listed are there only through the broad iOS-app rule. regress.py check runs Mac builds, which do not use DmGamepad.m, so none was run.
  • Device: Breath of the Wild 1.9.0 (01007EF00011E000) on an iPhone Duo, iOS 27.2, from an earlier launcher-integrated build of these settings: the left-stick dead zone removed the unintended movement, and 25 % and 400 % sensitivity feel clearly different.

Not checked: the floating left stick and this rebase on a device. At 400 % the camera is still limited by the game's own stick response: this is controller input, not direct camera rotation.

Docs

  • The finding is in its docs/reference/services.md entry, and python3 dm/tools/docs_symptoms.py was run
  • No game files, keys or data extracted from a game in the change or the attachments

@v0lumehi

Copy link
Copy Markdown
Contributor

Heads-up: seedds asked in #388 to agree on one settings schema and one store for player choices before the launchers are extended. I posted a proposal there (#388 (comment)): one optional meta column (key=value;…) in settings.tsv, a versioned launcher.json, and one store with one precedence. The touch-control settings would be rows with platform=ios, stored like every other choice. Input welcome in #388.

@harshulkuhar

Copy link
Copy Markdown
Author

Heads-up: seedds asked in #388 to agree on one settings schema and one store for player choices before the launchers are extended. I posted a proposal there (#388 (comment)): one optional meta column (key=value;…) in settings.tsv, a versioned launcher.json, and one store with one precedence. The touch-control settings would be rows with platform=ios, stored like every other choice. Input welcome in #388.

Hi, sorry this is my first time.

As far as I am aware, the four touch-control options are rows in settings.tsv, marked platform:ios, and go through the existing launcher.json generator.

They use the existing persistence, override precedence and reset handling, without introducing a separate format or store.

This builds on the shared settings infrastructure used by #355, #399 and #379.

Happy to have a conversation and make changes accordingly if need be.

@github-actions github-actions Bot added ios The iPhone and iPad app (dm/ios) tools Tools and scripts (dm/tools, scripts) docs Documentation game: 01007EF00011E000 The Legend of Zelda: Breath of the Wild labels Nov 3, 2026
@harshulkuhar
harshulkuhar force-pushed the ios-touch-control-settings branch from 9093aea to 69e9fee Compare November 3, 2026 15:11
@harshulkuhar

Copy link
Copy Markdown
Author

Rebased onto the latest main (8c80f39) and resolved the conflicts. The touch settings still use the shared settings list and existing persistence, override and reset handling from #388.

Local tests, lint and the iOS build checks pass with Xcode 27.2.

@scuan2

scuan2 commented Nov 10, 2026

Copy link
Copy Markdown

One complementary control option that may fit this PR: a floating left movement stick.

I’ve been using a very small DmGamepad variant on iPhone:

  • touch-down inside an enlarged home grab zone (1.65 × radius in my build) owns the left stick and that touch-down becomes its centre;
  • if the thumb moves beyond the stick radius, the centre follows just enough to keep the thumb at the radius — the same bounded-centre behaviour the current invisible right stick already uses — so reversing direction responds immediately;
  • the normal axis response/dead-zone can stay unchanged (and be handled independently by the setting this PR already adds);
  • physical controllers and the right stick are unchanged.

I also hide the left ring/knob in my personal layout, but I think upstream should make that/minimal artwork optional and keep the current gear/menu available.

The floating-left behaviour has been used on-device in Let's Go Pikachu. I also built and installed the same generalized controller path for Wonder, Link's Awakening and Luigi's Mansion 3; I haven't done dedicated gameplay validation across all three, so I don't want to overclaim that evidence.

Since this PR already owns iOS touch-control settings and adds the left-stick dead-zone setting, would an opt-in Floating left stick (default off) fit here? The core movement change is tiny and does not need a new polling or rendering path. Happy to provide the focused diff if useful.

seedds added a commit that referenced this pull request Nov 11, 2026
…orded as an open risk; not reproduced on an M4 Pro with 85 GB punched while Ocarina of Time rendered

S2Arm (jhaizhou-ops/S2Arm #258, #305) saw kIOGPUCommandBufferCallbackErrorHang in 6 of 6
Ocarina of Time runs on an M1 Pro (macOS 27.0.1) when it punched its guest files during
play, and now never punches. Here (M4 Pro, macOS 27.2, the Kokiri Forest deck save, 8
minutes a run) a probe wrote, fsynced and punched unmapped alias-region pages of the
guest files the game uses: 36.5 GB in 2.39 million 16 KB punches and 49 GB in 140,000
whole-run punches, no command buffer error; four runs with the sweep itself had none
either. Frames over 1 s came with and without punching. The entry names what may
differ (machine and OS; S2Arm punched ranges the moment they were unmapped, the sweep
only pages given back a whole interval earlier) and the iPad run still to do.
@harshulkuhar

Copy link
Copy Markdown
Author

One complementary control option that may fit this PR: a floating left movement stick.

I’ve been using a very small DmGamepad variant on iPhone:

  • touch-down inside an enlarged home grab zone (1.65 × radius in my build) owns the left stick and that touch-down becomes its centre;
  • if the thumb moves beyond the stick radius, the centre follows just enough to keep the thumb at the radius — the same bounded-centre behaviour the current invisible right stick already uses — so reversing direction responds immediately;
  • the normal axis response/dead-zone can stay unchanged (and be handled independently by the setting this PR already adds);
  • physical controllers and the right stick are unchanged.

I also hide the left ring/knob in my personal layout, but I think upstream should make that/minimal artwork optional and keep the current gear/menu available.

The floating-left behaviour has been used on-device in Let's Go Pikachu. I also built and installed the same generalized controller path for Wonder, Link's Awakening and Luigi's Mansion 3; I haven't done dedicated gameplay validation across all three, so I don't want to overclaim that evidence.

Since this PR already owns iOS touch-control settings and adds the left-stick dead-zone setting, would an opt-in Floating left stick (default off) fit here? The core movement change is tiny and does not need a new polling or rendering path. Happy to provide the focused diff if useful.

i'll work on it and get in-sync with main branch.

In the 3D touch layout the camera is a floating right stick: a finger held away from where it landed keeps the camera turning, and short drags give little input, unlike a mobile swipe camera. A left thumb that lands near the stick's centre also walks the character through unintended directions.

Four iOS rows in the launcher's Controls section, in settings.tsv and stored, reset and overridden like every other choice (seedds#388); every default keeps the current behaviour:

- DM_TOUCH_SWIPE (off): the camera takes finger velocity instead of the floating stick (DmSwipeLook.h). Timestamped velocity is resampled at each display callback rather than cleared every frame, which alternated tilt and neutral when touch, display and guest rates differed. It expires 40 ms after the last movement sample, since UIKit sends none while a finger rests; release, cancellation and reset clear it at once.
- DM_TOUCH_SWIPE_GAIN (1, 0.25 to 4): applied after the motion is normalized and clamped, so a fast swipe gives a quarter tilt at 25 % instead of saturating at every gain.
- DM_TOUCH_RIGHT_AREA (third, or half): where a camera touch may start, in either mode; buttons keep priority.
- DM_TOUCH_LEFT_DEADZONE (0.1, 0 to 0.4): the movement stick's radial dead zone, scaling the rest of the travel without snapping to axes.

DmTouchControls.h holds the parsing, bounds and stick math without UIKit, so t_touch_controls.c tests them natively: mixed 30/60/120 Hz sampling and phases, expiry, reversal, gain bounds, invalid values and the unchanged radial response. The launcher storage check covers loading, persistence, reset and env-file locking of the four choices. Physical controllers and game code are unchanged. The reference entry is "Relative swipe camera input on iOS" in docs/reference/services.md.

Checked on Breath of the Wild 1.9.0 on an iPhone Duo, iOS 27.2, from an earlier launcher-integrated build: the dead zone removes the unintended movement, and 25 % and 400 % feel clearly different. At 400 % the game's own stick response still limits the camera: this is controller input, not direct camera rotation.
With the movement stick fixed, a thumb that lands off the stick's centre starts already tilted, and a thumb that drifts past the ring has to come all the way back before reversing registers. As proposed in PR seedds#258's conversation, DM_TOUCH_LEFT_FLOAT (off by default, an iOS launcher row) makes it float the way the camera stick already does:

- a touch-down within 1.65 ring radii of the stick's home (1.4 for the fixed stick) grabs it and becomes its centre; the diamond and the D-pad keep their ownership rule (smallest distance over grab radius);
- past the ring the centre follows the thumb just far enough to keep it on the rim, so a reversal responds at once. dm_touch_follow in DmTouchControls.h is that rule, now used by both sticks;
- the ring is drawn at the centre while held and returns home on release; the dead zone applies around the new centre.

t_touch_controls.c checks the parsing and the follow rule (inside the ring the centre stays, past it the thumb stays on the rim at full tilt, a reversal turns the axis at once); the launcher tests cover the new row and its storage. Not yet checked on a device.
@harshulkuhar
harshulkuhar force-pushed the ios-touch-control-settings branch from 69e9fee to b3ffe8a Compare November 16, 2026 14:10
@harshulkuhar harshulkuhar changed the title iOS: add optional swipe camera and touch-control settings ios/gamepad: optional swipe camera, floating left stick, touch settings Nov 16, 2026
…h settings

beginLook: was never called, so the display link that turns finger velocity into right-stick tilt never ran and Swipe camera sent nothing. The camera touch now starts it where the right stick is claimed. The Duo harness test drives a swipe through DmGamepad and fails without that line.

Also:
- lookFrame: only recomputes when the camera value changed, so a resting finger does no work each frame.
- DM_TOUCH_FULL and DM_TOUCH_DEAD name the 0.8 full-tilt fraction and the 0.1 dead zone again, and the headers use DM_STICK_MAX.
- DM_TOUCH_SWIPE and DM_TOUCH_LEFT_FLOAT read with atoi, like DM_TOUCH_HAPTICS.
- Swipe sensitivity shows only with Swipe camera on (needs=, as Rumble strength does).
- t_touch_controls.c follows the CHECK style of the host tests.
- The reference entry uses Symptom / Cause / Fix / Check, and settings.md has one or two sentences per row.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation game: 01007EF00011E000 The Legend of Zelda: Breath of the Wild ios The iPhone and iPad app (dm/ios) tools Tools and scripts (dm/tools, scripts)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants