Skip to content

Fix theme selection and add Settings validation tests - #188

Merged
anod merged 2 commits into
masterfrom
anod-settings-validation-tests
Nov 21, 2026
Merged

anod merged 2 commits into
masterfrom
anod-settings-validation-tests

Conversation

@anod

@anod anod commented Nov 21, 2026 •

Copy link
Copy Markdown
Owner

Summary

Validates the Settings screen end to end and fills the test gaps around it: every setting is now covered from the stored preference, through the screen, to the code that consumes it. The tests exposed one real bug in the theme setting, which is fixed here.

Bug fix: theme selection

Preferences.selectedTheme built SelectedTheme from the theme's position in the list (themeIndex) instead of its stored id, and SelectedTheme.isDark was forced on whenever Black was selected.

  • "Black" never matched THEME_BLACK, so it painted the regular dark surface (#0B0F0C) instead of pure black (#000000).
  • "Follow system (Black)" ignored a light system theme, because Black implied dark.

The fix is two lines in Preferences.kt: pass theme (the id) and derive isDark from the mode alone. AppTheme already ORs in isSystem && isSystemInDarkTheme() and only applies the black surfaces in the dark scheme, so nothing else changes.

Tests

Unit tests: 274 → 475 (201 new, 0 failures)

Area Test classes
Preferences, items, clicks PreferencesTest, PreferenceItemsTest, PreferenceItemsDriveSyncTest, SettingsItemClickTest
View model SettingsViewModelTest
Screen and scene (Compose on Robolectric) SettingsScreenTest, SettingsScreenBackupTest, SettingsScreenDriveSyncTest, SettingsScreenIconShapeTest, SettingsScreenSceneTest
Consumers SettingsConsumersTest, AppThemeTest, UpdateCheckWifiOnlyTest, UpdateCheckNotificationSettingsTest, WatchListStateViewModelSettingsTest, MainViewModelSettingsTest, InstalledListViewModelSettingsTest, UploadServiceContentObserverTest

On-device Compose tests (12): SettingsUiTest (7) drives the real Settings screen and verifies persistence by reading WatcherPrefs.xml; ThemeRenderingUiTest (5 theme/mode variants) checks the painted colours. IsolatedWorkManagerRule swaps in an in-memory WorkManager so the real sync and upload workers cannot run during the tests, NotificationPermissionRule grants the notification permission that Android 13+ denies a fresh install (the update-check scheduling and sample-notification tests need it), and StableStartupPreferencesRule accepts preference overrides. The permission rule does nothing on a device that already allows notifications, and it leaves the grant in place because revoking a runtime permission kills the process that runs the tests.

Test infrastructure: FakeDfeApi.bulkDetailsResponse and androidTestImplementation(libs.work.testing).

Coverage (JaCoCo, line)

Before After
Overall 32.2% 41.3%
preferences 43.4% 82.3%
sync 35.2% 72.4%
backup 56.2% 76.1%
watchlist 25.9% 33.4%
SettingsViewModel 1.2% 82.3%
SettingsScreen 13.9% 60.8%
UpdateCheck 45.8% 85.8%
PreferenceItems 95.2% 100%
Preferences 89.2% 96.8%

"Before" is the same run with only the 274 tests that were on master. What is left uncovered in Settings: the @Preview, the notification-permission result callback and the SDK < 33 branch, and the Play Services-bound Drive sign-in, sign-out and sync-with-account flows.

Do the tests catch regressions?

I mutated the production code (reverted after each run) and checked that a test fails: 16 unit mutations, 2 scene mutations, 4 on-device mutations (theme index, Wi-Fi-only ignored, pull-to-refresh setter, refresh-history routing) and one for the diagnostics log failure handling. All were caught.

On-device results

Pixel 6a, Android 17: SettingsUiTest, ThemeRenderingUiTest, AppWatcherSmokeTest and SearchSurfacesUiTest pass (15/15) on 3 consecutive runs, and again after adding NotificationPermissionRule. One earlier run failed in the existing SearchSurfacesUiTest right after a fresh install; it did not recur in 6 later runs.

CI emulator (API 33, the instrumentation job): 16 tests, 15 pass. SettingsUiTest 7/7, ThemeRenderingUiTest 5/5, AppWatcherSmokeTest and DiagnosticRecorderDeviceTest pass. The one failure is the existing SearchSurfacesUiTest.searchSurfaces_composeMeasureAndAcceptInput (No menu with item "Settings" was found, raised by AppWatcherUiTestSupport.openMenuContaining), which this PR does not touch.

That test step is continue-on-error, so the job shows green even when tests fail. The failure only appears as a check-run annotation (The process '/usr/bin/sh' failed with exit code 1), and that annotation is on all 38 earlier Android CI runs (2026-08-28 to 2026-10-30), including master at this PR's base commit. So the step was already red before this PR. I could not tell which test failed on master, because its logs have expired.

The first CI run of this PR also reported my two notification-dependent SettingsUiTest tests as failed. A fresh API 33+ install denies notifications, so their assumeTrue guards threw an assumption violation, which AGP's report counts as a failure. NotificationPermissionRule now grants the permission up front, and both pass.

Findings, not changed here

  • The icon-style branch in onSettingsItemClick is dead code: that item is a placeholder.
  • lib IconShapeSelector onClickLabels are off by one (lib submodule).
  • lib PreferencesScreen keeps un-keyed remember widget state that can survive render().
  • "Updated notifications" (notify_installed_uptodate) has no effect in the sync pipeline since Fix installed update reconciliation #164: a sync-produced update always has !installed || remote > installed. Pinned by appsAlreadyUpdatedOnTheDeviceAreNeverAnnouncedWhateverUpdatedNotificationsIs.
  • Manual refresh (SyncScheduler.execute) always uses a connected network and no charging requirement, regardless of the Wi-Fi-only and charging settings.
  • Not unit-testable: Drive sync gating in UpdateCheck and the Drive account flows (Play Services), and crash-report collection (Firebase; only the toggle is tested).
  • The Android CI instrumentation step has been failing on every run since at least 2026-08-28, hidden by continue-on-error (see On-device results).
  • ktlintCheck reports one pre-existing unused import in AppItemStateTest.kt; it is untouched here.

Running it

  • Unit: ./gradlew :app:testDebugUnitTest
  • On device: ./gradlew :app:installDebug :app:installDebugAndroidTest, then adb shell am instrument -w -e class com.anod.appwatcher.SettingsUiTest,com.anod.appwatcher.ThemeRenderingUiTest com.anod.appwatcher.debug.test/androidx.test.runner.AndroidJUnitRunner. Prefer this over connectedDebugAndroidTest, which removes the debug app (and its data) from the device after the run.
  • Instrumentation force-stops the app, which clears its notifications and scheduled jobs; open the app once afterwards.

SelectedTheme was built from the theme's position in the list instead of its stored id, so the Black theme painted the regular dark surface instead of pure black, and isDark was forced on for Black, which made 'Follow system (Black)' ignore a light system theme. Pass the stored id and derive isDark from the mode alone.

Add 201 unit tests (475 in total) for every Settings item and its consumers: Preferences, preference items, the Settings view model and screen, AppTheme, the sync scheduler and update check (Wi-Fi only, charging, notifications), the watch list, main and installed view models, and the Drive upload observer. Add on-device Compose tests (SettingsUiTest, ThemeRenderingUiTest) that drive the real Settings screen and read the persisted preferences back.

Test infrastructure: FakeDfeApi can serve bulk details responses, StableStartupPreferencesRule accepts preference overrides, and the androidTest source set gets work-testing so on-device tests run against an isolated in-memory WorkManager.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings November 21, 2026 02:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block approval of the theme-selection fix and added Settings regression coverage.

No open issues 🎉

What changed in this PR

Fixes AppWatcher’s theme selection so Black uses pure-black surfaces and “Follow system (Black)” respects light mode. Adds layered Settings regression coverage from stored preferences through screen interactions to downstream consumers.

Changes:

  • Corrects SelectedTheme construction to use the stored theme ID and derive dark mode independently.
  • Adds preference, Compose, view-model, sync, notification, backup, and list-settings tests.
  • Adds device-level persistence and theme-rendering checks with shared test helpers and WorkManager test support.
File Description
app/​src/​test/​java/​com/​anod/​appwatcher/​watchlist/​WatchListStateViewModelSettingsTest.kt Tests list initialization and live settings changes.
app/​src/​test/​java/​com/​anod/​appwatcher/​watchlist/​MainViewModelSettingsTest.kt Tests preference propagation and notification-driven scheduling.
app/​src/​test/​java/​com/​anod/​appwatcher/​sync/​UpdateCheckWifiOnlyTest.kt Tests scheduled and manual refresh network policies.
app/​src/​test/​java/​com/​anod/​appwatcher/​sync/​UpdateCheckNotificationSettingsTest.kt Tests notification filtering through the sync pipeline.
app/​src/​test/​java/​com/​anod/​appwatcher/​sync/​UpdateCheckHarness.kt Provides shared sync test dependencies.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsViewModelTest.kt Tests Settings actions, persistence, scheduling, and failures.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsTestSupport.kt Adds shared bindings, fixtures, and test doubles.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsScreenTestBase.kt Provides reusable Compose screen test support.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsScreenTest.kt Tests displayed settings and control interactions.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsScreenSceneTest.kt Tests navigation, permissions, backup, and restart integration.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsScreenIconShapeTest.kt Tests icon-shape selection events.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsScreenDriveSyncTest.kt Tests Drive controls and availability states.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsScreenBackupTest.kt Tests document pickers, cancellation, and missing providers.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsItemClickTest.kt Tests preference-click routing and updates.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​SettingsConsumersTest.kt Tests notification filters and watchlist preference mapping.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​PreferencesTest.kt Tests defaults, persistence, change events, and theme mapping.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​PreferenceItemsTest.kt Tests Settings item structure, values, and enabled states.
app/​src/​test/​java/​com/​anod/​appwatcher/​preferences/​PreferenceItemsDriveSyncTest.kt Tests Drive item values and summaries.
app/​src/​test/​java/​com/​anod/​appwatcher/​installed/​InstalledListViewModelSettingsTest.kt Tests installed-list pull-to-refresh settings.
app/​src/​test/​java/​com/​anod/​appwatcher/​compose/​AppThemeTest.kt Tests theme colors across system light and dark modes.
app/​src/​test/​java/​com/​anod/​appwatcher/​backup/​gdrive/​UploadServiceContentObserverTest.kt Tests upload scheduling and preference-based constraints.
app/​src/​test/​java/​com/​anod/​appwatcher/​accounts/​FakeDfeApi.kt Adds configurable bulk-details responses.
app/​src/​main/​java/​com/​anod/​appwatcher/​preferences/​Preferences.kt Fixes theme identity and dark-mode interpretation.
app/​src/​androidTest/​java/​com/​anod/​appwatcher/​ThemeRenderingUiTest.kt Checks painted Settings colors on a device.
app/​src/​androidTest/​java/​com/​anod/​appwatcher/​StableStartupPreferencesRule.kt Supports startup preference overrides and restoration.
app/​src/​androidTest/​java/​com/​anod/​appwatcher/​SettingsUiTest.kt Tests real-screen persistence, scheduling, and navigation.
app/​src/​androidTest/​java/​com/​anod/​appwatcher/​SettingsScreenRobot.kt Adds reusable device interaction and assertion helpers.
app/​src/​androidTest/​java/​com/​anod/​appwatcher/​IsolatedWorkManagerRule.kt Provides test WorkManager setup and schedule inspection.
app/​build.gradle.kts Adds WorkManager testing support for instrumentation.

Review effort: Balanced

Android 13+ denies POST_NOTIFICATIONS to a fresh install, so on the CI emulator the two settings tests that need notifications (update frequency scheduling and the sample notification row) were reported as failed assumption violations. NotificationPermissionRule grants the permission before the activity starts, so the screen reads the allowed state. A device that already allows notifications is unchanged. The assumptions stay for a user who blocked the app in the system settings.

Co-authored-by: Copilot App <[email protected]>
Copilot AI balanced review requested due to automatic review settings November 21, 2026 02:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the theme-selection fix and expanded Settings regression coverage.

No open issues 🎉

Review effort: Balanced

@anod
anod merged commit 025ef80 into master Nov 21, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants