docs: record which SNMP communities are actually rotated - #23
Merged
Conversation
Three of the four devices were rotated and nothing said so. pfSense, the APC NMC and iLO each answer to their own new community and refuse the old shared one; the MokerLink switch does not, and cannot be verified, because it has never answered an SNMP poll at all. SECURITY.md is the ledger and carries the per-device status. docs/security.md carries the detail behind it — it also claimed SNMP was reachable only on the management VLAN, which is wrong: neo is polled at 10.7.7.2 on the switch-management LAN. secrets/README.md gets a pointer rather than a fifth copy of the status to keep in sync. The roadmap item stays unchecked but is renamed to name the switch. An unchecked box reading "all four devices" reads as though nothing happened, which is materially false. Refs #9, #22 Co-Authored-By: Claude Opus 5 <[email protected]>
Nothing in the runbook let an operator tell "my rotation broke this" from "this never worked", and that distinction decides whether to roll back. Adds the max_over_time(up[30d]) check to §2.4 and to the troubleshooting table. The neo row's hypothesis was right and is now evidence-backed, so it becomes a test rather than a hunch: ICMP and TCP/80 to 10.7.7.2 both succeed while UDP/161 does not, which narrows the pfSense rule from "missing" to "may not cover this protocol and port". §2.5 explained why --old skips a device but never what to do about it. A SKIP is an open question, not a pass deferred — a device that never answered may still accept the leaked community. Also records that --old refuses a pipe on purpose, so no script or agent can run it. The APC 15-character warning read as a property of the model. This lab's card is verified working at 16. The close-the-loop list was wrong by one: docs/security.md carries the detail SECURITY.md points at, and leaving it stale makes the two disagree. Refs #9, #22 Co-Authored-By: Claude Opus 5 <[email protected]>
7 tasks
6 tasks
Gerrrt
added a commit
that referenced
this pull request
Aug 19, 2026
docs: record which SNMP communities are actually rotated
Gerrrt
pushed a commit
that referenced
this pull request
Aug 19, 2026
It has. #23 wrote "never answered an SNMP poll at all" into four files on the strength of max_over_time(up{instance="10.7.7.2"}[30d]) == 0. That measured Prometheus retention, not the device: all four SNMP series carry ~23,127 samples and begin together when the current stack was built, so the TSDB holds no SNMP data from before that point and the window cannot speak to it. The switch last polled successfully in November 2025, under a hand-tuned minimal module. What is true is narrower and still enough to keep #9 open: neo is not answering now, and has not for the whole span of available data. Also records the sharper reading of the ICMP/TCP-pass, UDP-fail split — that is what a wedged SNMP agent looks like, not a dead device. Refs #9, #22 Co-Authored-By: Claude Opus 5 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Documentation only. Records which SNMP communities are actually rotated on the
hardware — three of the four were, and nothing in the repository said so — and
folds what the rotation taught us back into its runbook.
Why
Closing the loop on #9, which the rotation runbook's §3 requires and which was
never done.
SECURITY.mdstill asserted the communities were "not yet rotatedon the devices". That is materially false for pfSense, the APC NMC and iLO:
each answers to its own distinct new community and refuses the old shared one.
It is still true for the MokerLink switch, but not for the reason the issue
assumes.
neohas never answered an SNMP poll at all —max_over_time(up{job="snmp",instance="10.7.7.2"}[30d])is0across 23,070samples, so the failure predates the rotation. That is now tracked separately
in #22, and #9 stays open behind it: until the switch answers,
snmp-verify.sh --oldreturnsSKIPfor it, so whether it still accepts theleaked community is unknown rather than proven clean.
A published
SECURITY.mdthat understates what is fixed is as misleading asone that overstates it.
Two things found en route:
docs/security.mdclaimed "SNMP is only reachable on the management VLAN".It is not —
neois polled at10.7.7.2on the switch-management LAN.docs/security.md, which is the fileSECURITY.mddelegates detail to.Blast radius
None. No rendered config, no secret, no compose service, no Prometheus target
changes.
neois deliberately left inprometheus/targets/snmp.yaml— removingit to silence
SnmpTargetUnreachablewould be hiding the problem.secrets/*.sops.yamlVerification
./scripts/validate.sh— all checks pass (yamllint,markdownlint-cli2andgitleaksSKIP as not installed on this host;markdownlint-cli2runseparately via
npx, 0 issues).Live state, re-established immediately before this PR, since the merged wording
asserts it:
./scripts/snmp-verify.sh --oldhas since been run interactively (it needs aterminal by design) and confirms it —
rejectedfor morpheus, mjolnir andshiva,
SKIPfor neo. Full output attached to #9:make validatepassesdocs/, service README, ordocs/roadmap.md)Refs #9, #22