Skip to content

[BUG] Generated skills reference nonexistent commands and conflict on monitor deletion #798

Description

@gnapse

Describe the bug

The agent skills generated by pup skills install still document command groups and subcommands that do not exist in the same pup release. Two generated skills also give opposite instructions for deleting monitors.

This is the same general class of drift reported in #157 and #172. PR #428 fixed several examples, but the current generated sources still contain other unsupported commands.

To reproduce

Using pup 1.18.1:

pup skills install codex --name workflows --dir /tmp/pup-skills
pup skills install codex --name aws-integration --dir /tmp/pup-skills
pup skills install codex --name monitoring-alerting --dir /tmp/pup-skills

The generated skills document these unsupported commands:

  • agents/workflows.md: pup workflows list, pup workflows execute, and pup workflows app-keys ...
  • agents/aws-integration.md: pup aws ... and pup aws-logs ...
  • agents/monitoring-alerting.md: pup monitor-templates ..., pup notification-rules ..., and pup downtimes ...

For example:

$ pup workflows list
error: unrecognized subcommand 'list'

$ pup aws --help
error: unrecognized subcommand 'aws'

$ pup monitor-templates --help
error: unrecognized subcommand 'monitor-templates'

$ pup downtimes --help
error: unrecognized subcommand 'downtimes'

The implemented command surface uses pup workflows run, pup cloud aws, and singular pup downtime. Some documented capabilities do not appear to have a direct typed-command replacement and should be removed or explicitly described as unsupported rather than mechanically renamed.

There is also a policy conflict between generated skills:

  • dd-monitors says NEVER Delete Monitors Directly and instructs the agent to mark them for deletion.
  • monitoring-alerting presents pup monitors delete <id> as the normal confirmed deletion workflow.

The catalog also installs overlapping entry points without clear routing boundaries. For example, both dd-logs and logs claim log search/management, while dd-monitors and monitoring-alerting both claim monitor management. If these pairs are intentionally distinct, their descriptions should say when an agent should select each one so they do not both load for ordinary requests.

Expected behavior

Generated skills should describe commands exposed by the same release, give one consistent safety policy for monitor deletion, and distinguish intentionally overlapping entry points. Ideally, CI should validate command examples against the CLI command tree so this drift cannot recur.

Environment

  • OS: macOS, Darwin arm64
  • Pup version: 1.18.1, installed with Homebrew
  • Authentication: not required to reproduce; command parsing fails before authentication

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions