fix(generator): respect AWS enum description boundaries

[thomhurst]

Problem and fix

AWS enum extraction accepted nested attribute prose and treated the single bullet in Possible values: o PRE_APPROVED as a second allowed value. In the generated AWS PR #5163 and runtime PR #4367, this made valid SNS attribute names impossible to pass and exposed invalid O enum members.

Keep the original description indentation until enum detection. Accept enum headings at the option description level and at the start of a sentence, excluding nested attribute text. Recognize both o and * bullet lists, including one-item lists, and reject prose within their entries before applying the existing shared enum member limits. A single allowed value therefore retains the existing string fallback.

Regression coverage includes the SNS documentation excerpt, ACM authorization/contact values, the VPN ipsec.1 value, nested headings, inline and separate option-level headings, and generated constructor/enum output. Generated options are not edited in this PR.

Validation

• Before the fix: seven regression cases failed in the 101-case AWS scraper run.
• After the fix: all 322 AWS, gcloud, synopsis, description enum, and enum factory tests pass.
• Scoped formatting of the generator solution completed at severity info; existing diagnostics without code fixes remain.
• Release build of the generator solution passed with zero warnings and zero errors.

Regeneration

Fixes the generator portion of #5164. After this source PR merges, regenerate AWS and gcloud together from the latest main, review the replacement for closed #5163, and update #4367 with the authoritative output. Keep #5164 open until regeneration is verified.

Summary by CodeRabbit

• Bug Fixes
  • Improved AWS CLI option parsing to avoid incorrectly restricting values based on nested attribute documentation.
  • Correctly recognizes option-level “Possible values” sections, including inline and paragraph-separated headings.
  • Preserves complete top-level enum choices while rejecting structural or descriptive bullets as valid choices.
  • Ensures single-value fallback handling remains accurate.
  • Keeps SNS attribute names as required strings and excludes structural bullets from ACM option enums.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-16T04:49:29.151242Z	1b82365	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 497176f4-0590-406f-b567-41dce906a3f5

📥 Commits

Reviewing files that changed from the base of the PR and between 8b63d57 and 1b82365.

📒 Files selected for processing (2)

• tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.EnumBoundaries.cs
• tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/AwsCliScraper.cs

Included review availability: Your plan provides up to 4 included reviews per hour; 2 remain after this review.

---

📝 Walkthrough

Walkthrough

The AWS CLI scraper now uses preserved description indentation to distinguish option-level enum headings from nested attribute documentation. It also validates bullet markers more strictly. New tests cover SNS, ACM, and enum boundary cases.

Changes

AWS CLI enum boundary handling

Layer / File(s)	Summary
Scraper enum detection and parsing tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/AwsCliScraper.cs	The scraper preserves raw description whitespace, checks enum heading indentation, and validates repeated o or * bullet markers.
Boundary regression coverage tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.EnumBoundaries.cs	Tests cover nested attribute values, single-bullet values, bullet descriptions, option-level headings, top-level choices, SNS attribute names, and ACM structural bullets.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 1b823

The enum-boundary changes preserve the intended string fallback and are covered by focused regression tests, with no merge-blocking risk identified.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	Issue #5163 requires generated AWS option classes that reflect the latest AWS CLI documentation, including new commands, updated option types and descriptions, and expanded command coverage. The whole…	Run the options generator with the corrected AWS scraper against the AWS CLI help snapshot required by #5163. Commit the generated AWS option class changes, including new commands, updated option types and descriptions, and the correspondin…
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	The changes to AWS enum extraction and the related regression tests directly support accurate AWS option generation for #5163. The changes cover enum boundaries, bullet formats, nested headings, and g…
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: fixing AWS enum description boundary handling in the generator.

Full details: Linked Issues check

Explanation

Issue #5163 requires generated AWS option classes that reflect the latest AWS CLI documentation, including new commands, updated option types and descriptions, and expanded command coverage. The whole-PR diff changes only AwsCliScraper.cs and scraper tests. It adds no generated AWS option classes and no updated command coverage. The tests support the scraper fix, but they do not deliver the generated updates required by #5163.

Resolution

Run the options generator with the corrected AWS scraper against the AWS CLI help snapshot required by #5163. Commit the generated AWS option class changes, including new commands, updated option types and descriptions, and the corresponding command coverage evidence.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch issue-5164-aws-enum-boundaries

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{A rabbit checks each enum line,
Where nested notes no longer bind.
The bullets keep their proper place,
SNS strings retain their shape.
ACM choices now stay clear.}

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Review

Traced the new IsOptionLevelEnumHeading heuristic and ParseEnumChoices generalization by hand against every case in AwsCliScraperTests.EnumBoundaries.cs, and the logic checks out:

• Passing rawDescription (indentation preserved) into TryDetectEnum instead of the whitespace-collapsed description is scoped tightly — only the enum-detection call site changes; the flag/array/key-value/numeric detection still use the collapsed description, so no other behavior shifts.
• IsOptionLevelEnumHeading correctly distinguishes an option-level "Possible values:" heading (at the start of the description, or inline after a sentence-ending ./!/?) from a nested attribute's own heading, by comparing the heading line's indentation against the description's base indentation and rejecting lines that start with an o /* bullet. Walked through all four rejection cases (deeper indent, same-indent-but-bullet-prefixed, mid-sentence heading) and all three acceptance cases (inline, separate paragraph, indented) — all resolve as asserted.
• ParseEnumChoices's bullet-token generalization ("o" or "*", single-item lists included via the tokens.Length % 2 == 0 check now applied after picking tokens[0] rather than hardcoding "o") correctly falls through to an empty result (→ string fallback) for a single bullet value, relying on the existing shared single-member fallback in OptionEnumFactory.TryCreateFromHint — consistent with "a single allowed value retains the existing string fallback" in the PR description.
• Confirmed AwsCliScraper.TryDetectEnum is only called from within AwsCliScraper itself (gcloud/Cobra scrapers have their own separate enum-detection paths), so this change can't regress other tool scrapers.
• New tests reuse the existing TestAwsCliScraper/CreateToolDefinition test helpers already established in AwsCliScraperTests.cs, consistent with the existing partial-class test structure.

One non-blocking thought for future maintainability: IsOptionLevelEnumHeading computes indentation via index/substring arithmetic (LastIndexOf('\n'), slicing, TrimStart) anchored to "the first non-blank line's indent as the base level." That's correct for the cases tested, but a line-by-line indent-stack approach would generalize more safely if a future AWS renderer version produces descriptions with more than two indentation tiers (e.g., doubly-nested bullets) — worth keeping in mind if more regression cases surface, but not something this PR needs to solve now given its narrow, well-justified scope.

I wasn't able to run the build/tests in this sandboxed review environment, but the PR description's reported validation (322/322 passing, zero-warning release build) combined with the manual trace above gives me confidence in the fix.

Nice fix — good regression coverage for a genuinely subtle text-parsing boundary problem.

[thomhurst]

The final head is 1b82365. It also rejects prose inside bullet entries; all 322 related tests pass. @codex review

[thomhurst]

Thanks for the review in comment 5692160135. The indentation comparison already rejects any heading deeper than the option description, including more than two indentation tiers. An indentation stack would only be needed if future help formats require interpreting those nested values; this change intentionally keeps the existing string fallback for them. No additional parser state is needed for the current option-level enum contract.

[thomhurst]

Disposition of the two pre-merge warnings in comment 5692102410: #5163 is the closed generated PR, not the source issue. This PR implements source issue #5164. Repository guidance explicitly requires merging the source fix separately before regenerating a fresh AWS PR, so generated output belongs in the replacement PR after this merges. The new methods are named regression tests and a private helper; they follow the surrounding test/helper documentation convention, and the helper includes the non-obvious indentation rationale. @coderabbitai review the final head 1b82365, which additionally rejects prose inside bullet entries.

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 1b82365ef3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo '=== final commit and changed files ==='
git show --no-ext-diff --format=fuller --stat 1b82365ef330b98f20d9e51832f817e42b71ee57
git diff --no-ext-diff --unified=80 1b82365ef330b98f20d9e51832f817e42b71ee57^ 1b82365ef330b98f20d9e51832f817e42b71ee57 -- \
  tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/AwsCliScraper.cs \
  tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.EnumBoundaries.cs

echo '=== scraper outline ==='
ast-grep outline tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/AwsCliScraper.cs --items all

echo '=== tests outline ==='
ast-grep outline tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.EnumBoundaries.cs --items all

echo '=== enum parser call sites and definitions ==='
rg -n -C 6 -P '\b(TryDetectEnum|ParseEnumChoices|IsOptionLevelEnumHeading)\s*\(' \
  tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator \
  tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests

echo '=== adjacent scraper tests ==='
fd -a 'AwsCliScraperTests.*\.cs' tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers -x sh -c 'echo "--- $1"; rg -n -C 2 -P "\b(enum|Enum|Possible values|Valid values)\b" "$1"' _ {}

Length of output: 76181

---

🏁 Script executed:

#!/bin/bash
set -uo pipefail

base=$(git merge-base HEAD origin/main 2>/dev/null || git merge-base HEAD origin/master 2>/dev/null || true)
echo "=== base: ${base:-unavailable} ==="
if [ -n "$base" ]; then
  git diff --no-ext-diff --stat "$base"...1b82365ef330b98f20d9e51832f817e42b71ee57
  git diff --no-ext-diff --unified=30 "$base"...1b82365ef330b98f20d9e51832f817e42b71ee57 -- \
    tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/AwsCliScraper.cs \
    tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.EnumBoundaries.cs
fi

echo '=== ParseOptions and description extraction ==='
sed -n '390,480p' tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/AwsCliScraper.cs

echo '=== enum parsing and regex declarations ==='
sed -n '655,760p' tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/AwsCliScraper.cs
sed -n '800,890p' tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator/Scrapers/Cli/AwsCliScraper.cs

echo '=== full enum-boundary tests ==='
cat -n tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.EnumBoundaries.cs

echo '=== tests for bullet parse behavior ==='
sed -n '96,120p' tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.cs
sed -n '156,176p' tools/ModularPipelines.OptionsGenerator/src/ModularPipelines.OptionsGenerator.Tests/Scrapers/AwsCliScraperTests.cs

Length of output: 44459

---

@thomhurst Reviewed final head 1b82365ef330b98f20d9e51832f817e42b71ee57.

I found no blocking issues. The final parser logic rejects prose within o and * bullet sequences while it preserves valid repeated-marker enum lists. The regression tests cover the new boundary cases and the generator output contracts.

The source-fix-only scope is also correct. The generated AWS option changes belong in the replacement regeneration PR after this PR merges.

⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.