Skip to content

refactor(cli): rewrite files, models, adapters and inference on typed clients - #2000

Draft
maxdubrinsky wants to merge 17 commits into
mainfrom
cli-core-groups-2/mdubrinsky
Draft

maxdubrinsky wants to merge 17 commits into
mainfrom
cli-core-groups-2/mdubrinsky

Conversation

@maxdubrinsky

@maxdubrinsky maxdubrinsky commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Rewrites the files, models, adapters and inference command groups by hand on the typed clients from #1986, replacing their generated Stainless commands. CLI surface, output shapes and exit codes are unchanged; the generated modules are removed and skipped in the generator config.

Changes

  • cli/commands/files.py: filesets/files CRUD plus upload/download/list through filesets.transfer (SDK-free since feat(cli): typed-client plumbing for the Stainless-free CLI #1986); files list -f code refuses glob patterns with a usage error instead of emitting code that cannot run.
  • files filesets refresh (added to the generated tree by feat(files): serve filesets from GitHub and refresh a tracked revision [ASTD-592] #1957 while this PR was open) is carried over: FilesClient.refresh_fileset posts to the /refresh route and the command renders the returned fileset, same help text and position; wire and endpoint tests added.
  • cli/commands/models.py, adapters.py: entity CRUD, create --wait on the shared waiters (tests/cli/commands/test_create_wait.py keeps main's jobs tests and adds the deployment ones).
  • cli/commands/inference.py: providers, virtual models, provider readiness and the OpenAI proxy commands on InferenceGatewayClient; streaming uses the raw SSE stream.
  • The gateway patch/post/put proxy commands reject --input-data keys other than name/body (the generated commands failed loudly on stray keys; forwarding only body would have posted an empty request), and inference gateway openai v1 models list uses the OpenAI columns (id, owned_by, created) the endpoint returns, matching inference models list on the same route.
  • Bodies go through build_request_body; lists through collect_offset_pages.
  • manifest_registry.py entries, cli_config.yaml skip: true, cli/commands/api/{files,models,adapters,inference} deleted; wire tests per group.

Stack

Part of the AIRCORE-893 series that moves the nemo CLI off the generated Stainless SDK onto the typed clients in nemo_platform_plugin (blocking AIRCORE-827). Stacked on #1986 (plumbing) but targeted at main so CI runs (the CI workflow only triggers for PRs against main/release/*). Until #1986 merges, the diff includes its commits; this PR's own change is refactor(cli): rewrite files, models, adapters and inference on typed clients. Review that commit, or compare against cli-typed-client-plumbing/mdubrinsky.

  1. feat(cli): typed-client plumbing for the Stainless-free CLI #1986 plumbing (typed-client seam, bootstrap, pagination and request-body helpers)
  2. In parallel on top of it: refactor(cli): rewrite workspaces, projects and iam on typed clients #1999 core groups 1 (secrets, workspaces, projects, iam), refactor(cli): rewrite files, models, adapters and inference on typed clients #2000 core groups 2 (files, models, adapters, inference), refactor(cli): host guardrail, intake and experiments in their owning packages #2001 plugin-hosted groups (guardrail, intake, experiments), refactor(plugins): type platform handles as PlatformClient across the plugin runtime #2002 plugin runtime typing, refactor(cli): run nemo setup on typed clients #2003 setup
  3. refactor(cli)!: drop the generated Stainless command tree and the CLI generator #2004 final flip: drop get_client()'s Stainless path, delete cli/commands/api/* and the CLI generator

Status: draft. Being maintained by an agent on behalf of @mdubrinsky; it re-bases on #1986 as that moves and on @ironcommit's landing plugin migrations (#1971, #1973), and picks up CI and review feedback. It will be re-cut where those PRs overlap (noted per PR below).

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with documentation updates
  • Documentation only
  • Contributor tooling or automation
  • CI, build, or test infrastructure

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Documentation updated for user-visible behavior
  • Documentation not applicable — justification: no user-visible CLI change; --help output and docs/cli/reference.mdx are unchanged.

Verification

  • Pull request title follows the repository's Conventional Commit format
  • Every commit includes an appropriate Signed-off-by: trailer
  • uv run pre-commit run -a passes, or any blocked checks are identified below
  • Targeted tests pass, or tests are marked not applicable above
  • No secrets, API keys, or credentials are included

Targeted validation (on the current head, based on #1986 at 6d9822bd6):

  • uv run --frozen pytest packages/nemo_platform_ext/tests packages/nemo_platform_plugin/tests -q --deselect packages/nemo_platform_ext/tests/cli/commands/test_services.py --deselect packages/nemo_platform_ext/tests/local/test_services.py (the two deselected files bind ports 8080/9000, which are occupied on the host): 2241 passed (ext CLI, filesets) + 2752 (plugin, models) (plugin, filesets, models) + 1583 (ext CLI).
  • uv run --frozen pytest packages/filesets/tests services/core/models/tests/unit -q: passed.
  • uv run ruff check and uv run ruff format --check: clean.
  • uv run pre-commit run -a: all hooks pass except uv-lock (host uv 0.9.30 vs pinned 0.9.14); uvx --from uv==0.9.14 uv lock --check passes. helm-docs and yq are not installed on the host; both hooks are CI-covered.
  • NMP_CONFIG_FILE=/tmp/nope.yaml uv run _nemo <group> --help diffed against main for every rewritten group: identical, except a trailing period on group help lines.

Adds what hand-written CLI commands need from nemo_platform_plugin:

- InferenceGatewayClient (provider/model/openai proxy routes, provider_ready,
  raw SSE streams, OpenAI model listing) with endpoint and wire tests.
- client_from_platform accepts a NemoClient or AsyncNemoClient and derives
  the typed client with from_client, so callers no longer need to know which
  platform handle they hold; PlatformClient is the runtime-checkable
  structural type for that parameter and platform_default_headers reads
  identity headers off either shape.
- models.refs holds the pure model-reference helpers; packages/models
  re-exports them so the CLI does not import the Stainless-bound package.
- filesets.transfer holds SDK-free upload/download/list/delete; FilesResource
  delegates to it and filesets no longer imports .resources eagerly.
- Bearer tokens are resolved on every HTTP attempt rather than baked into the
  PreparedRequest, so retries and later pages never replay a stale token.
- A 409 on a create sent with exist_ok is no longer retried before send()
  resolves it by fetching the existing entity.
- Query-param TypedDicts and request models for files, iam, virtual models
  and workspaces gain the fields the CLI exposes; GuardrailConfig.data is
  optional to match the server entity.

Signed-off-by: Max Dubrinsky <[email protected]>
…ands

Additive plumbing so command groups can be rewritten on the typed clients
one PR at a time while the generated commands keep working:

- client/bootstrap.py resolves config, OIDC discovery and token providers
  without the generated SDK and builds NemoClient/AsyncNemoClient with the
  same retry policy, TLS verification and 5 s connect cap the SDK used;
  factory.py layers the NeMoPlatform constructors on top of it.
- CLIContext.typed_client(XClient) / async_typed_client derive a service
  client that shares the platform client's transport and auth; get_workspace
  exposes the configured default.
- pagination gains collect_offset_pages / collect_cursor_pages for typed
  paginated responses, carrying the server's envelope fields (sort, filter,
  grouped_by) into JSON output; fetch_all_pages stays for generated commands.
- errors maps typed-client and pydantic errors alongside the SDK ones
  (validation and unknown --input-data keys exit 2), and recognises every
  unresolved-workspace message.
- stdin_utils.build_request_body validates --input-data into a request
  model and rejects unknown keys instead of dropping them.
- code_generator renders typed-client snippets (request models as
  constructor calls, SecretStr masked, RootModel positional); generated
  commands are routed to legacy_code_generator until they are replaced.
- waiters accept either platform handle, poll with a monotonic clock and
  normalise str-enum statuses; formatters unwrap NemoResponse; the version
  flag reads distribution metadata; command groups no longer advertise
  shell completion (the root app owns it).

Signed-off-by: Max Dubrinsky <[email protected]>
…p calls it

The new test endpoint stubs raise NotImplementedError like their siblings
instead of an ellipsis body, which ty rejects as an implicit None return.
The auth-idp CLI refresh contract test patches discover_nmp_config on
client.bootstrap, where the CLI now resolves OIDC settings.

Signed-off-by: Max Dubrinsky <[email protected]>
AllPagesResponse and OffsetPageResponse emit the server envelope fields,
but callers that build them without an envelope (fetch_all_pages behind
the generated list commands, and nemo jobs list --all-pages) lost the
sort key that list output has always carried. sort is now always present,
null when the server did not echo one, and a real envelope keeps its own
field order.

Signed-off-by: Max Dubrinsky <[email protected]>
- Narrow the platform handle from object to PlatformClient in
  client_from_platform and platform_default_headers (adapter.py) and the
  nemo.sdk resource-factory owner in NemoPluginSDKResources (sdk.py), so
  the typed-client boundary stops erasing the type where plugins are built.
- build_request_body reports each field's accepted input alias in the
  unknown-input hint (schema, not the internal schema_) so the advertised
  key is one pydantic actually accepts.
- with_options now clears a clone's cached nemo.sdk plugin resources,
  which were built against the original client's transport, so typed
  clients accessed on the clone bind to its headers/retry/timeout.
- Drop the always-false _PLATFORM_JOB_LIFECYCLE watch exclusions from the
  legacy code generator's lifecycle helpers.

Signed-off-by: Max Dubrinsky <[email protected]>
Brings in the auth-off-Stainless SDK removal, customization typed-client
migration, and sandbox forward merges. Resolves conflicts in adapter.py and
client.py by combining both sides: keep main's _owns_http lifecycle
(owns_http_client=False) and _platform_default_headers header resolution,
and keep the PlatformClient narrowing plus the with_options cached-resource
clearing added for review.

Signed-off-by: Max Dubrinsky <[email protected]>
The main merge kept the typed resource factories on PlatformClient but took
main's __all__ which no longer re-exports NeMoPlatform/AsyncNeMoPlatform, so
the import became dead and tripped the ruff pre-commit hook (lint-python-style).

Signed-off-by: Max Dubrinsky <[email protected]>
Plugins (nemo-evaluator among others) import the generated NeMoPlatform /
AsyncNeMoPlatform classes from nemo_platform_plugin.sdk. My earlier drop of
that import broke them with an ImportError. Re-export lazily through a module
__getattr__ so plugin discovery still imports this module without requiring
the generated SDK at load time.

Signed-off-by: Max Dubrinsky <[email protected]>
Brings in guardrails typed-client (#1966) and other main changes. Resolved
guardrail/types.py to main's landed GuardrailConfig.data shape (RailsConfig,
typed) per the layered-merge rule, and took main's add/add
tests/guardrail/test_endpoints.py. Also fixes the sdk.py re-export regression
in the same push.

Signed-off-by: Max Dubrinsky <[email protected]>
@maxdubrinsky
maxdubrinsky force-pushed the cli-core-groups-2/mdubrinsky branch from db49818 to cb9679f Compare September 11, 2026 18:37
@maxdubrinsky
maxdubrinsky changed the base branch from cli-typed-client-plumbing/mdubrinsky to main September 11, 2026 18:50
@maxdubrinsky maxdubrinsky reopened this Sep 11, 2026
@github-actions

github-actions Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 46546/58300 79.8% 63.7%
Integration Tests 29187/55558 52.5% 23.9%

…bing-merge

Signed-off-by: Max Dubrinsky <[email protected]>

# Conflicts:
#	packages/nemo_platform_ext/src/nemo_platform_ext/cli/core/errors.py
@maxdubrinsky
maxdubrinsky force-pushed the cli-core-groups-2/mdubrinsky branch 2 times, most recently from 2a9fbea to 3cc73c6 Compare September 11, 2026 20:24
@maxdubrinsky
maxdubrinsky force-pushed the cli-core-groups-2/mdubrinsky branch from 3cc73c6 to 1f2ffb0 Compare September 11, 2026 20:46
…bing-merge

Signed-off-by: Max Dubrinsky <[email protected]>

# Conflicts:
#	packages/nemo_platform_plugin/tests/client/test_adapter.py
@maxdubrinsky
maxdubrinsky force-pushed the cli-core-groups-2/mdubrinsky branch from 1f2ffb0 to 4e7e278 Compare September 11, 2026 21:04
@maxdubrinsky
maxdubrinsky force-pushed the cli-core-groups-2/mdubrinsky branch from 4e7e278 to 23a194e Compare September 11, 2026 21:43
@maxdubrinsky
maxdubrinsky force-pushed the cli-core-groups-2/mdubrinsky branch from 23a194e to 3929456 Compare September 14, 2026 15:53
…bing-merge

Signed-off-by: Max Dubrinsky <[email protected]>

# Conflicts:
#	packages/models/src/models/resources.py
… clients

The four groups (including the inference sub-groups: providers, models,
prompts, deployments, deployment-configs, virtual-models, gateway) become
hand-written modules on FilesClient, ModelsClient and
InferenceGatewayClient via state.typed_client(), replacing their generated
Stainless commands and the generator overrides that patched them. Flags,
defaults, choices, table columns and help text are unchanged; the generator
skips these resources.

Fileset upload/download/list/delete go through filesets.transfer. files list
-f code renders the same query the command sends and refuses a glob
--remote-path it cannot express; nemo files otlp logs create sends real
payloads (it previously sent an empty body) labelled application/json.
--wait on deployments create polls through ModelsClient and the gateway
readiness endpoint with a monotonic clock.

Wire-level tests cover every command over a recorded transport; integration
tests run against the in-process Files and Models services.

`files filesets refresh` (added to the generated tree by #1957) is carried
over: FilesClient.refresh_fileset posts to the /refresh route and the command
renders the returned fileset, with the same help text and position.

The gateway proxy commands reject --input-data keys other than name and body
(the generated commands failed loudly on stray keys; forwarding only body
would have posted an empty request instead), and `inference gateway openai
v1 models list` uses the OpenAI columns (id, owned_by, created) that the
endpoint returns, matching `inference models list` on the same route.

Signed-off-by: Max Dubrinsky <[email protected]>
@maxdubrinsky
maxdubrinsky force-pushed the cli-core-groups-2/mdubrinsky branch from 3929456 to 56ac264 Compare September 14, 2026 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant