Skip to content

Add package manager command smoke tests - #1677

Open
edvilme wants to merge 2 commits into
mainfrom
package-manager-command-tests
Open

Add package manager command smoke tests#1677
edvilme wants to merge 2 commits into
mainfrom
package-manager-command-tests

Conversation

@edvilme

@edvilme edvilme commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds focused unit coverage for the concrete package manager command classes introduced in #1621, across Pip/UV, Conda, and Poetry. Also corrects the UvUninstallCommand doc comment.

What the tests cover

  • Read commands (list, list-direct-names, version, available-versions, Poetry show): stub the runner with representative output and assert the parsed result — package lists (with rows missing a version dropped), direct names with normalization, uv pip tree indentation filtering, version-string parsing, and available-version prerelease filtering.
  • Mutation commands (install / uninstall / add / remove): grouped as smoke tests that assert execute() resolves. They have no output to parse; their concrete command strings are exercised by integration tests and by the adoption PR Adopt package manager command classes #1686.

These deliberately avoid asserting the exact argument vector, which would merely restate buildCommand.

Doc fix

  • UvUninstallCommand: uv pip uninstall has no -y/--yes flag and is non-interactive by default — corrected the misleading comment.

Relationship

Testing

  • npm run compile-tests
  • npm run unittest (58 command tests passing)

@edvilme

edvilme commented Jul 28, 2026

Copy link
Copy Markdown
Contributor Author

Contributes to #1622

@edvilme
edvilme force-pushed the package-manager-command-tests branch from 469835c to 023bf16 Compare July 28, 2026 21:38
@edvilme
edvilme force-pushed the package-manager-command-tests branch from 023bf16 to 5a9aaac Compare July 28, 2026 22:01
@edvilme
edvilme force-pushed the package-manager-command-tests branch 2 times, most recently from 6128c10 to 51074af Compare July 28, 2026 22:20
@edvilme edvilme added the debt Code quality issues label Jul 28, 2026
@edvilme
edvilme force-pushed the package-manager-command-tests branch from 51074af to 7b78eef Compare July 28, 2026 22:42
edvilme added a commit that referenced this pull request Jul 29, 2026
## Summary

Introduces the reusable command-object layer for package management
while leaving existing package-manager call sites unchanged.

## Scope

- Adds the `PackageManagerCommand` base class and shared execution
options.
- Adds abstract templates for install, uninstall, list, version,
available versions, and direct package names.
- Adds concrete Pip/UV, Conda, and Poetry implementations.
- Adds command factories, exports, and command-local execution helpers
required by those implementations.
- Preserves the existing executables, flags, environment targeting,
prerelease defaults, and list-command timeouts.

## Non-goals

This PR does not add command tests or migrate package managers to the
new classes. Those changes are isolated in follow-up PRs #1677 and
#1678.

## PR stack

1. This PR: command classes and implementations, based on `main`.
2. #1677: command smoke tests, based on this branch.
3. #1678: adoption and cleanup, based on #1677.

## Testing

- `npm run lint`
- `npm run compile-tests`
- `npm run unittest` (1,448 passing, 4 pending)
Base automatically changed from package-manager-command-refactor to main July 29, 2026 23:03
@edvilme
edvilme force-pushed the package-manager-command-tests branch from 7b78eef to c229085 Compare July 30, 2026 17:09
@edvilme

edvilme commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Note: These tests only verify that the commands are valid, a further PR will add integration tests with the package managers and round-trips such as: fetching available versions for a package, installing it, listing (direct) packages, uninstalling, etc...

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds lightweight “smoke” unit tests for the concrete package manager command classes introduced in #1621, ensuring each command’s execute() method can run end-to-end under minimal mocked conditions without throwing/rejecting.

Changes:

  • Adds smoke unit tests for Pip/UV command classes, stubbing runPython/runUV and providing minimal parseable outputs where needed.
  • Adds smoke unit tests for Conda command classes, stubbing runCondaExecutable and providing minimal JSON/string outputs.
  • Adds smoke unit tests for Poetry command classes, stubbing runPoetry / getPoetryVersion and providing minimal outputs for parsing commands.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/test/managers/builtin/commands.unit.test.ts Smoke coverage for Pip/UV command classes with mocked execution helpers and minimal outputs.
src/test/managers/conda/commands.unit.test.ts Smoke coverage for Conda command classes with mocked conda execution and minimal JSON/string outputs.
src/test/managers/poetry/commands.unit.test.ts Smoke coverage for Poetry command classes with mocked runPoetry/getPoetryVersion and minimal outputs.

@StellaHuang95

Copy link
Copy Markdown
Contributor

Finding: The tests pass even when the commands do nothing or produce incorrect results

Severity: Medium

Locations:

Every test uses this general assertion:

await assert.doesNotReject(() => command.execute(...));

In simple terms, this asks only:

“Did this function finish without throwing an error?”

It does not ask:

  • Was the package-manager executable called?
  • Was the correct subcommand used?
  • Were the correct arguments passed?
  • Was the correct environment selected?
  • Was cancellation forwarded?
  • Was the returned output parsed correctly?
  • Did the command return the expected packages or versions?

Why that matters

For example, this test is named:

test('PipInstallCommand executes without error', async () => {
    const command = new PipInstallCommand(...);

    await assert.doesNotReject(() =>
        command.execute({ packages: [{ packageName: 'package' }] }),
    );
});

The real implementation should invoke something equivalent to:

python -m pip install package

But all of the following broken implementations would still pass:

async execute(): Promise<void> {
    // Do nothing.
}
async execute(): Promise<void> {
    await runPython('python', ['-m', 'pip', 'uninstall', 'package']);
}
async execute(): Promise<void> {
    await runPython('python', ['incorrect-command']);
}

Because runPython is stubbed to resolve successfully and the test never inspects its arguments, a completely wrong command looks identical to a correct one.

Concrete regressions these tests would miss

Pip and UV

The tests would not detect:

  • Pip using uninstall instead of install.
  • Pip omitting -m pip.
  • UV omitting --python <interpreter>.
  • UV using the wrong Python environment.
  • Upgrade requests omitting --upgrade.
  • Package versions being dropped.
  • Editable-install arguments being incorrectly transformed.
  • Uninstall omitting Pip’s noninteractive -y option.

For example, UvInstallCommand is expected to construct:

uv pip install --python python package

The test does not assert any part of that command.

Conda

The tests would not detect:

  • Installing into the wrong environment.
  • Omitting --prefix environment.
  • Omitting --yes and causing an interactive hang.
  • Using install instead of remove.
  • Incorrect formatting of versioned packages.
  • Upgrade operations omitting --update-all.

The environment path is particularly important:

condaEnvironmentPath: 'environment'

The test supplies it but never confirms it reaches the command runner. The implementation could silently install packages into Conda’s currently active environment and the test would still pass.

Poetry

The tests would not detect:

  • Running Poetry in the wrong working directory.
  • Omitting add, remove, show, or --top-level.
  • Formatting package@version incorrectly.
  • Dropping the project’s cwd.
  • Returning the wrong package names.

Poetry operates against a project’s pyproject.toml, so the working directory is behaviorally important. This test:

new PoetryAddCommand({
    pythonExecutable: 'poetry',
    cwd: 'project',
    log: mockLog,
});

never verifies that 'project' is passed to runPoetry. If it were omitted, Poetry could modify a different project or fail to find pyproject.toml.

Recommended improvement

At minimum, each mutation command should verify the runner invocation.

For example:

await command.execute({
    packages: [{ packageName: 'package' }],
});

assert.ok(
    runPythonStub.calledOnceWithExactly(
        'python',
        ['-m', 'pip', 'install', 'package'],
        undefined,
        mockLog,
        undefined,
        sinon.match.number,
    ),
);

The exact assertion should match the helper’s contract, but the important point is to verify the executable, arguments, environment/cwd, cancellation token, and timeout.

@eleanorjboyd eleanorjboyd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The added parsing assertions are a useful improvement. I left one inline comment on the remaining command-construction coverage gap and the inaccurate integration-test claim; this is the main issue I would address before treating these as meaningful command tests.

Comment on lines +33 to +34
// execute() resolves. Their concrete command strings are exercised by integration tests.
suite('mutation commands execute without error', () => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This integration-coverage claim is not currently true. packageManagement.integration.test.ts only selects venv environments for install/uninstall, so it never exercises the Conda or Poetry command classes; that path also dispatches to Pip or UV dynamically, so it does not deterministically cover either concrete command string. Because these unit tests do not inspect the runner invocation either, subcommand, flag, interpreter, environment-prefix, and cwd regressions remain uncovered. Please assert the runner arguments in these suites (including the analogous Conda/Poetry tests), or remove this claim until deterministic integration coverage exists.

@edvilme
edvilme force-pushed the package-manager-command-tests branch from fc8c0b6 to 2f904be Compare July 31, 2026 19:40
@edvilme
edvilme requested a review from Copilot July 31, 2026 19:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (3)

src/test/managers/builtin/commands.unit.test.ts:66

  • PR description says these command tests intentionally do not assert parsed values, but this test asserts parsing/filtering behavior (prerelease filtering). Either update the PR description to match the actual coverage, or reduce these to smoke-only tests that just ensure execute() resolves.
    test('PipAvailableVersionsCommand parses versions and filters prereleases', async () => {
        runPythonStub.resolves(JSON.stringify({ versions: ['2.0.0rc1', '1.0.0'] }));
        const command = new PipAvailableVersionsCommand({ pythonExecutable: 'python', log: mockLog });

        const result = await command.execute({
            packageName: 'package',
            pythonVersion: '3.13.1',
            includePrerelease: false,
        });

        assert.deepStrictEqual(result.map((v) => v.public), ['1.0.0']);

src/test/managers/poetry/commands.unit.test.ts:49

  • PR description says these smoke tests "intentionally do not assert ... parsed values", but this test asserts parsing behavior (and the test title explicitly states it parses fields). Either update the PR description/scope accordingly, or adjust this to a pure smoke test (only assert execute() resolves).
    test('PoetryShowCommand parses name, version and description', async () => {
        runPoetryStub.resolves(['requests 2.31.0 Python HTTP for Humans.', ''].join('\n'));
        const command = new PoetryShowCommand({ pythonExecutable: 'poetry', cwd: 'project', log: mockLog });

        const result = await command.execute();

src/test/managers/conda/commands.unit.test.ts:57

  • PR description states the new tests "verify only that each command executes without throwing or rejecting" and do not assert parsed values, but this test asserts parsed output (versions list). Consider either updating the PR description to reflect the added parser coverage, or converting this to a smoke-only doesNotReject test.
    test('CondaAvailableVersionsCommand parses versions keyed by package name', async () => {
        runCondaStub.resolves(JSON.stringify({ package: [{ version: '1.0.0' }, { version: '2.0.0' }] }));
        const command = new CondaAvailableVersionsCommand({ pythonExecutable: 'conda', log: mockLog });

        const result = await command.execute({ packageName: 'package', pythonVersion: '' });

        assert.deepStrictEqual(result.map((v) => v.public), ['1.0.0', '2.0.0']);

@edvilme

edvilme commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

The added parsing assertions are a useful improvement. I left one inline comment on the remaining command-construction coverage gap and the inaccurate integration-test claim; this is the main issue I would address before treating these as meaningful command tests.

I think you are right. I will try changing the order of these PRs. I think it would be better to have the proper integration first and then perform actual roundtrip integration tests; maybe even in the same PR.

… doc

Address PR review: strengthen command unit tests to verify output parsing behavior (package lists, direct names with normalization, version strings, available-version filtering) using representative runner output, instead of asserting the full argument vector (which merely restates buildCommand). Mutation commands remain smoke tests; their command strings are covered by integration tests.

Also correct UvUninstallCommand doc comment: 'uv pip uninstall' has no -y/--yes flag and is non-interactive by default.

Co-authored-by: Copilot <[email protected]>

Copilot-Session: 3f16397e-0917-4efb-8d75-566c71ebf9ba
@edvilme
edvilme force-pushed the package-manager-command-tests branch from 2f904be to 42b5e98 Compare July 31, 2026 20:55
edvilme added a commit that referenced this pull request Jul 31, 2026
- pipPackageManager.manage(): rethrow CancellationError instead of swallowing
  it, so callers (e.g. venv creation's pkgInstallationCancelled) can distinguish
  cancel from failure; restores parity with main and the conda/poetry managers.
- getDirectPackageNames: fix docstring to reflect uv uses 'uv pip tree --depth=0'
  (not 'uv pip list --not-required').
- parsePackageSpecs: omit undefined version property and clarify wording.
- runPython: restore 'python:' prefix on stderr log output for consistency.
- Remove overlapping command smoke tests; the reworked, parsed-value versions
  are owned by the tests PR (#1677).

Co-authored-by: Copilot <[email protected]>
Copilot-Session: 3f16397e-0917-4efb-8d75-566c71ebf9ba
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

debt Code quality issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants