feature/Cocode-declares-usage-stats-host · L-261011-cad92f

.github/workflows/tests-check.yml

@@ -21,6 +21,10 @@ jobs:
     env:
       VIRTUAL_ENV: ${{ github.workspace }}/.venv
       ENV: dev
+      # Our CI never sends usage statistics: the off switch every Pipelex tool honours. Safe on the pinned pipelex, which
+      # refuses DO_NOT_TRACK only when it boots with inference and the Pipelex Gateway enabled, because `make gha-tests`
+      # always passes --disable-inference.
+      DO_NOT_TRACK: "1"
     steps:
       - uses: actions/checkout@v4
 

CHANGELOG.md

@@ -2,6 +2,9 @@
 
 ## [Unreleased]
 
+### Added
+- **Usage statistics:** cocode sets `PIPELEX_USAGE_STATS_HOST=cocode` before it boots Pipelex, unless the environment already sets it, so that once cocode runs on the Pipelex release that introduces anonymous usage statistics, the runs it drives report as host `cocode`. cocode sends no statistics of its own, and a Pipelex release without them, like the one this version pins, sends none and ignores the variable. The new Usage Statistics page in the docs explains this and links to Pipelex's page for turning the statistics off.
+
 ### Fixed
 - **Docs:** `CLI_README.md`'s example of extracting Python files from the cookbook now reads its `recipes` directory. The cookbook's redesign removed the `examples` directory it pointed at, so the example extracted nothing.
 - **CI:** Bumped the Sigstore signing action in the publish workflow from `v3.0.0` to `v3.5.0`, SHA-pinned as `790bc6befb9d733738f18d8f895854b453640ec9`. The sigstore-python bundled with `v3.0.0` predates the Sigstore TUF trust-root rotation, so the "Sign the dists with Sigstore" step had started failing deterministically with `tuf.api.exceptions.UnsignedMetadataError: root was signed by 0/3 keys`. That step sits in the GitHub-release job, so the failure mode was a version published to PyPI with no matching GitHub release or tag.

cocode/cli/main.py

@@ -20,6 +20,7 @@
 from cocode.common import PIPELINE_LIBRARY_DIRS
 from cocode.github.github_cli import github_app
 from cocode.repox.repox_cli import repox_app
+from cocode.usage_stats import declare_cocode_usage_stats_host
 from cocode.validation_cli import validation_app
 
 
@@ -65,6 +66,8 @@ def get_command(self, ctx: Context, cmd_name: str) -> Optional[Command]:
 @app.callback(invoke_without_command=True)
 def main(ctx: TyperContext) -> None:
     """Initialize Pipelex system before any command runs."""
+    # Before the boot, which reads the declaration along with pipelex's other usage-statistics switches.
+    declare_cocode_usage_stats_host()
     Pipelex.make(library_dirs=PIPELINE_LIBRARY_DIRS)
 
     if ctx.invoked_subcommand is None:

cocode/usage_stats.py

@@ -0,0 +1,23 @@
+"""cocode's declaration to the anonymous usage statistics of the pipelex runtime it embeds.
+
+cocode sends no statistics of its own: the pipelex runtime reports the runs cocode drives through its own client, and
+cocode only names itself as their `host`, so those runs can be told apart from the rest. The declaration is the
+`PIPELEX_USAGE_STATS_HOST` variable, which pipelex reads with its other usage-statistics switches when it boots, and
+which every process cocode starts inherits. pipelex's statistics client also takes the declaration through a Python
+call, but the variable is the route pipelex documents, and it works whatever pipelex release cocode pins: a release
+that sends no statistics simply ignores it.
+"""
+
+import os
+
+PIPELEX_USAGE_STATS_HOST_VAR = "PIPELEX_USAGE_STATS_HOST"
+COCODE_USAGE_STATS_HOST = "cocode"
+
+
+def declare_cocode_usage_stats_host() -> None:
+    """Declare cocode as the program embedding pipelex, before cocode boots it.
+
+    A host the environment already declares is kept: a program that starts cocode with its own declaration is the one
+    the runs belong to.
+    """
+    os.environ.setdefault(PIPELEX_USAGE_STATS_HOST_VAR, COCODE_USAGE_STATS_HOST)

docs/pages/usage-statistics.md

@@ -0,0 +1,11 @@
+---
+title: Usage Statistics
+---
+
+# Usage Statistics
+
+cocode sends no usage statistics of its own. It runs its analyses on the Pipelex runtime, and Pipelex collects **anonymous usage statistics** from the Pipelex release that introduces them. Once cocode moves to that release, the runs it drives report through Pipelex's statistics and name cocode as their host, `"host": "cocode"`, so they can be told apart from the rest. Before that, nothing is sent.
+
+cocode declares itself by setting `PIPELEX_USAGE_STATS_HOST=cocode` in its own process before it boots Pipelex, and keeps any value the environment already sets.
+
+What Pipelex collects, what it never collects, where it goes and every way to turn it off are on Pipelex's [Usage Statistics](https://docs.pipelex.com/latest/setup/usage-statistics/) page. Each switch described there turns the statistics off for cocode too.

mkdocs.yml

@@ -71,6 +71,7 @@ nav:
   - Getting Started: pages/getting-started.md
   - Commands: pages/commands.md
   - Examples: pages/examples.md
+  - Usage Statistics: pages/usage-statistics.md
   - Contributing: contributing.md
 
 extra_css:

tests/conftest.py

@@ -1,5 +1,6 @@
 import asyncio
 import logging
+import os
 from pathlib import Path
 
 import pytest
@@ -20,6 +21,13 @@
 
 from cocode.common import PIPELINE_LIBRARY_DIRS
 
+# Our tests never report usage statistics, whatever pipelex they run against. `PIPELEX_USAGE_STATS=0` turns pipelex's
+# statistics off and touches nothing else, so a pipelex that has no statistics ignores it. `DO_NOT_TRACK` would turn
+# them off too, but the pinned pipelex refuses to boot with inference while it is set and the Pipelex Gateway is enabled.
+# Forced, not defaulted, so no value in the shell turns statistics back on, and set when this conftest loads, before any
+# fixture or plugin hook boots pipelex.
+os.environ["PIPELEX_USAGE_STATS"] = "0"
+
 PIPELINE_LIBRARY_DIRS_FOR_TESTS = [*PIPELINE_LIBRARY_DIRS, Path("tests/pipelines")]
 
 pytest_plugins = [

tests/unit/test_tests_never_report.py

@@ -0,0 +1,25 @@
+import importlib.util
+import os
+from pathlib import Path
+
+import pytest
+from pytest_mock import MockerFixture
+
+ROOT_CONFTEST_PATH = Path(__file__).parents[1] / "conftest.py"
+
+
+class TestTestsNeverReport:
+    @pytest.mark.parametrize("usage_stats_before", [None, "1"])
+    def test_root_conftest_turns_usage_stats_off(self, mocker: MockerFixture, usage_stats_before: str | None) -> None:
+        """Loading the root conftest sets `PIPELEX_USAGE_STATS=0`, whatever the environment held before, so no test sends usage statistics."""
+        mocker.patch.dict(os.environ)
+        os.environ.pop("PIPELEX_USAGE_STATS", None)
+        if usage_stats_before is not None:
+            os.environ["PIPELEX_USAGE_STATS"] = usage_stats_before
+
+        spec = importlib.util.spec_from_file_location("cocode_root_conftest_loaded_again", ROOT_CONFTEST_PATH)
+        assert spec is not None
+        assert spec.loader is not None
+        spec.loader.exec_module(importlib.util.module_from_spec(spec))
+
+        assert os.environ.get("PIPELEX_USAGE_STATS") == "0"

tests/unit/test_usage_stats.py

@@ -0,0 +1,37 @@
+import os
+
+import pytest
+from pytest_mock import MockerFixture
+from typer import Context as TyperContext
+
+from cocode.cli.main import main
+from cocode.usage_stats import COCODE_USAGE_STATS_HOST, PIPELEX_USAGE_STATS_HOST_VAR
+
+
+class TestUsageStatsHost:
+    @pytest.mark.parametrize(
+        ("declared_host", "expected_host_at_boot"),
+        [
+            (None, COCODE_USAGE_STATS_HOST),
+            ("mthds-agent", "mthds-agent"),
+        ],
+    )
+    def test_host_is_declared_before_pipelex_boots(self, mocker: MockerFixture, declared_host: str | None, expected_host_at_boot: str) -> None:
+        """The CLI declares `cocode` as the host before it boots pipelex, and keeps a host the environment already declares."""
+        mocker.patch.dict(os.environ)
+        os.environ.pop(PIPELEX_USAGE_STATS_HOST_VAR, None)
+        if declared_host is not None:
+            os.environ[PIPELEX_USAGE_STATS_HOST_VAR] = declared_host
+
+        hosts_seen_at_boot: list[str | None] = []
+
+        def record_host_at_boot(**_kwargs: object) -> None:
+            hosts_seen_at_boot.append(os.environ.get(PIPELEX_USAGE_STATS_HOST_VAR))
+
+        mocker.patch("cocode.cli.main.Pipelex.make", side_effect=record_host_at_boot)
+        ctx = mocker.Mock(spec=TyperContext)
+        ctx.invoked_subcommand = "validate"
+
+        main(ctx=ctx)
+
+        assert hosts_seen_at_boot == [expected_host_at_boot]