[Lincoln-developer]

Closes #48.

The bug

profiling/visualize.py had no if __name__ == "__main__" block and never imported argparse. Every documented invocation of it exited 0 and wrote nothing:

python profiling/visualize.py profiling/results/a100-modal-2026-10-10/benchmark_results.json --output-dir <tmp>
exit code 0, no directory created, no files written

A silent no-op is worse than a crash. Whoever ran it walked away believing the plots now existed. That is how the A100 result set came to have only benchmark_results.json and metadata.json while the T4 set had four plots and a summary, and it is why --output-dir appears in two documents despite never having been implemented.

What changed

Six commits.

Commit	Change
43eaa5a	main() with argparse: positional results_path, optional --run-dir
032d8bc	CI installs .[dev,profiling], since matplotlib lives in the profiling extra
7288a93	Four tests that execute the script as a subprocess
9b79674	A100 plots and summary generated, manifest.json updated
c2d7332	ruff format on the new test file
396918b	Four documents corrected

--run-dir defaults to the directory holding the JSON, so the common case needs no second argument. A missing input is an argparse error, not a traceback.

The flag is --run-dir, not the --output-dir the old docs claimed. Naming it --output-dir while it means "result set directory" is what produced the duplicate summary in #47, so the name now says what it takes.

The test executes the script rather than importing it

That distinction is the entire point. Importing visualize would have looked perfectly healthy while the bug was live, because the defect was the absence of an entry point. Only running it the way a contributor does catches that. The tests cover --help rendering, the layout manifest.json describes, --run-dir defaulting, and a missing file failing loudly.

This required widening testpaths to include profiling/tests, without which pytest collected none of it. That is a small piece of #35, which remains open for real profiling coverage.

A100 artifacts

Generated from the committed JSON. No GPU involved, and no measured value changes anywhere. The summary reports the A100 peaks from gpu_specs_db.py: 1555 GB/s, 19.5 TFLOP/s FP32, 312.0 TFLOP/s dense BF16 tensor.

Both result sets now declare the same seven artifact types, verified in both directions: every file listed in manifest.json exists, and every file on disk is listed.

One figure stated explicitly, because two documents quote different numbers for it. Average A100 attention compute utilization is 7.0%; the peak across the twelve attention configurations is 14.06%. The analysis document quotes the range and the peak, this summary quotes the mean. Recomputed from raw data: mean 6.9991, max 14.0571, min 0.9722. Neither is wrong and neither should be reconciled to the other.

Documentation

Four files described the broken command. profiling/README.md and README.md showed the bare invocation, profiling/results/README.md showed it twice with the nonexistent --output-dir, and CONTRIBUTING.md described the failure accurately but as an open defect. All four now show the working command and the layout it produces.

Two further corrections the stale docs were hiding:

• The profiling/README.md file structure tree still showed the flat pre-reorganization results layout and omitted gpu_specs_db.py, recompute_utilization.py, and tests/.
• The four charts in the root README did not say which GPU or run they came from, so a reader could not tell whether a figure described the T4 or the A100. Each caption now names both, and the section says the A100 run has the same four charts in its own directory.

Verification

• python -m ruff check . reports All checks passed!
• python -m ruff format --check . reports 25 files already formatted
• python -m pytest passes, 29 tests, up from 25
• CLI exercised directly: --help renders, and a real run against the A100 JSON produced four PNGs under plots/ and one benchmark_summary.txt at run-dir level, with nothing at plots/benchmark_summary.txt
• manifest.json checked against the filesystem for both result sets, both directions, zero mismatches

One thing worth flagging about my own process. Commit 7288a93 added the test file without running the formatter, so it failed ruff format --check and would have turned the lint job red on this branch. c2d7332 fixes it. Caught locally before pushing, but it is a real miss and the reason that commit exists separately.

Not included

The suite grows from about 10s to about 40-60s locally, because two of the new tests render four matplotlib figures each in a subprocess. That cost is deliberate: a silent no-op is the failure being prevented, and only real execution prevents it.

Full profiling test coverage is #35. This adds one test module, not a suite.