features and fixes - #204
Merged
Merged
Conversation
`by compile` wrote the generated C, the extensions and `by.h`, and nothing else. a compiled module reads its data files relative to itself, so a tree holding artefacts alone failed on the first `open`, naming a path in a directory the author never wrote anything to. carrying the rest of the project over is not enough on its own. the tree has to be one command's whole answer, because `finish` deletes what the last run wrote and this one did not — and that is only correct when this run authored the tree. so `compile` now writes everything `build` writes, the transpiled python and the sourcemap and the build record included, and the extensions besides. python's finder prefers an extension to source, so the modules that were named load natively and the rest are interpreted. three things follow, each of which was broken while the tree was artefacts alone: - `by compile one.by` produced a tree that could import `one` and nothing else written in basedpython, since the other modules arrived as `.by` - `by compile a.by` then `by compile b.by` left neither compiled: the second run pruned the first's artefact and put no python in its place - `by compile` into a directory `by build` had written deleted the sourcemap and the build record, and `by restage` then refused the tree — which took the language server's single-file re-stage down with it `Artifact` now reports what was written rather than what would have been. `--emit-c-only` used to name the extension it did not build, at a path no real build uses, so the caller found nothing there and pruned the extension a previous run had left. a build output is recognised by the `.by-manifest` it carries rather than by its name, so a tree written to any `--out` is turned away as a source. the directory is no longer created before the command knows it has anything to put in it, so a run that fails early leaves nothing behind. the default output directory is `build/` for both commands, which the standard python .gitignore already covers; `out/` never was. the editable install backend stages into the same constant and moves with it.
Contributor
ecosystem checkLinter (stable)✅ ecosystem check detected no linter changes. Linter (preview)✅ ecosystem check detected no linter changes. Formatter (stable)✅ ecosystem check detected no format changes. Formatter (preview)✅ ecosystem check detected no format changes. |
Contributor
by ecosystem round-tripbase: regressions: 0, changed: 63, improvements: 132, error changes: 0 (across 24927 files in 148 projects) ℹ️ changed round-trip outputpytest-autoprofile — .ci-helper-funcs.py(only produced on head)pytest-autoprofile — _by_sourcemap.py(only produced on head)pytest-autoprofile — lib/pytest_autoprofile/__init__.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_doctest.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_json.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_multiprocessing.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_patches.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_test_util_capture_warnings.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_test_utils.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_typing.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_warnings.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/_xdoctest.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/importers.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/option_hooks.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/plugin.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/profiler.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/rewriting.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/setup.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/startup_hook.py(only produced on base)pytest-autoprofile — lib/pytest_autoprofile/utils.py(only produced on base)pytest-autoprofile — pytest_autoprofile/__init__.py(only produced on head)pytest-autoprofile — pytest_autoprofile/_json.py(only produced on head)pytest-autoprofile — pytest_autoprofile/_patches.py(only produced on head)pytest-autoprofile — pytest_autoprofile/_test_util_capture_warnings.py(only produced on head)pytest-autoprofile — pytest_autoprofile/_typing.py(only produced on head)pytest-autoprofile — pytest_autoprofile/_warnings.py(only produced on head)pytest-autoprofile — pytest_autoprofile/_xdoctest.py(only produced on head)pytest-autoprofile — pytest_autoprofile/plugin.py(only produced on head)pytest-autoprofile — pytest_autoprofile/rewriting.py(only produced on head)pytest-autoprofile — pytest_autoprofile/setup.py(only produced on head)pytest-autoprofile — pytest_autoprofile/startup_hook.py(only produced on head)pytest-autoprofile — pytest_autoprofile/utils.py(only produced on head)pytest-autoprofile — tests/__init__.py(only produced on head)pytest-autoprofile — tests/conftest.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_autoprof/math_pkg/__init__.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_autoprof/math_pkg/arithmatics.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_autoprof/math_pkg/taylor_series.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_autoprof/my_test_package/__init__.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_autoprof/my_test_package/test_math.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_autoprof/my_test_package/test_string.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_autoprof/string_pkg/__init__.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_autoprof/string_pkg/utils.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_doctest/packages/my_pkg/__init__.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_doctest/packages/my_pkg/classes.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_doctest/packages/my_pkg/funcs.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_doctest/tests/test_misc.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_global_prof/complex_env/script.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_global_prof/complex_env/test_toggling.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_global_prof/simple_env/modules/mymod.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_global_prof/simple_env/tests/test_func.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_profiler/profiler_test_module.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_subprocess/packages/myfork.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_subprocess/packages/mymodule.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_subprocess/packages/myprocess.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_subprocess/tests/conftest.py(only produced on head)pytest-autoprofile — tests/test-modules-and-tests/test_subprocess/tests/test_func.py(only produced on head)pytest-autoprofile — tests/test_ast.py(only produced on head)pytest-autoprofile — tests/test_doctest.py(only produced on head)pytest-autoprofile — tests/test_global_prof.py(only produced on head)pytest-autoprofile — tests/test_misc.py(only produced on head)pytest-autoprofile — tests/test_patches.py(only produced on head)pytest-autoprofile — tests/test_profiler.py(only produced on head)pytest-autoprofile — tests/test_subprocess.py(only produced on head)💥 fails to round-trip (unchanged from base)
✅ improvements (failed on base, now builds)
⏭️ skipped
|
KotlinIsland
force-pushed
the
features-and-fixes
branch
from
September 7, 2026 13:26
7e566bf to
c45f2c1
Compare
a `by check` builds a project database, resolves an environment, parses and infers every file, and then exits and throws all of it away. a language server running for that project has that work done and kept current, so `by check` now asks it first and says "using project server information" when it does. on an 11,850-file project that is 3.8s down to 0.8s a server publishes a loopback port and a secret into a per-user directory as it starts, and takes the record away as it exits. a caller reads the directory, sends the project root and the configuration it resolved, and gets the diagnostics back already rendered — rendering needs the database, and relative paths need the caller's own directory, so both cross the wire the answer only crosses when it is the answer a cold check would have given, so what the two sides compare is what each actually ended up with: the merged options across every layer, the resolved python version, platform and search paths, and the two database inputs that are not options at all — `force-exclude` and whether the diagnostics explain where each rule was turned on. an environment is discovered as much as configured, out of `VIRTUAL_ENV`, a `.venv` beside the project or uv's answer, and a server started by an editor discovers it from the editor's environment: two processes that agree about every option can still be checking against different site-packages. the caller therefore asks after building its database rather than before, since resolving all of that is the database's job, and building one is cheap next to checking with it the server also refuses a different build, an open buffer the editor has not saved — every open document, notebooks included, and buffers outside the project root this database has read — and a session diagnosing only its open files; that last one is the default, so this needs `diagnosticMode: workspace`. it re-reads the file system first, because its own picture of the project is whatever the editor's watcher reported and the caller can see the disk directly, and it does so between the two passes over a request so a check that was going to be refused does not make the editor re-pull its diagnostics. `--no-server` and `BY_NO_PROJECT_SERVER` switch it off, the second on both halves the caller rules itself out for anything that is not a whole-project check: `--watch`, a fix mode, explicit paths and a memory report. a failure writing the answer is a failure rather than "no server answered", so a broken pipe no longer prints half the diagnostics and then checks the whole project again connections are served one thread each, both directions are bounded in size and time, the token is compared in constant time, and the answer repeats the token back: a record outlives a killed server, and anything that binds the freed port could otherwise have answered "All checks passed!". a build is identified by its executable as well as its version, since the version string does not change when a dirty tree is rebuilt the tests build their requests from a real cold database the way the command line does, and assert that the rendering matches that database's own — the claim the whole design rests on. `scripts/check_project_server.py` drives a real server over stdin and stdout and asserts the same thing across two processes, plus that every flag which changes a check refuses
`or` and `and` inside a type expression spell union and intersection, so the boolean rules have nothing to say about them: `type A = "foo" or "bar"` was SIM222 "use `"foo"` instead of `"foo" or ...`". A bare string in a type position is the literal type, not a forward reference, so UP037 offered to strip the quotes off `c: "c"` and leave a name. A match type's `case` arms are types, so their `:` opened no suite for E701 to report. `is` is a parametric type test rather than identity, and `===` is the spelling that always compares identity — F632 read the two as one, panicked on `1 === 1` because it could not find that operator's token, and offered to rewrite the type test `1 is T` as `1 == T`. E721 told a `.by` file to reach for python's two operators rather than basedpython's. A destructuring binder always binds, so `for Point(x, y) in points` and `def f(Point(x, y): Point)` now bind their captures the way the equivalent python unpacking does instead of as plain assignments F841 reports. A type parameter is not a constant, however the one-letter convention spells it, so `assert T == int` is no longer a yoda condition. ty spelled a `.by` file's types in python syntax whenever a `.py` file that imports it was checked first: the diagnostic that inference cached said `Literal["b"]` rather than `"b"`. The spelling now comes from the file being inferred rather than from whoever asked for the inference, which is what keeps it out of the salsa cache's blind spot.
`TypeVar("T", bound=A & B)` type checked clean and transpiled to itself, so the
emitted python evaluated `A.__and__(B)` at import. `NewType`, `TypeAliasType`,
`ParamSpec`, `TypeVarTuple` and the functional `NamedTuple` and `TypedDict` were
all silent the same way, for `?`, `not`, `or`/`and` and the rest of the type
syntax as much as for `&`: the transpiler's type-expression walker recognised
twelve positions, and none of them was an argument.
Which arguments those are is now the type checker's answer rather than a list the
transpiler keeps. `CallTypeForm` names the constructs and their type-expression
arguments in one place, inference dispatches on it, and the walker asks it, so a
form ty learns to check is one the transpiler lowers without further change.
The mdtest that covers this is one the divergence harness runs: with the walker's
hook removed it fails with `TypeError: unsupported operand type(s) for &`, which
is what the miscompilation did to anyone who wrote it.
the right-hand side of `is` in a `.by` file is a type position, but it was
inferred as a value expression — so none of the type-expression validation ran
on it. an annotation and a type test rejected different things: of 23 targets,
annotation position and `cast` position rejected the same 7 and `is` rejected
none, lowering each to a program that raises.
the target is now inferred as a type expression, and the type it names is
classified for a runtime form. what the runtime can check exactly is lowered to
that check; what it can only partly check is rejected rather than approximated,
since a test narrows and has to earn its `True`:
- a class, `type[C]`, a union, `None`, a literal, an enum member, a template
literal type and a conformance interface all lower to the check that decides
membership of the type. a literal pins the class too, since `1 == True`
would otherwise let a `bool` satisfy `Literal[1]`
- `Any`, a callable type, a `TypedDict`, an intersection and a protocol with an
unnameable member are rejected by `erased-type-check`, each with its own
reason. `float`, `complex`, `LiteralString` and `type[Any]` each report the
true reason rather than "has no runtime form", and a bare `Callable` lowers
to `callable()`
- a tuple target now means the tuple *type*. `v is (int, str)` answered `True`
for an `int`, because it lowered to `isinstance`'s classinfo tuple
the emitted check needs a *value*, and the type the source names parts company
with one more often than it looks like it does: `type AL = int` evaluates to a
`TypeAliasType`, `Literal[Color.RED]` and `Annotated[int, "x"]` to special
forms, `list[Any]` to a subscripted generic — none of which `isinstance` will
take. so the spelling is rebuilt from the type rather than passed through, and
the source is used only where rebuilding cannot reach: a class the emitting
module cannot name as a global, and only when the source wrote a plain dotted
name.
a sealed hierarchy now declares each variant in the enum body —
`Circle: ClassVar[type[_Shape_Circle]]`, `Point: ClassVar[_Shape_Point]` —
which is what lets the checker see that one is a class and the other a value,
and the lowering asks it rather than matching the written name. matching the
name both over-fired (a local binding shadowing the enum was rewritten anyway)
and under-fired (`S = Shape; s is S.Point` was missed, emitting an `isinstance`
against a singleton that raises). the declarations also clear the
`unresolved-attribute` a lowered enum used to report on its own variants.
`basedpython_is_keeps_identity` goes away with the value position it existed to
serve: `None`, an enum member and a literal all fall out of their own types.
the spelling now rides on the ast. `===` / `!==` and the `is` keyword parse to
one `CmpOp`, and four places told them apart by reading the source text between
the operands — so a parenthesised operand, a line continuation or a comment
silently turned a type test back into python identity, and `is` and `is not`
disagreed about the same spelling. `ExprCompare::identity_ops` records which
operators were written `===` / `!==`; the checker, the narrower, the formatter
and the transpiler all read it, and the parser builds the side table lazily so
an ordinary comparison allocates nothing. a chained type test is now a syntax
error, but only where a type test is something that gets handed on: python
chains `a is int is str` into `a is int and int is str`, whose second half asks
whether the class `int` has the type `str`, while `a < b is None` hands nothing
on. narrowing skips a chain the parser rejected instead of making its body
unreachable.
`generate_comparison` printed `is` for source that wrote `===`, so E713/E714
rewrote an identity comparison into a type test through a *safe* fix, and
E711/E712 destroyed every `===` in a chain; it takes the spelling now.
`assertIs` writes `===`, and BY003 rewrites an `isinstance` only when its
argument has the shape of a type expression — the predicate the reverse
direction already used. F632 reads the recorded spelling as well, rather than
scanning the tokens for it a second time; the scan is left to place the fix,
which is what lets `1 === 1 !== 2` offer `==` for one operator and `!=` for the
other. `identity_swap` no longer decides on ast shape, so an
f-string, a call or a unary-minus target takes the same path as a name; its
replacement range accounts for parentheses around an operand, which produced
invalid python for `(a) is not str`, and an unspaced `a===b` brings its own
spaces.
a settled test types as its answer rather than `bool`, which is what the
editor's data-flow view reads. the reverse direction writes an `isinstance`
back out only when its argument is something a type expression can say, mapping
a classinfo tuple to a union.
also: a recursive alias overflowed the stack on `v is A` — `type A = int | B`
with `type B = str | A` — so the alias recursion carries what it has already
opened; the template-literal pattern regex and the static matcher disagreed on
`-0`, where `str(-0)` is `"0"` so the sign belongs to the non-zero alternative;
and control characters are escaped rather than passed through, since CPython
refuses a source containing a NUL.
the runtime contract each target kind lowers to is executed end to end in
`parametric_is_runtime`, rather than only type-checked.
the entry is a folded scalar, which only folds the lines indented to match its first one — the deeper-indented continuations keep their newlines, so bash was handed `&& RUSTDOCFLAGS=...` at the start of a line and refused the whole command. every commit failed the hook with a syntax error rather than a doc error. the continuations line up with the first line again.
KotlinIsland
force-pushed
the
features-and-fixes
branch
from
September 7, 2026 16:31
c45f2c1 to
83b8f07
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.