From 8644a5eef62c19e1514cc6b2b609e89627fde505 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Wed, 12 Aug 2026 15:16:29 +0100 Subject: [PATCH 001/371] [ty] Improve ecosystem summary reports (#27695) --- .../summarise-ecosystem-results/SKILL.md | 15 +++++--- .../assets/report-template.md | 34 +++++++++++++++---- 2 files changed, 38 insertions(+), 11 deletions(-) diff --git a/.agents/skills/summarise-ecosystem-results/SKILL.md b/.agents/skills/summarise-ecosystem-results/SKILL.md index 1b258a151b..875f983d69 100644 --- a/.agents/skills/summarise-ecosystem-results/SKILL.md +++ b/.agents/skills/summarise-ecosystem-results/SKILL.md @@ -8,7 +8,7 @@ description: Use when a user says "summarise ecosystem results", "summarize this ## Priorities 1. Reproduce every retained behavior with the exact environment used by the Actions run. -2. Lead the report with new or changed project failures, then cover meaningful flaky behavior, diagnostic changes, and clear minimized examples. +2. Lead the report with new or meaningfully changed project failures, including intermittent severe failures, then cover stable diagnostic changes and clear minimized examples. 3. Keep execution, audit, and traceability bookkeeping out of the report. ## Deliverable @@ -19,13 +19,20 @@ Use the template's structure and omissions as the report contract. Remove all pl If summarising an ecosystem report is the only thing you're asked to do in a Codex App thread, you should rename that thread to "PR ecosystem summary". +## Reporting Policy + +- Focus on new or meaningfully changed behavior relative to the merge base. Evaluate individual diagnostics and failure outcomes, not a project's overall flaky or persistent status. +- Omit flaky diagnostic changes, unchanged failures, and frequency fluctuations that leave the observed outcomes unchanged. +- Report new or changed panics, crashes, overflows, and timeouts, including merge-base and PR run frequencies when intermittent behavior is involved. + ## Workflow 1. **Freeze the evidence.** Preserve any report URL or ecosystem-results comment explicitly supplied by the user before identifying the PR. For PR-only input, find its ecosystem-results comment and linked detailed report. Capture the matching Actions run and attempt as described in [references/evidence-acquisition.md](references/evidence-acquisition.md); never replace a supplied report with the PR's current report. Ignore later comment edits, PR updates, and workflow runs. Use the frozen detailed report as the authoritative change list and the comment for orientation when available. -2. **Identify changed outcomes.** Check the detailed report for new, fixed, or changed project failures, panics, timeouts, abnormal exits, and meaningful flaky diagnostic or exit-status changes. Omit unchanged persistent failures. If neither project outcomes nor diagnostics changed, say explicitly that the run had no ecosystem impact and omit project-specific sections and reproduction details. -3. **Reproduce from scratch.** Ignore retained memories and previous local artifacts. Load the `minimizing-ty-ecosystem-changes` skill, use its metadata helper and exact-run workflow, and reproduce each report entry before explaining or minimizing it. Reproduce flaky behavior with the reported run counts. +2. **Identify changed outcomes.** Check the detailed report for new, fixed, or changed project failures, panics, overflows, timeouts, abnormal exits, and diagnostic changes, applying the reporting policy to each entry and outcome. +3. **Reproduce from scratch.** Ignore retained memories and previous local artifacts. Load the `minimizing-ty-ecosystem-changes` skill, use its metadata helper and exact-run workflow, and reproduce each report entry before explaining or minimizing it. Reproduce intermittent severe failure changes with the reported run counts. 4. **Minimize with provenance.** Include a standalone reproducer only when a verified reduction chain connects it to a cited ecosystem entry and preserves the same underlying trigger. If either cannot be verified, retain the original source excerpt and identify it as unminimized. 5. **Group by cause.** Group entries only when the same base-to-PR behavior, underlying trigger, explanation, and reproducer account for every entry. Identical diagnostic text or displayed `@Todo` types do not establish equivalence. -6. **Write and verify.** Fill the report template, record each affected project's strict or non-strict analysis mode, and include both strict-analysis flags in the comparison method when applicable. Check every link, diagnostic, reproducer's source provenance, and causal fingerprint when required, then run `uv run --only-group dev --locked prek run --files PR__ECOSYSTEM_SUMMARY.md`. Present the Markdown file as the finished product. +6. **Find existing ty issues.** When a diagnostic change exposes a pre-existing shortcoming in ty, search the `astral-sh/ty` issue tracker for the precise underlying behavior. Link matching issues directly from the relevant report section; do not mistake incorrect or incomplete third-party annotations for ty shortcomings. +7. **Write and verify.** Fill the report template, record each affected project's strict or non-strict analysis mode, and include both strict-analysis flags in the comparison method when applicable. Check every change number, link, diagnostic, reproducer's source provenance, and causal fingerprint when required, then run `uv run --only-group dev --locked prek run --files PR__ECOSYSTEM_SUMMARY.md`. Present the Markdown file as the finished product. When parallelizing reproduction or minimization, read [references/subagent-handoff.md](references/subagent-handoff.md). Otherwise, keep batches small and work through them sequentially. diff --git a/.agents/skills/summarise-ecosystem-results/assets/report-template.md b/.agents/skills/summarise-ecosystem-results/assets/report-template.md index c77654181a..991a7c5bb7 100644 --- a/.agents/skills/summarise-ecosystem-results/assets/report-template.md +++ b/.agents/skills/summarise-ecosystem-results/assets/report-template.md @@ -1,22 +1,38 @@ - + # [PR #](https://github.com/astral-sh/ruff/pull/) ecosystem summary - + - + -## +## Project failures + +### 1. **Affected projects:** - [](): merge base: ``; PR: ``. - + + + + +## Intermittent severe failures + +### 1. + +**Affected projects:** - +- [](): merge base: ``; PR: ``. -## + + + + +## Affected projects + +### 1. **Report entries:** @@ -26,6 +42,10 @@ + + +**Existing ty issues:** [ty#](https://github.com/astral-sh/ty/issues/) + src/mdtest_snippet.py:1:7 + | +1 | while 1: # snapshot: while-one + | ^ +help: Replace with `True` + | + - while 1: # snapshot: while-one +1 + while True: # snapshot: while-one +2 | print("Hello, world!") + | +``` + +## Other spellings of one + +Any integer literal equal to one is flagged, whatever its base, and each is fixed to `True`. + +```py +while 0x1: # snapshot: while-one + ... + +while 0b1: # error: [while-one] + ... + +while 0o1: # error: [while-one] + ... + +while 1_0: # ten, not one, so this is left alone + ... +``` + +```snapshot +error[UP048]: Use `while True:` instead of `while 1:` + --> src/mdtest_snippet.py:1:7 + | +1 | while 0x1: # snapshot: while-one + | ^^^ +help: Replace with `True` + | + - while 0x1: # snapshot: while-one +1 + while True: # snapshot: while-one +2 | ... + | +``` + +## Parentheses and comments are preserved + +Only the literal itself is rewritten, so surrounding trivia survives the fix. + +```py +while ( + # keep me + 1 # snapshot: while-one +): + ... +``` + +```snapshot +error[UP048]: Use `while True:` instead of `while 1:` + --> src/mdtest_snippet.py:3:5 + | +3 | 1 # snapshot: while-one + | ^ +help: Replace with `True` + | +2 | # keep me + - 1 # snapshot: while-one +3 + True # snapshot: while-one +4 | ): + | +``` + +## Other conditions are left alone + +`while 0:` is unreachable rather than infinite, and rewriting it would change behavior. Non-literal +conditions are out of scope even when they are always truthy, because flagging them would collide +with rules that catch accidentally-constant conditions. + +```py +while 0: + ... + +while True: + ... + +while 1.0: + ... + +while "always": + ... + +while [1]: + ... + +while 2: + ... + +while -1: + ... +``` + +The rule targets the loop condition only, not integer literals elsewhere in a `while` statement. + +```py +x = 1 +while x == 1: + x = 1 +``` diff --git a/crates/ruff_linter/src/checkers/ast/analyze/statement.rs b/crates/ruff_linter/src/checkers/ast/analyze/statement.rs index 5d33fdb054..1d007eea2d 100644 --- a/crates/ruff_linter/src/checkers/ast/analyze/statement.rs +++ b/crates/ruff_linter/src/checkers/ast/analyze/statement.rs @@ -1242,6 +1242,9 @@ pub(crate) fn statement(stmt: &Stmt, checker: &mut Checker) { if checker.is_rule_enabled(Rule::NeedlessElse) { ruff::rules::needless_else(checker, while_stmt.into()); } + if checker.is_rule_enabled(Rule::WhileOne) { + pyupgrade::rules::while_one(checker, while_stmt); + } } Stmt::For( for_stmt @ ast::StmtFor { diff --git a/crates/ruff_linter/src/codes.rs b/crates/ruff_linter/src/codes.rs index 645b9266a2..e78302f81e 100644 --- a/crates/ruff_linter/src/codes.rs +++ b/crates/ruff_linter/src/codes.rs @@ -588,6 +588,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> { (Pyupgrade, "045") => rules::pyupgrade::rules::NonPEP604AnnotationOptional, (Pyupgrade, "046") => rules::pyupgrade::rules::NonPEP695GenericClass, (Pyupgrade, "047") => rules::pyupgrade::rules::NonPEP695GenericFunction, + (Pyupgrade, "048") => rules::pyupgrade::rules::WhileOne, (Pyupgrade, "049") => rules::pyupgrade::rules::PrivateTypeParameter, (Pyupgrade, "050") => rules::pyupgrade::rules::UselessClassMetaclassType, (Pyupgrade, "051") => rules::pyupgrade::rules::DeprecatedAbcDecorator, diff --git a/crates/ruff_linter/src/rules/pyupgrade/rules/mod.rs b/crates/ruff_linter/src/rules/pyupgrade/rules/mod.rs index 1ee8efc6fb..ca0e3d667e 100644 --- a/crates/ruff_linter/src/rules/pyupgrade/rules/mod.rs +++ b/crates/ruff_linter/src/rules/pyupgrade/rules/mod.rs @@ -41,6 +41,7 @@ pub(crate) use use_pep604_isinstance::*; pub(crate) use useless_class_metaclass_type::*; pub(crate) use useless_metaclass_type::*; pub(crate) use useless_object_inheritance::*; +pub(crate) use while_one::*; pub(crate) use yield_in_for_loop::*; mod convert_named_tuple_functional_to_class; @@ -86,4 +87,5 @@ mod use_pep604_isinstance; mod useless_class_metaclass_type; mod useless_metaclass_type; mod useless_object_inheritance; +mod while_one; mod yield_in_for_loop; diff --git a/crates/ruff_linter/src/rules/pyupgrade/rules/while_one.rs b/crates/ruff_linter/src/rules/pyupgrade/rules/while_one.rs new file mode 100644 index 0000000000..f94e56ec3e --- /dev/null +++ b/crates/ruff_linter/src/rules/pyupgrade/rules/while_one.rs @@ -0,0 +1,68 @@ +use ruff_macros::{ViolationMetadata, derive_message_formats}; +use ruff_python_ast::{self as ast, Expr, Number}; +use ruff_text_size::Ranged; + +use crate::checkers::ast::Checker; +use crate::{AlwaysFixableViolation, Edit, Fix}; + +/// ## What it does +/// Checks for `while` loops that use `1` as their condition. +/// +/// ## Why is this bad? +/// `while 1:` is a Python 2 idiom, where `True` was a global that could be +/// rebound and so had to be loaded and tested on every iteration. In Python 3 +/// `True` is a keyword, so both spellings compile to the same bytecode and +/// `while True:` is clearer about the loop being infinite. +/// +/// ## Example +/// ```python +/// while 1: +/// print("Hello, world!") +/// ``` +/// +/// Use instead: +/// ```python +/// while True: +/// print("Hello, world!") +/// ``` +/// +/// ## References +/// - [Python documentation: `while`](https://docs.python.org/3/reference/compound_stmts.html#the-while-statement) +/// - [PEP 285 – Adding a bool type](https://peps.python.org/pep-0285/) +#[derive(ViolationMetadata)] +#[violation_metadata(preview_since = "NEXT_RUFF_VERSION")] +pub(crate) struct WhileOne; + +impl AlwaysFixableViolation for WhileOne { + #[derive_message_formats] + fn message(&self) -> String { + "Use `while True:` instead of `while 1:`".to_string() + } + + fn fix_title(&self) -> String { + "Replace with `True`".to_string() + } +} + +/// UP048 +pub(crate) fn while_one(checker: &Checker, while_stmt: &ast::StmtWhile) { + let Expr::NumberLiteral(ast::ExprNumberLiteral { + value: Number::Int(value), + .. + }) = &*while_stmt.test + else { + return; + }; + + // Also covers other spellings of one, such as `0x1`. + if value.as_u8() != Some(1) { + return; + } + + let range = while_stmt.test.range(); + let mut diagnostic = checker.report_diagnostic(WhileOne, range); + diagnostic.set_fix(Fix::safe_edit(Edit::range_replacement( + "True".to_string(), + range, + ))); +} diff --git a/ruff.schema.json b/ruff.schema.json index 6ecca20f03..d9c6eccdd0 100644 --- a/ruff.schema.json +++ b/ruff.schema.json @@ -4535,6 +4535,7 @@ "UP045", "UP046", "UP047", + "UP048", "UP049", "UP05", "UP050", @@ -5508,6 +5509,7 @@ "verbose-raise", "wait-for-process-in-async-function", "weak-cryptographic-key", + "while-one", "whitespace-after-decorator", "whitespace-after-open-bracket", "whitespace-before-close-bracket", From 2b0d21094e2a55491bff60c07fd6f8803876cae5 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Wed, 12 Aug 2026 16:53:49 -0700 Subject: [PATCH 013/371] [ty] Centralize matched argument relations (#27705) ## Summary Part of astral-sh/ty#3557. This is the first focused step toward rebooting #26712 as a stack of smaller solver changes. Introduce `ArgumentRelation` as the shared representation of a matched argument/parameter pair and use it for generic specialization inference, unsatisfiable-constraint diagnostic recovery, and ordinary, positional-unpack, and keyword-unpack argument checking. Preserve effective per-element formal types, each checking path's existing actual-type lookup, synthetic receiver/source argument indices, gradual variadic filtering, and existing `TypeVarTuple` handling without changing solver behavior. ## Test plan Added mdtests covering generic bound-method diagnostics after an implicit receiver, positional and keyword source locations, homogeneous and heterogeneous unpacked generic parameters, unpacked positional element types, per-key `TypedDict` generic inference, and gradual variadic arguments that must not affect inference. --- .../resources/mdtest/call/function.md | 49 +++- .../mdtest/generics/pep695/functions.md | 21 ++ .../ty_python_semantic/src/types/call/bind.rs | 240 +++++++++++------- 3 files changed, 213 insertions(+), 97 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/call/function.md b/crates/ty_python_semantic/resources/mdtest/call/function.md index dfbcd9f3ef..2809aec140 100644 --- a/crates/ty_python_semantic/resources/mdtest/call/function.md +++ b/crates/ty_python_semantic/resources/mdtest/call/function.md @@ -12,15 +12,21 @@ reveal_type(get_int()) # revealed: int ## Gradual variadic parameters ```py -from typing import Any +from typing import Any, TypeVar + +T = TypeVar("T") def accepts_anything(first: int, *args: Any, **kwargs: Any) -> None: ... def accepts_only_gradual(*args: Any, **kwargs: Any) -> None: ... +def preserves_first(first: T, *args: Any, **kwargs: Any) -> T: + return first accepts_anything(1, "one", object(), keyword=object()) accepts_anything("not an int") # error: [invalid-argument-type] accepts_only_gradual(1, "one", keyword=object()) accepts_only_gradual(**{1: "one"}) # error: [invalid-argument-type] + +reveal_type(preserves_first(1, "other", keyword=object())) # revealed: Literal[1] ``` ## Object variadic parameters @@ -1370,8 +1376,9 @@ with_default(1, 2) ### Unpacked variadic elements preserve generic bounds -Ordinary type variables are inferred from individual unpacked elements, even beside an unresolved -type-variable tuple. Their upper bounds remain enforced. +Ordinary type variables are inferred from individual unpacked elements in homogeneous or +heterogeneous tuples, even beside an unresolved type-variable tuple. Their upper bounds remain +enforced. ```toml [environment] @@ -1390,6 +1397,19 @@ fixed(1) # error: [invalid-argument-type] reveal_type(suffix("prefix", "valid")) # revealed: Literal["valid"] suffix("prefix", 1) # error: [invalid-argument-type] + +def homogeneous[T: str](*args: *tuple[T, ...]) -> T: + return args[0] + +reveal_type(homogeneous("first", "second")) # revealed: Literal["first", "second"] +homogeneous("valid", 1) # error: [invalid-argument-type] + +def heterogeneous[T: str](*args: *tuple[int, T]) -> T: + return args[1] + +def _(valid: tuple[int, str], invalid: tuple[int, int]) -> None: + reveal_type(heterogeneous(*valid)) # revealed: str + heterogeneous(*invalid) # error: [invalid-argument-type] ``` ### Callable protocols enforce unpacked variadic requirements @@ -1593,6 +1613,29 @@ f(**dict(a=1, b=2)) f(**Foo(a=1, b=2)) ``` +### Unpacked keyword values retain their individual generic types + +Each named value in an unpacked `TypedDict` must be related to its own generic parameter instead of +using the type of the complete mapping. + +```py +from typing_extensions import TypedDict, TypeVar + +T = TypeVar("T") +U = TypeVar("U") + +class Values(TypedDict, closed=True): + first: int + second: str + +def combine(*, first: T, second: U) -> tuple[T, U]: + return first, second + +values: Values = {"first": 1, "second": "value"} + +reveal_type(combine(**values)) # revealed: tuple[int, str] +``` + ### Keyword-only parameters ```py diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md index e639338822..9455c740bf 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md @@ -1056,6 +1056,27 @@ def _(x: int): reveal_type(C().implicit_self(x)) # revealed: tuple[C, int] ``` +## Generic method errors account for the implicit receiver + +An implicit `self` participates in generic inference but is absent from the call-site argument list. +Bound violations must still identify the correct positional or keyword argument. + +```py +class Box: + def accept[T: int](self, value: T, *, other: T) -> T: + return value + +box = Box() + +reveal_type(box.accept(1, other=2)) # revealed: Literal[1, 2] + +# error: 12 [invalid-argument-type] "does not satisfy upper bound `int`" +box.accept("invalid", other=1) + +# error: 15 [invalid-argument-type] "does not satisfy upper bound `int`" +box.accept(1, other="invalid") +``` + ## `~T` is never assignable to `T` ```py diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index 7960593f7f..fa5d308e05 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -5413,6 +5413,44 @@ struct ArgumentTypeChecker<'a, 'db> { constraint_set_errors: Vec, } +/// The formal and actual types associated with one matched argument-parameter pair. +/// +/// An unpacked argument can produce multiple relations, each with its own matched parameter and +/// actual element type. +#[derive(Clone, Copy, Debug)] +struct ArgumentRelation<'db> { + argument_index: usize, + + /// The source argument index, or `None` for a synthetic receiver. + adjusted_argument_index: Option, + + matched_parameter: MatchedParameter<'db>, + declared_type: Type<'db>, + argument_type: Type<'db>, + has_starred_annotation: bool, +} + +impl<'db> ArgumentRelation<'db> { + fn new( + argument_index: usize, + adjusted_argument_index: Option, + parameter: &Parameter<'db>, + matched_parameter: MatchedParameter<'db>, + argument_type: Type<'db>, + ) -> Self { + Self { + argument_index, + adjusted_argument_index, + matched_parameter, + declared_type: matched_parameter + .expected_type + .unwrap_or_else(|| parameter.annotated_type()), + argument_type, + has_starred_annotation: parameter.has_starred_annotation(), + } + } +} + /// Result of checking only the key type of a keyword-unpack argument. enum KeywordUnpackKeyTypeCheck<'db> { /// The argument type is handled by a more specific path, or does not expose mapping keys. @@ -5493,8 +5531,14 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { fn enumerate_argument_types( &self, - ) -> impl Iterator, Argument<'a>, &CallArgumentTypes<'db>)> + 'a - { + ) -> impl Iterator< + Item = ( + usize, + Option, + Argument<'a>, + &'a CallArgumentTypes<'db>, + ), + > + 'a { let mut iter = self.arguments.iter().enumerate(); let mut num_synthetic_args = 0; std::iter::from_fn(move || { @@ -5519,6 +5563,44 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { }) } + /// Yields the effective formal and actual types for each matched argument-parameter pair. + /// + /// Gradual variadic parameters do not contribute constraints. For unpacked tuple parameters, + /// the matched parameter can provide a more specific formal type for the corresponding element. + fn argument_relations(&self) -> impl Iterator> + 'a { + let parameters: &'a Parameters<'db> = self.signature.parameters(); + let argument_matches: &'a [MatchedArgument<'db>] = self.argument_matches; + + self.enumerate_argument_types().flat_map( + move |(argument_index, adjusted_argument_index, _, argument_types)| { + argument_matches[argument_index] + .iter() + .filter_map(move |matched_parameter| { + let parameter_index = matched_parameter.index; + if Self::is_gradual_variadic_parameter(parameters, parameter_index) { + return None; + } + + let parameter = ¶meters[parameter_index]; + let declared_type = matched_parameter + .expected_type + .unwrap_or_else(|| parameter.annotated_type()); + let argument_type = matched_parameter + .argument_type + .unwrap_or_else(|| argument_types.get_for_declared_type(declared_type)); + + Some(ArgumentRelation::new( + argument_index, + adjusted_argument_index, + parameter, + matched_parameter, + argument_type, + )) + }) + }, + ) + } + /// Returns argument-index mappings for arguments matched to the `ParamSpec` component. /// /// `prefix_len` is the number of parameters before the `ParamSpec` components in a callable like @@ -5919,28 +6001,12 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { }; let inference = match builder.build_inference_with(generic_context, &mut choose) { Ok(inference) => inference, - Err(()) => { - let parameters = self.signature.parameters(); - let mut argument_relations = Vec::new(); - for (argument_index, _, _, argument_types) in self.enumerate_argument_types() { - for matched_parameter in self.argument_matches[argument_index].iter() { - let parameter_index = matched_parameter.index; - if self.is_gradual_variadic_parameter(parameter_index) { - continue; - } - - let formal = matched_parameter - .expected_type - .unwrap_or_else(|| parameters[parameter_index].annotated_type()); - let actual = matched_parameter - .argument_type - .unwrap_or_else(|| argument_types.get_for_declared_type(formal)); - argument_relations.push((formal, actual)); - } - } - - builder.build_diagnostic_inference_with(generic_context, argument_relations, choose) - } + Err(()) => builder.build_diagnostic_inference_with( + generic_context, + self.argument_relations() + .map(|relation| (relation.declared_type, relation.argument_type)), + choose, + ), }; let specialization = inference.specialization(db); @@ -5956,50 +6022,32 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { specialization_errors: &mut Vec>, ) -> bool { let db = self.db; - let parameters = self.signature.parameters(); - for (argument_index, adjusted_argument_index, _, argument_types) in - self.enumerate_argument_types() - { - for matched_parameter in self.argument_matches[argument_index].iter() { - let parameter_index = matched_parameter.index; - let parameter = ¶meters[parameter_index]; - let parameter_type = parameter.annotated_type(); - // TODO: Infer a `TypeVarTuple` from all matched positional arguments as a single - // tuple. Fixed elements beside that pack can still infer ordinary type variables. - if parameter.has_starred_annotation() - && matched_parameter.expected_type.is_none() - && (matches!( - parameter_type, - Type::TypeVar(typevar) if typevar.is_typevartuple(db) - ) || matches!( - parameter_type.exact_tuple_instance_spec(db).as_deref(), - Some(TupleSpec::Variable(variable)) - if matches!( - variable.variable(), - VariableSegment::TypeVarTuple(_) - ) - )) - { - continue; - } - if self.is_gradual_variadic_parameter(parameter_index) { - continue; - } - - let declared_type = matched_parameter.expected_type.unwrap_or(parameter_type); - let argument_type = argument_types.get_for_declared_type(declared_type); - let specialization_result = builder.infer( - declared_type, - matched_parameter.argument_type.unwrap_or(argument_type), - ); + for relation in self.argument_relations() { + // TODO: Infer a `TypeVarTuple` from all matched positional arguments as a single + // tuple. Fixed elements beside that pack can still infer ordinary type variables. + if relation.has_starred_annotation + && relation.matched_parameter.expected_type.is_none() + && (matches!( + relation.declared_type, + Type::TypeVar(typevar) if typevar.is_typevartuple(db) + ) || matches!( + relation.declared_type.exact_tuple_instance_spec(db).as_deref(), + Some(TupleSpec::Variable(variable)) + if matches!( + variable.variable(), + VariableSegment::TypeVarTuple(_) + ) + )) + { + continue; + } - if let Err(error) = specialization_result { - self.constraint_set_errors[argument_index] = true; - specialization_errors.push(BindingError::SpecializationError { - error, - argument_index: adjusted_argument_index, - }); - } + if let Err(error) = builder.infer(relation.declared_type, relation.argument_type) { + self.constraint_set_errors[relation.argument_index] = true; + specialization_errors.push(BindingError::SpecializationError { + error, + argument_index: relation.adjusted_argument_index, + }); } } @@ -6014,17 +6062,22 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { fn check_argument_type( &mut self, constraints: &ConstraintSetBuilder<'db>, - argument_index: usize, - adjusted_argument_index: Option, argument: Argument<'a>, - mut argument_type: Type<'db>, - matched_parameter: MatchedParameter<'db>, + relation: ArgumentRelation<'db>, ) { + let ArgumentRelation { + argument_index, + adjusted_argument_index, + matched_parameter, + declared_type, + mut argument_type, + has_starred_annotation, + } = relation; let db = self.db; let parameter_index = matched_parameter.index; let parameters = self.signature.parameters(); let parameter = ¶meters[parameter_index]; - if self.is_gradual_variadic_parameter(parameter_index) { + if Self::is_gradual_variadic_parameter(parameters, parameter_index) { return; } @@ -6068,9 +6121,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { Type::SubclassOf(subclass_of) if subclass_of.into_type_var().is_some() ); - let mut expected_ty = matched_parameter - .expected_type - .unwrap_or_else(|| parameter.annotated_type()); + let mut expected_ty = declared_type; if let Some(specialization) = self.specialization() { if !constructor_receiver { argument_type = argument_type.apply_specialization(db, specialization); @@ -6110,7 +6161,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { // An unresolved `*Ts` still has no per-element expected type. if !self.constraint_set_errors[argument_index] && !constructor_receiver - && (!parameter.has_starred_annotation() || matched_parameter.expected_type.is_some()) + && (!has_starred_annotation || matched_parameter.expected_type.is_some()) && !is_valid_isinstance_target() && argument_type .when_assignable_to( @@ -6171,8 +6222,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { } } - fn is_gradual_variadic_parameter(&self, parameter_index: usize) -> bool { - let parameters = self.signature.parameters(); + fn is_gradual_variadic_parameter(parameters: &Parameters<'db>, parameter_index: usize) -> bool { let parameter = ¶meters[parameter_index]; matches!(parameters.kind(), ParametersKind::Gradual) @@ -6325,18 +6375,18 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { continue; } - let declared_type = - self.signature.parameters()[parameter_index].annotated_type(); - let argument_type = argument_types.get_for_declared_type(declared_type); - - self.check_argument_type( - constraints, + let parameter = &self.signature.parameters()[parameter_index]; + let argument_type = + argument_types.get_for_declared_type(parameter.annotated_type()); + let relation = ArgumentRelation::new( argument_index, adjusted_argument_index, - argument, - argument_type, + parameter, matched_parameter, + argument_type, ); + + self.check_argument_type(constraints, argument, relation); } } } @@ -6506,16 +6556,17 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { continue; } - self.check_argument_type( - constraints, + let relation = ArgumentRelation::new( argument_index, adjusted_argument_index, - argument, + &self.signature.parameters()[parameter_index], + matched_parameter, matched_parameter .argument_type .unwrap_or_else(Type::unknown), - matched_parameter, ); + + self.check_argument_type(constraints, argument, relation); } } @@ -6580,14 +6631,15 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { .unwrap_or(Type::unknown()) }; - self.check_argument_type( - constraints, + let relation = ArgumentRelation::new( argument_index, adjusted_argument_index, - Argument::Keywords, - value_type, + &self.signature.parameters()[parameter_index], matched_parameter, + value_type, ); + + self.check_argument_type(constraints, Argument::Keywords, relation); } } From b8c5e73abe5b15a74fb066e474d30397d1421cfe Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Wed, 12 Aug 2026 17:44:59 -0700 Subject: [PATCH 014/371] [ty] Disable CodSpeed cycle estimation for instrumented benchmarks (#27706) ## Summary We've been seeing +/-5% false positives on many PRs all day today on one specific benchmark, `ty_micro[typevar_mapping_accumulation]`, across unrelated PRs that wouldn't plausibly impact that benchmark. The start of these false positives lines up very well with the rollout of CodSpeed v5 and its switch to "cycle estimation" rather than simple instruction counting by default. (Across 43 jobs before the CodSpeed v5 upgrade, we saw zero significant changes in this benchmark.) By examining all the various PRs on which this false positives has showed up, I ruled out other potential causes: - The false positive occurs on PRs where the benchmark ran v5 vs v5, so it's not an issue of mismatched CodSpeed version. - The false positive occurs on PRs where GitHub runner spec was identical from baseline to PR, so it's not a GitHub runner mismatch issue. This PR sets an option to disable the new cycle estimation mode, while staying on CodSpeed v5. This is easily reversible and seems like the most obvious thing to try to eliminate these false positives. - Disable CodSpeed v5's default cycle estimation for instrumented ty benchmarks while keeping the v5 action. - Leave Ruff benchmarks and wall-time benchmark jobs unchanged. The first comparison across this configuration change can be marked incomparable because its base and head use different runner settings. Once the updated workflow runs on `main`, subsequent runs will use the new matching baseline. ## Test plan - Existing instrumented ty simulation and memory benchmark matrix exercises the updated action configuration in CI. - Workflow validation, formatting, security, and typo hooks pass for the changed workflow. --- .github/workflows/ci.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 0ae975fe8a..4cf93d74f4 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -1206,6 +1206,7 @@ jobs: uses: CodSpeedHQ/action@0ca9cbbf4623b599a6c3ed4fc8a922942705d9f1 # v5.0.2 with: mode: ${{ matrix.mode }} + cycle-estimation: false run: cargo codspeed run --bench "${{ matrix.target }}" "${{ matrix.filter }}" benchmarks-walltime-build: From 59196baedf23c9876d1fcf1fa2ae78f80d306f94 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Wed, 12 Aug 2026 22:26:14 -0700 Subject: [PATCH 015/371] [ty] Unify polarity-aware relation construction (#27707) ## Summary Part of astral-sh/ty#3557 and the second step in rebooting #26712 as smaller focused changes. On `main`, generic inference does not consistently propagate the variance of an enclosing generic class into structural-protocol and callable comparisons. Those comparisons instead construct the forward `actual <= formal` relation even when the surrounding position is contravariant or invariant. This produces incorrect bounds and inferred types: ```py from typing import Callable class Middle: ... class Derived(Middle): ... class Sink[T]: def put(self, value: T) -> None: ... def infer[T](sink: Sink[Callable[[], T]], value: T) -> T: return value reveal_type(infer(Sink[Callable[[], Middle]](), Derived())) # main: Middle # this PR: Derived ``` This PR centralizes relation construction so inference consistently follows the surrounding polarity: covariance requires `actual <= formal`, contravariance requires `formal <= actual`, invariance requires both, and bivariance adds no constraints. The fix applies to nested generic protocols, callable signatures, callback protocols, TypedDict/protocol relations, and materialized protocols while preserving existing nominal inference, callable-union alternatives, TypedDict-union optimizations, and cycle-safe handling of recursive protocols. ## Limitation The new constraint solver does not yet support `ParamSpec` or `TypeVarTuple`. Whenever an inference context includes either variadic, all affected relations must continue through the old, forward-only inference path, including comparisons involving ordinary type variables alongside an unrelated variadic. This deliberately avoids regressions in parameter inference, structural protocols, descriptor signatures, and downstream argument diagnostics, but it also means the polarity fixes in this PR do not apply whenever variadics are involved. Focused TODO tests document cases that still incorrectly infer `Middle` instead of `Derived`; fixing those cases requires adding variadic support to the new solver. ## Test plan Added mdtests cover nested nominal classes, generic protocols, callable signatures, and callback protocols under covariance, contravariance, and invariance; double contravariance, callable-union alternatives, and invariant rejection; finite, recursive, and recursive-only materialized protocols; invariant, contravariant, and covariant `ParamSpec` inference through `Concatenate`; bounded and constrained prefixes, descriptor receivers, structural protocol unions, nominal callable objects, higher-order callbacks, and mixed ordinary/variadic inference; analogous `TypeVarTuple` callable, protocol, structural-union, and nominal-implementation cases; and explicit TODO coverage for unrelated variadics suppressing otherwise-correct polarity. Additional mdtests cover repeated comparisons of the same protocol or callable under opposite polarities, plus overloaded callable acceptance and rejection across covariant, contravariant, and invariant wrappers. --- .../resources/mdtest/descriptor_protocol.md | 23 + .../mdtest/generics/legacy/paramspec.md | 166 +++++++ .../mdtest/generics/pep695/functions.md | 433 ++++++++++++++++++ .../mdtest/generics/pep695/typevartuple.md | 99 ++++ .../mdtest/type_properties/materialization.md | 59 +++ .../ty_python_semantic/src/types/generics.rs | 210 ++++++--- 6 files changed, 919 insertions(+), 71 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md b/crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md index bb1ec3de05..82614f9966 100644 --- a/crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md +++ b/crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md @@ -1864,6 +1864,29 @@ class Example: pass ``` +An invalid descriptor receiver must not discard the inferred `ParamSpec` for its bound callable. +Even though `Concatenate` makes the receiver positional-only, the remaining parameters still retain +their precise types. + +```py +class Decorator(Generic[P]): + def __call__(self, *args: P.args, **kwargs: P.kwargs) -> None: ... + def __get__(self: "Decorator[Concatenate[Any, P2]]", instance: Any, owner: Any) -> "Decorator[P2]": + raise NotImplementedError + +def decorate(fn: Callable[P, Any]) -> Decorator[P]: + raise NotImplementedError + +class Decorated: + @decorate + def method(self, value: str) -> None: ... + +# error: [invalid-attribute-access] +bound = Decorated().method +reveal_type(bound) # revealed: Decorator[(value: str)] +bound(1) # error: [invalid-argument-type] +``` + [descriptors]: https://docs.python.org/3/howto/descriptor.html [precedence chain]: https://github.com/python/cpython/blob/3.13/Objects/typeobject.c#L5393-L5481 [simple example]: https://docs.python.org/3/howto/descriptor.html#simple-example-a-descriptor-that-returns-a-constant diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/paramspec.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/paramspec.md index 769951e064..846429959c 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/paramspec.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/paramspec.md @@ -746,6 +746,172 @@ takes_int_job(defaulted_job) takes_int_job(wrong_job) # error: [invalid-argument-type] ``` +## Inferring an invariant `ParamSpec` through `Concatenate` + +A `Concatenate` prefix is positional-only, so a callback whose first parameter also accepts a +keyword is not compatible with an invariant wrapper. Even though that argument is rejected, its +remaining parameters must still be inferred precisely. + +```py +from typing import Callable, Concatenate, Generic, ParamSpec + +P = ParamSpec("P") + +class Callback(Generic[P]): + def __init__(self, callback: Callable[P, None]) -> None: ... + +def without_first(callback: Callback[Concatenate[object, P]]) -> Callable[P, None]: + raise NotImplementedError + +def original(first: object, value: str) -> None: ... + +remaining = without_first(Callback(original)) # error: [invalid-argument-type] +reveal_type(remaining) # revealed: (value: str) -> None +remaining(1) # error: [invalid-argument-type] +``` + +## Inferring a contravariant `ParamSpec` through `Concatenate` + +The same callback is compatible with a contravariant wrapper, and its remaining parameters must +still be inferred precisely enough to reject an incompatible later argument. + +```py +from typing import Callable, Concatenate, Generic, ParamSpec, TypeVar + +P = ParamSpec("P", contravariant=True) + +class Callback(Generic[P]): + def __init__(self, callback: Callable[P, None]) -> None: ... + +def without_first(callback: Callback[Concatenate[object, P]]) -> Callable[P, None]: + raise NotImplementedError + +def original(first: object, value: str) -> None: ... + +remaining = without_first(Callback(original)) +reveal_type(remaining) # revealed: (value: str) -> None +remaining(1) # error: [invalid-argument-type] +``` + +A contravariant callback can accept a broader positional-only prefix than a bounded type variable +requires. The separate `Middle()` argument determines the narrower specialization without rejecting +the valid callback. + +```py +class Base: ... +class Middle(Base): ... +class Other: ... + +Bounded = TypeVar("Bounded", bound=Middle) +Q = ParamSpec("Q") + +def accepts_base(first: Base, /, value: str) -> None: ... +def bounded(callback: Callback[Concatenate[Bounded, Q]], witness: Bounded) -> tuple[Bounded, Callable[Q, None]]: + raise NotImplementedError + +bounded_result = bounded(Callback(accepts_base), Middle()) +reveal_type(bounded_result) # revealed: tuple[Middle, (value: str) -> None] +bounded_result[1](1) # error: [invalid-argument-type] +``` + +The same contravariant relationship remains valid when the prefix type variable has explicit +constraints instead of an upper bound. + +```py +Constrained = TypeVar("Constrained", Middle, Other) + +def constrained(callback: Callback[Concatenate[Constrained, Q]], witness: Constrained) -> tuple[Constrained, Callable[Q, None]]: + raise NotImplementedError + +constrained_result = constrained(Callback(accepts_base), Middle()) +reveal_type(constrained_result) # revealed: tuple[Middle, (value: str) -> None] +constrained_result[1](1) # error: [invalid-argument-type] +``` + +## Inferring a covariant `ParamSpec` through `Concatenate` + +A covariant wrapper containing a callback with a narrower positional-only prefix remains valid, +whether the wrapper is stored first or constructed inline. + +```py +from typing import Callable, Concatenate, Generic, ParamSpec + +P = ParamSpec("P", covariant=True) +Q = ParamSpec("Q") + +class Base: ... +class Middle(Base): ... + +class Callback(Generic[P]): + def __init__(self, callback: Callable[P, None]) -> None: ... + +def without_first(callback: Callback[Concatenate[Base, Q]]) -> Callable[Q, None]: + raise NotImplementedError + +def original(first: Middle, /, value: str) -> None: ... + +wrapped = Callback(original) +# TODO: Should reveal `(value: str) -> None`. Needs ParamSpecs in the new constraint solver. +reveal_type(without_first(wrapped)) # revealed: (...) -> None +# TODO: Should reveal `(value: str) -> None`. Needs ParamSpecs in the new constraint solver. +reveal_type(without_first(Callback(original))) # revealed: (...) -> None +``` + +## Inferring through unions of structural `ParamSpec` protocols + +Different protocols with the same parameter list can satisfy a target protocol structurally, even +when they appear in a union nested inside an invariant or contravariant wrapper. + +```py +from typing import Generic, ParamSpec, Protocol, TypeVar + +P = ParamSpec("P") +T = TypeVar("T") +TContra = TypeVar("TContra", contravariant=True) + +class Invariant(Generic[T]): + value: T + +class Contravariant(Generic[TContra]): + def put(self, value: TContra) -> None: ... + +class Target(Protocol[P]): + def call(self, *args: P.args, **kwargs: P.kwargs) -> None: ... + +class Actual(Protocol[P]): + def call(self, *args: P.args, **kwargs: P.kwargs) -> None: ... + +class Other(Protocol[P]): + def call(self, *args: P.args, **kwargs: P.kwargs) -> None: ... + +def invariant(value: Invariant[Target[P]]) -> Target[P]: + raise NotImplementedError + +def contravariant(value: Contravariant[Target[P]]) -> Target[P]: + raise NotImplementedError + +def compatible( + first: Invariant[Actual[[str]] | Other[[str]]], + second: Contravariant[Actual[[str]] | Other[[str]]], +) -> None: + reveal_type(invariant(first)) # revealed: Target[(str, /)] + reveal_type(contravariant(second)) # revealed: Target[(str, /)] +``` + +Union members with incompatible parameter lists cannot satisfy either wrapper. Their signatures must +remain visible in the inferred result instead of collapsing to a gradual parameter list. + +```py +def incompatible( + first: Invariant[Actual[[str]] | Other[[bytes]]], + second: Contravariant[Actual[[str]] | Other[[bytes]]], +) -> None: + # error: [invalid-argument-type] + reveal_type(invariant(first)) # revealed: Target[((str, /)) | ((bytes, /))] + # error: [invalid-argument-type] + reveal_type(contravariant(second)) # revealed: Target[((str, /)) | ((bytes, /))] +``` + ## `ParamSpec` cannot specialize a `TypeVar`, and vice versa diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md index 9455c740bf..a9d39cc2f5 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md @@ -186,6 +186,439 @@ def _(a: A, b: B, x: A | B): reveal_type(takes_in_supports_foo(x)) # revealed: A | B ``` +## Inferring through nested nominal generic classes + +When a nominal generic class is nested inside another, the outer class determines whether the inner +specialization contributes a lower bound, an upper bound, or both. + +```py +class Covariant[T]: + def get(self) -> T: + raise NotImplementedError + +class Contravariant[T]: + def put(self, value: T) -> None: ... + +class Invariant[T]: + value: T + +class Producer[T]: + def get(self) -> T: + raise NotImplementedError + +def covariant[T](container: Covariant[Producer[T]], value: T) -> T: + return value + +def contravariant[T](container: Contravariant[Producer[T]], value: T) -> T: + return value + +def invariant[T](container: Invariant[Producer[T]], value: T) -> T: + return value +``` + +Covariance permits the broader `Base`, contravariance accepts the narrower `Derived`, and invariance +preserves `Middle`. + +```py +class Base: ... +class Middle(Base): ... +class Derived(Middle): ... + +reveal_type(covariant(Covariant[Producer[Middle]](), Base())) # revealed: Base +reveal_type(contravariant(Contravariant[Producer[Middle]](), Derived())) # revealed: Derived +reveal_type(invariant(Invariant[Producer[Middle]](), Middle())) # revealed: Middle +``` + +## Inferring through nested generic protocols + +Generic protocols must preserve the variance of the outer class when constructing their structural +constraints, just as nominal generic classes do. + +```py +from typing import Protocol + +class Covariant[T]: + def get(self) -> T: + raise NotImplementedError + +class Contravariant[T]: + def put(self, value: T) -> None: ... + +class Invariant[T]: + value: T + +class Producer[T](Protocol): + def get(self) -> T: ... + +def covariant[T](container: Covariant[Producer[T]], value: T) -> T: + return value + +def contravariant[T](container: Contravariant[Producer[T]], value: T) -> T: + return value + +def invariant[T](container: Invariant[Producer[T]], value: T) -> T: + return value +``` + +A covariant outer class permits the broader `Base`, a contravariant class accepts the narrower +`Derived`, and an invariant class requires exactly `Middle`. + +```py +class Base: ... +class Middle(Base): ... +class Derived(Middle): ... + +reveal_type(covariant(Covariant[Producer[Middle]](), Base())) # revealed: Base +reveal_type(contravariant(Contravariant[Producer[Middle]](), Derived())) # revealed: Derived +reveal_type(invariant(Invariant[Producer[Middle]](), Middle())) # revealed: Middle +invariant(Invariant[Producer[Middle]](), Base()) # error: [invalid-argument-type] +``` + +When the same protocol specialization appears first contravariantly and then covariantly, both +relationships contribute their constraints even though the formal and actual types are identical. + +```py +class MixedVariance[First, Second]: + def put(self, value: First) -> None: ... + def get(self) -> Second: + raise NotImplementedError + +def repeated_polarity[T](container: MixedVariance[Producer[T], Producer[T]], witness: T) -> T: + return witness + +reveal_type(repeated_polarity(MixedVariance[Producer[Middle], Producer[Middle]](), Derived())) # revealed: Middle +``` + +A consuming protocol reverses the relationship once more. Nesting that protocol inside a +contravariant class therefore turns its upper bound back into a lower bound. + +```py +class Consumer[T](Protocol): + def put(self, value: T) -> None: ... + +def consumer[T](container: Covariant[Consumer[T]], value: T) -> T: + return value + +def double_contravariant[T](container: Contravariant[Consumer[T]], value: T) -> T: + return value + +reveal_type(consumer(Covariant[Consumer[Middle]](), Derived())) # revealed: Derived +reveal_type(double_contravariant(Contravariant[Consumer[Middle]](), Derived())) # revealed: Middle +``` + +A structural protocol method must also preserve its inferred parameters under either outer variance, +even when its nominal implementation is rejected by the wrapper. + +```py +from typing import Callable + +class Runner[**P](Protocol): + def run(self, *args: P.args, **kwargs: P.kwargs) -> None: ... + +class StringRunner: + def run(self, value: str) -> None: ... + +def invariant_runner[**P](container: Invariant[Runner[P]]) -> Callable[P, None]: + raise NotImplementedError + +def contravariant_runner[**P](container: Contravariant[Runner[P]]) -> Callable[P, None]: + raise NotImplementedError + +invariant_run = invariant_runner(Invariant[StringRunner]()) # error: [invalid-argument-type] +reveal_type(invariant_run) # revealed: (value: str) -> None +invariant_run(1) # error: [invalid-argument-type] + +contravariant_run = contravariant_runner(Contravariant[StringRunner]()) # error: [invalid-argument-type] +reveal_type(contravariant_run) # revealed: (value: str) -> None +contravariant_run(1) # error: [invalid-argument-type] +``` + +## Inferring through nested generic callables + +Callable return types are covariant, but nesting a callable inside a contravariant or invariant +generic class changes which constraints its return type supplies. + +```py +from typing import Callable, overload + +class Covariant[T]: + def __init__(self, *values: T) -> None: ... + def get(self) -> T: + raise NotImplementedError + +class Contravariant[T]: + def __init__(self, *values: T) -> None: ... + def put(self, value: T) -> None: ... + +class Invariant[T]: + def __init__(self, *values: T) -> None: ... + value: T + +def covariant[T](container: Covariant[Callable[[], T]], value: T) -> T: + return value + +def contravariant[T](container: Contravariant[Callable[[], T]], value: T) -> T: + return value + +def invariant[T](container: Invariant[Callable[[], T]], value: T) -> T: + return value + +class Base: ... +class Middle(Base): ... +class Derived(Middle): ... + +reveal_type(covariant(Covariant[Callable[[], Middle]](), Base())) # revealed: Base +reveal_type(contravariant(Contravariant[Callable[[], Middle]](), Derived())) # revealed: Derived +reveal_type(invariant(Invariant[Callable[[], Middle]](), Middle())) # revealed: Middle +invariant(Invariant[Callable[[], Middle]](), Base()) # error: [invalid-argument-type] +``` + +The same callable specialization can contribute both an upper and a lower bound when it appears +first in a contravariant position and then in a covariant position. + +```py +class MixedVariance[First, Second]: + def put(self, value: First) -> None: ... + def get(self) -> Second: + raise NotImplementedError + +def repeated_polarity[T](container: MixedVariance[Callable[[], T], Callable[[], T]], witness: T) -> T: + return witness + +reveal_type(repeated_polarity(MixedVariance[Callable[[], Middle], Callable[[], Middle]](), Derived())) # revealed: Middle +``` + +An unrelated variadic type parameter currently sends the entire inference context through the legacy +solver, so the ordinary callable loses the contravariant bound shown above. + +```py +def with_paramspec[T, **P](container: Contravariant[Callable[[], T]], value: T, unrelated: Callable[P, None]) -> T: + return value + +def with_typevartuple[T, *Ts](container: Contravariant[Callable[[], T]], value: T, unrelated: tuple[*Ts]) -> T: + return value + +def unrelated(value: str) -> None: ... + +# TODO: Should reveal `Derived` when an unrelated ParamSpec no longer disables contravariance. +reveal_type(with_paramspec(Contravariant[Callable[[], Middle]](), Derived(), unrelated)) # revealed: Middle +# TODO: Should reveal `Derived` when an unrelated TypeVarTuple no longer disables contravariance. +reveal_type(with_typevartuple(Contravariant[Callable[[], Middle]](), Derived(), ("value",))) # revealed: Middle +``` + +A union of callable return types offers alternative upper bounds; a compatible arm must not be +rejected just because another arm is incompatible. + +```py +reveal_type(contravariant(Contravariant[Callable[[], Middle] | Callable[[], str]](), Derived())) # revealed: Derived +``` + +Covariance accepts an overloaded callable when one overload matches. Contravariance and invariance +additionally require the formal callable to cover every overload, so an extra `str` overload is +incompatible with a callable accepting only `int`. + +```py +@overload +def overloaded(value: int, /) -> Middle: ... +@overload +def overloaded(value: str, /) -> Middle: ... +def overloaded(value: int | str, /) -> Middle: + raise NotImplementedError + +def covariant_overload[T](container: Covariant[Callable[[int], T]]) -> T: + raise NotImplementedError + +def contravariant_overload[T](container: Contravariant[Callable[[int], T]]) -> T: + raise NotImplementedError + +def invariant_overload[T](container: Invariant[Callable[[int], T]]) -> T: + raise NotImplementedError + +reveal_type(covariant_overload(Covariant(overloaded))) # revealed: Middle +contravariant_overload(Contravariant(overloaded)) # error: [invalid-argument-type] +invariant_overload(Invariant(overloaded)) # error: [invalid-argument-type] +``` + +When every overload is covered by the formal `int` parameter, all three wrapper variances accept it. + +```py +@overload +def covered(value: bool, /) -> Middle: ... +@overload +def covered(value: int, /) -> Middle: ... +def covered(value: int, /) -> Middle: + raise NotImplementedError + +reveal_type(covariant_overload(Covariant(covered))) # revealed: Middle +reveal_type(contravariant_overload(Contravariant(covered))) # revealed: Middle +reveal_type(invariant_overload(Invariant(covered))) # revealed: Middle +``` + +Callable parameter types are already contravariant. An outer contravariant class reverses their +relationship a second time, producing the same lower bound as a covariant callable return type. + +```py +def consumer[T](container: Covariant[Callable[[T], None]], value: T) -> T: + return value + +def double_contravariant[T](container: Contravariant[Callable[[T], None]], value: T) -> T: + return value + +reveal_type(consumer(Covariant[Callable[[Middle], None]](), Derived())) # revealed: Derived +reveal_type(double_contravariant(Contravariant[Callable[[Middle], None]](), Derived())) # revealed: Middle +``` + +A `Concatenate` prefix is positional-only, so these wrapped callbacks do not match the formal +parameter exactly. Their remaining parameters and ordinary return type variable must still be +inferred precisely under either outer polarity. + +```py +from typing import Concatenate + +class InvariantCallback[T]: + def __init__(self, callback: T) -> None: ... + callback: T + +class ContravariantCallback[T]: + def __init__(self, callback: T) -> None: ... + def put(self, callback: T) -> None: ... + +def invariant_tail[**P, R]( + container: InvariantCallback[Callable[Concatenate[object, P], R]], +) -> Callable[P, R]: + raise NotImplementedError + +def contravariant_tail[**P, R]( + container: ContravariantCallback[Callable[Concatenate[object, P], R]], +) -> Callable[P, R]: + raise NotImplementedError + +def original(first: object, value: str) -> int: + return len(value) + +invariant_remaining = invariant_tail(InvariantCallback(original)) # error: [invalid-argument-type] +reveal_type(invariant_remaining) # revealed: (value: str) -> int +invariant_remaining(1) # error: [invalid-argument-type] +invariant_remaining("valid").missing_attribute # error: [unresolved-attribute] + +contravariant_remaining = contravariant_tail(ContravariantCallback(original)) # error: [invalid-argument-type] +reveal_type(contravariant_remaining) # revealed: (value: str) -> int +contravariant_remaining(1) # error: [invalid-argument-type] +contravariant_remaining("valid").missing_attribute # error: [unresolved-attribute] +``` + +A higher-order callback must retain its inferred parameter list under both outer variances, even +when assigning the result causes the outer argument to be rejected. Its callback parameter can +accept either the declared prefix or a narrower derived prefix. + +```py +def accepts_exact(callback: Callable[[Base, str], None]) -> None: ... +def accepts_narrower(callback: Callable[[Derived, str], None]) -> None: ... +def invariant_higher_order[**P]( + container: InvariantCallback[Callable[[Callable[Concatenate[Base, P], None]], None]], +) -> Callable[P, None]: + raise NotImplementedError + +def contravariant_higher_order[**P]( + container: ContravariantCallback[Callable[[Callable[Concatenate[Base, P], None]], None]], +) -> Callable[P, None]: + raise NotImplementedError + +invariant_exact = invariant_higher_order(InvariantCallback(accepts_exact)) # error: [invalid-argument-type] +reveal_type(invariant_exact) # revealed: (str, /) -> None +invariant_exact(1) # error: [invalid-argument-type] + +invariant_narrower = invariant_higher_order(InvariantCallback(accepts_narrower)) # error: [invalid-argument-type] +reveal_type(invariant_narrower) # revealed: (str, /) -> None + +contravariant_exact = contravariant_higher_order(ContravariantCallback(accepts_exact)) # error: [invalid-argument-type] +reveal_type(contravariant_exact) # revealed: (str, /) -> None +contravariant_exact(1) # error: [invalid-argument-type] + +contravariant_narrower = contravariant_higher_order(ContravariantCallback(accepts_narrower)) # error: [invalid-argument-type] +reveal_type(contravariant_narrower) # revealed: (str, /) -> None +``` + +## Inferring through nested callable protocols + +A callable assigned to a callback protocol contributes the same return-type constraints through its +signature, including when an outer generic class reverses their direction. + +```py +from typing import Callable, Protocol + +class Covariant[T]: + def get(self) -> T: + raise NotImplementedError + +class Contravariant[T]: + def put(self, value: T) -> None: ... + +class Callback[T](Protocol): + def __call__(self) -> T: ... + +def covariant[T](container: Covariant[Callback[T]], value: T) -> T: + return value + +def contravariant[T](container: Contravariant[Callback[T]], value: T) -> T: + return value + +class Base: ... +class Middle(Base): ... +class Derived(Middle): ... + +reveal_type(covariant(Covariant[Callable[[], Middle]](), Base())) # revealed: Base +reveal_type(contravariant(Contravariant[Callable[[], Middle]](), Derived())) # revealed: Derived +``` + +A callback protocol with a positional-only prefix must likewise preserve its inferred parameter +tail, even when the wrapped callable is rejected. + +```py +class InvariantCallback[T]: + def __init__(self, callback: T) -> None: ... + callback: T + +class ContravariantCallback[T]: + def __init__(self, callback: T) -> None: ... + def put(self, callback: T) -> None: ... + +class VariadicCallback[**P](Protocol): + def __call__(self, first: object, /, *args: P.args, **kwargs: P.kwargs) -> None: ... + +def invariant_tail[**P](container: InvariantCallback[VariadicCallback[P]]) -> Callable[P, None]: + raise NotImplementedError + +def contravariant_tail[**P](container: ContravariantCallback[VariadicCallback[P]]) -> Callable[P, None]: + raise NotImplementedError + +def original(first: object, value: str) -> None: ... + +invariant_remaining = invariant_tail(InvariantCallback(original)) # error: [invalid-argument-type] +reveal_type(invariant_remaining) # revealed: (value: str) -> None +invariant_remaining(1) # error: [invalid-argument-type] + +contravariant_remaining = contravariant_tail(ContravariantCallback(original)) # error: [invalid-argument-type] +reveal_type(contravariant_remaining) # revealed: (value: str) -> None +contravariant_remaining(1) # error: [invalid-argument-type] +``` + +A nominal callable object's `__call__` method must likewise preserve the callback protocol's +inferred parameters under both wrapper variances. + +```py +class CallableObject: + def __call__(self, first: object, value: str) -> None: ... + +invariant_object = invariant_tail(InvariantCallback(CallableObject())) # error: [invalid-argument-type] +reveal_type(invariant_object) # revealed: (value: str) -> None +invariant_object(1) # error: [invalid-argument-type] + +contravariant_object = contravariant_tail(ContravariantCallback(CallableObject())) # error: [invalid-argument-type] +reveal_type(contravariant_object) # revealed: (value: str) -> None +contravariant_object(1) # error: [invalid-argument-type] +``` + ## Bound violations inferred through protocols If matching a protocol argument infers a type that violates a type variable's bound, the call should diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md index dea784f91f..073cb54187 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md @@ -392,6 +392,105 @@ reveal_type(simple(variadic2)) # revealed: tuple[Unknown, ...] reveal_type(simple(keyword_only)) # revealed: tuple[Unknown, ...] ``` +### Callable inference through invariant and contravariant wrappers + +An unpacked `TypeVarTuple` keeps its precise inferred parameter types when a callable or callable +protocol is nested inside an invariant or contravariant wrapper. + +```py +from typing import Callable, Protocol + +class Invariant[T]: + def __init__(self, callback: T) -> None: ... + callback: T + +class Contravariant[T]: + def __init__(self, callback: T) -> None: ... + def put(self, callback: T) -> None: ... + +def invariant[*Ts](wrapper: Invariant[Callable[[*Ts], None]]) -> tuple[*Ts]: + raise NotImplementedError + +def contravariant[*Ts](wrapper: Contravariant[Callable[[*Ts], None]]) -> tuple[*Ts]: + raise NotImplementedError + +def callback(first: object, value: str) -> None: ... + +reveal_type(invariant(Invariant(callback))) # revealed: tuple[object, str] +reveal_type(contravariant(Contravariant(callback))) # revealed: tuple[object, str] +``` + +A callable protocol preserves the same inferred parameters through both wrapper variances. + +```py +class Callback[*Ts](Protocol): + def __call__(self, *args: *Ts) -> None: ... + +def invariant_protocol[*Ts](wrapper: Invariant[Callback[*Ts]]) -> tuple[*Ts]: + raise NotImplementedError + +def contravariant_protocol[*Ts](wrapper: Contravariant[Callback[*Ts]]) -> tuple[*Ts]: + raise NotImplementedError + +reveal_type(invariant_protocol(Invariant(callback))) # revealed: tuple[object, str] +reveal_type(contravariant_protocol(Contravariant(callback))) # revealed: tuple[object, str] +``` + +Separately declared protocols with equivalent variadic methods also preserve the exact inferred +tuple under both wrapper variances. + +```py +class Target[*Ts](Protocol): + def call(self, *args: *Ts) -> None: ... + +class Actual[*Ts](Protocol): + def call(self, *args: *Ts) -> None: ... + +def invariant_structural[*Ts](wrapper: Invariant[Target[*Ts]]) -> tuple[*Ts]: + raise NotImplementedError + +def contravariant_structural[*Ts](wrapper: Contravariant[Target[*Ts]]) -> tuple[*Ts]: + raise NotImplementedError + +def check_structural( + invariant_wrapper: Invariant[Actual[str]], + contravariant_wrapper: Contravariant[Actual[str]], +) -> None: + reveal_type(invariant_structural(invariant_wrapper)) # revealed: tuple[str] + reveal_type(contravariant_structural(contravariant_wrapper)) # revealed: tuple[str] +``` + +Unions of structurally compatible protocols retain the same tuple. Incompatible alternatives are +rejected without widening their inferred tuple to an unknown-length tuple. + +```py +class Other[*Ts](Protocol): + def call(self, *args: *Ts) -> None: ... + +def check_unions( + invariant_match: Invariant[Actual[str] | Other[str]], + contravariant_match: Contravariant[Actual[str] | Other[str]], + invariant_mismatch: Invariant[Actual[str] | Other[bytes]], + contravariant_mismatch: Contravariant[Actual[str] | Other[bytes]], +) -> None: + reveal_type(invariant_structural(invariant_match)) # revealed: tuple[str] + reveal_type(contravariant_structural(contravariant_match)) # revealed: tuple[str] + # error: [invalid-argument-type] + reveal_type(invariant_structural(invariant_mismatch)) # revealed: tuple[()] + # error: [invalid-argument-type] + reveal_type(contravariant_structural(contravariant_mismatch)) # revealed: tuple[()] +``` + +A nominal class implementing the same variadic protocol retains its precise method parameter. + +```py +class StringRunner: + def call(self, value: str) -> None: ... + +reveal_type(invariant_structural(Invariant(StringRunner()))) # revealed: tuple[str] +reveal_type(contravariant_structural(Contravariant(StringRunner()))) # revealed: tuple[str] +``` + ### Callable return inference An unpacked `TypeVarTuple` in a callable return type is inferred as one packed tuple, including diff --git a/crates/ty_python_semantic/resources/mdtest/type_properties/materialization.md b/crates/ty_python_semantic/resources/mdtest/type_properties/materialization.md index c291cf8c4f..c6f42bb4e8 100644 --- a/crates/ty_python_semantic/resources/mdtest/type_properties/materialization.md +++ b/crates/ty_python_semantic/resources/mdtest/type_properties/materialization.md @@ -2007,7 +2007,25 @@ def materialized_inference(inherited: Top[InheritedInferenceAny]) -> None: def materialized_structural_inference(structural: Top[StructuralInferenceAny]) -> None: reveal_type(infer_item(structural)) # revealed: object +``` + +A top-materialized structural protocol nested inside a contravariant class supplies an upper bound +without widening a narrower argument. + +```py +class Contravariant[T]: + def put(self, value: T) -> None: ... + +def infer_contravariant_item[T](container: Contravariant[InferenceBase[T]], value: T) -> T: + return value + +def nested_inference(container: Contravariant[Top[StructuralInferenceAny]], value: bool) -> None: + reveal_type(infer_contravariant_item(container, value)) # revealed: bool +``` + +Bounds and constraints still reject a materialized `object` property when its type is incompatible. +```py def bounded_item[T: str](value: InferenceBase[T]) -> T: raise NotImplementedError @@ -2156,6 +2174,47 @@ def recursive_materialized_inference( reveal_type(infer_recursive_value(bottom)) # revealed: Never ``` +A materialized recursive protocol nested inside a contravariant class contributes an upper bound +without expanding its recursive property. + +```py +class Contravariant[T]: + def put(self, value: T) -> None: ... + +def infer_contravariant_value[T](container: Contravariant[RecursiveValue[T]], value: T) -> T: + return value + +def nested_recursive_inference(container: Contravariant[Top[RecursiveAny]], value: bool) -> None: + reveal_type(infer_contravariant_value(container, value)) # revealed: bool +``` + +When a materialized protocol has no nonrecursive members, inference must defer to a separate +argument rather than expand its recursive requirement or reject the call. + +```py +class RecursiveOnlyTarget[T](Protocol): + @property + def child(self) -> RecursiveOnlyTarget[T]: ... + +class RecursiveOnlySource[T](Protocol): + @property + def child(self) -> RecursiveOnlySource[T]: ... + +def infer_recursive_only[T](value: RecursiveOnlyTarget[T], witness: T) -> T: + return witness + +def infer_contravariant_recursive_only[T](value: Contravariant[RecursiveOnlyTarget[T]], witness: T) -> T: + return witness + +def no_finite_recursive_members( + top: Top[RecursiveOnlySource[Any]], + contravariant: Contravariant[Top[RecursiveOnlySource[Any]]], + witness: bool, +) -> None: + reveal_type(infer_recursive_only(top, witness)) # revealed: bool + reveal_type(infer_contravariant_recursive_only(contravariant, witness)) # revealed: bool +``` + The nonrecursive property is used only to infer the specialization. The complete protocol must still be checked, so a matching `value` cannot hide an incompatible `child`. diff --git a/crates/ty_python_semantic/src/types/generics.rs b/crates/ty_python_semantic/src/types/generics.rs index aa048715a0..ffdd7c00b4 100644 --- a/crates/ty_python_semantic/src/types/generics.rs +++ b/crates/ty_python_semantic/src/types/generics.rs @@ -19,9 +19,7 @@ use crate::types::relation::{ DisjointnessChecker, HasRelationToVisitor, IsDisjointVisitor, TypeRelation, TypeRelationChecker, TypeVarEvaluation, }; -use crate::types::signatures::{ - CallableSignature, Parameters, ReturnCallableTypeVarScope, SignatureRelationVisitor, -}; +use crate::types::signatures::{Parameters, ReturnCallableTypeVarScope, SignatureRelationVisitor}; use crate::types::tuple::{ TupleSpec, TupleSpecBuilder, TupleType, VariableSegment, walk_tuple_type, }; @@ -2441,6 +2439,24 @@ enum ConstraintSetInferenceError<'db> { Unsatisfiable, } +/// Returns the directional comparisons required by this comparison's polarity. +/// +/// A covariant comparison requires `actual <= formal`, a contravariant comparison requires +/// `formal <= actual`, and an invariant comparison requires both. Bivariant comparisons add +/// no constraints. +fn relation_directions( + formal: T, + actual: T, + polarity: TypeVarVariance, +) -> impl Iterator { + [ + (!polarity.is_contravariant()).then_some((actual, formal)), + (!polarity.is_covariant()).then_some((formal, actual)), + ] + .into_iter() + .flatten() +} + impl<'db, 'c> SpecializationBuilder<'db, 'c> { pub(crate) fn new( db: &'db dyn Db, @@ -3019,6 +3035,24 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { } } + /// Returns the assignability constraints required by this comparison's polarity. + fn constraint_for_relation( + &self, + formal: Type<'db>, + actual: Type<'db>, + polarity: TypeVarVariance, + ) -> ConstraintSet<'db, 'c> { + let db = self.db; + relation_directions(formal, actual, polarity).when_all( + db, + self.constraints, + |(source, target)| { + let when = source.when_constraint_set_assignable_to_owned(db, self.env, target); + self.constraints.load(db, self.env, &when) + }, + ) + } + /// Returns common protocol constraints for the `TypedDict` members of a union when every such /// member has the same constraints as their shared `Mapping[str, object]` fallback. fn common_typed_dict_protocol_constraints( @@ -3223,12 +3257,22 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { /// are returned for immediate diagnosis. fn infer_from_callable_signature( &mut self, - formal_signature: &CallableSignature<'db>, - actual_callables: &CallableTypes<'db>, + formal: CallableType<'db>, + actual_callables: CallableTypes<'db>, + polarity: TypeVarVariance, ) -> Result<(), SpecializationError<'db>> { let db = self.db; - let formal_is_single_paramspec = formal_signature.is_single_paramspec().is_some(); + if !matches!(polarity, TypeVarVariance::Covariant) { + let actual = actual_callables + .map(|callable| callable.into_regular(db)) + .into_type(db, self.env); + let formal = Type::Callable(formal.into_regular(db)); + let when = self.constraint_for_relation(formal, actual, polarity); + return self.infer_from_constraint_set(when); + } + let formal_signature = formal.signatures(db); + let formal_is_single_paramspec = formal_signature.is_single_paramspec().is_some(); for actual_callable in actual_callables.as_slice() { if formal_is_single_paramspec { let when = actual_callable @@ -3300,7 +3344,7 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { formal: Type<'db>, actual: Type<'db>, polarity: TypeVarVariance, - seen: &mut FxHashSet<(Type<'db>, Type<'db>)>, + seen: &mut FxHashSet<(Type<'db>, Type<'db>, TypeVarVariance)>, ) -> Result<(), SpecializationError<'db>> { let db = self.db; // TODO: Eventually, the builder will maintain a constraint set, instead of a hash-map of @@ -3316,8 +3360,8 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { return Ok(()); } - // Avoid infinite recursion - if !seen.insert((formal, actual)) { + // Avoid infinite recursion while retaining comparisons under different polarities. + if !seen.insert((formal, actual, polarity)) { return Ok(()); } @@ -3329,6 +3373,22 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { let actual = actual.filter_disjoint_elements(db, self.env, formal, self.inferable); let formal = formal.filter_disjoint_elements(db, self.env, actual, self.inferable); + // ParamSpecs and TypeVarTuples still use the forward-only legacy mapping table. Keep + // their entire inference context on the existing signature path, and use forward + // structural relations so nested variadics and ordinary type variables retain their + // mappings. Preserve the original polarity for recursive and ordinary inference. + // TODO: Apply full polarity once variadics are supported by the new constraint solver. + let relation_polarity = if !polarity.is_covariant() + && self + .inferable + .iter(db) + .any(|typevar| typevar.is_paramspec(db) || typevar.is_typevartuple(db)) + { + TypeVarVariance::Covariant + } else { + polarity + }; + match (formal, actual) { // Expand PEP 695 type aliases in the formal type. // This is necessary for solving generics like `def head[T](my_list: MyList[T]) -> T`. @@ -3737,24 +3797,8 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { || matches!(formal, Type::SubclassOf(subclass) if matches!(subclass.subclass_of(), SubclassOfInner::Class(_))) => { - let when = match polarity { - TypeVarVariance::Covariant | TypeVarVariance::Invariant => { - actual.when_constraint_set_assignable_to_owned(db, self.env, formal) - } - TypeVarVariance::Contravariant => { - formal.when_constraint_set_assignable_to_owned(db, self.env, actual) - } - TypeVarVariance::Bivariant => return Ok(()), - }; - let mut when = self.constraints.load(db, self.env, &when); - if matches!(polarity, TypeVarVariance::Invariant) { - let reverse = - formal.when_constraint_set_assignable_to_owned(db, self.env, actual); - let reverse = self.constraints.load(db, self.env, &reverse); - when.intersect(db, self.constraints, reverse); - } - self.infer_from_constraint_set(when)?; - return Ok(()); + let when = self.constraint_for_relation(formal, actual, relation_polarity); + return self.infer_from_constraint_set(when); } (Type::SubclassOf(subclass_of), ty) | (ty, Type::SubclassOf(subclass_of)) @@ -3798,31 +3842,57 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { && let Some(actual_origin) = actual_protocol.materialized_origin(db) && let Some(formal_origin) = formal_protocol.class_origin(db) { - let nominally_inherited = actual_origin - .iter_mro(db) - .filter_map(ClassBase::into_class) - .any(|base| base.class_literal(db) == formal_origin.class_literal(db)); - let when = if nominally_inherited - || formal_protocol.interface(db).has_only_finite_members(db) - { - Some(actual.when_constraint_set_assignable_to_owned(db, self.env, formal)) - } else { - actual_protocol - .when_non_recursive_members_assignable_to_owned(db, formal_protocol) - .map(Cow::Borrowed) - }; - // Materialized protocols cannot be replaced by their nominal origin: doing // so would recover the original `Any` requirements. Infer from the complete // interface when doing so is cycle-safe; otherwise use its nonrecursive // requirements and leave full recursive compatibility to argument checking. + let when = relation_directions( + (formal_protocol, formal_origin), + (actual_protocol, actual_origin), + relation_polarity, + ) + .try_fold( + ConstraintSet::from_bool(self.constraints, true), + |mut combined, ((source, source_origin), (target, target_origin))| { + if combined.is_trivially_never_satisfied() { + return Some(combined); + } + + let when = if source_origin + .is_subtype_of_class_literal(db, target_origin.class_literal(db)) + || target.interface(db).has_only_finite_members(db) + { + Type::ProtocolInstance(source) + .when_constraint_set_assignable_to_owned( + db, + self.env, + Type::ProtocolInstance(target), + ) + } else { + Cow::Borrowed( + source.when_non_recursive_members_assignable_to_owned( + db, target, + )?, + ) + }; + let next = self.constraints.load(db, self.env, &when); + Some(combined.intersect(db, self.constraints, next)) + }, + ); + if let Some(when) = when { - let when = self.constraints.load(db, self.env, &when); - self.infer_from_constraint_set(when)?; - return Ok(()); + return self.infer_from_constraint_set(when); } } + // Converting the actual protocol to its nominal origin makes the reversed + // comparison impossible: a protocol cannot be assignable to a nominal class. + if matches!(formal, Type::ProtocolInstance(_)) && !relation_polarity.is_covariant() + { + let when = self.constraint_for_relation(formal, actual, relation_polarity); + return self.infer_from_constraint_set(when); + } + // TODO: This will only handle protocol classes that explicit inherit // from other generic protocol classes by listing it as a base class. // To handle classes that implicitly implement a generic protocol, we @@ -3944,11 +4014,8 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { // will handle implicitly implemented protocols and generic protocols. We // eventually want this logic to be used for _all_ nominal instances // (replacing the logic below). - let when = - actual.when_constraint_set_assignable_to_owned(db, self.env, formal); - let when = self.constraints.load(db, self.env, &when); - self.infer_from_constraint_set(when)?; - return Ok(()); + let when = self.constraint_for_relation(formal, actual, relation_polarity); + return self.infer_from_constraint_set(when); } _ => None, @@ -3986,25 +4053,22 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { // union, but the old solver isn't well-equipped to handle that (due to side effects // from even failed matches), so for now we handle this particular case. (formal @ Type::ProtocolInstance(_), actual @ Type::Union(actual_union)) => { - let when = self - .common_typed_dict_protocol_constraints(formal, actual_union) - .unwrap_or_else(|| { - actual.when_constraint_set_assignable_to( - db, - self.env, - formal, - self.constraints, - ) - }); - self.infer_from_constraint_set(when)?; - return Ok(()); + // Common TypedDict constraints prove only `actual <= formal`. Contravariance + // reverses that relation, while invariance additionally requires the reverse. + let when = if matches!(relation_polarity, TypeVarVariance::Covariant) + && let Some(common) = + self.common_typed_dict_protocol_constraints(formal, actual_union) + { + common + } else { + self.constraint_for_relation(formal, actual, relation_polarity) + }; + return self.infer_from_constraint_set(when); } (formal @ Type::ProtocolInstance(_), actual @ Type::TypedDict(_)) => { - let when = actual.when_constraint_set_assignable_to_owned(db, self.env, formal); - let when = self.constraints.load(db, self.env, &when); - self.infer_from_constraint_set(when)?; - return Ok(()); + let when = self.constraint_for_relation(formal, actual, relation_polarity); + return self.infer_from_constraint_set(when); } // When the formal type is a protocol with a `__call__` method, infer the specialization @@ -4021,18 +4085,22 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { // The protocol interface exposes the callable signature already bound for // instance access. - let formal_signature = call_method.signatures(db); - - self.infer_from_callable_signature(formal_signature, &actual_callables)?; + self.infer_from_callable_signature( + call_method, + actual_callables, + relation_polarity, + )?; } (Type::Callable(formal_callable), _) => { let Some(actual_callables) = actual.try_upcast_to_callable(db, self.env) else { return Ok(()); }; - let formal_signature = formal_callable.signatures(db); - - self.infer_from_callable_signature(formal_signature, &actual_callables)?; + self.infer_from_callable_signature( + formal_callable, + actual_callables, + relation_polarity, + )?; } // Expand type aliases in the actual type. From 126352467217bebfa4cb86fd3c4d20820322d9e3 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Thu, 13 Aug 2026 12:35:45 +0100 Subject: [PATCH 016/371] [ty] Simplify display implementations with std::fmt::from_fn (#27718) --- crates/ty_ide/src/hover.rs | 48 +-- crates/ty_module_resolver/src/environment.rs | 40 +- crates/ty_project/src/db.rs | 220 +++++------ .../src/types/call/arguments.rs | 100 ++--- .../src/types/class/known.rs | 112 ++---- .../src/types/class_base.rs | 45 +-- .../src/types/constraints.rs | 363 ++++++------------ .../ty_python_semantic/src/types/display.rs | 84 ++-- .../src/types/protocol_class.rs | 108 ++---- 9 files changed, 384 insertions(+), 736 deletions(-) diff --git a/crates/ty_ide/src/hover.rs b/crates/ty_ide/src/hover.rs index a1dc3a3f4b..fde3411e74 100644 --- a/crates/ty_ide/src/hover.rs +++ b/crates/ty_ide/src/hover.rs @@ -5,8 +5,7 @@ use ruff_db::files::FileRange; use ruff_db::parsed::parsed_module; use ruff_python_ast as ast; use ruff_text_size::{Ranged, TextSize}; -use std::fmt; -use std::fmt::Formatter; +use std::fmt::{self, Display}; use ty_python_core::ProgramFile; use ty_python_semantic::ProgramEnvironment; use ty_python_semantic::types::ide_support::{resolved_call_signature, typed_dict_key_hover}; @@ -230,12 +229,21 @@ pub struct Hover<'db> { impl<'db> Hover<'db> { /// Renders the hover to a string using the specified markup kind. - pub const fn display<'a>(&'a self, db: &'db dyn Db, kind: MarkupKind) -> DisplayHover<'db, 'a> { - DisplayHover { - db, - hover: self, - kind, - } + pub const fn display<'a>(&'a self, db: &'db dyn Db, kind: MarkupKind) -> impl Display { + std::fmt::from_fn(move |f| { + let mut first = true; + let env = ProgramEnvironment::from_file(self.program_file); + for content in &self.contents { + if !first { + kind.horizontal_line().fmt(f)?; + } + + content.display(db, &env, kind).fmt(f)?; + first = false; + } + + Ok(()) + }) } fn iter(&self) -> std::slice::Iter<'_, HoverContent<'db>> { @@ -261,30 +269,6 @@ impl<'a, 'db> IntoIterator for &'a Hover<'db> { } } -pub struct DisplayHover<'db, 'a> { - db: &'db dyn Db, - hover: &'a Hover<'db>, - kind: MarkupKind, -} - -impl fmt::Display for DisplayHover<'_, '_> { - fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result { - let db = self.db; - let mut first = true; - let env = ProgramEnvironment::from_file(self.hover.program_file); - for content in &self.hover.contents { - if !first { - self.kind.horizontal_line().fmt(f)?; - } - - content.display(db, &env, self.kind).fmt(f)?; - first = false; - } - - Ok(()) - } -} - #[derive(Debug, Clone)] pub enum HoverContent<'db> { Signature(String), diff --git a/crates/ty_module_resolver/src/environment.rs b/crates/ty_module_resolver/src/environment.rs index 478a854672..8218ca2587 100644 --- a/crates/ty_module_resolver/src/environment.rs +++ b/crates/ty_module_resolver/src/environment.rs @@ -1,5 +1,3 @@ -use std::fmt; - use ruff_db::files::File; use ruff_python_ast::PythonVersion; @@ -22,34 +20,20 @@ impl<'db> ResolverEnvironment<'db> { self, db: &'db dyn Db, mode: ModuleResolveMode, - ) -> DisplaySearchPaths<'db> { - DisplaySearchPaths { - db, - resolver_environment: self, - mode, - } - } -} - -pub struct DisplaySearchPaths<'db> { - db: &'db dyn Db, - resolver_environment: ResolverEnvironment<'db>, - mode: ModuleResolveMode, -} - -impl fmt::Display for DisplaySearchPaths<'_> { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - let mut paths = search_paths(self.db, self.resolver_environment, self.mode).peekable(); + ) -> impl std::fmt::Display { + std::fmt::from_fn(move |f| { + let mut paths = search_paths(db, self, mode).peekable(); - if paths.peek().is_none() { - return f.write_str("[]"); - } + if paths.peek().is_none() { + return f.write_str("[]"); + } - writeln!(f, "[")?; - for path in paths { - writeln!(f, " {path},")?; - } - f.write_str("]") + writeln!(f, "[")?; + for path in paths { + writeln!(f, " {path},")?; + } + f.write_str("]") + }) } } diff --git a/crates/ty_project/src/db.rs b/crates/ty_project/src/db.rs index ae2491a36d..50f23c2390 100644 --- a/crates/ty_project/src/db.rs +++ b/crates/ty_project/src/db.rs @@ -325,137 +325,125 @@ fn bytes_to_mb(total: usize) -> f64 { impl SalsaMemoryDump { /// Returns a short report that provides total memory usage information. - pub fn display_short(&self) -> impl fmt::Display + '_ { - struct DisplayShort<'a>(&'a SalsaMemoryDump); - - impl fmt::Display for DisplayShort<'_> { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - let SalsaMemoryDump { - total_fields, - total_metadata, - total_memo_fields, - total_memo_metadata, - ref ingredients, - ref memos, - } = *self.0; - - writeln!(f, "=======SALSA SUMMARY=======")?; + pub fn display_short(self) -> impl fmt::Display { + std::fmt::from_fn(move |f| { + let SalsaMemoryDump { + total_fields, + total_metadata, + total_memo_fields, + total_memo_metadata, + ref ingredients, + ref memos, + } = self; + + writeln!(f, "=======SALSA SUMMARY=======")?; + + writeln!( + f, + "TOTAL MEMORY USAGE: {:.2}MB", + bytes_to_mb( + total_metadata + total_fields + total_memo_fields + total_memo_metadata + ) + )?; + + writeln!( + f, + " struct metadata = {:.2}MB", + bytes_to_mb(total_metadata), + )?; + writeln!(f, " struct fields = {:.2}MB", bytes_to_mb(total_fields))?; + writeln!( + f, + " memo metadata = {:.2}MB", + bytes_to_mb(total_memo_metadata), + )?; + writeln!( + f, + " memo fields = {:.2}MB", + bytes_to_mb(total_memo_fields), + )?; + + writeln!(f, "QUERY COUNT: {}", memos.len())?; + writeln!(f, "STRUCT COUNT: {}", ingredients.len())?; + + Ok(()) + }) + } - writeln!( - f, - "TOTAL MEMORY USAGE: {:.2}MB", - bytes_to_mb( - total_metadata + total_fields + total_memo_fields + total_memo_metadata - ) - )?; + /// Returns a short report that provides fine-grained memory usage information per + /// Salsa ingredient. + pub fn display_full(self) -> impl fmt::Display { + std::fmt::from_fn(move |f| { + let SalsaMemoryDump { + total_fields, + total_metadata, + total_memo_fields, + total_memo_metadata, + ref ingredients, + ref memos, + } = self; + + writeln!(f, "=======SALSA STRUCTS=======")?; + + for ingredient in ingredients { + let size_of_fields = + ingredient.size_of_fields() + ingredient.heap_size_of_fields().unwrap_or(0); writeln!( f, - " struct metadata = {:.2}MB", - bytes_to_mb(total_metadata), - )?; - writeln!(f, " struct fields = {:.2}MB", bytes_to_mb(total_fields))?; - writeln!( - f, - " memo metadata = {:.2}MB", - bytes_to_mb(total_memo_metadata), + "{:<50} metadata={:<8} fields={:<8} count={}", + format!("`{}`", ingredient.debug_name()), + format!("{:.2}MB", bytes_to_mb(ingredient.size_of_metadata())), + format!("{:.2}MB", bytes_to_mb(size_of_fields)), + ingredient.count() )?; - writeln!( - f, - " memo fields = {:.2}MB", - bytes_to_mb(total_memo_fields), - )?; - - writeln!(f, "QUERY COUNT: {}", memos.len())?; - writeln!(f, "STRUCT COUNT: {}", ingredients.len())?; - - Ok(()) } - } - DisplayShort(self) - } + writeln!(f, "=======SALSA QUERIES=======")?; - /// Returns a short report that provides fine-grained memory usage information per - /// Salsa ingredient. - pub fn display_full(&self) -> impl fmt::Display + '_ { - struct DisplayFull<'a>(&'a SalsaMemoryDump); - - impl fmt::Display for DisplayFull<'_> { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - let SalsaMemoryDump { - total_fields, - total_metadata, - total_memo_fields, - total_memo_metadata, - ref ingredients, - ref memos, - } = *self.0; - - writeln!(f, "=======SALSA STRUCTS=======")?; - - for ingredient in ingredients { - let size_of_fields = - ingredient.size_of_fields() + ingredient.heap_size_of_fields().unwrap_or(0); - - writeln!( - f, - "{:<50} metadata={:<8} fields={:<8} count={}", - format!("`{}`", ingredient.debug_name()), - format!("{:.2}MB", bytes_to_mb(ingredient.size_of_metadata())), - format!("{:.2}MB", bytes_to_mb(size_of_fields)), - ingredient.count() - )?; - } - - writeln!(f, "=======SALSA QUERIES=======")?; - - for (query_fn, memo) in memos { - let size_of_fields = - memo.size_of_fields() + memo.heap_size_of_fields().unwrap_or(0); + for (query_fn, memo) in memos { + let size_of_fields = + memo.size_of_fields() + memo.heap_size_of_fields().unwrap_or(0); - writeln!(f, "`{query_fn} -> {}`", memo.debug_name())?; - - writeln!( - f, - " metadata={:<8} fields={:<8} count={}", - format!("{:.2}MB", bytes_to_mb(memo.size_of_metadata())), - format!("{:.2}MB", bytes_to_mb(size_of_fields)), - memo.count() - )?; - } - - writeln!(f, "=======SALSA SUMMARY=======")?; - writeln!( - f, - "TOTAL MEMORY USAGE: {:.2}MB", - bytes_to_mb( - total_metadata + total_fields + total_memo_fields + total_memo_metadata - ) - )?; + writeln!(f, "`{query_fn} -> {}`", memo.debug_name())?; writeln!( f, - " struct metadata = {:.2}MB", - bytes_to_mb(total_metadata), + " metadata={:<8} fields={:<8} count={}", + format!("{:.2}MB", bytes_to_mb(memo.size_of_metadata())), + format!("{:.2}MB", bytes_to_mb(size_of_fields)), + memo.count() )?; - writeln!(f, " struct fields = {:.2}MB", bytes_to_mb(total_fields))?; - writeln!( - f, - " memo metadata = {:.2}MB", - bytes_to_mb(total_memo_metadata), - )?; - writeln!( - f, - " memo fields = {:.2}MB", - bytes_to_mb(total_memo_fields), - )?; - - Ok(()) } - } - DisplayFull(self) + writeln!(f, "=======SALSA SUMMARY=======")?; + writeln!( + f, + "TOTAL MEMORY USAGE: {:.2}MB", + bytes_to_mb( + total_metadata + total_fields + total_memo_fields + total_memo_metadata + ) + )?; + + writeln!( + f, + " struct metadata = {:.2}MB", + bytes_to_mb(total_metadata), + )?; + writeln!(f, " struct fields = {:.2}MB", bytes_to_mb(total_fields))?; + writeln!( + f, + " memo metadata = {:.2}MB", + bytes_to_mb(total_memo_metadata), + )?; + writeln!( + f, + " memo fields = {:.2}MB", + bytes_to_mb(total_memo_fields), + )?; + + Ok(()) + }) } /// Serializes the memory dump to JSON. diff --git a/crates/ty_python_semantic/src/types/call/arguments.rs b/crates/ty_python_semantic/src/types/call/arguments.rs index ae4bea09c4..387ebd1173 100644 --- a/crates/ty_python_semantic/src/types/call/arguments.rs +++ b/crates/ty_python_semantic/src/types/call/arguments.rs @@ -459,85 +459,35 @@ impl<'a, 'db> CallArguments<'a, 'db> { } } - struct DisplayCallArguments<'env, 'a, 'db> { - call_arguments: &'a CallArguments<'a, 'db>, - db: &'db dyn Db, - env: &'env ProgramEnvironment<'db>, - } - - impl std::fmt::Display for DisplayCallArguments<'_, '_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - f.write_str("(")?; - for (index, (argument, types)) in self.call_arguments.iter().enumerate() { - if index > 0 { - write!(f, ", ")?; + std::fmt::from_fn(move |f| { + f.write_str("(")?; + for (index, (argument, types)) in self.iter().enumerate() { + if index > 0 { + write!(f, ", ")?; + } + match argument { + Argument::Synthetic => { + write!(f, "self: {}", DisplayCallArgumentTypes { types, db, env })?; } - match argument { - Argument::Synthetic => { - write!( - f, - "self: {}", - DisplayCallArgumentTypes { - types, - db: self.db, - env: self.env, - } - )?; - } - Argument::Positional => { - write!( - f, - "{}", - DisplayCallArgumentTypes { - types, - db: self.db, - env: self.env, - } - )?; - } - Argument::Variadic => { - write!( - f, - "*{}", - DisplayCallArgumentTypes { - types, - db: self.db, - env: self.env, - } - )?; - } - Argument::Keyword(name) => write!( - f, - "{}={}", - name, - DisplayCallArgumentTypes { - types, - db: self.db, - env: self.env, - } - )?, - Argument::Keywords => { - write!( - f, - "**{}", - DisplayCallArgumentTypes { - types, - db: self.db, - env: self.env, - } - )?; - } + Argument::Positional => { + write!(f, "{}", DisplayCallArgumentTypes { types, db, env })?; + } + Argument::Variadic => { + write!(f, "*{}", DisplayCallArgumentTypes { types, db, env })?; + } + Argument::Keyword(name) => write!( + f, + "{}={}", + name, + DisplayCallArgumentTypes { types, db, env } + )?, + Argument::Keywords => { + write!(f, "**{}", DisplayCallArgumentTypes { types, db, env })?; } } - f.write_str(")") } - } - - DisplayCallArguments { - call_arguments: self, - db, - env, - } + f.write_str(")") + }) } } diff --git a/crates/ty_python_semantic/src/types/class/known.rs b/crates/ty_python_semantic/src/types/class/known.rs index 66d8846b35..a75792d696 100644 --- a/crates/ty_python_semantic/src/types/class/known.rs +++ b/crates/ty_python_semantic/src/types/class/known.rs @@ -1017,30 +1017,14 @@ impl KnownClass { } pub(crate) fn display(self, python_version: PythonVersion) -> impl std::fmt::Display { - struct KnownClassDisplay { - class: KnownClass, - python_version: PythonVersion, - } - - impl std::fmt::Display for KnownClassDisplay { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let KnownClassDisplay { - class: known_class, - python_version, - } = *self; - write!( - f, - "{module}.{class}", - module = known_class.canonical_module(python_version), - class = known_class.name(python_version) - ) - } - } - - KnownClassDisplay { - class: self, - python_version, - } + std::fmt::from_fn(move |f| { + write!( + f, + "{module}.{class}", + module = self.canonical_module(python_version), + class = self.name(python_version) + ) + }) } /// Look up a [`KnownClass`] in its canonical module and return a [`Type`] representing all @@ -2007,62 +1991,42 @@ impl<'db> KnownClassLookupError<'db> { } fn display<'env>( - &self, + self, db: &'db dyn Db, env: &'env ProgramEnvironment<'db>, class: KnownClass, ) -> impl std::fmt::Display + 'env { - struct ErrorDisplay<'env, 'db> { - db: &'db dyn Db, - env: &'env ProgramEnvironment<'db>, - class: KnownClass, - error: KnownClassLookupError<'db>, - } - - impl std::fmt::Display for ErrorDisplay<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let db = self.db; - let ErrorDisplay { - db: _, - env, - class, - error, - } = self; - - let python_version = env.python_version(db); - let class = class.display(python_version); - let location = if error.is_third_party() { - "" - } else { - " in typeshed" - }; + std::fmt::from_fn(move |f| { + let python_version = env.python_version(db); + let class = class.display(python_version); + let location = if self.is_third_party() { + "" + } else { + " in typeshed" + }; - match error { - KnownClassLookupError::ClassNotFound { .. } => write!( - f, - "Could not find class `{class}`{location} on Python {python_version}", - ), - KnownClassLookupError::SymbolNotAClass { found_type, .. } => write!( - f, - "Error looking up `{class}`{location}: expected to find a class definition \ - on Python {python_version}, but found a symbol of type `{found_type}` instead", - found_type = found_type.display(db, env), - ), - KnownClassLookupError::ClassPossiblyUnbound { .. } => write!( - f, - "Error looking up `{class}`{location} on Python {python_version}: expected \ - to find a fully bound symbol, but found one that is possibly unbound", - ), - } + match self { + KnownClassLookupError::ClassNotFound { .. } => write!( + f, + "Could not find class `{class}`{location} on Python {python_version}", + ), + KnownClassLookupError::SymbolNotAClass { found_type, .. } => write!( + f, + "Error looking up `{class}`{location}: \ + expected to find a class definition \ + on Python {python_version}, \ + but found a symbol of type `{found_type}` instead", + found_type = found_type.display(db, env), + ), + KnownClassLookupError::ClassPossiblyUnbound { .. } => write!( + f, + "Error looking up `{class}`{location} \ + on Python {python_version}: expected \ + to find a fully bound symbol, \ + but found one that is possibly unbound", + ), } - } - - ErrorDisplay { - db, - env, - class, - error: *self, - } + }) } } diff --git a/crates/ty_python_semantic/src/types/class_base.rs b/crates/ty_python_semantic/src/types/class_base.rs index 3722d9fde0..d848b47f46 100644 --- a/crates/ty_python_semantic/src/types/class_base.rs +++ b/crates/ty_python_semantic/src/types/class_base.rs @@ -1,3 +1,5 @@ +use std::fmt::Display; + use crate::ProgramEnvironment; use crate::types::class::CodeGeneratorKind; use crate::types::generics::{ApplySpecialization, Specialization}; @@ -490,37 +492,18 @@ impl<'db> ClassBase<'db> { db: &'db dyn Db, env: &'env ProgramEnvironment<'db>, display_settings: DisplaySettings<'db>, - ) -> impl std::fmt::Display + 'env { - struct ClassBaseDisplay<'env, 'db> { - db: &'db dyn Db, - env: &'env ProgramEnvironment<'db>, - base: ClassBase<'db>, - settings: DisplaySettings<'db>, - } - - impl std::fmt::Display for ClassBaseDisplay<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let db = self.db; - match self.base { - ClassBase::Any => f.write_str("Any"), - ClassBase::Dynamic(dynamic) => dynamic.fmt(f), - ClassBase::Divergent(_) => f.write_str("Divergent"), - ClassBase::Class(class) => Type::from(class) - .display_with(db, self.env, self.settings.clone()) - .fmt(f), - ClassBase::Protocol => f.write_str("typing.Protocol"), - ClassBase::Generic => f.write_str("typing.Generic"), - ClassBase::TypedDict(_) => f.write_str("typing.TypedDict"), - } - } - } - - ClassBaseDisplay { - db, - env, - base: self, - settings: display_settings, - } + ) -> impl Display + 'env { + std::fmt::from_fn(move |f| match self { + ClassBase::Any => f.write_str("Any"), + ClassBase::Dynamic(dynamic) => dynamic.fmt(f), + ClassBase::Divergent(_) => f.write_str("Divergent"), + ClassBase::Class(class) => Type::from(class) + .display_with(db, env, display_settings.clone()) + .fmt(f), + ClassBase::Protocol => f.write_str("typing.Protocol"), + ClassBase::Generic => f.write_str("typing.Generic"), + ClassBase::TypedDict(_) => f.write_str("typing.TypedDict"), + }) } } diff --git a/crates/ty_python_semantic/src/types/constraints.rs b/crates/ty_python_semantic/src/types/constraints.rs index 76a8939dac..b89052ae48 100644 --- a/crates/ty_python_semantic/src/types/constraints.rs +++ b/crates/ty_python_semantic/src/types/constraints.rs @@ -877,26 +877,10 @@ impl<'db, 'c> ConstraintSet<'db, 'c> { db: &'db dyn Db, env: &'c ProgramEnvironment<'db>, ) -> impl Display + 'c { - struct DisplayConstraintSet<'c, 'db> { - node: NodeId, - db: &'db dyn Db, - env: &'c ProgramEnvironment<'db>, - builder: &'c ConstraintSetBuilder<'db>, - } - - impl Display for DisplayConstraintSet<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let storage = self.builder.storage.borrow(); - Display::fmt(&self.node.display(self.db, self.env, &storage), f) - } - } - - DisplayConstraintSet { - node: self.node, - db, - env, - builder: self.builder, - } + std::fmt::from_fn(move |f| { + let storage = self.builder.storage.borrow(); + self.node.display(db, env, &storage).fmt(f) + }) } #[expect(dead_code)] // Keep this around for debugging purposes @@ -910,33 +894,10 @@ impl<'db, 'c> ConstraintSet<'db, 'c> { 'db: 'a, 'c: 'a, { - struct DisplayConstraintSet<'a, 'c, 'db> { - node: NodeId, - prefix: &'a dyn Display, - db: &'db dyn Db, - env: &'a ProgramEnvironment<'db>, - builder: &'c ConstraintSetBuilder<'db>, - } - - impl Display for DisplayConstraintSet<'_, '_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let storage = self.builder.storage.borrow(); - Display::fmt( - &self - .node - .display_graph(self.db, self.env, &storage, self.prefix), - f, - ) - } - } - - DisplayConstraintSet { - node: self.node, - prefix, - db, - env, - builder: self.builder, - } + std::fmt::from_fn(move |f| { + let storage = self.builder.storage.borrow(); + self.node.display_graph(db, env, &storage, prefix).fmt(f) + }) } } @@ -3321,36 +3282,14 @@ impl NodeId { // Render the BDD directly as an unsimplified DNF formula. Each root-to-true path becomes // one clause, with true, uncertain, and false edges contributing positive, unconstrained, // and negative assignments respectively. - struct DisplayNode<'db, 'c> { - node: NodeId, - db: &'db dyn Db, - env: &'c ProgramEnvironment<'db>, - storage: &'c ConstraintSetStorage<'db>, - } - - impl Display for DisplayNode<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self.node.node() { - Node::AlwaysTrue => f.write_str("always"), - Node::AlwaysFalse => f.write_str("never"), - Node::Interior(_) => Display::fmt( - &self.node.satisfied_clauses(self.storage).display( - self.db, - self.env, - self.storage, - ), - f, - ), - } - } - } - - DisplayNode { - node: self, - db, - env, - storage, - } + std::fmt::from_fn(move |f| match self.node() { + Node::AlwaysTrue => f.write_str("always"), + Node::AlwaysFalse => f.write_str("never"), + Node::Interior(_) => Display::fmt( + &self.satisfied_clauses(storage).display(db, env, storage), + f, + ), + }) } /// Displays the full graph structure of this BDD. `prefix` will be output before each line @@ -3378,15 +3317,6 @@ impl NodeId { storage: &'a ConstraintSetStorage<'db>, prefix: &'a dyn Display, ) -> impl Display + 'a { - struct DisplayNode<'a, 'db> { - db: &'db dyn Db, - env: &'a ProgramEnvironment<'db>, - storage: &'a ConstraintSetStorage<'db>, - node: NodeId, - prefix: &'a dyn Display, - seen: RefCell>, - } - fn format_node<'db>( db: &'db dyn Db, env: &ProgramEnvironment<'db>, @@ -3447,29 +3377,9 @@ impl NodeId { } } - impl Display for DisplayNode<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let db = self.db; - format_node( - db, - self.env, - self.storage, - self.node, - self.prefix, - &self.seen, - f, - ) - } - } - - DisplayNode { - db, - env, - storage, - node: self, - prefix, - seen: RefCell::default(), - } + std::fmt::from_fn(move |f| { + format_node(db, env, storage, self, prefix, &RefCell::default(), f) + }) } } @@ -4689,91 +4599,61 @@ impl ConstraintAssignment { env: &'a ProgramEnvironment<'db>, storage: &'a ConstraintSetStorage<'db>, ) -> impl Display + 'a { - struct DisplayConstraintAssignment<'db, 'c> { - assignment: ConstraintAssignment, - db: &'db dyn Db, - env: &'c ProgramEnvironment<'db>, - storage: &'c ConstraintSetStorage<'db>, - } + let (equality_sign, range_prefix) = match self { + ConstraintAssignment::Positive(_) => ("=", ""), + ConstraintAssignment::Negative(_) => ("≠", "¬"), + ConstraintAssignment::Unconstrained(_) => ("=?", "?"), + }; - impl DisplayConstraintAssignment<'_, '_> { - fn equality_sign(&self) -> &'static str { - match self.assignment { - ConstraintAssignment::Positive(_) => "=", - ConstraintAssignment::Negative(_) => "≠", - ConstraintAssignment::Unconstrained(_) => "=?", + std::fmt::from_fn(move |f| { + let constraint_data = storage.constraint_data(self.constraint()); + let lower = constraint_data.bounds.materialized_lower(); + let upper = constraint_data.bounds.materialized_upper(); + let typevar = constraint_data.typevar; + if lower.is_equivalent_to(db, env, upper) { + // If this typevar is equivalent to another, output the constraint in a + // consistent alphabetical order, regardless of the salsa ordering that we are + // using the in BDD. + if let Type::TypeVar(bound) = lower { + let bound = bound.identity(db).display(db).to_string(); + let typevar = typevar.identity(db).display(db).to_string(); + let (smaller, larger) = if bound < typevar { + (bound, typevar) + } else { + (typevar, bound) + }; + return write!(f, "({smaller} {equality_sign} {larger})"); } - } - fn range_prefix(&self) -> &'static str { - match self.assignment { - ConstraintAssignment::Positive(_) => "", - ConstraintAssignment::Negative(_) => "¬", - ConstraintAssignment::Unconstrained(_) => "?", - } + return write!( + f, + "({} {} {})", + typevar.identity(db).display(db), + equality_sign, + lower.display(db, env) + ); } - } - - impl Display for DisplayConstraintAssignment<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let db = self.db; - - let constraint_data = self.storage.constraint_data(self.assignment.constraint()); - let lower = constraint_data.bounds.materialized_lower(); - let upper = constraint_data.bounds.materialized_upper(); - let typevar = constraint_data.typevar; - if lower.is_equivalent_to(db, self.env, upper) { - // If this typevar is equivalent to another, output the constraint in a - // consistent alphabetical order, regardless of the salsa ordering that we are - // using the in BDD. - if let Type::TypeVar(bound) = lower { - let bound = bound.identity(db).display(db).to_string(); - let typevar = typevar.identity(db).display(db).to_string(); - let (smaller, larger) = if bound < typevar { - (bound, typevar) - } else { - (typevar, bound) - }; - return write!(f, "({} {} {})", smaller, self.equality_sign(), larger); - } - - return write!( - f, - "({} {} {})", - typevar.identity(db).display(db), - self.equality_sign(), - lower.display(db, self.env) - ); - } - if lower.is_never() && upper.is_object() { - return write!( - f, - "({} {} *)", - typevar.identity(db).display(db), - self.equality_sign() - ); - } - - f.write_str(self.range_prefix())?; - f.write_str("(")?; - if !lower.is_never() { - write!(f, "{} ≤ ", lower.display(db, self.env))?; - } - typevar.identity(db).display(db).fmt(f)?; - if !upper.is_object() { - write!(f, " ≤ {}", upper.display(db, self.env))?; - } - f.write_str(")") + if lower.is_never() && upper.is_object() { + return write!( + f, + "({} {} *)", + typevar.identity(db).display(db), + equality_sign + ); } - } - DisplayConstraintAssignment { - assignment: self, - db, - env, - storage, - } + f.write_str(range_prefix)?; + f.write_str("(")?; + if !lower.is_never() { + write!(f, "{} ≤ ", lower.display(db, env))?; + } + typevar.identity(db).display(db).fmt(f)?; + if !upper.is_object() { + write!(f, " ≤ {}", upper.display(db, env))?; + } + f.write_str(")") + }) } } @@ -5932,78 +5812,59 @@ impl SequentMap { storage: &'a ConstraintSetStorage<'db>, prefix: &'a dyn Display, ) -> impl Display + 'a { - struct DisplaySequentMap<'a, 'db> { - map: &'a SequentMap, - prefix: &'a dyn Display, - db: &'db dyn Db, - env: &'a ProgramEnvironment<'db>, - storage: &'a ConstraintSetStorage<'db>, - } + std::fmt::from_fn(move |f| { + let mut first = true; + let mut maybe_write_prefix = |f: &mut std::fmt::Formatter<'_>| { + if first { + first = false; + Ok(()) + } else { + write!(f, "\n{prefix}") + } + }; - impl Display for DisplaySequentMap<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let db = self.db; - let mut first = true; - let mut maybe_write_prefix = |f: &mut std::fmt::Formatter<'_>| { - if first { - first = false; - Ok(()) - } else { - write!(f, "\n{}", self.prefix) + for sequent in &self.sequents { + match sequent { + Sequent::SingleTautology { .. } => {} + + Sequent::PairImpossibility { ante1, ante2 } => { + maybe_write_prefix(f)?; + write!( + f, + "{} ∧ {} → false", + ante1.display(db, env, storage), + ante2.display(db, env, storage), + )?; } - }; - - for sequent in &self.map.sequents { - match sequent { - Sequent::SingleTautology { .. } => {} - - Sequent::PairImpossibility { ante1, ante2 } => { - maybe_write_prefix(f)?; - write!( - f, - "{} ∧ {} → false", - ante1.display(db, self.env, self.storage), - ante2.display(db, self.env, self.storage), - )?; - } - Sequent::PairImplication { ante1, ante2, post } => { - maybe_write_prefix(f)?; - write!( - f, - "{} ∧ {} → {}", - ante1.display(db, self.env, self.storage), - ante2.display(db, self.env, self.storage), - post.display(db, self.env, self.storage), - )?; - } - - Sequent::SingleImplication { ante, post } => { - maybe_write_prefix(f)?; - write!( - f, - "{} → {}", - ante.display(db, self.env, self.storage), - post.display(db, self.env, self.storage) - )?; - } + Sequent::PairImplication { ante1, ante2, post } => { + maybe_write_prefix(f)?; + write!( + f, + "{} ∧ {} → {}", + ante1.display(db, env, storage), + ante2.display(db, env, storage), + post.display(db, env, storage), + )?; } - } - if first { - f.write_str("[no sequents]")?; + Sequent::SingleImplication { ante, post } => { + maybe_write_prefix(f)?; + write!( + f, + "{} → {}", + ante.display(db, env, storage), + post.display(db, env, storage) + )?; + } } - Ok(()) } - } - DisplaySequentMap { - map: self, - prefix, - db, - env, - storage, - } + if first { + f.write_str("[no sequents]")?; + } + Ok(()) + }) } } diff --git a/crates/ty_python_semantic/src/types/display.rs b/crates/ty_python_semantic/src/types/display.rs index 689803522d..b477932a5f 100644 --- a/crates/ty_python_semantic/src/types/display.rs +++ b/crates/ty_python_semantic/src/types/display.rs @@ -1586,42 +1586,24 @@ impl<'db> FmtDetailed<'db> for DisplayRepresentation<'_, 'db> { impl<'db> BoundTypeVarIdentity<'db> { pub(crate) fn display(self, db: &'db dyn Db) -> impl Display { - DisplayBoundTypeVarIdentity { - bound_typevar_identity: self, - db, - settings: DisplaySettings::default(), - } + self.display_with(db, DisplaySettings::default()) } fn display_with(self, db: &'db dyn Db, settings: DisplaySettings<'db>) -> impl Display { - DisplayBoundTypeVarIdentity { - bound_typevar_identity: self, - db, - settings, - } - } -} - -struct DisplayBoundTypeVarIdentity<'db> { - bound_typevar_identity: BoundTypeVarIdentity<'db>, - db: &'db dyn Db, - settings: DisplaySettings<'db>, -} - -impl Display for DisplayBoundTypeVarIdentity<'_> { - fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result { - f.write_str(self.bound_typevar_identity.identity.name(self.db))?; - let binding_context = self.bound_typevar_identity.binding_context; - if let Some(binding_context_name) = binding_context.name(self.db) - && let Some(definition) = binding_context.definition() - && !self.settings.active_scopes.contains(&definition) - { - write!(f, "@{binding_context_name}")?; - } - if let Some(paramspec_attr) = self.bound_typevar_identity.paramspec_attr { - write!(f, ".{paramspec_attr}")?; - } - Ok(()) + std::fmt::from_fn(move |f| { + f.write_str(self.identity.name(db))?; + let binding_context = self.binding_context; + if let Some(binding_context_name) = binding_context.name(db) + && let Some(definition) = binding_context.definition() + && !settings.active_scopes.contains(&definition) + { + write!(f, "@{binding_context_name}")?; + } + if let Some(paramspec_attr) = self.paramspec_attr { + write!(f, ".{paramspec_attr}")?; + } + Ok(()) + }) } } @@ -3596,29 +3578,19 @@ impl Display for DisplayTypeArray<'_, '_> { } impl<'db> StringLiteralType<'db> { - fn display(self, db: &'db dyn Db) -> DisplayStringLiteralType<'db> { - DisplayStringLiteralType { - string: self.value(db), - } - } -} - -struct DisplayStringLiteralType<'db> { - string: &'db str, -} - -impl Display for DisplayStringLiteralType<'_> { - fn fmt(&self, f: &mut Formatter<'_>) -> fmt::Result { - f.write_char('"')?; - for ch in self.string.chars() { - match ch { - // `escape_debug` will escape even single quotes, which is not necessary for our - // use case as we are already using double quotes to wrap the string. - '\'' => f.write_char('\''), - _ => ch.escape_debug().fmt(f), - }?; - } - f.write_char('"') + fn display(self, db: &'db dyn Db) -> impl std::fmt::Display { + std::fmt::from_fn(move |f| { + f.write_char('"')?; + for ch in self.value(db).chars() { + match ch { + // `escape_debug` will escape even single quotes, which is not necessary for our + // use case as we are already using double quotes to wrap the string. + '\'' => f.write_char('\''), + _ => ch.escape_debug().fmt(f), + }?; + } + f.write_char('"') + }) } } diff --git a/crates/ty_python_semantic/src/types/protocol_class.rs b/crates/ty_python_semantic/src/types/protocol_class.rs index d2135c6f9d..238f6d608d 100644 --- a/crates/ty_python_semantic/src/types/protocol_class.rs +++ b/crates/ty_python_semantic/src/types/protocol_class.rs @@ -903,31 +903,16 @@ impl<'db> ProtocolInterface<'db> { db: &'db dyn Db, env: &'env ProgramEnvironment<'db>, ) -> impl std::fmt::Display + 'env { - struct ProtocolInterfaceDisplay<'env, 'db> { - db: &'db dyn Db, - env: &'env ProgramEnvironment<'db>, - interface: ProtocolInterface<'db>, - } - - impl std::fmt::Display for ProtocolInterfaceDisplay<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let db = self.db; - f.write_char('{')?; - for (i, (name, data)) in self.interface.inner(db).iter().enumerate() { - write!(f, "\"{name}\": {data}", data = data.display(db, self.env))?; - if i < self.interface.inner(db).len() - 1 { - f.write_str(", ")?; - } + std::fmt::from_fn(move |f| { + f.write_char('{')?; + for (i, (name, data)) in self.inner(db).iter().enumerate() { + write!(f, "\"{name}\": {data}", data = data.display(db, env))?; + if i < self.inner(db).len() - 1 { + f.write_str(", ")?; } - f.write_char('}') } - } - - ProtocolInterfaceDisplay { - db, - env, - interface: self, - } + f.write_char('}') + }) } } @@ -1617,60 +1602,37 @@ impl<'db> ProtocolMemberData<'db> { } } - fn display<'env>( - &self, + fn display<'a, 'env>( + &'a self, db: &'db dyn Db, env: &'env ProgramEnvironment<'db>, - ) -> impl std::fmt::Display + 'env { - struct ProtocolMemberDataDisplay<'env, 'db> { - db: &'db dyn Db, - env: &'env ProgramEnvironment<'db>, - kind: ProtocolMemberKind<'db>, - qualifiers: TypeQualifiers, - } - - impl std::fmt::Display for ProtocolMemberDataDisplay<'_, '_> { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - let db = self.db; - match self.kind { - ProtocolMemberKind::Method(member, _) => { - write!(f, "MethodMember(`{}`)", member.ty().display(db, self.env)) - } - ProtocolMemberKind::Property { read, write } => { - let env = self.env; - let mut d = f.debug_struct("PropertyMember"); - if let Some(read) = read.and_then(|read| read.resolve(db, env)) { - d.field( - "read", - &format_args!("`{}`", read.ty().display(db, self.env)), - ); - } - if let Some(write) = write.and_then(|write| write.display_type(db, env)) { - d.field( - "write", - &format_args!("`{}`", write.ty().display(db, self.env)), - ); - } - d.finish() - } - ProtocolMemberKind::Attribute(attribute) => { - f.write_str("AttributeMember(")?; - write!(f, "`{}`", attribute.ty().display(db, self.env))?; - if self.qualifiers.contains(TypeQualifiers::CLASS_VAR) { - f.write_str("; ClassVar")?; - } - f.write_char(')') - } + ) -> impl std::fmt::Display + 'a + where + 'env: 'a, + { + std::fmt::from_fn(move |f| match self.kind { + ProtocolMemberKind::Method(member, _) => { + write!(f, "MethodMember(`{}`)", member.ty().display(db, env)) + } + ProtocolMemberKind::Property { read, write } => { + let mut d = f.debug_struct("PropertyMember"); + if let Some(read) = read.and_then(|read| read.resolve(db, env)) { + d.field("read", &format_args!("`{}`", read.ty().display(db, env))); } + if let Some(write) = write.and_then(|write| write.display_type(db, env)) { + d.field("write", &format_args!("`{}`", write.ty().display(db, env))); + } + d.finish() } - } - - ProtocolMemberDataDisplay { - db, - env, - kind: self.kind, - qualifiers: self.qualifiers, - } + ProtocolMemberKind::Attribute(attribute) => { + f.write_str("AttributeMember(")?; + write!(f, "`{}`", attribute.ty().display(db, env))?; + if self.qualifiers.contains(TypeQualifiers::CLASS_VAR) { + f.write_str("; ClassVar")?; + } + f.write_char(')') + } + }) } } From ecdd401fdbc5b0b22e18759c8bd25cda452e8b32 Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Thu, 13 Aug 2026 14:04:12 +0200 Subject: [PATCH 017/371] [ty] Separate script and uv modules from project metadata (#27720) Move standalone script handling and uv workspace metadata into dedicated top-level project modules. This establishes the module structure used by the stacked PEP 723 work without changing behavior. Testing: Ran the project test suite, all affected CLI/LSP tests, Clippy, and repository hooks. --- crates/ty_project/src/lib.rs | 2 ++ crates/ty_project/src/metadata.rs | 6 +++--- crates/ty_project/src/metadata/settings.rs | 2 +- crates/ty_project/src/{metadata => }/script.rs | 0 crates/ty_project/src/uv.rs | 5 +++++ .../src/{metadata/uv.rs => uv/metadata.rs} | 16 ++++++++-------- crates/ty_project/src/walk.rs | 2 +- 7 files changed, 20 insertions(+), 13 deletions(-) rename crates/ty_project/src/{metadata => }/script.rs (100%) create mode 100644 crates/ty_project/src/uv.rs rename crates/ty_project/src/{metadata/uv.rs => uv/metadata.rs} (95%) diff --git a/crates/ty_project/src/lib.rs b/crates/ty_project/src/lib.rs index 45c84dd18a..d80f5b631c 100644 --- a/crates/ty_project/src/lib.rs +++ b/crates/ty_project/src/lib.rs @@ -37,6 +37,8 @@ mod files; pub mod glob; pub mod metadata; pub mod parallel; +mod script; +mod uv; mod walk; pub mod watch; diff --git a/crates/ty_project/src/metadata.rs b/crates/ty_project/src/metadata.rs index e97c390a4d..04afe3fb72 100644 --- a/crates/ty_project/src/metadata.rs +++ b/crates/ty_project/src/metadata.rs @@ -17,6 +17,7 @@ use crate::metadata::options::{ use crate::metadata::pyproject::{Project, PyProject, PyProjectError, ResolveRequiresPythonError}; use crate::metadata::settings::Settings; use crate::metadata::value::RelativePathBuf; +use crate::uv; pub use options::Options; use options::TyTomlError; @@ -24,9 +25,7 @@ mod configuration_file; pub mod options; pub mod pyproject; pub mod python_version; -pub(crate) mod script; pub mod settings; -mod uv; pub mod value; #[derive(Debug, Clone, PartialEq, Eq, get_size2::GetSize)] @@ -640,7 +639,8 @@ mod tests { use ruff_ranged_value::ValueSource; use ty_static::EnvVars; - use crate::metadata::{Options, uv::UvWorkspace, value::RelativePathBuf}; + use crate::metadata::{Options, value::RelativePathBuf}; + use crate::uv::UvWorkspace; use crate::{ProjectMetadata, ProjectMetadataError}; #[test] diff --git a/crates/ty_project/src/metadata/settings.rs b/crates/ty_project/src/metadata/settings.rs index e939243e88..5a0b6cd69e 100644 --- a/crates/ty_project/src/metadata/settings.rs +++ b/crates/ty_project/src/metadata/settings.rs @@ -6,7 +6,7 @@ use ty_python_semantic::AnalysisSettings; use ty_python_semantic::lint::RuleSelection; use crate::metadata::options::{InnerOverrideOptions, Options, OutputFormat}; -use crate::metadata::script::script_metadata; +use crate::script::script_metadata; use crate::{Db, glob::IncludeExcludeFilter}; /// The resolved [`super::Options`] for the project. diff --git a/crates/ty_project/src/metadata/script.rs b/crates/ty_project/src/script.rs similarity index 100% rename from crates/ty_project/src/metadata/script.rs rename to crates/ty_project/src/script.rs diff --git a/crates/ty_project/src/uv.rs b/crates/ty_project/src/uv.rs new file mode 100644 index 0000000000..3294582273 --- /dev/null +++ b/crates/ty_project/src/uv.rs @@ -0,0 +1,5 @@ +//! Integrates uv with project discovery. + +pub(crate) use metadata::UvWorkspace; + +mod metadata; diff --git a/crates/ty_project/src/metadata/uv.rs b/crates/ty_project/src/uv/metadata.rs similarity index 95% rename from crates/ty_project/src/metadata/uv.rs rename to crates/ty_project/src/uv/metadata.rs index 8bda3071e1..e44d3c8584 100644 --- a/crates/ty_project/src/metadata/uv.rs +++ b/crates/ty_project/src/uv/metadata.rs @@ -7,17 +7,17 @@ use serde::Deserialize; use thiserror::Error; use ty_static::EnvVars; -use super::python_version::SupportedPythonVersion; +use crate::metadata::python_version::SupportedPythonVersion; #[derive(Debug, Clone, PartialEq, Eq, get_size2::GetSize)] -pub(super) struct UvWorkspace { +pub(crate) struct UvWorkspace { root: SystemPathBuf, environment: Option, python_version: Option>, } impl UvWorkspace { - pub(super) fn discover( + pub(crate) fn discover( path: &SystemPath, system: &dyn System, ) -> Result { @@ -50,7 +50,7 @@ impl UvWorkspace { Self::from_metadata(&output.stdout, system) } - pub(super) fn from_metadata( + pub(crate) fn from_metadata( metadata: &[u8], system: &dyn System, ) -> Result { @@ -78,15 +78,15 @@ impl UvWorkspace { }) } - pub(super) fn root(&self) -> &SystemPath { + pub(crate) fn root(&self) -> &SystemPath { &self.root } - pub(super) fn environment(&self) -> Option<&SystemPath> { + pub(crate) fn environment(&self) -> Option<&SystemPath> { self.environment.as_deref() } - pub(super) fn python_version(&self) -> Option<&RangedValue> { + pub(crate) fn python_version(&self) -> Option<&RangedValue> { self.python_version.as_ref() } } @@ -129,7 +129,7 @@ fn existing_directory( } #[derive(Debug, Error)] -pub(super) enum UvWorkspaceError { +pub(crate) enum UvWorkspaceError { #[error("Failed to invoke `uv workspace metadata`: {0}")] Invocation(#[source] std::io::Error), diff --git a/crates/ty_project/src/walk.rs b/crates/ty_project/src/walk.rs index 869742c220..0c70da7c33 100644 --- a/crates/ty_project/src/walk.rs +++ b/crates/ty_project/src/walk.rs @@ -1,5 +1,5 @@ use crate::glob::IncludeExcludeFilter; -use crate::metadata::script::script_metadata; +use crate::script::script_metadata; use crate::{Db, GlobFilterCheckMode, IncludeResult, Project}; use ruff_db::diagnostic::{Diagnostic, DiagnosticId, Severity}; use ruff_db::files::{File, system_path_to_file}; From b0e47022cfce4f3594aa26d15ea792681430b6f6 Mon Sep 17 00:00:00 2001 From: Brent Westbrook <36778786+ntBre@users.noreply.github.com> Date: Thu, 13 Aug 2026 09:39:16 -0400 Subject: [PATCH 018/371] Bump 0.16.3 (#27723) --- CHANGELOG.md | 53 +++++++++++++ Cargo.lock | 74 +++++++++---------- Cargo.toml | 72 +++++++++--------- README.md | 6 +- crates/ruff/Cargo.toml | 2 +- crates/ruff/README.md | 2 +- crates/ruff_annotate_snippets/Cargo.toml | 2 +- crates/ruff_cache/Cargo.toml | 2 +- crates/ruff_cache/README.md | 4 +- crates/ruff_db/Cargo.toml | 2 +- crates/ruff_db/README.md | 4 +- crates/ruff_diagnostics/Cargo.toml | 2 +- crates/ruff_diagnostics/README.md | 4 +- crates/ruff_formatter/Cargo.toml | 2 +- crates/ruff_formatter/README.md | 4 +- crates/ruff_graph/Cargo.toml | 2 +- crates/ruff_graph/README.md | 4 +- crates/ruff_index/Cargo.toml | 2 +- crates/ruff_index/README.md | 4 +- crates/ruff_linter/Cargo.toml | 2 +- crates/ruff_linter/README.md | 4 +- .../src/rules/pyupgrade/rules/while_one.rs | 2 +- crates/ruff_macros/Cargo.toml | 2 +- crates/ruff_macros/README.md | 4 +- crates/ruff_markdown/Cargo.toml | 2 +- crates/ruff_markdown/README.md | 4 +- crates/ruff_memory_usage/Cargo.toml | 2 +- crates/ruff_memory_usage/README.md | 4 +- crates/ruff_notebook/Cargo.toml | 2 +- crates/ruff_notebook/README.md | 4 +- crates/ruff_options_metadata/Cargo.toml | 2 +- crates/ruff_options_metadata/README.md | 4 +- crates/ruff_python_ast/Cargo.toml | 2 +- crates/ruff_python_ast/README.md | 4 +- crates/ruff_python_codegen/Cargo.toml | 2 +- crates/ruff_python_codegen/README.md | 4 +- crates/ruff_python_formatter/Cargo.toml | 2 +- crates/ruff_python_formatter/README.md | 4 +- crates/ruff_python_importer/Cargo.toml | 2 +- crates/ruff_python_importer/README.md | 4 +- crates/ruff_python_index/Cargo.toml | 2 +- crates/ruff_python_index/README.md | 4 +- crates/ruff_python_literal/Cargo.toml | 2 +- crates/ruff_python_literal/README.md | 4 +- crates/ruff_python_parser/Cargo.toml | 2 +- crates/ruff_python_parser/README.md | 4 +- crates/ruff_python_semantic/Cargo.toml | 2 +- crates/ruff_python_semantic/README.md | 4 +- crates/ruff_python_stdlib/Cargo.toml | 2 +- crates/ruff_python_stdlib/README.md | 4 +- crates/ruff_python_trivia/Cargo.toml | 2 +- crates/ruff_python_trivia/README.md | 4 +- crates/ruff_ranged_value/Cargo.toml | 2 +- crates/ruff_ranged_value/README.md | 4 +- crates/ruff_server/Cargo.toml | 2 +- crates/ruff_server/README.md | 4 +- crates/ruff_source_file/Cargo.toml | 2 +- crates/ruff_source_file/README.md | 4 +- crates/ruff_text_size/Cargo.toml | 2 +- crates/ruff_text_size/README.md | 4 +- crates/ruff_wasm/Cargo.toml | 2 +- crates/ruff_wasm/README.md | 4 +- crates/ruff_workspace/Cargo.toml | 2 +- crates/ruff_workspace/README.md | 4 +- crates/ty_combine/Cargo.toml | 2 +- crates/ty_combine/README.md | 4 +- crates/ty_module_resolver/Cargo.toml | 2 +- crates/ty_module_resolver/README.md | 4 +- crates/ty_python_core/Cargo.toml | 2 +- crates/ty_python_core/README.md | 4 +- crates/ty_python_semantic/Cargo.toml | 2 +- crates/ty_python_semantic/README.md | 4 +- crates/ty_site_packages/Cargo.toml | 2 +- crates/ty_site_packages/README.md | 4 +- crates/ty_static/Cargo.toml | 2 +- crates/ty_static/README.md | 4 +- crates/ty_vendored/Cargo.toml | 2 +- docs/formatter.md | 2 +- docs/integrations.md | 8 +- docs/tutorial.md | 2 +- pyproject.toml | 2 +- scripts/benchmarks/pyproject.toml | 2 +- uv.lock | 2 +- 83 files changed, 245 insertions(+), 192 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b871d0c132..1879a950d7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,58 @@ # Changelog +## 0.16.3 + +Released on 2026-08-13. + +### Preview features + +- \[`pylint`\] Fix false negatives on negative numbers (`PLR6104`) ([#27251](https://github.com/astral-sh/ruff/pull/27251)) +- \[`pyupgrade`\] Add rule to replace `while 1` with `while True` (`UP048`) ([#27190](https://github.com/astral-sh/ruff/pull/27190)) + +### Bug fixes + +- \[`flake8-bandit`\] Also check keyword arguments (`S602`, `S603`, `S607`, `S609`) ([#27687](https://github.com/astral-sh/ruff/pull/27687)) +- \[`pylint`\] Allow `continue` in `finally` on Python 3.8 ([#27626](https://github.com/astral-sh/ruff/pull/27626)) +- \[`pylint`\] Fix `PLE1307` false positive with bools ([#27651](https://github.com/astral-sh/ruff/pull/27651)) +- \[`pylint`\] Fix false positives and negatives with `%b` format character (`PLE1300`, `PLE1307`) ([#27560](https://github.com/astral-sh/ruff/pull/27560)) +- \[`pylint`\] Improve handling of concatenated strings (`PLE1300`) ([#27659](https://github.com/astral-sh/ruff/pull/27659)) + +### Rule changes + +- \[`numpy`\] Make `np.chararray` autofix backwards-compatible (`NPY201`) ([#27527](https://github.com/astral-sh/ruff/pull/27527)) + +### Performance + +- Enable PGO for Linux x86-64 Ruff releases ([#27570](https://github.com/astral-sh/ruff/pull/27570)) +- Enable PGO for Linux ARM64 Ruff releases ([#27574](https://github.com/astral-sh/ruff/pull/27574)) +- Enable PGO for Windows x86-64 Ruff releases ([#27573](https://github.com/astral-sh/ruff/pull/27573)) +- Enable PGO for macOS ARM64 Ruff releases ([#27572](https://github.com/astral-sh/ruff/pull/27572)) +- Reduce `Expr` size to 64 bytes ([#27591](https://github.com/astral-sh/ruff/pull/27591)) + +### CLI + +- Hyperlink rule codes in `ruff check --statistics` output ([#27646](https://github.com/astral-sh/ruff/pull/27646)) + +### Documentation + +- \[`ruff`\] Also suggest `asyncio.TaskGroup` (`RUF006`) ([#27461](https://github.com/astral-sh/ruff/pull/27461)) + +### Other changes + +- Use mimalloc v3 ([#27586](https://github.com/astral-sh/ruff/pull/27586)) + +### Contributors + +- [@Andrej730](https://github.com/Andrej730) +- [@alonfaraj](https://github.com/alonfaraj) +- [@romero-deshaw](https://github.com/romero-deshaw) +- [@Avasam](https://github.com/Avasam) +- [@tjkuson](https://github.com/tjkuson) +- [@charliermarsh](https://github.com/charliermarsh) +- [@chirizxc](https://github.com/chirizxc) +- [@saberoueslati](https://github.com/saberoueslati) +- [@MichaReiser](https://github.com/MichaReiser) + ## 0.16.2 Released on 2026-08-06. diff --git a/Cargo.lock b/Cargo.lock index 9ddea2aca0..c4d507d8c8 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3065,7 +3065,7 @@ dependencies = [ [[package]] name = "ruff" -version = "0.16.2" +version = "0.16.3" dependencies = [ "anyhow", "argfile", @@ -3129,7 +3129,7 @@ dependencies = [ [[package]] name = "ruff_annotate_snippets" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anstream 1.0.0", "anstyle", @@ -3167,7 +3167,7 @@ dependencies = [ [[package]] name = "ruff_cache" -version = "0.0.8" +version = "0.0.9" dependencies = [ "char_str", "filetime", @@ -3181,7 +3181,7 @@ dependencies = [ [[package]] name = "ruff_db" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anstyle", "arc-swap", @@ -3272,7 +3272,7 @@ dependencies = [ [[package]] name = "ruff_diagnostics" -version = "0.0.8" +version = "0.0.9" dependencies = [ "get-size2", "is-macro", @@ -3282,7 +3282,7 @@ dependencies = [ [[package]] name = "ruff_formatter" -version = "0.0.8" +version = "0.0.9" dependencies = [ "drop_bomb", "ruff_cache", @@ -3298,7 +3298,7 @@ dependencies = [ [[package]] name = "ruff_graph" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "clap", @@ -3319,7 +3319,7 @@ dependencies = [ [[package]] name = "ruff_index" -version = "0.0.8" +version = "0.0.9" dependencies = [ "get-size2", "ruff_macros", @@ -3329,7 +3329,7 @@ dependencies = [ [[package]] name = "ruff_linter" -version = "0.16.2" +version = "0.16.3" dependencies = [ "aho-corasick", "anyhow", @@ -3392,7 +3392,7 @@ dependencies = [ [[package]] name = "ruff_macros" -version = "0.0.8" +version = "0.0.9" dependencies = [ "heck", "itertools 0.15.0", @@ -3405,7 +3405,7 @@ dependencies = [ [[package]] name = "ruff_markdown" -version = "0.0.8" +version = "0.0.9" dependencies = [ "insta", "regex", @@ -3436,14 +3436,14 @@ dependencies = [ [[package]] name = "ruff_memory_usage" -version = "0.0.8" +version = "0.0.9" dependencies = [ "get-size2", ] [[package]] name = "ruff_notebook" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "rand 0.10.2", @@ -3459,14 +3459,14 @@ dependencies = [ [[package]] name = "ruff_options_metadata" -version = "0.0.8" +version = "0.0.9" dependencies = [ "serde", ] [[package]] name = "ruff_python_ast" -version = "0.0.8" +version = "0.0.9" dependencies = [ "aho-corasick", "arrayvec", @@ -3503,7 +3503,7 @@ dependencies = [ [[package]] name = "ruff_python_codegen" -version = "0.0.8" +version = "0.0.9" dependencies = [ "ruff_python_ast", "ruff_python_literal", @@ -3515,7 +3515,7 @@ dependencies = [ [[package]] name = "ruff_python_formatter" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "clap", @@ -3548,7 +3548,7 @@ dependencies = [ [[package]] name = "ruff_python_importer" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "insta", @@ -3563,7 +3563,7 @@ dependencies = [ [[package]] name = "ruff_python_index" -version = "0.0.8" +version = "0.0.9" dependencies = [ "ruff_python_ast", "ruff_python_parser", @@ -3574,7 +3574,7 @@ dependencies = [ [[package]] name = "ruff_python_literal" -version = "0.0.8" +version = "0.0.9" dependencies = [ "bitflags 2.13.1", "icu_properties", @@ -3584,7 +3584,7 @@ dependencies = [ [[package]] name = "ruff_python_parser" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "bitflags 2.13.1", @@ -3613,7 +3613,7 @@ dependencies = [ [[package]] name = "ruff_python_semantic" -version = "0.0.8" +version = "0.0.9" dependencies = [ "bitflags 2.13.1", "insta", @@ -3634,7 +3634,7 @@ dependencies = [ [[package]] name = "ruff_python_stdlib" -version = "0.0.8" +version = "0.0.9" dependencies = [ "bitflags 2.13.1", "unicode-ident", @@ -3642,7 +3642,7 @@ dependencies = [ [[package]] name = "ruff_python_trivia" -version = "0.0.8" +version = "0.0.9" dependencies = [ "itertools 0.15.0", "ruff_source_file", @@ -3663,7 +3663,7 @@ dependencies = [ [[package]] name = "ruff_ranged_value" -version = "0.0.8" +version = "0.0.9" dependencies = [ "get-size2", "ruff_db", @@ -3675,7 +3675,7 @@ dependencies = [ [[package]] name = "ruff_server" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "crossbeam", @@ -3718,7 +3718,7 @@ dependencies = [ [[package]] name = "ruff_source_file" -version = "0.0.8" +version = "0.0.9" dependencies = [ "get-size2", "memchr", @@ -3728,7 +3728,7 @@ dependencies = [ [[package]] name = "ruff_text_size" -version = "0.0.8" +version = "0.0.9" dependencies = [ "get-size2", "schemars", @@ -3739,7 +3739,7 @@ dependencies = [ [[package]] name = "ruff_wasm" -version = "0.16.2" +version = "0.16.3" dependencies = [ "console_error_panic_hook", "console_log", @@ -3766,7 +3766,7 @@ dependencies = [ [[package]] name = "ruff_workspace" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "colored", @@ -4633,7 +4633,7 @@ dependencies = [ [[package]] name = "ty_combine" -version = "0.0.8" +version = "0.0.9" dependencies = [ "ordermap", "ruff_db", @@ -4715,7 +4715,7 @@ dependencies = [ [[package]] name = "ty_module_resolver" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "camino", @@ -4789,7 +4789,7 @@ dependencies = [ [[package]] name = "ty_python_core" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "bitflags 2.13.1", @@ -4823,7 +4823,7 @@ dependencies = [ [[package]] name = "ty_python_semantic" -version = "0.0.8" +version = "0.0.9" dependencies = [ "anyhow", "bitflags 2.13.1", @@ -4918,7 +4918,7 @@ dependencies = [ [[package]] name = "ty_site_packages" -version = "0.0.8" +version = "0.0.9" dependencies = [ "camino", "colored", @@ -4939,7 +4939,7 @@ dependencies = [ [[package]] name = "ty_static" -version = "0.0.8" +version = "0.0.9" dependencies = [ "ruff_macros", ] @@ -4971,7 +4971,7 @@ dependencies = [ [[package]] name = "ty_vendored" -version = "0.0.8" +version = "0.0.9" dependencies = [ "path-slash", "ruff_db", diff --git a/Cargo.toml b/Cargo.toml index 6279cc5a5d..b11a50e23c 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -14,51 +14,51 @@ license = "MIT" [workspace.dependencies] char_str = { version = "0.0.2" } -ruff = { version = "0.16.2", path = "crates/ruff" } -ruff_annotate_snippets = { version = "0.0.8", path = "crates/ruff_annotate_snippets" } -ruff_cache = { version = "0.0.8", path = "crates/ruff_cache" } -ruff_db = { version = "0.0.8", path = "crates/ruff_db", default-features = false } -ruff_diagnostics = { version = "0.0.8", path = "crates/ruff_diagnostics" } -ruff_formatter = { version = "0.0.8", path = "crates/ruff_formatter" } -ruff_graph = { version = "0.0.8", path = "crates/ruff_graph" } -ruff_index = { version = "0.0.8", path = "crates/ruff_index" } -ruff_linter = { version = "0.16.2", path = "crates/ruff_linter" } -ruff_macros = { version = "0.0.8", path = "crates/ruff_macros" } -ruff_markdown = { version = "0.0.8", path = "crates/ruff_markdown" } -ruff_memory_usage = { version = "0.0.8", path = "crates/ruff_memory_usage" } -ruff_notebook = { version = "0.0.8", path = "crates/ruff_notebook" } -ruff_options_metadata = { version = "0.0.8", path = "crates/ruff_options_metadata" } -ruff_python_ast = { version = "0.0.8", path = "crates/ruff_python_ast" } -ruff_python_codegen = { version = "0.0.8", path = "crates/ruff_python_codegen" } -ruff_python_formatter = { version = "0.0.8", path = "crates/ruff_python_formatter" } -ruff_python_importer = { version = "0.0.8", path = "crates/ruff_python_importer" } -ruff_python_index = { version = "0.0.8", path = "crates/ruff_python_index" } -ruff_python_literal = { version = "0.0.8", path = "crates/ruff_python_literal" } -ruff_python_parser = { version = "0.0.8", path = "crates/ruff_python_parser" } -ruff_python_semantic = { version = "0.0.8", path = "crates/ruff_python_semantic" } -ruff_python_stdlib = { version = "0.0.8", path = "crates/ruff_python_stdlib" } -ruff_python_trivia = { version = "0.0.8", path = "crates/ruff_python_trivia" } -ruff_server = { version = "0.0.8", path = "crates/ruff_server" } -ruff_source_file = { version = "0.0.8", path = "crates/ruff_source_file" } +ruff = { version = "0.16.3", path = "crates/ruff" } +ruff_annotate_snippets = { version = "0.0.9", path = "crates/ruff_annotate_snippets" } +ruff_cache = { version = "0.0.9", path = "crates/ruff_cache" } +ruff_db = { version = "0.0.9", path = "crates/ruff_db", default-features = false } +ruff_diagnostics = { version = "0.0.9", path = "crates/ruff_diagnostics" } +ruff_formatter = { version = "0.0.9", path = "crates/ruff_formatter" } +ruff_graph = { version = "0.0.9", path = "crates/ruff_graph" } +ruff_index = { version = "0.0.9", path = "crates/ruff_index" } +ruff_linter = { version = "0.16.3", path = "crates/ruff_linter" } +ruff_macros = { version = "0.0.9", path = "crates/ruff_macros" } +ruff_markdown = { version = "0.0.9", path = "crates/ruff_markdown" } +ruff_memory_usage = { version = "0.0.9", path = "crates/ruff_memory_usage" } +ruff_notebook = { version = "0.0.9", path = "crates/ruff_notebook" } +ruff_options_metadata = { version = "0.0.9", path = "crates/ruff_options_metadata" } +ruff_python_ast = { version = "0.0.9", path = "crates/ruff_python_ast" } +ruff_python_codegen = { version = "0.0.9", path = "crates/ruff_python_codegen" } +ruff_python_formatter = { version = "0.0.9", path = "crates/ruff_python_formatter" } +ruff_python_importer = { version = "0.0.9", path = "crates/ruff_python_importer" } +ruff_python_index = { version = "0.0.9", path = "crates/ruff_python_index" } +ruff_python_literal = { version = "0.0.9", path = "crates/ruff_python_literal" } +ruff_python_parser = { version = "0.0.9", path = "crates/ruff_python_parser" } +ruff_python_semantic = { version = "0.0.9", path = "crates/ruff_python_semantic" } +ruff_python_stdlib = { version = "0.0.9", path = "crates/ruff_python_stdlib" } +ruff_python_trivia = { version = "0.0.9", path = "crates/ruff_python_trivia" } +ruff_server = { version = "0.0.9", path = "crates/ruff_server" } +ruff_source_file = { version = "0.0.9", path = "crates/ruff_source_file" } ruff_mdtest = { path = "crates/ruff_mdtest" } -ruff_ranged_value = { version = "0.0.8", path = "crates/ruff_ranged_value" } -ruff_text_size = { version = "0.0.8", path = "crates/ruff_text_size" } -ruff_workspace = { version = "0.0.8", path = "crates/ruff_workspace" } +ruff_ranged_value = { version = "0.0.9", path = "crates/ruff_ranged_value" } +ruff_text_size = { version = "0.0.9", path = "crates/ruff_text_size" } +ruff_workspace = { version = "0.0.9", path = "crates/ruff_workspace" } ty = { path = "crates/ty" } -ty_combine = { version = "0.0.8", path = "crates/ty_combine" } +ty_combine = { version = "0.0.9", path = "crates/ty_combine" } ty_completion_bench = { path = "crates/ty_completion_bench" } ty_completion_eval = { path = "crates/ty_completion_eval" } ty_ide = { path = "crates/ty_ide" } -ty_module_resolver = { version = "0.0.8", path = "crates/ty_module_resolver" } +ty_module_resolver = { version = "0.0.9", path = "crates/ty_module_resolver" } ty_project = { path = "crates/ty_project", default-features = false } -ty_python_semantic = { version = "0.0.8", path = "crates/ty_python_semantic" } -ty_python_core = { version = "0.0.8", path = "crates/ty_python_core" } +ty_python_semantic = { version = "0.0.9", path = "crates/ty_python_semantic" } +ty_python_core = { version = "0.0.9", path = "crates/ty_python_core" } ty_server = { path = "crates/ty_server" } -ty_site_packages = { version = "0.0.8", path = "crates/ty_site_packages" } -ty_static = { version = "0.0.8", path = "crates/ty_static" } +ty_site_packages = { version = "0.0.9", path = "crates/ty_site_packages" } +ty_static = { version = "0.0.9", path = "crates/ty_static" } ty_test = { path = "crates/ty_test" } -ty_vendored = { version = "0.0.8", path = "crates/ty_vendored" } +ty_vendored = { version = "0.0.9", path = "crates/ty_vendored" } mdtest = { path = "crates/mdtest" } diff --git a/README.md b/README.md index 2b8366f4aa..bf3f8704d2 100644 --- a/README.md +++ b/README.md @@ -152,8 +152,8 @@ curl -LsSf https://astral.sh/ruff/install.sh | sh powershell -c "irm https://astral.sh/ruff/install.ps1 | iex" # For a specific version. -curl -LsSf https://astral.sh/ruff/0.16.2/install.sh | sh -powershell -c "irm https://astral.sh/ruff/0.16.2/install.ps1 | iex" +curl -LsSf https://astral.sh/ruff/0.16.3/install.sh | sh +powershell -c "irm https://astral.sh/ruff/0.16.3/install.ps1 | iex" ``` You can also install Ruff via [Homebrew](https://formulae.brew.sh/formula/ruff), [Conda](https://anaconda.org/conda-forge/ruff), @@ -186,7 +186,7 @@ Ruff can also be used as a [pre-commit](https://pre-commit.com/) hook via [`ruff ```yaml - repo: https://github.com/astral-sh/ruff-pre-commit # Ruff version. - rev: v0.16.2 + rev: v0.16.3 hooks: # Run the linter. - id: ruff-check diff --git a/crates/ruff/Cargo.toml b/crates/ruff/Cargo.toml index 7a1f33dc66..f1d2522982 100644 --- a/crates/ruff/Cargo.toml +++ b/crates/ruff/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff" -version = "0.16.2" +version = "0.16.3" description = "An extremely fast Python linter and code formatter" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff/README.md b/crates/ruff/README.md index 2364cf7d26..3dc4398c55 100644 --- a/crates/ruff/README.md +++ b/crates/ruff/README.md @@ -10,7 +10,7 @@ See the [documentation](https://docs.astral.sh/ruff/) or This crate is the entry point to the Ruff command-line interface. The Rust API exposed here is not considered public interface. -This is version 0.16.2. The source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff). +This is version 0.16.3. The source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff). The following Ruff workspace members are also available: diff --git a/crates/ruff_annotate_snippets/Cargo.toml b/crates/ruff_annotate_snippets/Cargo.toml index 888ee9b3d9..99bad240ae 100644 --- a/crates/ruff_annotate_snippets/Cargo.toml +++ b/crates/ruff_annotate_snippets/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_annotate_snippets" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_cache/Cargo.toml b/crates/ruff_cache/Cargo.toml index 977ebef415..3bcfdd8715 100644 --- a/crates/ruff_cache/Cargo.toml +++ b/crates/ruff_cache/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_cache" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_cache/README.md b/crates/ruff_cache/README.md index 380cd45f4d..6810965cc1 100644 --- a/crates/ruff_cache/README.md +++ b/crates/ruff_cache/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_cache). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_cache). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_db/Cargo.toml b/crates/ruff_db/Cargo.toml index 52d668e9af..6024c5898d 100644 --- a/crates/ruff_db/Cargo.toml +++ b/crates/ruff_db/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_db" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_db/README.md b/crates/ruff_db/README.md index 4ef95696f3..e1bdb38bbe 100644 --- a/crates/ruff_db/README.md +++ b/crates/ruff_db/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_db). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_db). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_diagnostics/Cargo.toml b/crates/ruff_diagnostics/Cargo.toml index 5d1e392155..5e77a0df6e 100644 --- a/crates/ruff_diagnostics/Cargo.toml +++ b/crates/ruff_diagnostics/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_diagnostics" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_diagnostics/README.md b/crates/ruff_diagnostics/README.md index ecec0c72c0..54c7bd62fa 100644 --- a/crates/ruff_diagnostics/README.md +++ b/crates/ruff_diagnostics/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_diagnostics). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_diagnostics). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_formatter/Cargo.toml b/crates/ruff_formatter/Cargo.toml index f6a9b7cce8..231b11b10b 100644 --- a/crates/ruff_formatter/Cargo.toml +++ b/crates/ruff_formatter/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_formatter" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_formatter/README.md b/crates/ruff_formatter/README.md index f75009dcac..9a876529e6 100644 --- a/crates/ruff_formatter/README.md +++ b/crates/ruff_formatter/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_formatter). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_formatter). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_graph/Cargo.toml b/crates/ruff_graph/Cargo.toml index aa1bcc33d0..a86926d037 100644 --- a/crates/ruff_graph/Cargo.toml +++ b/crates/ruff_graph/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_graph" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" edition.workspace = true rust-version.workspace = true diff --git a/crates/ruff_graph/README.md b/crates/ruff_graph/README.md index ff4feff470..39867b396a 100644 --- a/crates/ruff_graph/README.md +++ b/crates/ruff_graph/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_graph). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_graph). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_index/Cargo.toml b/crates/ruff_index/Cargo.toml index 1f62f61968..d81f3aaf18 100644 --- a/crates/ruff_index/Cargo.toml +++ b/crates/ruff_index/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_index" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_index/README.md b/crates/ruff_index/README.md index faa6312359..c88e391887 100644 --- a/crates/ruff_index/README.md +++ b/crates/ruff_index/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_index). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_index). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_linter/Cargo.toml b/crates/ruff_linter/Cargo.toml index 94aebb3b7b..66624d8497 100644 --- a/crates/ruff_linter/Cargo.toml +++ b/crates/ruff_linter/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_linter" -version = "0.16.2" +version = "0.16.3" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_linter/README.md b/crates/ruff_linter/README.md index 124078abe7..22e0e97f1b 100644 --- a/crates/ruff_linter/README.md +++ b/crates/ruff_linter/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.16.2) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_linter). +This version (0.16.3) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_linter). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_linter/src/rules/pyupgrade/rules/while_one.rs b/crates/ruff_linter/src/rules/pyupgrade/rules/while_one.rs index f94e56ec3e..6c2bb51ffb 100644 --- a/crates/ruff_linter/src/rules/pyupgrade/rules/while_one.rs +++ b/crates/ruff_linter/src/rules/pyupgrade/rules/while_one.rs @@ -30,7 +30,7 @@ use crate::{AlwaysFixableViolation, Edit, Fix}; /// - [Python documentation: `while`](https://docs.python.org/3/reference/compound_stmts.html#the-while-statement) /// - [PEP 285 – Adding a bool type](https://peps.python.org/pep-0285/) #[derive(ViolationMetadata)] -#[violation_metadata(preview_since = "NEXT_RUFF_VERSION")] +#[violation_metadata(preview_since = "0.16.3")] pub(crate) struct WhileOne; impl AlwaysFixableViolation for WhileOne { diff --git a/crates/ruff_macros/Cargo.toml b/crates/ruff_macros/Cargo.toml index 13d495f8bb..1162c8eb53 100644 --- a/crates/ruff_macros/Cargo.toml +++ b/crates/ruff_macros/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_macros" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_macros/README.md b/crates/ruff_macros/README.md index ac925be274..66775787d5 100644 --- a/crates/ruff_macros/README.md +++ b/crates/ruff_macros/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_macros). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_macros). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_markdown/Cargo.toml b/crates/ruff_markdown/Cargo.toml index ec98be0bc9..4f126d2005 100644 --- a/crates/ruff_markdown/Cargo.toml +++ b/crates/ruff_markdown/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_markdown" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" edition = { workspace = true } rust-version = { workspace = true } diff --git a/crates/ruff_markdown/README.md b/crates/ruff_markdown/README.md index 9320691af7..4e4474474e 100644 --- a/crates/ruff_markdown/README.md +++ b/crates/ruff_markdown/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_markdown). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_markdown). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_memory_usage/Cargo.toml b/crates/ruff_memory_usage/Cargo.toml index 852b0e46b5..c6bea81213 100644 --- a/crates/ruff_memory_usage/Cargo.toml +++ b/crates/ruff_memory_usage/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_memory_usage" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_memory_usage/README.md b/crates/ruff_memory_usage/README.md index bdb69d27c9..77f21f3583 100644 --- a/crates/ruff_memory_usage/README.md +++ b/crates/ruff_memory_usage/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_memory_usage). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_memory_usage). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_notebook/Cargo.toml b/crates/ruff_notebook/Cargo.toml index 477f3fa14c..2f87131549 100644 --- a/crates/ruff_notebook/Cargo.toml +++ b/crates/ruff_notebook/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_notebook" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_notebook/README.md b/crates/ruff_notebook/README.md index bcf6f56751..a696cc342d 100644 --- a/crates/ruff_notebook/README.md +++ b/crates/ruff_notebook/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_notebook). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_notebook). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_options_metadata/Cargo.toml b/crates/ruff_options_metadata/Cargo.toml index 203bf5f5ce..78a4c95b53 100644 --- a/crates/ruff_options_metadata/Cargo.toml +++ b/crates/ruff_options_metadata/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_options_metadata" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_options_metadata/README.md b/crates/ruff_options_metadata/README.md index 6ae0120c4a..26b536feb3 100644 --- a/crates/ruff_options_metadata/README.md +++ b/crates/ruff_options_metadata/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_options_metadata). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_options_metadata). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_ast/Cargo.toml b/crates/ruff_python_ast/Cargo.toml index f3f65747e1..69db26a227 100644 --- a/crates/ruff_python_ast/Cargo.toml +++ b/crates/ruff_python_ast/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_ast" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_python_ast/README.md b/crates/ruff_python_ast/README.md index 77078685db..18b656a99c 100644 --- a/crates/ruff_python_ast/README.md +++ b/crates/ruff_python_ast/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_ast). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_ast). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_codegen/Cargo.toml b/crates/ruff_python_codegen/Cargo.toml index 15eb55c7fe..25a25e4028 100644 --- a/crates/ruff_python_codegen/Cargo.toml +++ b/crates/ruff_python_codegen/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_codegen" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_python_codegen/README.md b/crates/ruff_python_codegen/README.md index eb260851fa..f4a8044f85 100644 --- a/crates/ruff_python_codegen/README.md +++ b/crates/ruff_python_codegen/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_codegen). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_codegen). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_formatter/Cargo.toml b/crates/ruff_python_formatter/Cargo.toml index cfa6732104..45640f3f14 100644 --- a/crates/ruff_python_formatter/Cargo.toml +++ b/crates/ruff_python_formatter/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_formatter" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_python_formatter/README.md b/crates/ruff_python_formatter/README.md index c9a942f1bc..b226fc83da 100644 --- a/crates/ruff_python_formatter/README.md +++ b/crates/ruff_python_formatter/README.md @@ -32,8 +32,8 @@ Head to [The Ruff Formatter](https://docs.astral.sh/ruff/formatter/) for usage i This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_formatter). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_formatter). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_importer/Cargo.toml b/crates/ruff_python_importer/Cargo.toml index 98b44197b0..01178771cd 100644 --- a/crates/ruff_python_importer/Cargo.toml +++ b/crates/ruff_python_importer/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_importer" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_python_importer/README.md b/crates/ruff_python_importer/README.md index 31a810a657..176b77a6c0 100644 --- a/crates/ruff_python_importer/README.md +++ b/crates/ruff_python_importer/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_importer). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_importer). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_index/Cargo.toml b/crates/ruff_python_index/Cargo.toml index a177107337..1052e6358e 100644 --- a/crates/ruff_python_index/Cargo.toml +++ b/crates/ruff_python_index/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_index" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_python_index/README.md b/crates/ruff_python_index/README.md index dc83a2c02e..dfbfb2c564 100644 --- a/crates/ruff_python_index/README.md +++ b/crates/ruff_python_index/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_index). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_index). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_literal/Cargo.toml b/crates/ruff_python_literal/Cargo.toml index 7aa7cdd0a4..a27c252382 100644 --- a/crates/ruff_python_literal/Cargo.toml +++ b/crates/ruff_python_literal/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_literal" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = ["Charlie Marsh ", "RustPython Team"] edition = { workspace = true } diff --git a/crates/ruff_python_literal/README.md b/crates/ruff_python_literal/README.md index 94ee9e81bc..aeac41ede3 100644 --- a/crates/ruff_python_literal/README.md +++ b/crates/ruff_python_literal/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_literal). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_literal). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_parser/Cargo.toml b/crates/ruff_python_parser/Cargo.toml index 9766dbb59c..df30c68c67 100644 --- a/crates/ruff_python_parser/Cargo.toml +++ b/crates/ruff_python_parser/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_parser" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = ["Charlie Marsh ", "RustPython Team"] edition = { workspace = true } diff --git a/crates/ruff_python_parser/README.md b/crates/ruff_python_parser/README.md index 20d8066dc4..4e5827c8d2 100644 --- a/crates/ruff_python_parser/README.md +++ b/crates/ruff_python_parser/README.md @@ -19,8 +19,8 @@ Refer to the [contributing guidelines](./CONTRIBUTING.md) to get started and Git This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_parser). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_parser). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_semantic/Cargo.toml b/crates/ruff_python_semantic/Cargo.toml index c73d7ea449..475da07046 100644 --- a/crates/ruff_python_semantic/Cargo.toml +++ b/crates/ruff_python_semantic/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_semantic" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_python_semantic/README.md b/crates/ruff_python_semantic/README.md index b0275afaa4..aba0c3c710 100644 --- a/crates/ruff_python_semantic/README.md +++ b/crates/ruff_python_semantic/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_semantic). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_semantic). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_stdlib/Cargo.toml b/crates/ruff_python_stdlib/Cargo.toml index c3e59fa976..0b1c8ab46e 100644 --- a/crates/ruff_python_stdlib/Cargo.toml +++ b/crates/ruff_python_stdlib/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_stdlib" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_python_stdlib/README.md b/crates/ruff_python_stdlib/README.md index d38692821e..56af682dc1 100644 --- a/crates/ruff_python_stdlib/README.md +++ b/crates/ruff_python_stdlib/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_stdlib). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_stdlib). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_python_trivia/Cargo.toml b/crates/ruff_python_trivia/Cargo.toml index 465877e15c..b9a4d1974d 100644 --- a/crates/ruff_python_trivia/Cargo.toml +++ b/crates/ruff_python_trivia/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_python_trivia" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_python_trivia/README.md b/crates/ruff_python_trivia/README.md index b467967292..26deecdc06 100644 --- a/crates/ruff_python_trivia/README.md +++ b/crates/ruff_python_trivia/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_python_trivia). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_python_trivia). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_ranged_value/Cargo.toml b/crates/ruff_ranged_value/Cargo.toml index 8f699f3a38..2c628f2c86 100644 --- a/crates/ruff_ranged_value/Cargo.toml +++ b/crates/ruff_ranged_value/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_ranged_value" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_ranged_value/README.md b/crates/ruff_ranged_value/README.md index 068105df68..d5001d83ac 100644 --- a/crates/ruff_ranged_value/README.md +++ b/crates/ruff_ranged_value/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_ranged_value). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_ranged_value). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_server/Cargo.toml b/crates/ruff_server/Cargo.toml index 465d5b7d10..3c2b61815e 100644 --- a/crates/ruff_server/Cargo.toml +++ b/crates/ruff_server/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_server" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_server/README.md b/crates/ruff_server/README.md index fbb2f6b627..ed0aab459a 100644 --- a/crates/ruff_server/README.md +++ b/crates/ruff_server/README.md @@ -24,8 +24,8 @@ You can also join us on [**Discord**](https://discord.com/invite/astral-sh). This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_server). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_server). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_source_file/Cargo.toml b/crates/ruff_source_file/Cargo.toml index e918a959a0..5854e5c49f 100644 --- a/crates/ruff_source_file/Cargo.toml +++ b/crates/ruff_source_file/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_source_file" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_source_file/README.md b/crates/ruff_source_file/README.md index 2d61cd4f33..9893d3bc3d 100644 --- a/crates/ruff_source_file/README.md +++ b/crates/ruff_source_file/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_source_file). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_source_file). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_text_size/Cargo.toml b/crates/ruff_text_size/Cargo.toml index 3ad9d4da5f..dcbd64f4e7 100644 --- a/crates/ruff_text_size/Cargo.toml +++ b/crates/ruff_text_size/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_text_size" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_text_size/README.md b/crates/ruff_text_size/README.md index e14fefa0f7..6a564d9228 100644 --- a/crates/ruff_text_size/README.md +++ b/crates/ruff_text_size/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_text_size). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_text_size). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_wasm/Cargo.toml b/crates/ruff_wasm/Cargo.toml index 1e3b3000f7..4b31b98251 100644 --- a/crates/ruff_wasm/Cargo.toml +++ b/crates/ruff_wasm/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_wasm" -version = "0.16.2" +version = "0.16.3" description = "WebAssembly bindings for Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_wasm/README.md b/crates/ruff_wasm/README.md index 29b092ebfd..353122ec63 100644 --- a/crates/ruff_wasm/README.md +++ b/crates/ruff_wasm/README.md @@ -55,8 +55,8 @@ const formatted = workspace.format(exampleDocument); This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.16.2) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_wasm). +This version (0.16.3) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_wasm). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ruff_workspace/Cargo.toml b/crates/ruff_workspace/Cargo.toml index 2ad0e6af61..b60fb3f1ec 100644 --- a/crates/ruff_workspace/Cargo.toml +++ b/crates/ruff_workspace/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ruff_workspace" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ruff_workspace/README.md b/crates/ruff_workspace/README.md index 996e2998cb..61f6796e4c 100644 --- a/crates/ruff_workspace/README.md +++ b/crates/ruff_workspace/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ruff_workspace). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ruff_workspace). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ty_combine/Cargo.toml b/crates/ty_combine/Cargo.toml index 99241b4673..a39708b620 100644 --- a/crates/ty_combine/Cargo.toml +++ b/crates/ty_combine/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ty_combine" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" edition.workspace = true rust-version.workspace = true diff --git a/crates/ty_combine/README.md b/crates/ty_combine/README.md index de9f6647aa..e72179baef 100644 --- a/crates/ty_combine/README.md +++ b/crates/ty_combine/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ty_combine). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ty_combine). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ty_module_resolver/Cargo.toml b/crates/ty_module_resolver/Cargo.toml index dbd4c2c350..837cbe610a 100644 --- a/crates/ty_module_resolver/Cargo.toml +++ b/crates/ty_module_resolver/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ty_module_resolver" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ty_module_resolver/README.md b/crates/ty_module_resolver/README.md index 1c094d7d90..91b375cc80 100644 --- a/crates/ty_module_resolver/README.md +++ b/crates/ty_module_resolver/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ty_module_resolver). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ty_module_resolver). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ty_python_core/Cargo.toml b/crates/ty_python_core/Cargo.toml index ca9f0e95fe..4ca276094e 100644 --- a/crates/ty_python_core/Cargo.toml +++ b/crates/ty_python_core/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ty_python_core" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ty_python_core/README.md b/crates/ty_python_core/README.md index c25f0aed08..10537760eb 100644 --- a/crates/ty_python_core/README.md +++ b/crates/ty_python_core/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ty_python_core). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ty_python_core). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ty_python_semantic/Cargo.toml b/crates/ty_python_semantic/Cargo.toml index 9f2867e94b..991b92d01e 100644 --- a/crates/ty_python_semantic/Cargo.toml +++ b/crates/ty_python_semantic/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ty_python_semantic" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ty_python_semantic/README.md b/crates/ty_python_semantic/README.md index 96f36cc39e..e271499346 100644 --- a/crates/ty_python_semantic/README.md +++ b/crates/ty_python_semantic/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ty_python_semantic). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ty_python_semantic). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ty_site_packages/Cargo.toml b/crates/ty_site_packages/Cargo.toml index cb95e5ffd3..eb16be63e4 100644 --- a/crates/ty_site_packages/Cargo.toml +++ b/crates/ty_site_packages/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ty_site_packages" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/crates/ty_site_packages/README.md b/crates/ty_site_packages/README.md index c2ce5db95e..f420faff67 100644 --- a/crates/ty_site_packages/README.md +++ b/crates/ty_site_packages/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ty_site_packages). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ty_site_packages). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ty_static/Cargo.toml b/crates/ty_static/Cargo.toml index 365a9854c6..7625e02aca 100644 --- a/crates/ty_static/Cargo.toml +++ b/crates/ty_static/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ty_static" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" edition = { workspace = true } rust-version = { workspace = true } diff --git a/crates/ty_static/README.md b/crates/ty_static/README.md index 6952f9a0ff..5f1103c4b1 100644 --- a/crates/ty_static/README.md +++ b/crates/ty_static/README.md @@ -5,8 +5,8 @@ This crate is an internal component of [Ruff](https://crates.io/crates/ruff). The Rust API exposed here is unstable and will have frequent breaking changes. -This version (0.0.8) is a component of [Ruff 0.16.2](https://crates.io/crates/ruff/0.16.2). The -source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.2/crates/ty_static). +This version (0.0.9) is a component of [Ruff 0.16.3](https://crates.io/crates/ruff/0.16.3). The +source can be found [here](https://github.com/astral-sh/ruff/blob/0.16.3/crates/ty_static). See Ruff's [crate versioning policy](https://docs.astral.sh/ruff/versioning/#crate-versioning) for details on versioning. diff --git a/crates/ty_vendored/Cargo.toml b/crates/ty_vendored/Cargo.toml index ef78599682..feb6dcc376 100644 --- a/crates/ty_vendored/Cargo.toml +++ b/crates/ty_vendored/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "ty_vendored" -version = "0.0.8" +version = "0.0.9" description = "This is an internal component crate of Ruff" authors = { workspace = true } edition = { workspace = true } diff --git a/docs/formatter.md b/docs/formatter.md index f1dbc9c726..e70761de3d 100644 --- a/docs/formatter.md +++ b/docs/formatter.md @@ -303,7 +303,7 @@ support needs to be explicitly included by adding it to `types_or`: ```yaml title=".pre-commit-config.yaml" repos: - repo: https://github.com/astral-sh/ruff-pre-commit - rev: v0.16.2 + rev: v0.16.3 hooks: - id: ruff-format types_or: [python, pyi, jupyter, markdown] diff --git a/docs/integrations.md b/docs/integrations.md index c4b1fdb036..7dd5c1951d 100644 --- a/docs/integrations.md +++ b/docs/integrations.md @@ -80,7 +80,7 @@ You can add the following configuration to `.gitlab-ci.yml` to run a `ruff forma stage: build interruptible: true image: - name: ghcr.io/astral-sh/ruff:0.16.2-alpine + name: ghcr.io/astral-sh/ruff:0.16.3-alpine before_script: - cd $CI_PROJECT_DIR - ruff --version @@ -106,7 +106,7 @@ Ruff can be used as a [pre-commit](https://pre-commit.com) hook via [`ruff-pre-c ```yaml - repo: https://github.com/astral-sh/ruff-pre-commit # Ruff version. - rev: v0.16.2 + rev: v0.16.3 hooks: # Run the linter. - id: ruff-check @@ -119,7 +119,7 @@ To enable lint fixes, add the `--fix` argument to the lint hook: ```yaml - repo: https://github.com/astral-sh/ruff-pre-commit # Ruff version. - rev: v0.16.2 + rev: v0.16.3 hooks: # Run the linter. - id: ruff-check @@ -133,7 +133,7 @@ To avoid running on Jupyter Notebooks, remove `jupyter` from the list of allowed ```yaml - repo: https://github.com/astral-sh/ruff-pre-commit # Ruff version. - rev: v0.16.2 + rev: v0.16.3 hooks: # Run the linter. - id: ruff-check diff --git a/docs/tutorial.md b/docs/tutorial.md index c58e24ec8f..da45c0f454 100644 --- a/docs/tutorial.md +++ b/docs/tutorial.md @@ -372,7 +372,7 @@ This tutorial has focused on Ruff's command-line interface, but Ruff can also be ```yaml - repo: https://github.com/astral-sh/ruff-pre-commit # Ruff version. - rev: v0.16.2 + rev: v0.16.3 hooks: # Run the linter. - id: ruff-check diff --git a/pyproject.toml b/pyproject.toml index c0816c28e1..5595f75bb0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,7 +4,7 @@ build-backend = "maturin" [project] name = "ruff" -version = "0.16.2" +version = "0.16.3" description = "An extremely fast Python linter and code formatter, written in Rust." authors = [{ name = "Astral Software Inc.", email = "hey@astral.sh" }] readme = "README.md" diff --git a/scripts/benchmarks/pyproject.toml b/scripts/benchmarks/pyproject.toml index 6a1171ac92..059e341502 100644 --- a/scripts/benchmarks/pyproject.toml +++ b/scripts/benchmarks/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "scripts" -version = "0.16.2" +version = "0.16.3" description = "" authors = ["Charles Marsh "] diff --git a/uv.lock b/uv.lock index c789281c0d..489bb3460d 100644 --- a/uv.lock +++ b/uv.lock @@ -623,7 +623,7 @@ wheels = [ [[package]] name = "ruff" -version = "0.16.2" +version = "0.16.3" source = { editable = "." } [package.dev-dependencies] From 27e91ed05f004458f34194ae50e630c9ec36ef92 Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Thu, 13 Aug 2026 09:44:29 -0400 Subject: [PATCH 019/371] [ty] Avoid treating augmented assignments as attribute definitions (#27633) ## Summary Previously, we treated augmented assignments as ordinary bindings when inferring implicit attributes. However, `self.value += 1` first reads `self.value`, so it cannot establish an otherwise missing attribute: ```python class Counter: def __init__(self) -> None: self.value = 0 def increment(self) -> None: self.value += 1 class UninitializedCounter: def increment(self) -> None: self.value += 1 # error: [unresolved-attribute] ``` We now distinguish assignments that establish an attribute from augmented assignments that require an existing value. Class and instance member lookup collects augmented assignments while traversing the MRO and incorporates their inferred results only after resolving an independent attribute binding. This preserves missing-attribute diagnostics and allows augmented assignments to contribute to inferred attribute types. --- .../resources/mdtest/assignment/augmented.md | 24 +- .../resources/mdtest/attributes.md | 484 +++++++++++++++++- .../resources/mdtest/protocols.md | 8 +- crates/ty_python_semantic/src/types/class.rs | 229 ++++++++- .../src/types/class/static_literal.rs | 123 ++++- crates/ty_python_semantic/src/types/narrow.rs | 16 +- 6 files changed, 829 insertions(+), 55 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/assignment/augmented.md b/crates/ty_python_semantic/resources/mdtest/assignment/augmented.md index 9fde35b37a..592c49e306 100644 --- a/crates/ty_python_semantic/resources/mdtest/assignment/augmented.md +++ b/crates/ty_python_semantic/resources/mdtest/assignment/augmented.md @@ -426,23 +426,24 @@ def update(counter: Counter | None) -> None: counter.count += 1 ``` -An augmented assignment should not define an otherwise missing instance attribute, because it must -read an existing value before writing its result. We currently treat it like an ordinary -self-referential assignment instead. +An augmented assignment cannot define an otherwise missing instance attribute, because it must read +an existing value before writing its result. ```py class UninitializedCounter: def increment(self) -> None: - # TODO: Report an unresolved-attribute error instead of implicitly defining the attribute. + # error: [unresolved-attribute] self.value += 1 -reveal_type(UninitializedCounter().value) # revealed: Divergent +# error: [unresolved-attribute] +reveal_type(UninitializedCounter().value) # revealed: Unknown ``` ## Dynamically provided attributes -A dynamic attribute hook can provide the initial value read by an augmented assignment. The -assignment currently infers a divergent attribute type instead of preserving the hook's return type. +A dynamic attribute hook can provide the initial value read by an augmented assignment. Ordinary +attribute lookup preserves the hook's return type, but the resulting assignment is not yet +recognized as establishing instance storage. ```py class DynamicCounter: @@ -450,10 +451,11 @@ class DynamicCounter: return 0 def increment(self) -> None: + # TODO: Recognize the instance attribute established after reading from a dynamic hook. + # error: [unresolved-attribute] self.value += 1 -# TODO: Infer `int` from the dynamic attribute hook. -reveal_type(DynamicCounter().value) # revealed: Divergent +reveal_type(DynamicCounter().value) # revealed: int ``` The same behavior applies when the attribute is provided by `__getattribute__`. @@ -464,9 +466,11 @@ class InterceptedCounter: return 0 def increment(self) -> None: + # TODO: Recognize the instance attribute established after reading from a dynamic hook. + # error: [unresolved-attribute] self.value += 1 -reveal_type(InterceptedCounter().value) # revealed: Divergent +reveal_type(InterceptedCounter().value) # revealed: int ``` ## Class-level defaults in diamond inheritance diff --git a/crates/ty_python_semantic/resources/mdtest/attributes.md b/crates/ty_python_semantic/resources/mdtest/attributes.md index 07c14d19c7..fc45a569fe 100644 --- a/crates/ty_python_semantic/resources/mdtest/attributes.md +++ b/crates/ty_python_semantic/resources/mdtest/attributes.md @@ -259,8 +259,8 @@ reveal_type(c_instance.b) # revealed: int #### Augmented assignments -An augmented assignment contributes its result to the inferred type of an unannotated instance -attribute. +An augmented assignment contributes its result to an instance attribute that already has an +independent binding. ```py class Weird: @@ -276,6 +276,375 @@ class C: reveal_type(C().w) # revealed: Weird | str ``` +#### Augmented assignments with stable recursive inference + +An independently initialized buffer updated from multiple methods must retain its concrete type, +even when augmented assignments recursively look up that attribute. + +```toml +[rules] +unsound-return-statement = "error" +``` + +```py +class Buffer: + def __init__(self) -> None: + self.reset() + + def append(self, value: bytes) -> None: + if value: + self.content += b"," + self.content += value + + def reset(self) -> None: + self.content = bytearray() + + def finish(self) -> bytearray: + self.content += b"]" + return self.content + +reveal_type(Buffer().content) # revealed: bytearray +``` + +The same cycle recovery also preserves a concrete integer attribute. + +```py +class Counter: + def __init__(self) -> None: + self.reset() + + def increment(self, value: int) -> None: + self.value += value + + def reset(self) -> None: + self.value = 0 + + def finish(self) -> int: + self.value += 1 + return self.value + +reveal_type(Counter().value) # revealed: int +``` + +#### Augmented assignments to narrowed optional attributes + +Once an optional attribute has been narrowed to its non-`None` value, augmented assignments must not +introduce `Unknown` into its instance attribute type. + +```toml +[rules] +unsound-return-statement = "error" +``` + +```py +class Counter: + def __init__(self, value: int | None) -> None: + self.value = value + + def update(self, decrement: bool) -> None: + if self.value is None: + return + + if decrement: + self.value -= 1 + else: + self.value += 1 + + def current(self) -> int | None: + return self.value + +reveal_type(Counter(0).value) # revealed: int | None +``` + +#### Augmented assignments to unannotated class-level defaults + +An unannotated class-level default can supply the initial value read by an augmented assignment. The +instance attribute can then contain either the original value or the result of the operation. + +```py +class After: + def __iadd__(self, other: int) -> "After": + return self + +class Before: + def __iadd__(self, other: int) -> After: + return After() + +class C: + value = Before() + + def update(self) -> None: + self.value += 1 # error: [invalid-assignment] + +reveal_type(C().value) # revealed: Before | After +``` + +#### Augmented assignments to conditionally defined class-level defaults + +A conditional class default must not hide the dynamic fallback used when that default is absent. + +```py +class After: + def __iadd__(self, other: int) -> "After": + return self + +class Before: + def __iadd__(self, other: int) -> After: + return After() + +class FallbackAfter: + def __iadd__(self, other: int) -> "FallbackAfter": + return self + +class Fallback: + def __iadd__(self, other: int) -> FallbackAfter: + return FallbackAfter() + +def flag() -> bool: + return True + +class C: + if flag(): + value = Before() + + def __getattr__(self, name: str) -> Fallback: + return Fallback() + + def update(self) -> None: + # error: [invalid-assignment] + # error: [possibly-missing-attribute] + self.value += 1 + +reveal_type(C().value) # revealed: Before | After | FallbackAfter | Fallback +``` + +#### Augmented assignments with expanding generic results + +An augmented assignment can repeatedly expand a generic attribute's type arguments. Inference must +still converge when the initial value comes from a class-level default. + +```py +from __future__ import annotations + +from typing import Generic, TypeVar + +T = TypeVar("T") + +class Grow(Generic[T]): + def __iadd__(self, other: int) -> Grow[list[T]]: + raise NotImplementedError + +class Counter: + value = Grow[int]() + + def update(self) -> None: + self.value += 1 # error: [invalid-assignment] + +reveal_type(Counter().value) # revealed: Grow[int] | Grow[list[int]] +``` + +An independently initialized attribute must use the same bounded cycle recovery. + +```py +class InitializedCounter: + def __init__(self) -> None: + self.value = Grow[int]() + + def update(self) -> None: + self.value += 1 # error: [invalid-assignment] + +reveal_type(InitializedCounter().value) # revealed: Grow[int] | Grow[list[int]] +``` + +#### Augmented assignments with expanding tuple results + +Repeatedly nesting an independently initialized tuple must converge instead of exhausting Salsa's +cycle-iteration limit. + +```py +class C: + def __init__(self) -> None: + self.value = (1,) + + def update(self) -> None: + self.value += (self.value,) + +reveal_type(C().value) # revealed: tuple[int] | tuple[Divergent, ...] +``` + +#### Augmented assignments to inherited instance attributes + +An instance attribute established by a superclass can supply the initial value read by an augmented +assignment in a subclass. + +```py +class After: + def __iadd__(self, other: int) -> "After": + return self + +class Before: + def __iadd__(self, other: int) -> After: + return After() + +class Base: + def __init__(self) -> None: + self.value = Before() + +class Child(Base): + def update(self) -> None: + self.value += 1 + +reveal_type(Child().value) # revealed: Before | After +``` + +#### Augmented assignments preserve inherited instance bindings beneath class defaults + +A superclass initializer writes instance storage even when a subclass defines a class-level default +with the same name. Both initial values and their augmented-assignment results remain possible. + +```py +class AfterA: + def __iadd__(self, other: int) -> "AfterA": + return self + +class AfterB: + def __iadd__(self, other: int) -> "AfterB": + return self + +class BeforeA: + def __iadd__(self, other: int) -> AfterA: + return AfterA() + +class BeforeB: + def __iadd__(self, other: int) -> AfterB: + return AfterB() + +class Base: + def __init__(self) -> None: + self.value = BeforeA() + +class Child(Base): + value = BeforeB() + + def update(self) -> None: + self.value += 1 # error: [invalid-assignment] + +reveal_type(Child().value) # revealed: BeforeB | AfterB | AfterA | BeforeA +``` + +#### Augmented assignments preserve subclass attribute bindings + +An augmented assignment inherited from an intermediate class must not discard instance attributes +that subclasses establish independently. + +```py +from typing import Any + +class Base: + value = 0 + +class Middle(Base): + def increment(self) -> None: + self.value += 1 + +class Child(Middle): + def set(self, value: Any) -> None: + self.value = value + +reveal_type(Child().value) # revealed: int | Any +``` + +An untyped subclass binding is likewise preserved. + +```py +class UnknownChild(Middle): + def set(self, value) -> None: + self.value = value + +reveal_type(UnknownChild().value) # revealed: int | Unknown +``` + +An explicitly annotated class-level default also preserves subclass bindings. + +```py +class AnnotatedBase: + value: int = 0 + +class AnnotatedMiddle(AnnotatedBase): + def increment(self) -> None: + self.value += 1 + +class AnnotatedChild(AnnotatedMiddle): + def set(self, value: Any) -> None: + self.value = value + +reveal_type(AnnotatedChild().value) # revealed: int | Any +``` + +#### Augmented assignments with gradual operands + +An augmented assignment with an `Any` or untyped operand contributes its gradual result to the +inferred instance attribute. + +```py +from typing import Any + +class C: + def __init__(self, any_value: Any, unknown_value) -> None: + self.from_any = 0.0 + self.from_any += any_value + + self.from_unknown = 0 + self.from_unknown += unknown_value + +reveal_type(C(0, 0).from_any) # revealed: float | Any +reveal_type(C(0, 0).from_unknown) # revealed: int | Unknown +``` + +#### Augmented assignments to possible data descriptors + +An augmented assignment to a data descriptor passes its result to `__set__` rather than creating +instance storage. When a class default might be a descriptor, preserve the existing attribute types +without exposing the descriptor's write-only result. + +```py +class After: + def __iadd__(self, other: int) -> "After": + return self + +class Before: + def __iadd__(self, other: int) -> After: + return After() + +class DescriptorAfter: + def __iadd__(self, other: int) -> "DescriptorAfter": + return self + +class DescriptorValue: + def __iadd__(self, other: int) -> DescriptorAfter: + return DescriptorAfter() + +class Descriptor: + def __get__(self, instance: object, owner: type[object]) -> DescriptorValue: + return DescriptorValue() + + def __set__(self, instance: object, value: DescriptorAfter) -> None: ... + +def flag() -> bool: + return True + +class C: + value = Descriptor() if flag() else Before() + + def update(self) -> None: + # error: [invalid-assignment] + # error: [invalid-assignment] + self.value += 1 + +# TODO: Include `After` from the non-descriptor branch without including `DescriptorAfter`. +reveal_type(C().value) # revealed: DescriptorValue | Before +``` + #### Nested augmented assignments after narrowing Augmented assignments to nested attributes (e.g., `self.inner.value += ...`) should work correctly @@ -834,6 +1203,53 @@ reveal_type(c_instance.pure_class_variable) # revealed: str c_instance.pure_class_variable = "value set on instance" ``` +#### Augmented assignments in class methods + +A classmethod can establish an implicit class variable and then augment it with an operation that +changes its type. Both the initial value and the augmented result remain possible. + +```py +class After: ... + +class Before: + def __iadd__(self, other: int) -> After: + return After() + +class Example: + @classmethod + def update(cls) -> None: + cls.value = Before() + cls.value += 1 + +reveal_type(Example.value) # revealed: Before | After +``` + +#### Augmented assignments to inherited class variables + +A classmethod can read an inherited class variable before storing its augmented result on the +subclass. Class-member lookup must preserve the deferred assignment until it finds that inherited +value. + +```py +class After: + def __iadd__(self, other: int) -> "After": + return self + +class Before: + def __iadd__(self, other: int) -> After: + return After() + +class Parent: + value = Before() + +class Child(Parent): + @classmethod + def update(cls) -> None: + cls.value += 1 + +reveal_type(Child.value) # revealed: Before | After +``` + ### Instance variables with class-level default values These are instance attributes, but the fact that we can see that they have a binding (not a @@ -871,6 +1287,29 @@ reveal_type(C.variable_with_class_default1) # revealed: Literal["overwritten on reveal_type(c_instance.variable_with_class_default1) # revealed: Literal["value set on instance"] ``` +#### Augmented assignments to overriding class-level defaults + +An unannotated class-level default supplies the initial value for an augmented assignment, even when +another branch of a diamond declares a wider instance attribute. + +```py +class Base: + value: int | None = None + +class First(Base): ... + +class Second(Base): + value: int | None + +class Child(First, Second): + value = 1 + + def update(self) -> None: + self.value |= 2 + +reveal_type(Child().value) # revealed: int +``` + #### Descriptor attributes as class variables Whether they are explicitly qualified as `ClassVar`, or just have a class level default, we treat @@ -1571,6 +2010,47 @@ class UsesGeneratedDescriptor(metaclass=DescriptorMeta): reveal_type(UsesGeneratedDescriptor().generated_descriptor) # revealed: Literal["descriptor"] ``` +An augmented assignment to a data descriptor on a metaclass calls the descriptor's `__set__` method. +It does not store an attribute on the class, so the attribute is unavailable on instances. + +```py +class AugmentedDescriptor: + def __get__(self, instance: object, owner: type[object]) -> int: + return 1 + + def __set__(self, instance: object, value: int) -> None: ... + +class AugmentedDescriptorMeta(type): + descriptor_value = AugmentedDescriptor() + + def update(cls) -> None: + cls.descriptor_value += 1 + +class UsesAugmentedDescriptor(metaclass=AugmentedDescriptorMeta): ... + +# error: [unresolved-attribute] +reveal_type(UsesAugmentedDescriptor().descriptor_value) # revealed: Unknown +``` + +A metaclass default that might be a data descriptor likewise must not expose a class attribute on +constructed instances. + +```py +def choose_descriptor() -> bool: + return True + +class MaybeAugmentedDescriptorMeta(type): + descriptor_value = AugmentedDescriptor() if choose_descriptor() else 1 + + def update(cls) -> None: + cls.descriptor_value += 1 # error: [invalid-assignment] + +class UsesMaybeAugmentedDescriptor(metaclass=MaybeAugmentedDescriptorMeta): ... + +# error: [unresolved-attribute] +reveal_type(UsesMaybeAugmentedDescriptor().descriptor_value) # revealed: Unknown +``` + When a metaclass declaration uses a union, only the data descriptors in that union take precedence over an instance attribute. A non-descriptor member and the instance attribute both remain possible: diff --git a/crates/ty_python_semantic/resources/mdtest/protocols.md b/crates/ty_python_semantic/resources/mdtest/protocols.md index a04242d07a..e57b740e07 100644 --- a/crates/ty_python_semantic/resources/mdtest/protocols.md +++ b/crates/ty_python_semantic/resources/mdtest/protocols.md @@ -1046,7 +1046,8 @@ class AnySelf(Protocol): ``` Assignments in a comprehension and augmented assignments are also writes to the instance. -`__getattr__` provides the read side of `+=` below, so that case tests only the write: +`__getattr__` provides the read side of `+=` below, although the write is not yet recognized as +establishing an instance attribute: ```py class AssignmentForms(Protocol): @@ -1057,14 +1058,15 @@ class AssignmentForms(Protocol): [None for self.from_comprehension in [1]] # error: [ambiguous-protocol-member] def augmented_assignment(self) -> None: + # error: [unresolved-attribute] self.augmented += 1 # snapshot: ambiguous-protocol-member ``` ```snapshot warning[ambiguous-protocol-member]: Cannot assign to an undeclared attribute in a protocol method - --> src/mdtest_snippet.py:326:9 + --> src/mdtest_snippet.py:327:9 | -326 | self.augmented += 1 # snapshot: ambiguous-protocol-member +327 | self.augmented += 1 # snapshot: ambiguous-protocol-member | ^^^^^^^^^^^^^^ `augmented` is not declared as a protocol member info: Assigning to an undeclared attribute in a protocol method leads to an ambiguous interface --> src/mdtest_snippet.py:318:7 diff --git a/crates/ty_python_semantic/src/types/class.rs b/crates/ty_python_semantic/src/types/class.rs index 3c75c17777..08c587b92a 100644 --- a/crates/ty_python_semantic/src/types/class.rs +++ b/crates/ty_python_semantic/src/types/class.rs @@ -10,6 +10,7 @@ use self::named_tuple::synthesize_namedtuple_class_member; pub(super) use self::named_tuple::{ DynamicNamedTupleAnchor, DynamicNamedTupleLiteral, NamedTupleField, NamedTupleSpec, }; +use self::static_literal::{AugmentedBindings, ImplicitAttribute}; pub(crate) use self::static_literal::{ ExpandedClassBaseEntry, FrozenDataclassDispatch, StaticClassLiteral, expanded_class_base_entries, @@ -31,6 +32,7 @@ use crate::types::constraints::{ use crate::types::enums::enum_metadata; use crate::types::function::{AbstractMethodKind, DataclassTransformerParams}; use crate::types::generics::{GenericContext, Specialization, walk_specialization}; +use crate::types::infer::infer_definition_types; use crate::types::known_instance::DeprecatedInstance; use crate::types::member::Member; use crate::types::relation::{ @@ -2199,6 +2201,49 @@ impl<'db> ClassType<'db> { } } + /// Pair an ordinary member lookup with augmented assignments that first read their target. + /// + /// ```python + /// class Counter: + /// value = 0 + /// + /// def increment(self): + /// self.value += 1 + /// + /// @classmethod + /// def increment_class(cls): + /// cls.value += 1 + /// ``` + /// + /// MRO lookup can infer either assignment only after locating an existing `value`. If ordinary + /// lookup suppressed an implicit attribute, such as a generated `NamedTuple` field, its writes + /// must remain suppressed too. + fn member_with_augmented_bindings( + self, + db: &'db dyn Db, + member: Member<'db>, + name: &str, + target_method_decorator: MethodDecorator, + ) -> ImplicitAttribute<'db> { + let augmented_bindings = self + .static_class_literal(db) + .map(|(class, _)| { + StaticClassLiteral::implicit_attribute_bindings( + db, + class.body_scope(db), + name, + target_method_decorator, + ) + }) + .filter(|implicit| member.is_undefined() == implicit.member.is_undefined()) + .and_then(|implicit| implicit.augmented_bindings); + + ImplicitAttribute { + member, + augmented_bindings, + } + } + /// Return a callable type (or union of callable types) that represents the callable /// constructor signature of this class. pub(super) fn into_callable(self, db: &'db dyn Db) -> CallableTypes<'db> { @@ -2746,6 +2791,60 @@ impl<'db, I: Iterator>> MroLookup<'db, I> { } } + /// Infer augmented-assignment results after finding the existing attribute they read. + /// + /// ```python + /// class Counter: + /// def __init__(self): + /// self.value = (1,) + /// + /// def update(self): + /// self.value += (self.value,) + /// ``` + /// + /// Inferring these bindings earlier would recursively look up the same attribute and + /// allow an augmented assignment to incorrectly establish an otherwise missing attribute. + /// Once recursive inference produces a concrete result, top-level cycle placeholders do not + /// represent additional runtime values. Other inferred alternatives remain intact, as do + /// nested placeholders in genuinely expanding recursive types such as the tuple above. + fn infer_augmented_bindings( + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + bindings: &[(ClassType<'db>, AugmentedBindings<'db>)], + ) -> (Type<'db>, Provenance<'db>) { + let mut union = UnionBuilder::new(db, env); + let mut provenance = Provenance::Unknown; + + for (class, bindings) in bindings { + let (_, specialization) = class.class_literal_and_specialization(db); + + for definition in bindings.definitions(db) { + let inferred_ty = infer_definition_types(db, *definition) + .binding_type(*definition) + .apply_optional_specialization(db, specialization); + union = union.add(inferred_ty); + provenance = provenance.or(Provenance::SingleDefinition(*definition)); + } + } + + let inferred_ty = union.build().promote(db, env).promote_singletons(db, env); + let inferred_ty = if let Some(elements) = + inferred_ty.as_union().map(|union| union.elements(db)) + && elements.iter().any(Type::is_divergent) + && elements.iter().any(|ty| !ty.is_divergent()) + { + UnionType::from_elements( + db, + env, + elements.iter().copied().filter(|ty| !ty.is_divergent()), + ) + } else { + inferred_ty + }; + + (inferred_ty, provenance) + } + /// Look up a class member by iterating through the MRO. /// /// Parameters: @@ -2772,6 +2871,7 @@ impl<'db, I: Iterator>> MroLookup<'db, I> { let mut dynamic_type: Option> = None; let mut lookup_result: LookupResult<'db> = Err(LookupError::Undefined(TypeQualifiers::empty())); + let mut pending_augmented_bindings = Vec::new(); for superclass in self.mro_iter { match superclass { @@ -2809,14 +2909,40 @@ impl<'db, I: Iterator>> MroLookup<'db, I> { continue; } + let implicit = class.member_with_augmented_bindings( + db, + class.own_class_member(db, &self.env, inherited_generic_context, name), + name, + MethodDecorator::ClassMethod, + ); + if let Some(bindings) = implicit.augmented_bindings { + pending_augmented_bindings.push((class, bindings)); + } + + let mut member = implicit.member.inner; + if let Place::Defined(defined) = &mut member.place + && !pending_augmented_bindings.is_empty() + { + if !defined.origin.is_declared() { + let (inferred_ty, inferred_provenance) = Self::infer_augmented_bindings( + db, + &self.env, + &pending_augmented_bindings, + ); + defined.ty = UnionType::from_two_elements( + db, + &self.env, + defined.ty, + inferred_ty, + ); + defined.provenance = defined.provenance.or(inferred_provenance); + } + + pending_augmented_bindings.clear(); + } + lookup_result = lookup_result.or_else(|lookup_error| { - lookup_error.or_fall_back_to( - db, - &self.env, - class - .own_class_member(db, &self.env, inherited_generic_context, name) - .inner, - ) + lookup_error.or_fall_back_to(db, &self.env, member) }); } ClassBase::TypedDict(module) => { @@ -2847,8 +2973,9 @@ impl<'db, I: Iterator>> MroLookup<'db, I> { let db = self.db; let mut union = UnionBuilder::new(db, &self.env); let mut union_qualifiers = TypeQualifiers::empty(); - let mut is_definitely_bound = false; + let mut definitely_bound_member: Option> = None; let mut provenance = Provenance::Unknown; + let mut pending_augmented_bindings = Vec::new(); for superclass in self.mro_iter { match superclass { @@ -2862,6 +2989,16 @@ impl<'db, I: Iterator>> MroLookup<'db, I> { return InstanceMemberResult::Done(PlaceAndQualifiers::unbound()); } ClassBase::Class(class) => { + let implicit = class.member_with_augmented_bindings( + db, + class.own_instance_member(db, &self.env, name), + name, + MethodDecorator::None, + ); + if let Some(bindings) = implicit.augmented_bindings { + pending_augmented_bindings.push((class, bindings)); + } + if let member @ PlaceAndQualifiers { place: Place::Defined(DefinedPlace { @@ -2872,16 +3009,28 @@ impl<'db, I: Iterator>> MroLookup<'db, I> { .. }), qualifiers, - } = class.own_instance_member(db, &self.env, name).inner + } = implicit.member.inner { if boundness == Definedness::AlwaysDefined { if origin.is_declared() { + if definitely_bound_member.is_some_and(|member| { + !member + .qualifiers + .contains(TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE) + }) && !qualifiers + .contains(TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE) + { + // An overriding class default shadows inherited declarations, + // but inherited instance assignments must still be collected. + continue; + } + // We found a definitely-declared attribute. Discard possibly collected // inferred types from subclasses and return the declared type. return InstanceMemberResult::Done(member); } - is_definitely_bound = true; + definitely_bound_member = Some(member); } // If the attribute is not definitely declared on this class, keep looking @@ -2893,6 +3042,64 @@ impl<'db, I: Iterator>> MroLookup<'db, I> { // TODO: We could raise a diagnostic here if there are conflicting type // qualifiers union_qualifiers |= qualifiers; + + if !pending_augmented_bindings.is_empty() { + let (inferred_ty, inferred_provenance) = Self::infer_augmented_bindings( + db, + &self.env, + &pending_augmented_bindings, + ); + union = union.add(inferred_ty); + provenance = provenance.or(inferred_provenance); + union_qualifiers |= TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE; + pending_augmented_bindings.clear(); + } + } + + if !pending_augmented_bindings.is_empty() + && let class_member @ Member { + inner: + PlaceAndQualifiers { + place: + Place::Defined(DefinedPlace { + ty: class_member_ty, + origin, + definedness: class_member_definedness, + provenance: class_member_provenance, + .. + }), + .. + }, + } = class.own_class_member(db, &self.env, None, name) + { + if !class_member_ty.is_definitely_non_data_descriptor(db, &self.env) { + pending_augmented_bindings.clear(); + continue; + } + + if origin.is_declared() { + if union.is_empty() { + return InstanceMemberResult::Done(class_member.inner); + } + + union = union.add(class_member_ty); + provenance = provenance.or(class_member_provenance); + union_qualifiers |= class_member.inner.qualifiers; + } else { + let (inferred_ty, inferred_provenance) = Self::infer_augmented_bindings( + db, + &self.env, + &pending_augmented_bindings, + ); + union = union.add(inferred_ty); + provenance = provenance.or(inferred_provenance); + union_qualifiers |= TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE; + } + + pending_augmented_bindings.clear(); + if class_member_definedness == Definedness::AlwaysDefined { + definitely_bound_member = Some(class_member.inner); + } } } ClassBase::TypedDict(_) => { @@ -2904,7 +3111,7 @@ impl<'db, I: Iterator>> MroLookup<'db, I> { let result = if union.is_empty() { Place::Undefined.with_qualifiers(TypeQualifiers::empty()) } else { - let boundness = if is_definitely_bound { + let boundness = if definitely_bound_member.is_some() { Definedness::AlwaysDefined } else { Definedness::PossiblyUndefined diff --git a/crates/ty_python_semantic/src/types/class/static_literal.rs b/crates/ty_python_semantic/src/types/class/static_literal.rs index c6b23967dd..8dd37c408e 100644 --- a/crates/ty_python_semantic/src/types/class/static_literal.rs +++ b/crates/ty_python_semantic/src/types/class/static_literal.rs @@ -48,7 +48,7 @@ use crate::{ is_implicit_staticmethod, }, generics::Specialization, - infer::{infer_definition_types, infer_unpack_types}, + infer::infer_unpack_types, infer_expression_type, inferred_declaration, known_instance::DeprecatedInstance, member::{Member, class_member}, @@ -2848,12 +2848,35 @@ impl<'db> StaticClassLiteral<'db> { name: &str, target_method_decorator: MethodDecorator, ) -> Member<'db> { + Self::implicit_attribute_bindings(db, class_body_scope, name, target_method_decorator) + .member + } + + /// Separate assignments that establish an attribute from assignments that must first read it. + /// + /// ```python + /// class Counter: + /// def increment(self): + /// self.value += 1 + /// ``` + /// + /// Here, `value` remains undefined until MRO lookup finds an independent class or instance + /// attribute. The same rule applies to `cls.value` in a classmethod. + pub(super) fn implicit_attribute_bindings( + db: &'db dyn Db, + class_body_scope: ScopeId<'db>, + name: &str, + target_method_decorator: MethodDecorator, + ) -> ImplicitAttribute<'db> { // Collect names in a tracked query so unrelated edits can preserve dependent member // lookups, and avoid retaining query entries for names that no method can define. let names = implicit_attribute_names(db, class_body_scope); let Ok(name_index) = names.binary_search_by(|candidate| candidate.as_str().cmp(name)) else { - return Member::unbound(); + return ImplicitAttribute { + member: Member::unbound(), + augmented_bindings: None, + }; }; Self::implicit_attribute_inner( @@ -2870,15 +2893,18 @@ impl<'db> StaticClassLiteral<'db> { #[salsa::tracked( returns(copy), cycle_fn=implicit_attribute_cycle_recover, - cycle_initial=|_, id, _| Member { - inner: Place::bound(Type::divergent(id)).into(), + cycle_initial=|_, id, _| ImplicitAttribute { + member: Member { + inner: Place::bound(Type::divergent(id)).into(), + }, + augmented_bindings: None, }, heap_size=ruff_memory_usage::heap_size, )] fn implicit_attribute_inner( db: &'db dyn Db, attribute: ImplicitAttributeName<'db>, - ) -> Member<'db> { + ) -> ImplicitAttribute<'db> { let class_body_scope = attribute.class_body_scope(db); let name = attribute.name(db).as_str(); let target_method_decorator = attribute.target_method_decorator(db); @@ -2893,6 +2919,7 @@ impl<'db> StaticClassLiteral<'db> { let mut qualifiers = TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE; let mut is_attribute_bound = false; + let mut augmented_bindings = Vec::new(); let mut provenance = Provenance::Unknown; let module = parsed_module(db, python_file).load(db); @@ -2986,10 +3013,13 @@ impl<'db> StaticClassLiteral<'db> { index.expression(value), TypeContext::default(), ); - return Member { - inner: Place::bound(inferred_ty) - .with_definition(declaration) - .with_qualifiers(all_qualifiers), + return ImplicitAttribute { + member: Member { + inner: Place::bound(inferred_ty) + .with_definition(declaration) + .with_qualifiers(all_qualifiers), + }, + augmented_bindings: None, }; } @@ -2998,7 +3028,10 @@ impl<'db> StaticClassLiteral<'db> { continue; } - return Member { inner: annotation }; + return ImplicitAttribute { + member: Member { inner: annotation }, + augmented_bindings: None, + }; } } @@ -3056,6 +3089,11 @@ impl<'db> StaticClassLiteral<'db> { continue; }; + if matches!(binding.kind(db), DefinitionKind::AugmentedAssignment(_)) { + augmented_bindings.push(binding); + continue; + } + if !is_method_reachable.is_always_false() { is_attribute_bound = true; } @@ -3172,9 +3210,6 @@ impl<'db> StaticClassLiteral<'db> { } } } - DefinitionKind::AugmentedAssignment(_) => { - Some(infer_definition_types(db, binding).binding_type(binding)) - } DefinitionKind::NamedExpression(_) => { // A named expression whose target is an attribute is syntactically prohibited None @@ -3189,19 +3224,25 @@ impl<'db> StaticClassLiteral<'db> { } } - Member { - inner: if is_attribute_bound { - Place::bound( + let member = if is_attribute_bound { + Member { + inner: Place::bound( union_of_inferred_types .build() .promote(db, env) .promote_singletons(db, env), ) .with_provenance(provenance) - .with_qualifiers(qualifiers) - } else { - Place::Undefined.with_qualifiers(qualifiers) - }, + .with_qualifiers(qualifiers), + } + } else { + Member::unbound() + }; + + ImplicitAttribute { + member, + augmented_bindings: (!augmented_bindings.is_empty()) + .then(|| AugmentedBindings::new(db, augmented_bindings.into_boxed_slice())), } } @@ -3885,6 +3926,29 @@ fn explicit_bases_cycle_fn<'db>( } } +/// Attributes assigned by instance methods or classmethods on a single class. +/// +/// Ordinary assignments such as `self.value = 1` or `cls.value = 1` establish an attribute +/// directly. Augmented assignments first require an existing instance or class attribute to supply +/// the value they read. +#[derive(Debug, Clone, Copy, PartialEq, Eq, get_size2::GetSize, salsa::SalsaValue)] +pub(super) struct ImplicitAttribute<'db> { + /// The attribute established by assignments that do not depend on an existing value. + pub(super) member: Member<'db>, + /// Augmented assignments that require an existing instance or class attribute. + pub(super) augmented_bindings: Option>, +} + +/// Augmented assignments deferred until MRO lookup finds the attribute they read. +#[salsa::interned(debug, heap_size=ruff_memory_usage::heap_size)] +pub(super) struct AugmentedBindings<'db> { + #[returns(deref)] + pub(super) definitions: Box<[Definition<'db>]>, +} + +// The Salsa heap is tracked separately. +impl get_size2::GetSize for AugmentedBindings<'_> {} + #[salsa::interned(debug, heap_size=ruff_memory_usage::heap_size)] struct ImplicitAttributeName<'db> { #[returns(copy)] @@ -3920,13 +3984,18 @@ fn implicit_attribute_names<'db>(db: &'db dyn Db, class_body_scope: ScopeId<'db> fn implicit_attribute_cycle_recover<'db>( db: &'db dyn Db, cycle: &salsa::Cycle, - previous_member: &Member<'db>, - member: Member<'db>, + previous: &ImplicitAttribute<'db>, + attribute_member: ImplicitAttribute<'db>, attribute: ImplicitAttributeName<'db>, -) -> Member<'db> { +) -> ImplicitAttribute<'db> { let env = ProgramEnvironment::from_scope(attribute.class_body_scope(db)); - let inner = member - .inner - .cycle_normalized(db, &env, previous_member.inner, cycle); - Member { inner } + let inner = + attribute_member + .member + .inner + .cycle_normalized(db, &env, previous.member.inner, cycle); + ImplicitAttribute { + member: Member { inner }, + ..attribute_member + } } diff --git a/crates/ty_python_semantic/src/types/narrow.rs b/crates/ty_python_semantic/src/types/narrow.rs index 49a3f56916..15f226acbf 100644 --- a/crates/ty_python_semantic/src/types/narrow.rs +++ b/crates/ty_python_semantic/src/types/narrow.rs @@ -4088,11 +4088,23 @@ impl<'db> NarrowingConstraintsBuilder<'db, '_> { } } + // Expression-inference cycles can replace every subexpression's type, including literals, + // with a cycle placeholder. This can prevent comparisons against `None` from narrowing + // recursively inferred attributes. Other literals can encounter the same issue, but a + // general solution would require broader changes to cycle recovery. For now, intentionally + // preserve only `None`, whose type can be recovered directly. + let expression_type = |expr: &ast::Expr, env: &ProgramEnvironment<'db>| { + if expr.is_none_literal_expr() { + Type::none(db, env) + } else { + inference.expression_type(expr) + } + }; let mut last_rhs_ty: Option = None; for (op, (left, right)) in std::iter::zip(&**ops, comparator_tuples) { - let lhs_ty = last_rhs_ty.unwrap_or_else(|| inference.expression_type(left)); - let rhs_ty = inference.expression_type(right); + let lhs_ty = last_rhs_ty.unwrap_or_else(|| expression_type(left, &self.env)); + let rhs_ty = expression_type(right, &self.env); let lhs_narrowing_rhs_ty = if matches!(op, ast::CmpOp::In | ast::CmpOp::NotIn) { self.inline_membership_rhs_type(right, inference) .unwrap_or(rhs_ty) From cf358a909dab9394d92102b4d5562c9b1c1c7d30 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Thu, 13 Aug 2026 14:58:08 +0100 Subject: [PATCH 020/371] [ty] Update typing conformance suite pin (#27725) --- .github/workflows/typing_conformance.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/typing_conformance.yaml b/.github/workflows/typing_conformance.yaml index 222fc89a97..15d122d566 100644 --- a/.github/workflows/typing_conformance.yaml +++ b/.github/workflows/typing_conformance.yaml @@ -36,7 +36,7 @@ env: RUST_BACKTRACE: 1 # Line-tables-only debug info: faster builds, backtraces still work. CARGO_PROFILE_DEV_DEBUG: line-tables-only - CONFORMANCE_SUITE_COMMIT: f4f2952f3ac94d7af819c5c71b60a50a100370e0 + CONFORMANCE_SUITE_COMMIT: bee91c2646261629c9835dc0adad16e0d554b4a5 PYTHON_VERSION: 3.12 jobs: From 73d3b6f8c14b98b023bbcf00cf0bfb2ce548a4bc Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Thu, 13 Aug 2026 10:44:47 -0400 Subject: [PATCH 021/371] Validate the PGO pipeline in CI (#27669) ## Summary This PR runs Ruff's complete PGO pipeline in CI when release-build inputs change, catching failures before they reach a release without running on ordinary code changes. The new `--debug` flag performs the same instrumented build, linting and formatting workloads, profile merging, and profile-guided rebuild without the cost of release optimization. The optional job follows the existing release-build path filters, uses an eight-core runner, and respects the `no-build` label. --- .github/workflows/ci.yaml | 39 +++++++++++++++++++++++++++++++++++++++ scripts/build_ruff_pgo.py | 23 +++++++++++++++-------- 2 files changed, 54 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 4cf93d74f4..cb27525daa 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -51,6 +51,8 @@ jobs: playground: ${{ steps.check_playground.outputs.changed }} # Flag that is set to "true" when code related to the benchmarks changes. benchmarks: ${{ steps.check_benchmarks.outputs.changed }} + # Flag that is set to "true" when release build inputs change. + release: ${{ steps.check_release.outputs.changed }} steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: @@ -68,6 +70,21 @@ jobs: sha=$(git merge-base HEAD "origin/${BASE_REF}") echo "sha=${sha}" >> "$GITHUB_OUTPUT" + - name: Check if release build inputs changed + id: check_release + env: + MERGE_BASE: ${{ steps.merge_base.outputs.sha }} + run: | + if git diff --quiet "${MERGE_BASE}...HEAD" -- \ + ':pyproject.toml' \ + ':.github/workflows/build-binaries.yml' \ + ':scripts/build_ruff_pgo.py' \ + ; then + echo "changed=false" >> "$GITHUB_OUTPUT" + else + echo "changed=true" >> "$GITHUB_OUTPUT" + fi + - name: Check if the parser code changed id: check_parser env: @@ -872,6 +889,28 @@ jobs: - name: "Remove wheels from cache" run: rm -rf target/wheels + pgo: + name: "PGO" + runs-on: depot-ubuntu-24.04-8 + needs: determine_changes + if: ${{ needs.determine_changes.outputs.release == 'true' && !contains(github.event.pull_request.labels.*.name, 'no-build') }} + timeout-minutes: 20 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: ${{ env.PYTHON_VERSION }} + architecture: x64 + - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 + with: + save-if: ${{ github.ref == 'refs/heads/main' }} + - name: "Install LLVM profiling tools" + run: rustup component add llvm-tools-preview + - name: "Run PGO pipeline" + run: python scripts/build_ruff_pgo.py --debug + prek: name: "prek" runs-on: ${{ github.repository == 'astral-sh/ruff' && 'depot-ubuntu-22.04-16' || 'ubuntu-latest' }} diff --git a/scripts/build_ruff_pgo.py b/scripts/build_ruff_pgo.py index 7966e6257c..9bcb7d533b 100644 --- a/scripts/build_ruff_pgo.py +++ b/scripts/build_ruff_pgo.py @@ -128,6 +128,11 @@ def url(self) -> str: def main() -> None: parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--target", help="Host-native Rust target triple") + parser.add_argument( + "--debug", + action="store_true", + help="Use debug builds to validate the complete PGO pipeline", + ) parser.add_argument( "--target-dir", type=Path, @@ -201,11 +206,12 @@ def main() -> None: environment.get("RUSTFLAGS"), f"-Cprofile-generate={profile_dir}" ), } - print("Building instrumented release Ruff", flush=True) - run(cargo_command(target), environment=instrumented_environment) + profile = "debug" if args.debug else "release" + print(f"Building instrumented {profile} Ruff", flush=True) + run(cargo_command(target, debug=args.debug), environment=instrumented_environment) binary_name = "ruff.exe" if "windows" in target else "ruff" - instrumented_binary = instrumented_target_dir / target / "release" / binary_name + instrumented_binary = instrumented_target_dir / target / profile / binary_name if not instrumented_binary.is_file(): raise RuntimeError(f"Instrumented Ruff binary not found: {instrumented_binary}") @@ -227,10 +233,11 @@ def main() -> None: environment.get("RUSTFLAGS"), f"-Cprofile-use={merged_profile}" ), } - print("Building optimized release Ruff", flush=True) - run(cargo_command(target), environment=optimized_environment) + print(f"Building profile-guided {profile} Ruff", flush=True) + run(cargo_command(target, debug=args.debug), environment=optimized_environment) print( - f"Optimized Ruff: {target_dir / target / 'release' / binary_name}", flush=True + f"Profile-guided Ruff: {target_dir / target / profile / binary_name}", + flush=True, ) @@ -504,11 +511,11 @@ def write_corpus_arguments(target_directory: Path, corpus: list[str]) -> Path: return arguments -def cargo_command(target: str) -> list[str]: +def cargo_command(target: str, *, debug: bool = False) -> list[str]: return [ "cargo", "rustc", - "--release", + *(() if debug else ("--release",)), "--locked", "--package", "ruff", From 9b5c5bc497aef06e1cd5093c6e29d5b918312daf Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Thu, 13 Aug 2026 16:09:38 +0100 Subject: [PATCH 022/371] [ty] Improve diagnostic hints for assignability mismatches with protocols and TypedDicts (#27717) --- .../mdtest/diagnostics/error_context.md | 164 +++++++++++++++++- .../src/types/protocol_class.rs | 6 +- .../src/types/relation_error.rs | 112 +++++++++--- 3 files changed, 253 insertions(+), 29 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/diagnostics/error_context.md b/crates/ty_python_semantic/resources/mdtest/diagnostics/error_context.md index 36e9aa6f9f..43e9a702fb 100644 --- a/crates/ty_python_semantic/resources/mdtest/diagnostics/error_context.md +++ b/crates/ty_python_semantic/resources/mdtest/diagnostics/error_context.md @@ -796,7 +796,7 @@ error[invalid-assignment]: Object of type `PersonWithAge` is not assignable to ` | | | Declared type info: field "age" is required in TypedDict `PersonWithAge` but not required and mutable in TypedDict `PersonWithOptionalAge` -help: The required field could be removed through a destructive operation like `del` on the target. +help: The required field could be removed through a destructive operation like `del` on the target ``` Assigning a `TypedDict` to a `dict` @@ -815,8 +815,8 @@ error[invalid-assignment]: Object of type `Person` is not assignable to `dict[st | | | Declared type info: TypedDict `Person` is not assignable to `dict` -help: A TypedDict is not usually assignable to any `dict[..]` type; `dict` types allow destructive operations like `clear()`. -help: Consider using `Mapping[..]` instead of `dict[..]`. +help: A TypedDict is not usually assignable to any `dict[..]` type; `dict` types allow destructive operations like `clear()` +help: Consider using `Mapping[..]` instead of `dict[..]` ``` Assigning an open `TypedDict` to a specialized `Mapping`: @@ -842,8 +842,41 @@ error[invalid-return-type]: Return type does not match returned value 40 | return d # snapshot | ^ expected `Mapping[str, int]`, found `D` info: TypedDict `D` is not assignable to `Mapping[str, int]` -help: `D` would be assignable to this `Mapping` type if it were declared with `closed=True`, but TypedDicts are open by default. -help: A subclass of `D` could validly add a new field of an arbitrary type, violating subtyping with the `Mapping` type +help: `D` would be assignable to `Mapping[str, int]` if it were declared with `closed=True`, but TypedDicts are open by default +help: A subclass of `D` could validly add a new field of an arbitrary type, violating subtyping with `Mapping[str, int]` +``` + +## Open `TypedDict` and a union of specialized mappings + +Each mapping in a union receives its own explanation when an open `TypedDict` is incompatible with +every alternative. + +```py +from collections.abc import Mapping +from typing import TypedDict + +class Empty(TypedDict): + pass + +def _(value: Empty) -> Mapping[str, int] | Mapping[str, str]: + return value # snapshot +``` + +```snapshot +error[invalid-return-type]: Return type does not match returned value + --> src/mdtest_snippet.py:8:12 + | +7 | def _(value: Empty) -> Mapping[str, int] | Mapping[str, str]: + | ------------------------------------- Expected `Mapping[str, int] | Mapping[str, str]` because of return type +8 | return value # snapshot + | ^^^^^ expected `Mapping[str, int] | Mapping[str, str]`, found `Empty` +info: type `Empty` is not assignable to any element of the union `Mapping[str, int] | Mapping[str, str]` +info: ├── TypedDict `Empty` is not assignable to `Mapping[str, int]` +info: └── TypedDict `Empty` is not assignable to `Mapping[str, str]` +help: `Empty` would be assignable to `Mapping[str, int]` if it were declared with `closed=True`, but TypedDicts are open by default +help: A subclass of `Empty` could validly add a new field of an arbitrary type, violating subtyping with `Mapping[str, int]` +help: `Empty` would be assignable to `Mapping[str, str]` if it were declared with `closed=True`, but TypedDicts are open by default +help: A subclass of `Empty` could validly add a new field of an arbitrary type, violating subtyping with `Mapping[str, str]` ``` ## Generic `TypedDict` field conflicts in overload diagnostics @@ -1238,6 +1271,126 @@ info: └── protocol member `check` is incompatible info: └── parameter `y` has an incompatible type: `str` is not assignable to `bytes` ``` +## Protocol method parameter names + +Assignability errors against protocols are often caused because a method in the protocol class +should have used positional-only parameters, but didn't. In this situation, we point out the likely +cause of the assignability error in a dedicated `help:` message that points out that the issue may +be due to the protocol itself rather than the type being assigned to the protocol: + +```py +from typing import Protocol + +class Target(Protocol): + def run(self, expected: int) -> None: ... + +class Source: + def run(self, actual: int) -> None: ... + +target: Target = Source() # snapshot +``` + +```snapshot +error[invalid-assignment]: Object of type `Source` is not assignable to `Target` + --> src/mdtest_snippet.py:9:18 + | +9 | target: Target = Source() # snapshot + | ------ ^^^^^^^^ Incompatible value of type `Source` + | | + | Declared type +info: type `Source` is not assignable to protocol `Target` +info: └── protocol member `run` is incompatible +info: └── the parameter named `actual` does not match `expected` (and can be used as a keyword parameter) +help: `Source` might be assignable to `Target` if the parameter `expected` were made positional-only in `Target.run` +``` + +The same suggestion applies for the case where a positional-or-keyword parameter was apparently +demanded by a protocol member, but only a positional-only parameter was supplied in the type that +was assigned to the protocol: + +```py +class Target2(Protocol): + def run(self, expected: int) -> None: ... + +class Source2: + def run(self, actual: int, /) -> None: ... + +target: Target2 = Source2() # snapshot +``` + +```snapshot +error[invalid-assignment]: Object of type `Source2` is not assignable to `Target2` + --> src/mdtest_snippet.py:16:19 + | +16 | target: Target2 = Source2() # snapshot + | ------- ^^^^^^^^^ Incompatible value of type `Source2` + | | + | Declared type +info: type `Source2` is not assignable to protocol `Target2` +info: └── protocol member `run` is incompatible +info: └── parameter `actual` is positional-only but must also accept keyword arguments +help: `Source2` might be assignable to `Target2` if the parameter `expected` were made positional-only in `Target2.run` +``` + +Making a parameter positional-only resolves a name mismatch but does not necessarily make the method +compatible, because its parameter type can still be incorrect. For this reason, we hedge our bets a +little in our `help:` message (we say "*might* be assignable", rather than "*will* be assignable"): + +```py +class Target3(Protocol): + def run(self, expected: int) -> None: ... + +class Source3: + def run(self, actual: str) -> None: ... + +target: Target3 = Source3() # snapshot +``` + +```snapshot +error[invalid-assignment]: Object of type `Source3` is not assignable to `Target3` + --> src/mdtest_snippet.py:23:19 + | +23 | target: Target3 = Source3() # snapshot + | ------- ^^^^^^^^^ Incompatible value of type `Source3` + | | + | Declared type +info: type `Source3` is not assignable to protocol `Target3` +info: └── protocol member `run` is incompatible +info: └── the parameter named `actual` does not match `expected` (and can be used as a keyword parameter) +help: `Source3` might be assignable to `Target3` if the parameter `expected` were made positional-only in `Target3.run` +``` + +Suggestions for inherited protocol methods name the protocol that actually declares the method. + +```py +from typing import Protocol + +class Parent(Protocol): + def run(self, expected: int) -> None: ... + +class Child(Parent, Protocol): + pass + +class Source4: + def run(self, actual: int) -> None: ... + +target: Child = Source4() # snapshot +``` + +```snapshot +error[invalid-assignment]: Object of type `Source4` is not assignable to `Child` + --> src/mdtest_snippet.py:35:17 + | +35 | target: Child = Source4() # snapshot + | ----- ^^^^^^^^^ Incompatible value of type `Source4` + | | + | Declared type +info: type `Source4` is not assignable to protocol `Child` +info: └── protocol member `run` is incompatible +info: └── the parameter named `actual` does not match `expected` (and can be used as a keyword parameter) +help: `Source4` might be assignable to `Child` if the parameter `expected` were made positional-only in `Parent.run` +``` + ## Type aliases Type aliases should be expanded in diagnostics to understand the underlying incompatibilities: @@ -1441,6 +1594,7 @@ error[invalid-assignment]: Object of type `IncompatibleFoo` is not assignable to info: type `IncompatibleFoo` is not assignable to protocol `SupportsFooAndBar` info: └── protocol member `foo` is incompatible info: └── the parameter named `name_` does not match `name` (and can be used as a keyword parameter) +help: `IncompatibleFoo` might be assignable to `SupportsFooAndBar` if the parameter `name` were made positional-only in `SupportsFooAndBar.foo` ``` ## Assigning to `Iterable` diff --git a/crates/ty_python_semantic/src/types/protocol_class.rs b/crates/ty_python_semantic/src/types/protocol_class.rs index 238f6d608d..be49967184 100644 --- a/crates/ty_python_semantic/src/types/protocol_class.rs +++ b/crates/ty_python_semantic/src/types/protocol_class.rs @@ -400,7 +400,11 @@ impl<'db> ProtocolInterfaceView<'db> { }) } - fn member_by_name<'a>(self, db: &'db dyn Db, name: &'a str) -> Option> { + pub(super) fn member_by_name<'a>( + self, + db: &'db dyn Db, + name: &'a str, + ) -> Option> { self.interface .inner(db) .get(name) diff --git a/crates/ty_python_semantic/src/types/relation_error.rs b/crates/ty_python_semantic/src/types/relation_error.rs index 06570acce6..2098cd934b 100644 --- a/crates/ty_python_semantic/src/types/relation_error.rs +++ b/crates/ty_python_semantic/src/types/relation_error.rs @@ -6,8 +6,10 @@ use std::cell::{Cell, RefCell}; use std::rc::Rc; use ruff_python_ast::name::Name; +use ty_python_core::semantic_index; use crate::types::context::LintDiagnosticGuard; +use crate::types::infer::nearest_enclosing_class; use crate::types::tuple::TupleLength; use crate::types::{DisplaySettings, Type, TypedDictType}; use crate::{FxOrderSet, ProgramEnvironment}; @@ -177,7 +179,7 @@ impl<'db> ErrorContext<'db> { db: &'db dyn Db, env: &ProgramEnvironment<'db>, relation: TypeRelation, - help_messages: &mut FxOrderSet, + help_messages: &mut FxOrderSet>, ) -> Option { let typed_dict_name = |typed_dict: &TypedDictType<'db>| match typed_dict { TypedDictType::Class(class) => format!("TypedDict `{}`", class.name(db)), @@ -311,11 +313,12 @@ impl<'db> ErrorContext<'db> { Self::OpenTypedDictNotAssignableToMapping { source, target } => { let name = source.defining_class().map(|class| class.name(db)); help_messages.insert(HelpMessages::OpenTypedDictNotAssignableToMapping { - typed_dict_name: name.cloned(), - relation, + typed_dict_name: name, + mapping_target: *target, }); help_messages.insert(HelpMessages::ExplainOpenTypedDictUnsoundness { - typed_dict_name: name.cloned(), + typed_dict_name: name, + mapping_target: *target, }); format!( @@ -481,7 +484,7 @@ impl<'db> ErrorContext<'db> { } #[derive(Clone, Debug, PartialEq, Eq, Hash)] -enum HelpMessages { +enum HelpMessages<'db> { RequiredFieldCouldBeRemoved, TypedDictNotAssignableToDict(TypeRelation), ConsiderUsingMappingInsteadOfDict, @@ -490,35 +493,48 @@ enum HelpMessages { parameter_name: Option, }, OpenTypedDictNotAssignableToMapping { - typed_dict_name: Option, - relation: TypeRelation, + typed_dict_name: Option<&'db Name>, + mapping_target: Type<'db>, }, ExplainOpenTypedDictUnsoundness { - typed_dict_name: Option, + typed_dict_name: Option<&'db Name>, + mapping_target: Type<'db>, + }, + SuggestMakingParameterPositionalOnly { + ty: Type<'db>, + protocol: Type<'db>, + declaring_protocol_name: &'db Name, + method_name: Name, + parameter_name: Name, }, } -impl std::fmt::Display for HelpMessages { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - match self { +impl<'db> HelpMessages<'db> { + fn display( + &self, + db: &'db dyn Db, + env: &ProgramEnvironment, + relation: TypeRelation, + ) -> impl std::fmt::Display { + std::fmt::from_fn(move |f| match self { HelpMessages::RequiredFieldCouldBeRemoved => f.write_str( "The required field could be removed through a destructive operation \ - like `del` on the target.", + like `del` on the target", ), HelpMessages::TypedDictNotAssignableToDict(relation) => { write!( f, "A TypedDict is not usually {} any `dict[..]` type; \ - `dict` types allow destructive operations like `clear()`.", + `dict` types allow destructive operations like `clear()`", relation.description() ) } HelpMessages::ConsiderUsingMappingInsteadOfDict => { - f.write_str("Consider using `Mapping[..]` instead of `dict[..]`.") + f.write_str("Consider using `Mapping[..]` instead of `dict[..]`") } HelpMessages::OpenTypedDictNotAssignableToMapping { typed_dict_name, - relation, + mapping_target, } => { let name = typed_dict_name .as_ref() @@ -526,13 +542,17 @@ impl std::fmt::Display for HelpMessages { .unwrap_or_else(|| "this TypedDict".to_string()); write!( f, - "{name} would be {relation} this `Mapping` type \ + "{name} would be {relation} `{mapping}` \ if it were declared with `closed=True`, \ - but TypedDicts are open by default.", - relation = relation.description() + but TypedDicts are open by default", + relation = relation.description(), + mapping = mapping_target.display(db, env) ) } - HelpMessages::ExplainOpenTypedDictUnsoundness { typed_dict_name } => { + HelpMessages::ExplainOpenTypedDictUnsoundness { + typed_dict_name, + mapping_target, + } => { let name = typed_dict_name .as_ref() .map(|name| format!("`{name}`")) @@ -540,7 +560,8 @@ impl std::fmt::Display for HelpMessages { write!( f, "A subclass of {name} could validly add a new field \ - of an arbitrary type, violating subtyping with the `Mapping` type" + of an arbitrary type, violating subtyping with `{mapping_type}`", + mapping_type = mapping_target.display(db, env) ) } HelpMessages::TopCallableExplanation => f.write_str( @@ -552,7 +573,26 @@ impl std::fmt::Display for HelpMessages { Some(name) => write!(f, "Parameter `{name}` must have a default value"), None => f.write_str("The parameter must have a default value"), }, - } + HelpMessages::SuggestMakingParameterPositionalOnly { + ty, + protocol, + declaring_protocol_name, + method_name, + parameter_name, + } => { + let settings = + DisplaySettings::from_possibly_ambiguous_types(db, env, [*ty, *protocol]); + write!( + f, + "`{source}` might be {relation} `{target}` \ + if the parameter `{parameter_name}` were made positional-only \ + in `{declaring_protocol_name}.{method_name}`", + source = ty.display_with(db, env, settings.clone()), + relation = relation.description(), + target = protocol.display_with(db, env, settings), + ) + } + }) } } @@ -584,7 +624,7 @@ impl<'db> ErrorContextNode<'db> { env: &ProgramEnvironment<'db>, relation: TypeRelation, output_lines: &mut Vec, - help_messages: &mut FxOrderSet, + help_messages: &mut FxOrderSet>, prefix: &str, continuation: &str, ) { @@ -592,6 +632,32 @@ impl<'db> ErrorContextNode<'db> { output_lines.push(format!("{prefix}{line}")); } + if let ErrorContext::TypeNotCompatibleWithProtocol { ty, protocol } = &self.context + && let Type::ProtocolInstance(proto_instance) = protocol + && let [single_child] = self.children.as_slice() + && let ErrorContext::ProtocolMemberIncompatible { member_name } = &single_child.context + && let [single_grandchild] = single_child.children.as_slice() + && let ErrorContext::ParameterNameMismatch { target_name, .. } + | ErrorContext::ParameterMustAcceptKeywordArguments { target_name, .. } = + &single_grandchild.context + && let Some(protocol_member) = + proto_instance.interface(db).member_by_name(db, member_name) + && let Some(definition) = protocol_member.definition() + && let Some(declaring_protocol) = nearest_enclosing_class( + db, + semantic_index(db, definition.program_file(db)), + definition.scope(db), + ) + { + help_messages.insert(HelpMessages::SuggestMakingParameterPositionalOnly { + ty: *ty, + protocol: *protocol, + declaring_protocol_name: declaring_protocol.name(db), + method_name: member_name.clone(), + parameter_name: target_name.clone(), + }); + } + let num_children = self.children.len(); for (index, child) in self.children.iter().enumerate() { let is_last = index == num_children - 1; @@ -722,7 +788,7 @@ impl<'db> ErrorContextTree<'db> { diag.info(line); } for help_message in help_messages { - diag.help(help_message.to_string()); + diag.help(help_message.display(db, env, self.relation)); } } } From 4df8eaee5676e0c02dc6d149132a286b063b5b5f Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Thu, 13 Aug 2026 08:42:33 -0700 Subject: [PATCH 023/371] [ty] Preserve tuple types containing Never (#27580) ## Summary Stop simplifying fixed-length tuple types containing `Never` to `Never`. Tuple annotations can describe user-defined subclasses, and ty also uses tuple types to represent `TypeVarTuple` specializations. Collapsing those specializations erased variadic type arguments and rejected valid `returns` container callbacks. Make tuple construction, tuple type-expression inference, and tuple specialization mapping infallible now that fixed `Never` elements no longer cause construction to fail. Preserve the existing simplification of the homogeneous `tuple[Never, ...]` to `tuple[()]`. Fixes astral-sh/ty#4209. ## Test plan - Add mdtests covering `Never`-containing inherited variadic bases and aliases with both legacy and PEP 695 syntax, a `returns`-style generic callback, and tuple expressions that preserve `Never`-containing element shapes. - Update mdtests for `Never`/`NoReturn` tuple equivalence, top and bottom materializations, negation, assignability, and union behavior while retaining homogeneous-`Never` coverage. - Verify the original `returns` reproduction and the complete `ty_python_semantic` test suite. --- .../mdtest/generics/legacy/typevartuple.md | 63 +++++++++++++++++++ .../mdtest/generics/pep695/typevartuple.md | 31 +++++++++ .../resources/mdtest/ty_extensions.md | 2 +- .../resources/mdtest/type_compendium/never.md | 9 +-- .../resources/mdtest/type_compendium/tuple.md | 19 ++++-- .../type_properties/is_assignable_to.md | 6 +- .../mdtest/type_properties/materialization.md | 16 ++--- .../tuples_containing_never.md | 43 ++++++------- .../resources/mdtest/union_types.md | 7 ++- .../src/types/class/named_tuple.rs | 10 +-- .../src/types/class_base.rs | 2 +- .../ty_python_semantic/src/types/generics.rs | 11 ++-- .../src/types/infer/builder/subscript.rs | 7 +-- .../types/infer/builder/type_expression.rs | 9 ++- .../ty_python_semantic/src/types/instance.rs | 16 ++--- .../ty_python_semantic/src/types/relation.rs | 12 +--- crates/ty_python_semantic/src/types/tuple.rs | 31 +++------ 17 files changed, 174 insertions(+), 120 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/typevartuple.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/typevartuple.md index 832cb3dda7..9755910ddd 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/typevartuple.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/typevartuple.md @@ -327,6 +327,50 @@ reveal_type(Between().attr) # revealed: tuple[Unknown, *tuple[Unknown, ...], Un reveal_type(Between[int]().attr) # revealed: tuple[Unknown, *tuple[Unknown, ...], Unknown] ``` +### Inherited specializations containing `Never` + +A `Never` argument in a variadic generic must retain its position when a subclass forwards its type +arguments to a generic base. + +```py +from typing import Any, Generic, Never, TypeVarTuple + +Ts = TypeVarTuple("Ts") + +class Kind(Generic[*Ts]): ... +class SupportsKind(Kind[*Ts]): ... +class Container(SupportsKind[int, Never]): ... + +def _(value: Container) -> None: + expected: Kind[int, Any] = value +``` + +### Callbacks returning containers with `Never` arguments + +A callback can return a concrete container whose variadic base contains `Never` when the expected +container type uses `Any` in that position. + +```py +from collections.abc import Callable +from typing import Any, Generic, Never, TypeVar, TypeVarTuple + +T = TypeVar("T") +U = TypeVar("U") +Ts = TypeVarTuple("Ts") + +class Kind(Generic[T, *Ts]): ... + +class Result(Kind[T, Never]): + def bind(self, callback: Callable[[T], Kind[U, Any]]) -> "Result[U]": + raise NotImplementedError + +def parse(value: str) -> Result[int]: + raise NotImplementedError + +def _(result: Result[str]) -> None: + reveal_type(result.bind(parse)) # revealed: Result[int] +``` + ### `TypeVarTuple` with `ParamSpec` ```py @@ -486,6 +530,25 @@ def _( reveal_type(a10) # revealed: tuple[Unknown, *tuple[Unknown, ...], Unknown] ``` +### Legacy aliases containing `Never` + +A legacy alias must retain free type variables that appear alongside a `Never` argument in a +variadic specialization. + +```py +from typing import Generic, Never, TypeVar, TypeVarTuple + +T = TypeVar("T") +Ts = TypeVarTuple("Ts") + +class Container(Generic[*Ts]): ... + +Padded = Container[T, Never] + +def _(value: Padded[int]) -> None: + reveal_type(value) # revealed: Container[int, Never] +``` + ### Variadic arguments require variadic aliases An unpacked type variable tuple or arbitrary-length tuple cannot be used to specialize a diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md index 073cb54187..f414064d7d 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md @@ -99,6 +99,22 @@ reveal_type(Between().attr) # revealed: tuple[Unknown, *tuple[Unknown, ...], Un reveal_type(Between[int]().attr) # revealed: tuple[Unknown, *tuple[Unknown, ...], Unknown] ``` +### Inherited specializations containing `Never` + +A `Never` argument in a variadic generic must retain its position when a subclass forwards its type +arguments to a generic base. + +```py +from typing import Any, Never + +class Kind[*Ts]: ... +class SupportsKind[*Ts](Kind[*Ts]): ... +class Container(SupportsKind[int, Never]): ... + +def _(value: Container) -> None: + expected: Kind[int, Any] = value +``` + ### `TypeVarTuple` with `ParamSpec` ```py @@ -955,6 +971,21 @@ def _( reveal_type(a10) # revealed: tuple[Unknown, *tuple[Unknown, ...], Unknown] ``` +### Aliases containing `Never` + +A variadic alias retains each specialized argument even when a later argument is `Never`. + +```py +from typing import Never + +class Container[*Ts]: ... + +type Padded[T] = Container[T, Never] + +def _(value: Padded[int]) -> None: + reveal_type(value) # revealed: Container[int, Never] +``` + ### Unpacked tuple type arguments ```py diff --git a/crates/ty_python_semantic/resources/mdtest/ty_extensions.md b/crates/ty_python_semantic/resources/mdtest/ty_extensions.md index 331cf1c1ae..a5247b14e5 100644 --- a/crates/ty_python_semantic/resources/mdtest/ty_extensions.md +++ b/crates/ty_python_semantic/resources/mdtest/ty_extensions.md @@ -359,7 +359,7 @@ from ty_extensions._internal import is_equivalent_to from typing_extensions import Never, Union static_assert(is_equivalent_to(type, type[object])) -static_assert(is_equivalent_to(tuple[int, Never], Never)) +static_assert(is_equivalent_to(tuple[Never, ...], tuple[()])) static_assert(is_equivalent_to(int | str, Union[int, str])) static_assert(not is_equivalent_to(int, str)) diff --git a/crates/ty_python_semantic/resources/mdtest/type_compendium/never.md b/crates/ty_python_semantic/resources/mdtest/type_compendium/never.md index 917b2ec2eb..bfef09447b 100644 --- a/crates/ty_python_semantic/resources/mdtest/type_compendium/never.md +++ b/crates/ty_python_semantic/resources/mdtest/type_compendium/never.md @@ -171,18 +171,19 @@ x: list[Never] = [] ## Tuples involving `Never` -A type like `tuple[int, Never]` has no inhabitants, and so it is equivalent to `Never`: +A type like `tuple[int, Never]` remains distinct from `Never`. A tuple annotation can describe +user-defined subclasses, so its element types remain part of the type: ```py from ty_extensions import static_assert from ty_extensions._internal import is_equivalent_to from typing_extensions import Never -static_assert(is_equivalent_to(tuple[int, Never], Never)) +static_assert(not is_equivalent_to(tuple[int, Never], Never)) ``` -Note that this is not the case for the homogenous tuple type `tuple[Never, ...]` though, because -that type is inhabited by the empty tuple: +The homogeneous tuple type `tuple[Never, ...]` is also distinct from `Never`: it is inhabited by the +empty tuple. ```py static_assert(not is_equivalent_to(tuple[Never, ...], Never)) diff --git a/crates/ty_python_semantic/resources/mdtest/type_compendium/tuple.md b/crates/ty_python_semantic/resources/mdtest/type_compendium/tuple.md index 69606757af..ef8e5d5c0e 100644 --- a/crates/ty_python_semantic/resources/mdtest/type_compendium/tuple.md +++ b/crates/ty_python_semantic/resources/mdtest/type_compendium/tuple.md @@ -227,17 +227,26 @@ static_assert(not is_singleton(tuple[None])) python-version = "3.11" ``` -The `Never` type contains no inhabitants, so a tuple type that contains `Never` as a mandatory -element also contains no inhabitants. +The `Never` type contains no inhabitants, but a tuple annotation can also describe user-defined +subclasses. A tuple type containing `Never` as a mandatory element therefore retains its shape +instead of simplifying to `Never`. ```py from typing import Never from ty_extensions import static_assert from ty_extensions._internal import is_equivalent_to -static_assert(is_equivalent_to(tuple[Never], Never)) -static_assert(is_equivalent_to(tuple[int, Never], Never)) -static_assert(is_equivalent_to(tuple[Never, *tuple[int, ...]], Never)) +static_assert(not is_equivalent_to(tuple[Never], Never)) +static_assert(not is_equivalent_to(tuple[int, Never], Never)) +static_assert(not is_equivalent_to(tuple[Never, *tuple[int, ...]], Never)) +``` + +Tuple expressions also preserve their element types when an element has type `Never`. + +```py +def tuple_from_never(value: Never) -> None: + reveal_type((value,)) # revealed: tuple[Never] + reveal_type((1, value)) # revealed: tuple[Literal[1], Never] ``` If the variable-length portion of a tuple is `Never`, then that portion of the tuple must always be diff --git a/crates/ty_python_semantic/resources/mdtest/type_properties/is_assignable_to.md b/crates/ty_python_semantic/resources/mdtest/type_properties/is_assignable_to.md index 061f8285ca..8b4f56be1b 100644 --- a/crates/ty_python_semantic/resources/mdtest/type_properties/is_assignable_to.md +++ b/crates/ty_python_semantic/resources/mdtest/type_properties/is_assignable_to.md @@ -839,9 +839,9 @@ from typing_extensions import Any, Never, Sequence from ty_extensions import static_assert from ty_extensions._internal import is_assignable_to -# The bottom materialization of `tuple[Any]` is `tuple[Never]`, -# which simplifies to `Never`, so `tuple[int]` and `tuple[()]` are -# both assignable to `~tuple[Any]` +# The bottom materialization of `tuple[Any]` is `tuple[Never]`. Both +# `tuple[int]` and `tuple[()]` are disjoint from `tuple[Never]`, so they are +# assignable to `~tuple[Any]`. static_assert(is_assignable_to(tuple[int], ~tuple[Any])) static_assert(is_assignable_to(tuple[()], ~tuple[Any])) diff --git a/crates/ty_python_semantic/resources/mdtest/type_properties/materialization.md b/crates/ty_python_semantic/resources/mdtest/type_properties/materialization.md index c6f42bb4e8..a639e0d2b6 100644 --- a/crates/ty_python_semantic/resources/mdtest/type_properties/materialization.md +++ b/crates/ty_python_semantic/resources/mdtest/type_properties/materialization.md @@ -169,7 +169,7 @@ type C2 = Callable[[int, tuple[int | Any]], tuple[Any]] def _(top: Top[C2], bottom: Bottom[C2]) -> None: reveal_type(top) # revealed: (int, tuple[int], /) -> tuple[object] - reveal_type(bottom) # revealed: (int, tuple[object], /) -> Never + reveal_type(bottom) # revealed: (int, tuple[object], /) -> tuple[Never] ``` But, if the callable itself is in a contravariant position, then the variance is flipped i.e., if @@ -294,13 +294,13 @@ from ty_extensions import Bottom, Top, static_assert from ty_extensions._internal import Unknown, is_equivalent_to static_assert(is_equivalent_to(Top[tuple[Any, int]], tuple[object, int])) -static_assert(is_equivalent_to(Bottom[tuple[Any, int]], Never)) +static_assert(is_equivalent_to(Bottom[tuple[Any, int]], tuple[Never, int])) static_assert(is_equivalent_to(Top[tuple[Unknown, int]], tuple[object, int])) -static_assert(is_equivalent_to(Bottom[tuple[Unknown, int]], Never)) +static_assert(is_equivalent_to(Bottom[tuple[Unknown, int]], tuple[Never, int])) static_assert(is_equivalent_to(Top[tuple[Any, int, Unknown]], tuple[object, int, object])) -static_assert(is_equivalent_to(Bottom[tuple[Any, int, Unknown]], Never)) +static_assert(is_equivalent_to(Bottom[tuple[Any, int, Unknown]], tuple[Never, int, Never])) ``` Except for when the tuple itself is in a contravariant position, then all positions in the tuple @@ -313,7 +313,7 @@ from ty_extensions._internal import TypeOf type C = Callable[[tuple[Any, int], tuple[str, Unknown]], None] def _(top: Top[C], bottom: Bottom[C]) -> None: - reveal_type(top) # revealed: (Never, Never, /) -> None + reveal_type(top) # revealed: (tuple[Never, int], tuple[str, Never], /) -> None reveal_type(bottom) # revealed: (tuple[object, int], tuple[str, object], /) -> None ``` @@ -469,9 +469,9 @@ from ty_extensions._internal import Unknown, is_equivalent_to static_assert(is_equivalent_to(Top[~Any], object)) static_assert(is_equivalent_to(Bottom[~Any], Never)) -# tuple[Any, int] is in a contravariant position, so the -# top materialization is Never and the negation of it -static_assert(is_equivalent_to(Top[~tuple[Any, int]], object)) +# tuple[Any, int] is in a contravariant position, so its top +# materialization negates the tuple's bottom materialization. +static_assert(is_equivalent_to(Top[~tuple[Any, int]], ~tuple[Never, int])) static_assert(is_equivalent_to(Bottom[~tuple[Any, int]], ~tuple[object, int])) ``` diff --git a/crates/ty_python_semantic/resources/mdtest/type_properties/tuples_containing_never.md b/crates/ty_python_semantic/resources/mdtest/type_properties/tuples_containing_never.md index d5f289c9d2..038e7a52b8 100644 --- a/crates/ty_python_semantic/resources/mdtest/type_properties/tuples_containing_never.md +++ b/crates/ty_python_semantic/resources/mdtest/type_properties/tuples_containing_never.md @@ -1,30 +1,25 @@ # Tuples containing `Never` -A heterogeneous `tuple[…]` type that contains `Never` as a type argument simplifies to `Never`. One -way to think about this is the following: in order to construct a tuple, you need to have an object -of every element type. But since there is no object of type `Never`, you cannot construct the tuple. -Such a tuple type is therefore uninhabited and equivalent to `Never`. - -In the language of algebraic data types, a tuple type is a product type and `Never` acts like the -zero element in multiplication, similar to how a Cartesian product with the empty set is the empty -set. +A heterogeneous `tuple[…]` type that contains `Never` remains distinct from `Never`. Tuple types +include user-defined subclasses, so their element types must not be discarded solely because an +ordinary tuple with those elements cannot be constructed. ```py from ty_extensions import static_assert from ty_extensions._internal import is_equivalent_to from typing_extensions import Never, NoReturn -static_assert(is_equivalent_to(Never, tuple[Never])) -static_assert(is_equivalent_to(Never, tuple[Never, int])) -static_assert(is_equivalent_to(Never, tuple[int, Never])) -static_assert(is_equivalent_to(Never, tuple[int, Never, str])) -static_assert(is_equivalent_to(Never, tuple[int, tuple[str, Never]])) -static_assert(is_equivalent_to(Never, tuple[tuple[str, Never], int])) +static_assert(not is_equivalent_to(Never, tuple[Never])) +static_assert(not is_equivalent_to(Never, tuple[Never, int])) +static_assert(not is_equivalent_to(Never, tuple[int, Never])) +static_assert(not is_equivalent_to(Never, tuple[int, Never, str])) +static_assert(not is_equivalent_to(Never, tuple[int, tuple[str, Never]])) +static_assert(not is_equivalent_to(Never, tuple[tuple[str, Never], int])) def _(x: tuple[Never], y: tuple[int, Never], z: tuple[Never, int]): - reveal_type(x) # revealed: Never - reveal_type(y) # revealed: Never - reveal_type(z) # revealed: Never + reveal_type(x) # revealed: tuple[Never] + reveal_type(y) # revealed: tuple[int, Never] + reveal_type(z) # revealed: tuple[Never, int] ``` The empty `tuple` is *not* equivalent to `Never`! @@ -33,13 +28,13 @@ The empty `tuple` is *not* equivalent to `Never`! static_assert(not is_equivalent_to(Never, tuple[()])) ``` -`NoReturn` is just a different spelling of `Never`, so the same is true for `NoReturn`: +`NoReturn` is just a different spelling of `Never`, so these tuple types also retain their shape: ```py -static_assert(is_equivalent_to(NoReturn, tuple[NoReturn])) -static_assert(is_equivalent_to(NoReturn, tuple[NoReturn, int])) -static_assert(is_equivalent_to(NoReturn, tuple[int, NoReturn])) -static_assert(is_equivalent_to(NoReturn, tuple[int, NoReturn, str])) -static_assert(is_equivalent_to(NoReturn, tuple[int, tuple[str, NoReturn]])) -static_assert(is_equivalent_to(NoReturn, tuple[tuple[str, NoReturn], int])) +static_assert(not is_equivalent_to(NoReturn, tuple[NoReturn])) +static_assert(not is_equivalent_to(NoReturn, tuple[NoReturn, int])) +static_assert(not is_equivalent_to(NoReturn, tuple[int, NoReturn])) +static_assert(not is_equivalent_to(NoReturn, tuple[int, NoReturn, str])) +static_assert(not is_equivalent_to(NoReturn, tuple[int, tuple[str, NoReturn]])) +static_assert(not is_equivalent_to(NoReturn, tuple[tuple[str, NoReturn], int])) ``` diff --git a/crates/ty_python_semantic/resources/mdtest/union_types.md b/crates/ty_python_semantic/resources/mdtest/union_types.md index ebf3edf134..0ea3eddaf2 100644 --- a/crates/ty_python_semantic/resources/mdtest/union_types.md +++ b/crates/ty_python_semantic/resources/mdtest/union_types.md @@ -393,8 +393,9 @@ def gradual_aliases( reveal_type(nested_last) # revealed: Covariant[NestedGradualAlias] ``` -Matching materialization endpoints do not establish that gradual tuple arguments have the same -shape. A bounded generic must preserve which tuple position contains the gradual element. +Matching top materializations do not establish that gradual tuple arguments have the same shape. A +bounded generic must preserve which tuple position contains the gradual element, including in its +bottom materialization. ```py from ty_extensions import Bottom, Top, static_assert @@ -408,7 +409,7 @@ class C[T: tuple[int, int]]: raise NotImplementedError static_assert(is_equivalent_to(Top[C[L]], Top[C[R]])) -static_assert(is_equivalent_to(Bottom[C[L]], Bottom[C[R]])) +static_assert(not is_equivalent_to(Bottom[C[L]], Bottom[C[R]])) static_assert(not is_equivalent_to(C[L], C[R])) static_assert(not is_equivalent_to(C[L] | C[R], C[L])) static_assert(not is_equivalent_to(C[R] | C[L], C[R])) diff --git a/crates/ty_python_semantic/src/types/class/named_tuple.rs b/crates/ty_python_semantic/src/types/class/named_tuple.rs index 0d150fb4fe..3d94e9a9df 100644 --- a/crates/ty_python_semantic/src/types/class/named_tuple.rs +++ b/crates/ty_python_semantic/src/types/class/named_tuple.rs @@ -286,15 +286,7 @@ impl<'db> DynamicNamedTupleLiteral<'db> { } let field_types = self.fields(db).iter().map(|field| field.ty); - TupleType::heterogeneous(db, env, field_types) - .map(|tuple| tuple.to_class_type(db)) - .unwrap_or_else(|| { - KnownClass::Tuple - .to_class_literal(db, env) - .as_class_literal() - .expect("tuple should be a class literal") - .default_specialization(db) - }) + TupleType::heterogeneous(db, env, field_types).to_class_type(db) } /// Look up an instance member defined directly on this class (not inherited). diff --git a/crates/ty_python_semantic/src/types/class_base.rs b/crates/ty_python_semantic/src/types/class_base.rs index d848b47f46..f9796ac60a 100644 --- a/crates/ty_python_semantic/src/types/class_base.rs +++ b/crates/ty_python_semantic/src/types/class_base.rs @@ -308,7 +308,7 @@ impl<'db> ClassBase<'db> { db, env, fields.values().map(|field| field.declared_ty), - )? + ) .to_class_type(db) .into(), subclass, diff --git a/crates/ty_python_semantic/src/types/generics.rs b/crates/ty_python_semantic/src/types/generics.rs index ffdd7c00b4..2fe3031b78 100644 --- a/crates/ty_python_semantic/src/types/generics.rs +++ b/crates/ty_python_semantic/src/types/generics.rs @@ -1494,7 +1494,7 @@ impl<'db> Specialization<'db> { }); let original_tuple_inner = self.tuple_inner(db); - let tuple_inner = original_tuple_inner.and_then(|tuple| { + let tuple_inner = original_tuple_inner.map(|tuple| { tuple.apply_type_mapping_impl(db, type_mapping, TypeContext::default(), visitor) }); @@ -1651,7 +1651,7 @@ impl<'db> Specialization<'db> { } }); let original_tuple_inner = self.tuple_inner(db); - let tuple_inner = original_tuple_inner.and_then(|tuple| { + let tuple_inner = original_tuple_inner.map(|tuple| { // Tuples are immutable, so tuple element types are always in covariant position. tuple.apply_type_mapping_impl( db, @@ -1778,11 +1778,8 @@ impl<'c, 'db> TypeRelationChecker<'_, 'c, 'db> { // Materializing the source otherwise loses the bottom needed to // simplify `Covariant[Any] | Covariant[Any | str]`. Comparing both // top and bottom is a possible alternative, but it gets more complex - // due to the need to preserve Divergent markers. Also the fact that we currently - // simplify tuples containing `Never` to `Never` means that for - // `class C[T: tuple[int, int]]`, `C[tuple[Any, int]]` and `C[tuple[int, Any]]` - // have the same top and bottom but expose `Any` in different tuple positions. - // TODO: Try resolving the above issues so we can compare top/bottom subtyping here. + // due to the need to preserve Divergent markers. + // TODO: Resolve that issue so we can compare top/bottom subtyping here. !matches!(self.relation, TypeRelation::Redundancy { pure: false }) || target == target.materialize_impl( diff --git a/crates/ty_python_semantic/src/types/infer/builder/subscript.rs b/crates/ty_python_semantic/src/types/infer/builder/subscript.rs index 77dadab58c..0461705187 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/subscript.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/subscript.rs @@ -233,10 +233,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { } } - let tuple_generic_alias = |env: &ProgramEnvironment<'db>, tuple: Option>| { - let tuple = tuple.unwrap_or_else(|| TupleType::homogeneous(db, env, Type::unknown())); - Type::from(tuple.to_class_type(db)) - }; + let tuple_generic_alias = |tuple: TupleType<'db>| Type::from(tuple.to_class_type(db)); match value_ty { Type::ClassLiteral(class) => { @@ -248,7 +245,6 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // special cases, too. if class.is_tuple(db) { return Ok(tuple_generic_alias( - env, self.infer_tuple_type_expression(subscript), )); } else if class.is_known(db, KnownClass::Type) { @@ -282,7 +278,6 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { Type::SpecialForm(special_form) => match special_form { SpecialFormType::Tuple => { return Ok(tuple_generic_alias( - env, self.infer_tuple_type_expression(subscript), )); } diff --git a/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs b/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs index 0d92e39c31..33187fcbac 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs @@ -1071,7 +1071,7 @@ impl<'db> TypeInferenceBuilder<'db, '_> { pub(super) fn infer_tuple_type_expression( &mut self, tuple: &ast::ExprSubscript, - ) -> Option> { + ) -> TupleType<'db> { let db = self.db(); let env = self.program_environment(); match &*tuple.slice { @@ -1099,8 +1099,8 @@ impl<'db> TypeInferenceBuilder<'db, '_> { ); } let result = TupleType::homogeneous(db, env, element_ty); - self.store_expression_type(&tuple.slice, Type::tuple(Some(result))); - return Some(result); + self.store_expression_type(&tuple.slice, Type::tuple(result)); + return result; } let mut element_types = TupleSpecBuilder::with_capacity(elements.len()); @@ -1326,8 +1326,7 @@ impl<'db> TypeInferenceBuilder<'db, '_> { if class_literal.is_tuple(self.db()) { let class_type = self .infer_tuple_type_expression(subscript) - .map(|tuple_type| tuple_type.to_class_type(self.db())) - .unwrap_or_else(|| class_literal.default_specialization(db)); + .to_class_type(self.db()); SubclassOfType::from(db, env, class_type) } else { match class_literal.generic_context(db) { diff --git a/crates/ty_python_semantic/src/types/instance.rs b/crates/ty_python_semantic/src/types/instance.rs index ac26e7c9b1..624ba85e5f 100644 --- a/crates/ty_python_semantic/src/types/instance.rs +++ b/crates/ty_python_semantic/src/types/instance.rs @@ -99,11 +99,8 @@ impl<'db> Type<'db> { } } - pub(crate) fn tuple(tuple: Option>) -> Self { - let Some(tuple) = tuple else { - return Type::Never; - }; - Type::tuple_instance(tuple) + pub(crate) fn tuple(tuple: TupleType<'db>) -> Self { + Type::NominalInstance(NominalInstanceType(NominalInstanceInner::ExactTuple(tuple))) } pub fn homogeneous_tuple( @@ -111,7 +108,7 @@ impl<'db> Type<'db> { env: &ProgramEnvironment<'db>, element: Type<'db>, ) -> Self { - Type::tuple_instance(TupleType::homogeneous(db, env, element)) + Type::tuple(TupleType::homogeneous(db, env, element)) } pub(crate) fn heterogeneous_tuple( @@ -131,12 +128,7 @@ impl<'db> Type<'db> { } pub(crate) fn empty_tuple(db: &'db dyn Db, env: &ProgramEnvironment<'db>) -> Self { - Type::tuple_instance(TupleType::empty(db, env)) - } - - /// **Private** helper function to create a `Type::NominalInstance` from a tuple. - fn tuple_instance(tuple: TupleType<'db>) -> Self { - Type::NominalInstance(NominalInstanceType(NominalInstanceInner::ExactTuple(tuple))) + Type::tuple(TupleType::empty(db, env)) } pub(crate) const fn sys_version_info() -> Self { diff --git a/crates/ty_python_semantic/src/types/relation.rs b/crates/ty_python_semantic/src/types/relation.rs index d35916794b..4535c0e345 100644 --- a/crates/ty_python_semantic/src/types/relation.rs +++ b/crates/ty_python_semantic/src/types/relation.rs @@ -1713,11 +1713,7 @@ impl<'a, 'c, 'db> TypeRelationChecker<'a, 'c, 'db> { { self.check_type_pair( db, - Type::tuple(Some(TupleType::unpacked_typevartuple( - db, - env, - bound_typevar, - ))), + Type::tuple(TupleType::unpacked_typevartuple(db, env, bound_typevar)), target, ) } @@ -1729,11 +1725,7 @@ impl<'a, 'c, 'db> TypeRelationChecker<'a, 'c, 'db> { self.check_type_pair( db, source, - Type::tuple(Some(TupleType::unpacked_typevartuple( - db, - env, - bound_typevar, - ))), + Type::tuple(TupleType::unpacked_typevartuple(db, env, bound_typevar)), ) } diff --git a/crates/ty_python_semantic/src/types/tuple.rs b/crates/ty_python_semantic/src/types/tuple.rs index d1927128e4..12fd27b7bc 100644 --- a/crates/ty_python_semantic/src/types/tuple.rs +++ b/crates/ty_python_semantic/src/types/tuple.rs @@ -10,11 +10,8 @@ //! //! The description of which elements can appear in a `tuple` is called a [`TupleSpec`]. Other //! things besides `tuple` instances can be described by a tuple spec — for instance, the targets -//! of an unpacking assignment. A `tuple` specialization that includes `Never` as one of its -//! fixed-length elements cannot be instantiated. We reduce the entire `tuple` type down to -//! `Never`. The same is not true of tuple specs in general. (That means that it is [`TupleType`] -//! that adds that "collapse `Never`" behavior, whereas [`TupleSpec`] allows you to add any element -//! types, including `Never`.) +//! of an unpacking assignment. A `tuple` specialization can include `Never` as a fixed-length +//! element because a user-defined tuple subclass can inhabit that type. use crate::{Program, ProgramEnvironment}; use std::cmp::Ordering; @@ -176,13 +173,7 @@ impl<'db> TupleType<'db> { db: &'db dyn Db, env: &ProgramEnvironment<'db>, spec: &TupleSpec<'db>, - ) -> Option { - // If a fixed-length (i.e., mandatory) element of the tuple is `Never`, then it's not - // possible to instantiate the tuple as a whole. - if spec.fixed_elements().any(Type::is_never) { - return None; - } - + ) -> Self { // If the variable-length portion is Never, it can only be instantiated with zero elements. // That means this isn't a variable-length tuple after all! if let TupleSpec::Variable(tuple) = spec @@ -193,10 +184,10 @@ impl<'db> TupleType<'db> { .iter_prefix_elements() .chain(tuple.iter_suffix_elements()), )); - return Some(TupleType::new_internal(db, env.program(db), tuple)); + return TupleType::new_internal(db, env.program(db), tuple); } - Some(TupleType::new_internal(db, env.program(db), spec)) + TupleType::new_internal(db, env.program(db), spec) } pub(crate) fn empty(db: &'db dyn Db, env: &ProgramEnvironment<'db>) -> Self { @@ -211,7 +202,7 @@ impl<'db> TupleType<'db> { db: &'db dyn Db, env: &ProgramEnvironment<'db>, types: impl IntoIterator>, - ) -> Option { + ) -> Self { TupleType::new(db, env, &TupleSpec::heterogeneous(types)) } @@ -221,7 +212,7 @@ impl<'db> TupleType<'db> { prefix: impl IntoIterator>, variable: Type<'db>, suffix: impl IntoIterator>, - ) -> Option { + ) -> Self { Self::mixed_with_segment( db, env, @@ -237,7 +228,7 @@ impl<'db> TupleType<'db> { prefix: impl IntoIterator>, variable: VariableSegment<'db>, suffix: impl IntoIterator>, - ) -> Option { + ) -> Self { TupleType::new( db, env, @@ -311,7 +302,7 @@ impl<'db> TupleType<'db> { type_mapping: &TypeMapping<'a, 'db>, tcx: TypeContext<'db>, visitor: &ApplyTypeMappingVisitor<'_, 'db>, - ) -> Option { + ) -> Self { TupleType::new( db, visitor.env, @@ -782,10 +773,6 @@ fn to_class_type_cycle_initial<'db>( } /// A tuple spec describes the contents of a tuple type, which might be fixed- or variable-length. -/// -/// Tuple specs are used for more than just `tuple` instances, so they allow `Never` to appear as a -/// fixed-length element type. [`TupleType`] adds that additional invariant (since a tuple that -/// must contain an element that can't be instantiated, can't be instantiated itself). pub(crate) type TupleSpec<'db> = Tuple, VariableSegment<'db>>; /// The variable-length portion of a [`TupleSpec`]. From 432402503d4af400e48cbaa551589346c70386ee Mon Sep 17 00:00:00 2001 From: Brent Westbrook <36778786+ntBre@users.noreply.github.com> Date: Thu, 13 Aug 2026 13:04:30 -0400 Subject: [PATCH 024/371] Add default indicator to rules table (#27724) Summary -- I saw this suggestion on [Discord]: > Would it make sense to put a mark in the full rule page for rules that are selected by default? I feel like that would be very useful in seeing for example what rules are not selected from a group like UP by default. The extra "default rules" page doesn't help with that, for example. > Like a blue dot, maybe in front of the rules in the table. This PR implements this suggestion with a couple of modifications. I included a check mark instead of a blue dot and as an additional indicator at the end of each row, near the preview, fix, and other indicators, rather than at the start. Test Plan -- Local build: image [Discord]: https://discord.com/channels/1039017663004942429/1070132471699607623/1536942546972839957 --- crates/ruff_dev/src/generate_rules_table.rs | 37 +++++++++++++++++---- 1 file changed, 30 insertions(+), 7 deletions(-) diff --git a/crates/ruff_dev/src/generate_rules_table.rs b/crates/ruff_dev/src/generate_rules_table.rs index 82b6458f85..9e23e72c3b 100644 --- a/crates/ruff_dev/src/generate_rules_table.rs +++ b/crates/ruff_dev/src/generate_rules_table.rs @@ -10,23 +10,31 @@ use strum::IntoEnumIterator; use ruff_linter::FixAvailability; use ruff_linter::registry::{Linter, Rule, RuleNamespace}; +use ruff_linter::settings::LinterSettings; +use ruff_linter::settings::rule_table::RuleTable; use ruff_linter::upstream_categories::UpstreamCategoryAndPrefix; use ruff_options_metadata::OptionsMetadata; use ruff_workspace::options::Options; +const DEFAULT_SYMBOL: &str = "✅"; const FIX_SYMBOL: &str = "🛠️"; const PREVIEW_SYMBOL: &str = "🧪"; const REMOVED_SYMBOL: &str = "❌"; const WARNING_SYMBOL: &str = "⚠️"; const SPACER: &str = "    "; -/// Style for the rule's fixability and status icons. +/// Style for the rule's default selection, fixability, and status icons. const SYMBOL_STYLE: &str = "style='width: 1em; display: inline-block;'"; -/// Style for the container wrapping the fixability and status icons. +/// Style for the container wrapping the default selection, fixability, and status icons. const SYMBOLS_CONTAINER: &str = "style='display: flex; gap: 0.5rem; justify-content: end;'"; -fn generate_table(table_out: &mut String, rules: impl IntoIterator, linter: &Linter) { - table_out.push_str("| Code { scope='col' } | Name { scope='col' } | Message { scope='col' } | Fix/Status { scope='col' .sr-only } |"); +fn generate_table( + table_out: &mut String, + rules: impl IntoIterator, + linter: &Linter, + default_rules: &RuleTable, +) { + table_out.push_str("| Code { scope='col' } | Name { scope='col' } | Message { scope='col' } | Status/Fix/Default { scope='col' .sr-only } |"); table_out.push('\n'); table_out.push_str("| ---- | ---- | ------- | -: |"); table_out.push('\n'); @@ -63,6 +71,14 @@ fn generate_table(table_out: &mut String, rules: impl IntoIterator, FixAvailability::None => format!(""), }; + let default_token = if default_rules.enabled(rule) { + format!( + "Enabled by default" + ) + } else { + format!("") + }; + let rule_name = rule.name(); // If the message ends in a bracketed expression (like: "Use {replacement}"), escape the @@ -89,7 +105,7 @@ fn generate_table(table_out: &mut String, rules: impl IntoIterator, #[expect(clippy::or_fun_call)] let _ = write!( table_out, - "| {ss}{prefix}{code}{se} {{ #{prefix}{code} }} | {ss}{explanation}{se} | {ss}{message}{se} |
{status_token}{fix_token}
|", + "| {ss}{prefix}{code}{se} {{ #{prefix}{code} }} | {ss}{explanation}{se} | {ss}{message}{se} |
{status_token}{fix_token}{default_token}
|", prefix = linter.common_prefix(), code = linter.code_for_rule(rule).unwrap(), explanation = rule @@ -132,10 +148,17 @@ pub(crate) fn generate() -> String { &mut table_out, "{SPACER}{FIX_SYMBOL}{SPACER} The rule is automatically fixable by the `--fix` command-line option." ); + table_out.push_str("
"); + + let _ = write!( + &mut table_out, + "{SPACER}{DEFAULT_SYMBOL}{SPACER} The rule is enabled by default." + ); table_out.push_str("\n\n"); table_out.push_str("All rules not marked as preview, deprecated or removed are stable."); table_out.push('\n'); + let default_rules = LinterSettings::default().rules; for linter in Linter::iter() { let codes_csv: String = match linter.common_prefix() { "" => linter @@ -222,10 +245,10 @@ pub(crate) fn generate() -> String { } table_out.push('\n'); table_out.push('\n'); - generate_table(&mut table_out, rules.clone(), &linter); + generate_table(&mut table_out, rules.clone(), &linter, &default_rules); } } else { - generate_table(&mut table_out, linter.all_rules(), &linter); + generate_table(&mut table_out, linter.all_rules(), &linter, &default_rules); } } From a46f553de35bf43ef2ed1a8fc68c0feaf8e262f4 Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Thu, 13 Aug 2026 13:27:12 -0400 Subject: [PATCH 025/371] [ty] Treat generator-expression exceptions as eagerly evaluated (#27735) ## Summary Previously, generator expressions were assumed to execute eagerly for name resolution and assignment expressions, but lazily when determining whether exceptions could reach surrounding handlers: ```python def may_raise() -> None: ... caught = False try: ((value := 1, may_raise()) for _ in [0]) except Exception: caught = True reveal_type(value) # int reveal_type(caught) # Previously: Literal[False]; now: bool ``` We now apply the same eager-execution assumption to exception flow, allowing potentially raising operations inside generator expressions to reach enclosing exception handlers. Generator expressions now follow the existing exception-flow behavior of list, set, and dictionary comprehensions. --- crates/ty_python_core/src/builder.rs | 16 ++++++---------- .../resources/mdtest/exception/control_flow.md | 18 +++++++++++------- 2 files changed, 17 insertions(+), 17 deletions(-) diff --git a/crates/ty_python_core/src/builder.rs b/crates/ty_python_core/src/builder.rs index 440a59cc8b..953a83dc6f 100644 --- a/crates/ty_python_core/src/builder.rs +++ b/crates/ty_python_core/src/builder.rs @@ -2213,22 +2213,20 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { } /// Returns whether an exception raised while evaluating `scope` can propagate directly to its - /// enclosing scope. Generator-expression bodies are lazy even though they share the - /// comprehension scope kind; their eagerly evaluated first iterable is visited before entering - /// the generator scope. + /// enclosing scope. /// - /// Only the list comprehension's call can reach the handler immediately: + /// Generator expressions follow the eager comprehension-scope convention used throughout our + /// flow model. Although generators are lazy at runtime, their bodies are assumed to execute + /// immediately, since in practice they are almost always eagerly iterated over. /// /// ```python /// try: - /// [may_raise() for _ in [0]] /// (may_raise() for _ in [0]) /// except Exception: /// ... /// ``` fn exception_checkpoint_crosses_scope_boundary(&self, scope_id: FileScopeId) -> bool { - let scope = &self.scopes[scope_id]; - scope.is_eager() && !matches!(scope.node(), NodeWithScopeKind::GeneratorExpression(_)) + self.scopes[scope_id].is_eager() } /// Records the current flow state immediately before an operation that may raise an exception. @@ -2896,9 +2894,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { self.record_exception_checkpoint_if(loopback_can_raise); let nested_bindings = self.pop_scope(); self.synthesize_comprehension_binding_definitions(nested_bindings); - if !matches!(scope, NodeWithScopeRef::GeneratorExpression(_)) { - self.record_exception_checkpoint(); - } + self.record_exception_checkpoint(); self.current_assignments = saved_assignments; diff --git a/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md b/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md index c4eec2a8fe..96adadc938 100644 --- a/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md +++ b/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md @@ -791,7 +791,8 @@ except: reveal_type(y) # revealed: Literal[0, 1] ``` -A generator expression does not run its body until the generator is consumed: +Generator expressions are also assumed to run eagerly for exception-flow analysis, since in practice +they are almost always eagerly consumed in real-world code: ```py z = 0 @@ -800,7 +801,7 @@ try: except: z = 1 -reveal_type(z) # revealed: Literal[0] +reveal_type(z) # revealed: Literal[0, 1] ``` A nested function body also runs later, so its exceptions cannot reach the handler surrounding its @@ -870,22 +871,25 @@ def dict_comprehension_assignment() -> None: reveal_type(state) # revealed: Literal["before"] | int ``` -## Assignments in lazy generator expressions +## Assignments in generator expressions -Assignments and calls in a generator expression do not execute when the generator is created, so -they do not make the surrounding exception handler reachable: +Generator expressions are assumed to run eagerly, so their assignments and calls can reach the +surrounding exception handler. Strictly speaking generator expressions *can* be lazy, but in +practice they are almost always eagerly consumed in real-world code: ```py def generator_may_raise() -> None: ... -def lazy_generator_assignment() -> None: +def generator_assignment() -> None: state = 0 caught = False try: ((state := 1, generator_may_raise()) for _ in [0]) except: + reveal_type(state) # revealed: int caught = True - reveal_type(caught) # revealed: Literal[False] + reveal_type(caught) # revealed: bool + reveal_type(state) # revealed: int ``` ## Nested comprehension assignments From d332d20d3f1f784f80c4f1adf7b08897f3e058e4 Mon Sep 17 00:00:00 2001 From: Swayam Mhaskar Date: Thu, 13 Aug 2026 23:18:40 +0530 Subject: [PATCH 026/371] [syntax-errors] Name is parameter and nonlocal (#27628) ## Summary Part of #17412 Detects semantic syntax error where name is parameter and nonlocal ## Test Plan Added tests in `nonlocal_parameter.py` --- .../semantic_errors/nonlocal_parameter.py | 33 +++++++++++ crates/ruff_linter/src/checkers/ast/mod.rs | 1 + crates/ruff_linter/src/linter.rs | 1 + ...ntax_error_nonlocal_parameter.py_3.10.snap | 56 +++++++++++++++++++ .../ruff_python_parser/src/semantic_errors.rs | 21 ++++++- crates/ty_python_core/src/builder.rs | 4 +- .../diagnostics/semantic_syntax_errors.md | 56 +++++++++++++++++++ 7 files changed, 170 insertions(+), 2 deletions(-) create mode 100644 crates/ruff_linter/resources/test/fixtures/semantic_errors/nonlocal_parameter.py create mode 100644 crates/ruff_linter/src/snapshots/ruff_linter__linter__tests__semantic_syntax_error_nonlocal_parameter.py_3.10.snap diff --git a/crates/ruff_linter/resources/test/fixtures/semantic_errors/nonlocal_parameter.py b/crates/ruff_linter/resources/test/fixtures/semantic_errors/nonlocal_parameter.py new file mode 100644 index 0000000000..b83f9ee608 --- /dev/null +++ b/crates/ruff_linter/resources/test/fixtures/semantic_errors/nonlocal_parameter.py @@ -0,0 +1,33 @@ +def f(a): + nonlocal a + +def g(a): + if True: + nonlocal a + +def h(a): + def inner(): + nonlocal a + +def i(a): + try: + nonlocal a + except Exception: + pass + +def f(a): + a = 1 + a = 2 + nonlocal a + +def f(a): + class Inner: + nonlocal a # ok + +def f(a): + def inner(a): + nonlocal a + +def f(a=1): + def inner(): + nonlocal a # ok \ No newline at end of file diff --git a/crates/ruff_linter/src/checkers/ast/mod.rs b/crates/ruff_linter/src/checkers/ast/mod.rs index aad92bd5e6..abb8ffd40a 100644 --- a/crates/ruff_linter/src/checkers/ast/mod.rs +++ b/crates/ruff_linter/src/checkers/ast/mod.rs @@ -810,6 +810,7 @@ impl SemanticSyntaxContext for Checker<'_> { | SemanticSyntaxErrorKind::DifferentMatchPatternBindings | SemanticSyntaxErrorKind::InvalidExpression(..) | SemanticSyntaxErrorKind::GlobalParameter(_) + | SemanticSyntaxErrorKind::NonlocalParameter(_) | SemanticSyntaxErrorKind::DuplicateMatchKey(_) | SemanticSyntaxErrorKind::DuplicateMatchClassAttribute(_) | SemanticSyntaxErrorKind::InvalidStarExpression diff --git a/crates/ruff_linter/src/linter.rs b/crates/ruff_linter/src/linter.rs index d1491ad0b2..fc07c4e93c 100644 --- a/crates/ruff_linter/src/linter.rs +++ b/crates/ruff_linter/src/linter.rs @@ -1032,6 +1032,7 @@ mod tests { #[test_case(Path::new("write_to_debug.py"), PythonVersion::PY310)] #[test_case(Path::new("invalid_expression.py"), PythonVersion::PY312)] #[test_case(Path::new("global_parameter.py"), PythonVersion::PY310)] + #[test_case(Path::new("nonlocal_parameter.py"), PythonVersion::PY310)] #[test_case(Path::new("annotated_global.py"), PythonVersion::PY314)] #[test_case(Path::new("lazy_future_import.py"), PythonVersion::PY315)] fn test_semantic_errors(path: &Path, python_version: PythonVersion) -> Result<()> { diff --git a/crates/ruff_linter/src/snapshots/ruff_linter__linter__tests__semantic_syntax_error_nonlocal_parameter.py_3.10.snap b/crates/ruff_linter/src/snapshots/ruff_linter__linter__tests__semantic_syntax_error_nonlocal_parameter.py_3.10.snap new file mode 100644 index 0000000000..0331d685a0 --- /dev/null +++ b/crates/ruff_linter/src/snapshots/ruff_linter__linter__tests__semantic_syntax_error_nonlocal_parameter.py_3.10.snap @@ -0,0 +1,56 @@ +--- +source: crates/ruff_linter/src/linter.rs +--- +invalid-syntax: name `a` cannot refer to a parameter and a nonlocal variable + --> resources/test/fixtures/semantic_errors/nonlocal_parameter.py:2:14 + | +1 | def f(a): +2 | nonlocal a + | ^ +3 | +4 | def g(a): + | + +invalid-syntax: name `a` cannot refer to a parameter and a nonlocal variable + --> resources/test/fixtures/semantic_errors/nonlocal_parameter.py:6:18 + | +4 | def g(a): +5 | if True: +6 | nonlocal a + | ^ +7 | +8 | def h(a): + | + +invalid-syntax: name `a` cannot refer to a parameter and a nonlocal variable + --> resources/test/fixtures/semantic_errors/nonlocal_parameter.py:14:18 + | +12 | def i(a): +13 | try: +14 | nonlocal a + | ^ +15 | except Exception: +16 | pass + | + +invalid-syntax: name `a` cannot refer to a parameter and a nonlocal variable + --> resources/test/fixtures/semantic_errors/nonlocal_parameter.py:21:14 + | +19 | a = 1 +20 | a = 2 +21 | nonlocal a + | ^ +22 | +23 | def f(a): + | + +invalid-syntax: name `a` cannot refer to a parameter and a nonlocal variable + --> resources/test/fixtures/semantic_errors/nonlocal_parameter.py:29:18 + | +27 | def f(a): +28 | def inner(a): +29 | nonlocal a + | ^ +30 | +31 | def f(a=1): + | diff --git a/crates/ruff_python_parser/src/semantic_errors.rs b/crates/ruff_python_parser/src/semantic_errors.rs index 9a9273a750..4d41444c82 100644 --- a/crates/ruff_python_parser/src/semantic_errors.rs +++ b/crates/ruff_python_parser/src/semantic_errors.rs @@ -324,7 +324,13 @@ impl SemanticSyntaxChecker { if !ctx.in_module_scope() { for name in names { - if !ctx.has_nonlocal_binding(name) { + if ctx.is_bound_parameter(name) { + Self::add_error( + ctx, + SemanticSyntaxErrorKind::NonlocalParameter(name.to_string()), + name.range, + ); + } else if !ctx.has_nonlocal_binding(name) { Self::add_error( ctx, SemanticSyntaxErrorKind::NonlocalWithoutBinding(name.to_string()), @@ -1432,6 +1438,12 @@ impl Display for SemanticSyntaxError { "name `{name}` cannot refer to a parameter and a global variable" ) } + SemanticSyntaxErrorKind::NonlocalParameter(name) => { + write!( + f, + "name `{name}` cannot refer to a parameter and a nonlocal variable" + ) + } SemanticSyntaxErrorKind::DifferentMatchPatternBindings => { write!(f, "alternative patterns bind different names") } @@ -1871,6 +1883,13 @@ pub enum SemanticSyntaxErrorKind { /// ambiguity and will result in a `SyntaxError`. GlobalParameter(String), + /// Represents a function parameter that is also declared as `nonlocal`. + /// + /// Declaring a parameter as `nonlocal` is invalid, since parameters are already + /// bound in a local scope of the function. using `nonlocal` on them introduces + /// ambiguity and will result in a `SyntaxError`. + NonlocalParameter(String), + /// Represents the use of alternative patterns in a `match` statement that bind different names. /// /// Python requires all alternatives in an OR pattern (`|`) to bind the same set of names. diff --git a/crates/ty_python_core/src/builder.rs b/crates/ty_python_core/src/builder.rs index 953a83dc6f..c812523590 100644 --- a/crates/ty_python_core/src/builder.rs +++ b/crates/ty_python_core/src/builder.rs @@ -4589,7 +4589,9 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { let symbol_id = self.add_symbol(name.id.clone()); let symbol = self.current_place_table().symbol(symbol_id); // Check whether the variable has already been accessed in this scope. - if symbol.is_bound() || symbol.is_declared() || symbol.is_used() { + if (symbol.is_bound() || symbol.is_declared() || symbol.is_used()) + && !symbol.is_parameter() + { self.report_semantic_error(SemanticSyntaxError { kind: SemanticSyntaxErrorKind::LoadBeforeNonlocalDeclaration { name: name.to_string(), diff --git a/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md b/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md index a12a65f664..433d0b08bb 100644 --- a/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md +++ b/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md @@ -603,3 +603,59 @@ error[invalid-syntax]: name `a` cannot refer to a parameter and a global variabl 27 | global a # snapshot: invalid-syntax | ^ ``` + +## name cannot refer to a parameter and a nonlocal variable + +```py +a = None + +def outer(): + a = None + def f(a): + nonlocal a # snapshot: invalid-syntax + +def outer(): + a = None + def g(a): + if True: + nonlocal a # error: [invalid-syntax] + +def h(a): + def inner(): + nonlocal a + +def outer(): + a = None + def i(a): + try: + nonlocal a # error: [invalid-syntax] + except Exception: + pass + +def outer(): + a = None + def f(a): + a = 1 + a = 2 + nonlocal a # error: [invalid-syntax] + +def f(a): + class Inner: + nonlocal a + +def f(a): + def inner(a): + nonlocal a # error: [invalid-syntax] + +def f(a=1): + def inner(): + nonlocal a +``` + +```snapshot +error[invalid-syntax]: name `a` cannot refer to a parameter and a nonlocal variable + --> src/mdtest_snippet.py:6:18 + | +6 | nonlocal a # snapshot: invalid-syntax + | ^ +``` From 04d59f43abf8e6e6df8643202ef3f03fb39a6ed2 Mon Sep 17 00:00:00 2001 From: Andrej730 Date: Fri, 14 Aug 2026 01:21:59 +0500 Subject: [PATCH 027/371] [ty] Report deprecated unary operations (#27584) ## Summary Adding support for deprecated unary operations, supported cases: - flagging general deprecated unary dunder methods on any class - flagging `__invert__` as deprecated for `Literal[True]`, `Literal[False]`, while preserving the bitwise math applied to the underlying value - flagging deprecated unary ops on type variables constraints Known issues (both will be addressed separately): - flagging deprecated `__bool__` during `not x` unary operation is not considered - possibly duplicated `deprecated` diagnostics in case of unions, when deprecated method on both classes is coming from the same common ancestor Some examples: ```python # Reported before this PR as `deprecated` True.__invert__ # deprecated # Reported after this PR: def test(a: bool): ~a # deprecated ~True # deprecated from typing_extensions import deprecated class A: @deprecated("deprecated!") def __invert__(self): ... ~A() # deprecated ```
Example diagnostics ```python warning[deprecated]: The function `__invert__` is deprecated --> example_bool:2:6 | 2 | True.__invert__ # deprecated | ^^^^^^^^^^ Will throw an error in Python 3.16. Use `not` for logical negation of bools instead. warning[deprecated]: The function `__invert__` is deprecated --> example_bool:7:5 | 7 | ~a # deprecated | ^^ Will throw an error in Python 3.16. Use `not` for logical negation of bools instead. warning[deprecated]: The function `__invert__` is deprecated --> example_bool:9:1 | 9 | ~True # deprecated | ^^^^^ Will throw an error in Python 3.16. Use `not` for logical negation of bools instead. warning[deprecated]: The function `__invert__` is deprecated --> example_bool:19:1 | 19 | ~A() # deprecated | ^^^^ deprecated! Found 4 diagnostics ```
## Test Plan Added mdtests to `deprecated.md`, updated `integers.md` test to consider deprecation diagnostic. All previous tests also pass. --------- Co-authored-by: Carl Meyer Co-authored-by: Carl Meyer --- .../resources/mdtest/deprecated.md | 182 ++++++++++++++++++ .../resources/mdtest/unary/integers.md | 6 +- .../ty_python_semantic/src/types/call/bind.rs | 9 +- .../src/types/infer/builder.rs | 108 +++++++++-- 4 files changed, 288 insertions(+), 17 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/deprecated.md b/crates/ty_python_semantic/resources/mdtest/deprecated.md index 55e9d41237..d209095f80 100644 --- a/crates/ty_python_semantic/resources/mdtest/deprecated.md +++ b/crates/ty_python_semantic/resources/mdtest/deprecated.md @@ -393,6 +393,8 @@ AliasClass() # error: [deprecated] "Use OtherType instead" ## Dunders +### Binary operators + If a dunder like `__add__` is deprecated, then the equivalent syntactic sugar like `+` should fire a diagnostic. @@ -412,6 +414,186 @@ y = MyInt(2) z = x + y # TODO error: [deprecated] "MyInt `+` support is broken" ``` +### Unary operators + +If a dunder like `__invert__` is deprecated, then the equivalent `~` operator should fire a +diagnostic. + +#### Custom operator + +```py +from typing_extensions import deprecated + +class MyBits: + @deprecated("MyBits `~` support is broken") + def __invert__(self): + return self + +x = MyBits() +~x # error: [deprecated] "MyBits `~` support is broken" +``` + +#### Possibly unbound operator + +If the operand's type is a union and the dunder is missing on some members, it's possibly unbound. +This should still report the deprecation on the members where it is found and is deprecated, +alongside `unsupported-operator` diagnostic. + +```py +from typing_extensions import deprecated + +class MyBits: + @deprecated("MyBits `~` support is broken") + def __invert__(self): + return self + +class NoBits: ... + +def f(x: MyBits | NoBits): + # error: [unsupported-operator] + # error: [deprecated] + ~x +``` + +#### Unions and intersections + +A union reports a deprecated operator when any alternative is deprecated. An intersection reports +deprecated operators only when every applicable implementation is deprecated. + +```py +from typing_extensions import deprecated + +class Deprecated: + @deprecated("old inversion") + def __invert__(self) -> int: + return 1 + +class AlsoDeprecated: + @deprecated("another old inversion") + def __invert__(self) -> int: + return 2 + +class Ordinary: + def __invert__(self) -> int: + return 3 + +def mixed_union(value: Deprecated | Ordinary) -> None: + ~value # error: [deprecated] "old inversion" + +def mixed_intersection(value: Deprecated) -> None: + if isinstance(value, Ordinary): + ~value + +def deprecated_intersection(value: Deprecated) -> None: + if isinstance(value, AlsoDeprecated): + # error: [deprecated] "old inversion" + # error: [deprecated] "old inversion" + ~value +``` + +A gradually typed comparison can produce an intersection of `bool` and `Any`. The unknown +alternative might provide a nondeprecated operator, so inverting it should not warn. + +```py +from typing import Any + +def gradual_intersection(value: Any) -> None: + if value is None: + return + + mask = value == 0 + ~mask +``` + +#### Bool literals + +`bool.__invert__` is one such case in typeshed. This applies both to `bool` literals and to +arbitrary values of type `bool`. + +```py +~True # error: [deprecated] + +def f(x: bool): + ~x # error: [deprecated] +``` + +#### Constrained TypeVars + +Type variable constraints also should be checked. + +```py +from typing import TypeVar +from typing_extensions import deprecated + +class First: + @deprecated("first") + def __invert__(self) -> int: + return 42 + +class Second: + @deprecated("second") + def __invert__(self) -> int: + return 42 + +T = TypeVar("T", First, Second) + +def f(value: T) -> None: + # error: [deprecated] "first" + # error: [deprecated] "second" + ~value +``` + +Deprecation reporting for one constraint does not depend on whether another constraint supports the +operator or on the order of the constraints. + +```py +class Third: ... + +U = TypeVar("U", Third, First) +V = TypeVar("V", First, Third) + +def g(value: U) -> None: + # error: [unsupported-operator] + # error: [deprecated] + ~value + +def h(value: V) -> None: + # error: [unsupported-operator] + # error: [deprecated] + ~value +``` + +A constraint that is itself a union may contain a deprecated operator even when that operator is +missing from another union member. + +```py +W = TypeVar("W", First | Third, Second) + +def nested_union(value: W) -> None: + # error: [unsupported-operator] + # error: [deprecated] "first" + # error: [deprecated] "second" + ~value +``` + +A deprecated operator should also be reported when its signature cannot accept the implicit unary +call. + +```py +class Invalid: + @deprecated("invalid inversion") + def __invert__(self, required: int) -> int: + return required + +X = TypeVar("X", Invalid, Second) + +def invalid_operator(value: X) -> None: + # error: [unsupported-operator] + # error: [deprecated] "invalid inversion" + # error: [deprecated] "second" + ~value +``` + ## Overloads Overloads can be deprecated, but only trigger warnings when invoked. diff --git a/crates/ty_python_semantic/resources/mdtest/unary/integers.md b/crates/ty_python_semantic/resources/mdtest/unary/integers.md index ec439977ed..596e6f951f 100644 --- a/crates/ty_python_semantic/resources/mdtest/unary/integers.md +++ b/crates/ty_python_semantic/resources/mdtest/unary/integers.md @@ -21,5 +21,9 @@ reveal_type(-True) # revealed: Literal[-1] ```py reveal_type(~0) # revealed: Literal[-1] reveal_type(~1) # revealed: Literal[-2] -reveal_type(~True) # revealed: Literal[-2] + +# `~` on a `bool` is currently deprecated in typeshed. +# error: [deprecated] +# revealed: Literal[-2] +reveal_type(~True) ``` diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index fa5d308e05..cd935720b4 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -453,7 +453,7 @@ impl<'db> BindingsElement<'db> { self.items.iter_mut() } - fn callables(&self) -> impl Iterator> { + fn callables(&self) -> impl Iterator> + Clone { self.items.iter().map(CallableItem::callable) } @@ -891,6 +891,13 @@ impl<'db> Bindings<'db> { self.implicit_dunder_init_is_possibly_unbound } + /// Returns the callable bindings for each union element without flattening intersections. + pub(crate) fn iter_union_elements( + &self, + ) -> impl Iterator> + Clone> + '_ { + self.elements.iter().map(BindingsElement::callables) + } + /// Returns an iterator over all `CallableBinding`s, flattening the two-level structure. /// /// Note: This loses the union/intersection distinction. The returned iterator yields diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index 9c04ec76d1..4957cb307a 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -9747,6 +9747,26 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { diag.add_primary_tag(ruff_db::diagnostic::DiagnosticTag::Deprecated); } + /// Report a deprecated callable only when its union alternative has no non-deprecated + /// intersection member that could provide the implementation instead. + fn check_deprecated_bindings(&self, ranged: &T, bindings: &Bindings<'db>) { + let db = self.db(); + + for callables in bindings.iter_union_elements() { + if callables.clone().all(|callable| { + let ty = match callable.callable_type { + Type::BoundMethod(bound) => Type::FunctionLiteral(bound.function(db)), + ty => ty, + }; + ty.is_deprecated(db) + }) { + for callable in callables { + self.check_deprecated(ranged, callable.callable_type); + } + } + } + } + fn infer_name_load(&mut self, name_node: &ast::ExprName) -> Type<'db> { let db = self.db(); let expr = PlaceExpr::from_expr_name(name_node); @@ -10548,8 +10568,19 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { CallArguments::none(), TypeContext::default(), ) { - Ok(outcome) => outcome.return_type(db, env), + Ok(outcome) => { + self.check_deprecated_bindings(unary, &outcome); + outcome.return_type(db, env) + } Err(e) => { + let bindings = match &e { + CallDunderError::PossiblyUnbound { bindings, .. } => Some(bindings), + CallDunderError::CallError(_, bindings, _) => Some(bindings), + CallDunderError::MethodNotAvailable => None, + }; + if let Some(bindings) = bindings { + self.check_deprecated_bindings(unary, bindings); + } self.report_unsupported_unary_operator( unary, op, @@ -10589,7 +10620,26 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { (ast::UnaryOp::Invert, Type::LiteralValue(literal)) => match literal.kind() { LiteralValueTypeKind::Int(value) => Type::int_literal(!value.as_i64()), - LiteralValueTypeKind::Bool(value) => Type::int_literal(!i64::from(value)), + LiteralValueTypeKind::Bool(value) => { + // `~bool` is currently deprecated in typeshed. Technically we should + // similarly check for deprecation of dunder methods on all our literal + // type fast paths, but we choose not to pay that extra cost, since it is + // implausible that e.g. `int.__neg__` would ever be deprecated. + if let Some(dunder) = literal + .fallback_instance(db, env) + .member_lookup_with_policy( + db, + env, + "__invert__", + MemberLookupPolicy::NO_INSTANCE_FALLBACK, + ) + .place + .ignore_possibly_undefined() + { + self.check_deprecated(unary, dunder); + } + Type::int_literal(!i64::from(value)) + } _ => fallback_unary_expression_type(), }, @@ -10631,24 +10681,52 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { match tvar.typevar(self.db()).bound_or_constraints(db, env) { Some(TypeVarBoundOrConstraints::Constraints(constraints)) => { - match Self::map_constrained_typevar_constraints( + // Call the dunder method for every constraint up front so deprecation + // reporting doesn't depend on whether any constraint fails. + let outcomes: Vec<_> = constraints + .elements(db) + .iter() + .map(|constraint| { + constraint.try_call_dunder( + db, + env, + unary_dunder_method, + CallArguments::none(), + TypeContext::default(), + ) + }) + .collect(); + for outcome in &outcomes { + let bindings = match outcome { + Ok(bindings) => Some(bindings), + // A method can be deprecated even if it is missing from some + // union members or its signature rejects the implicit call. + // Preserve those bindings so the deprecation is reported + // alongside the unsupported-operator diagnostic. + Err( + CallDunderError::PossiblyUnbound { bindings, .. } + | CallDunderError::CallError(_, bindings, _), + ) => Some(bindings.as_ref()), + // A completely missing method has no bindings to inspect. + Err(CallDunderError::MethodNotAvailable) => None, + }; + if let Some(bindings) = bindings { + self.check_deprecated_bindings(unary, bindings); + } + } + + let mut outcomes = outcomes.into_iter(); + let result = Self::map_constrained_typevar_constraints( db, env, operand_type, constraints, - |constraint| { - constraint - .try_call_dunder( - db, - env, - unary_dunder_method, - CallArguments::none(), - TypeContext::default(), - ) - .map(|outcome| outcome.return_type(db, env)) - .ok() + |_constraint| { + let outcome = outcomes.next()?.ok()?; + Some(outcome.return_type(db, env)) }, - ) { + ); + match result { Some(ty) => ty, None => { // At least one constraint failed; report error. From 41e88e3984b85f68942605246834d0def3c719da Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 14 Aug 2026 10:50:46 +0100 Subject: [PATCH 028/371] Document file-watcher sandbox limitations (#27727) --- AGENTS.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 6324173da2..73d93ad54c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,6 +24,8 @@ Run all tests (using `nextest` for faster execution and setting `INSTA_FORCE_PAS CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run ``` +File-watcher tests do not work inside the sandbox. It is usually unnecessary to run them locally before filing a change unless you are certain that the change affects file-watching behavior. + Run tests for a specific crate: ```sh From 5794a52bee9118ee0749d3ce87095bcd49989432 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 14 Aug 2026 11:25:22 +0100 Subject: [PATCH 029/371] [ty] Ignore generic declaration metadata in staticness checks (#27692) ## Summary - Treat generic aliases as fully static when their actual type arguments are fully static, even if their type parameters have gradual bounds, constraints, or defaults. - Visit only a generic alias's concrete specialization when checking dynamic content, matching the existing protocol-specific traversal. - Cover modern and legacy generic declarations, generator return statements, and genuinely gradual specializations. This fixes the false negatives for the `unsound-*` rules identified in https://github.com/astral-sh/ty/issues/4238. --- .../resources/mdtest/directives/cast.md | 14 ++ .../resources/mdtest/function/return_type.md | 121 ++++++++++++++++++ .../ty_python_semantic/src/types/generics.rs | 10 ++ .../ty_python_semantic/src/types/visitor.rs | 9 ++ 4 files changed, 154 insertions(+) diff --git a/crates/ty_python_semantic/resources/mdtest/directives/cast.md b/crates/ty_python_semantic/resources/mdtest/directives/cast.md index 0311c79766..5f699a3484 100644 --- a/crates/ty_python_semantic/resources/mdtest/directives/cast.md +++ b/crates/ty_python_semantic/resources/mdtest/directives/cast.md @@ -97,6 +97,20 @@ def f(x: RecursiveAlias): cast(RecursiveAlias, x) ``` +## Redundant casts of tuple classes with unknown elements + +A tuple class with an `Unknown` element is not fully static, even when its other element is `object` +and their union simplifies to `object`. A cast involving that tuple class must not be reported as +redundant. + +```py +from typing import cast +from ty_extensions._internal import Unknown + +def cast_gradual_tuple_class(value: type[tuple[object, Unknown]]) -> None: + cast(type[tuple[object, Unknown]], value) +``` + ## Diagnostic snapshots ```py diff --git a/crates/ty_python_semantic/resources/mdtest/function/return_type.md b/crates/ty_python_semantic/resources/mdtest/function/return_type.md index d4ef4520c6..6215d3870a 100644 --- a/crates/ty_python_semantic/resources/mdtest/function/return_type.md +++ b/crates/ty_python_semantic/resources/mdtest/function/return_type.md @@ -836,6 +836,127 @@ def returns_list_containing_any() -> list[int]: return [returns_any()] ``` +## Regression test: `unsound-return-statement` with gradual generic declarations + +A specialized generic type is fully static if it has been specialized with fully static types, even +if the type parameter(s) it is generic over have non-fully-static bounds, constraints, or defaults. +A previous version of the rule incorrectly considered these specialized generic types as being +non-fully-static, leading to false negatives in the below examples: + +```toml +[environment] +python-version = "3.13" + +[rules] +unsound-return-statement = "error" +``` + +```py +from typing import Any, Generator, Generic, TypeVar + +class Bounded[T: Any]: ... +class Constrained[T: (int, Any)]: ... +class Defaulted[T = Any]: ... + +# `Bounded[int]`, `Constrained[int]` and `Defaulted[int]` are all fully static, +# despite their bounds/constraints/defaults not being fully static +def returns_bounded(value: Any) -> Bounded[int]: + return value # error: [unsound-return-statement] + +def returns_constrained(value: Any) -> Constrained[int]: + return value # error: [unsound-return-statement] + +def returns_defaulted(value: Any) -> Defaulted[int]: + return value # error: [unsound-return-statement] +``` + +The same applies to classes declared with legacy type variables: + +```py +BoundedT = TypeVar("BoundedT", bound=Any) +ConstrainedT = TypeVar("ConstrainedT", int, Any) +DefaultedT = TypeVar("DefaultedT", default=Any) + +class LegacyBounded(Generic[BoundedT]): ... +class LegacyConstrained(Generic[ConstrainedT]): ... +class LegacyDefaulted(Generic[DefaultedT]): ... + +def returns_legacy_bounded(value: Any) -> LegacyBounded[int]: + return value # error: [unsound-return-statement] + +def returns_legacy_constrained(value: Any) -> LegacyConstrained[int]: + return value # error: [unsound-return-statement] + +def returns_legacy_defaulted(value: Any) -> LegacyDefaulted[int]: + return value # error: [unsound-return-statement] +``` + +and to `return` statements in generator functions: + +```py +def generator_returns_bounded(value: Any) -> Generator[None, None, Bounded[int]]: + yield + return value # error: [unsound-return-statement] +``` + +A specialized generic type is nonetheless considered to be non-fully-static if it is specialized +with non-fully-static types: + +```py +def returns_gradual_bounded(value: Any) -> Bounded[Any]: + # no error + return value + +def returns_gradual_constrained(value: Any) -> Constrained[Any]: + # no error + return value + +def returns_gradual_defaulted(value: Any) -> Defaulted[Any]: + # no error + return value + +def returns_nested_gradual_bounded(value: Any) -> Bounded[list[Any]]: + # no error + return value +``` + +## Regression test: `unsound-return-statement` with tuple class objects + +A tuple class has only one generic parameter, so its element types are combined into a union. Its +original element types must still determine whether the tuple class is fully static. + +```toml +[environment] +python-version = "3.11" + +[rules] +unsound-return-statement = "error" +``` + +A tuple class with fully static elements forms a fully static return boundary: + +```py +from typing import Any + +def returns_static_tuple_class(value: Any) -> type[tuple[int, object]]: + return value # error: [unsound-return-statement] +``` + +A tuple class with an `Any` element remains gradual even though the union `object | Any` simplifies +to `object`: + +```py +def returns_gradual_tuple_class(value: Any) -> type[tuple[object, Any]]: + return value +``` + +An unpacked gradual tuple likewise makes the entire tuple class gradual: + +```py +def returns_gradual_variadic_tuple_class(value: Any) -> type[tuple[object, *tuple[Any, ...]]]: + return value +``` + ## Regression test: `unsound-return-statement` uses "pure redundancy" Internally, the rule uses "pure redundancy" rather than "impure redundancy". The following example diff --git a/crates/ty_python_semantic/src/types/generics.rs b/crates/ty_python_semantic/src/types/generics.rs index 2fe3031b78..68a707b836 100644 --- a/crates/ty_python_semantic/src/types/generics.rs +++ b/crates/ty_python_semantic/src/types/generics.rs @@ -1227,12 +1227,22 @@ pub struct Specialization<'db> { // The Salsa heap is tracked separately. impl get_size2::GetSize for Specialization<'_> {} +/// Visit specialization arguments and the generic declaration. pub(super) fn walk_specialization<'db, V: TypeVisitor<'db> + ?Sized>( db: &'db dyn Db, specialization: Specialization<'db>, visitor: &V, ) { walk_generic_context(db, specialization.generic_context(db), visitor); + walk_specialization_types(db, specialization, visitor); +} + +/// Visit specialization arguments without walking the generic declaration. +pub(super) fn walk_specialization_types<'db, V: TypeVisitor<'db> + ?Sized>( + db: &'db dyn Db, + specialization: Specialization<'db>, + visitor: &V, +) { for ty in specialization.types(db) { visitor.visit_type(db, *ty); } diff --git a/crates/ty_python_semantic/src/types/visitor.rs b/crates/ty_python_semantic/src/types/visitor.rs index cf7c4c2294..bfea95b34a 100644 --- a/crates/ty_python_semantic/src/types/visitor.rs +++ b/crates/ty_python_semantic/src/types/visitor.rs @@ -17,6 +17,7 @@ use crate::types::{ class::walk_generic_alias, cyclic::ActiveRecursionDetector, function::{FunctionType, walk_function_type}, + generics::walk_specialization_types, instance::{walk_nominal_instance_type, walk_protocol_instance_type}, known_instance::walk_known_instance_type, method::{walk_bound_method_type, walk_method_wrapper_type}, @@ -495,6 +496,14 @@ fn dynamic_content_impl<'db>( walk_type_with_recursion_guard(db, ty, self, &self.recursion_guard); } + fn visit_generic_alias_type(&self, db: &'db dyn Db, alias: GenericAlias<'db>) { + // Use `walk_specialization_types` rather than `walk_specialization` to avoid walking + // the bounds/constraints/defaults of the generic context. + // Only the types the class was actually specialized with are relevant to whether + // the `GenericAlias` contains a dynamic type. + walk_specialization_types(db, alias.specialization(db), self); + } + fn visit_type_alias_type(&self, db: &'db dyn Db, alias: TypeAliasType<'db>) { self.active_type_aliases.visit( &alias.definition(db), From 9d3dc933020d9d55945966c6069d745aaeb45231 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 14 Aug 2026 11:25:22 +0100 Subject: [PATCH 030/371] [ty] Simplify fully static property tests (#27693) Stacked on #27692. Properties such as reflexivity of subtyping only hold for fully static types. Generating arbitrary types and checking `Type::is_fully_static()` as a precondition silently reduces property-test coverage: non-static inputs count toward the iteration target without exercising the property, and nested types are disproportionately excluded. Preserve the dedicated `FullyStaticTy` generator so every static-only property-test iteration uses a fully static type, while making the generator easier to maintain: - Replace the manually maintained static/dynamic boundary with explicitly named candidate groups while preserving uniform selection. - Assert that every generated `FullyStaticTy` actually satisfies `Type::is_fully_static()`. - Share the property-test macro implementation between ordinary and fully static inputs. - Document why fully static inputs must be generated directly instead of filtered after generation. ## Test plan `QUICKCHECK_TESTS=100000 cargo test --locked --release --package ty_python_semantic -- --ignored types::property_tests::stable` still passes --- .../src/types/property_tests.rs | 24 +-- .../types/property_tests/type_generation.rs | 202 +++++++++++------- 2 files changed, 130 insertions(+), 96 deletions(-) diff --git a/crates/ty_python_semantic/src/types/property_tests.rs b/crates/ty_python_semantic/src/types/property_tests.rs index 04e9a744f0..5c235fa80f 100644 --- a/crates/ty_python_semantic/src/types/property_tests.rs +++ b/crates/ty_python_semantic/src/types/property_tests.rs @@ -39,10 +39,10 @@ use type_generation::{intersection, union}; /// where `t1`, `t2`, ..., `tn` are identifiers that represent arbitrary types, and `` /// is an expression using these identifiers. macro_rules! type_property_test { - ($test_name:ident, $db:ident, $env:ident, forall types $($types:ident),+ . $property:expr) => { + (@impl $test_name:ident, $db:ident, $env:ident, $input_type:ty, $($types:ident),+ . $property:expr) => { #[quickcheck_macros::quickcheck] #[ignore] - fn $test_name($($types: Ty),+) -> bool { + fn $test_name($($types: $input_type),+) -> bool { let $db = &get_cached_db(); let $env = &$db.program_environment(); $(let $types = $types.into_type($db, $env);)+ @@ -57,22 +57,12 @@ macro_rules! type_property_test { } }; - ($test_name:ident, $db:ident, $env:ident, forall fully_static_types $($types:ident),+ . $property:expr) => { - #[quickcheck_macros::quickcheck] - #[ignore] - fn $test_name($($types: FullyStaticTy),+) -> bool { - let $db = &get_cached_db(); - let $env = &$db.program_environment(); - $(let $types = $types.into_type($db, $env);)+ - let result = $property; - - if !result { - println!("\nFailing types were:"); - $(println!("{}", $types.display($db, $env));)+ - } + ($test_name:ident, $db:ident, $env:ident, forall types $($types:ident),+ . $property:expr) => { + type_property_test!(@impl $test_name, $db, $env, Ty, $($types),+ . $property); + }; - result - } + ($test_name:ident, $db:ident, $env:ident, forall fully_static_types $($types:ident),+ . $property:expr) => { + type_property_test!(@impl $test_name, $db, $env, FullyStaticTy, $($types),+ . $property); }; // A property test with a logical implication. diff --git a/crates/ty_python_semantic/src/types/property_tests/type_generation.rs b/crates/ty_python_semantic/src/types/property_tests/type_generation.rs index 6fc0a7a1ba..9bef4c30d7 100644 --- a/crates/ty_python_semantic/src/types/property_tests/type_generation.rs +++ b/crates/ty_python_semantic/src/types/property_tests/type_generation.rs @@ -384,6 +384,28 @@ fn newtype_instance<'db>(db: &'db dyn Db, env: &ProgramEnvironment<'db>, name: & } } +/// A `QuickCheck` input generated without dynamic components, including in nested unions, tuples, +/// and callables. +/// +/// Some type properties, such as reflexivity of subtyping, only hold for fully static types. It is +/// tempting to generate an arbitrary [`Ty`] and express such a property as an implication: +/// +/// ```text +/// t.is_fully_static(db, env) => t.is_subtype_of(db, env, t) +/// ``` +/// +/// However, the property-test macro implements implications as `!premise || conclusion`. Every +/// non-static input therefore counts as a successful `QuickCheck` iteration even though the property +/// itself was never checked. If `QUICKCHECK_TESTS=100000`, the test can report 100,000 successful +/// iterations while checking reflexivity for far fewer types. Properties with two fully static +/// inputs lose even more coverage because both inputs must satisfy the premise. +/// +/// Filtering also disproportionately removes nested unions, tuples, and callables: each additional +/// component gives the generated type another opportunity to contain a dynamic type. Generating +/// fully static components directly ensures that every `QuickCheck` iteration checks the property +/// and that complex types remain represented alongside simple ones. +/// +/// See for the discussion of this coverage problem. #[derive(Debug, Clone, PartialEq)] pub(crate) struct FullyStaticTy(Ty); @@ -393,94 +415,116 @@ impl FullyStaticTy { db: &'db dyn Db, env: &ProgramEnvironment<'db>, ) -> Type<'db> { - self.0.into_type(db, env) + let ty = self.0.into_type(db, env); + assert!( + ty.is_fully_static(db, env), + "FullyStaticTy generated a non-static type: {}", + ty.display(db, env), + ); + ty } } +// A single draw across both groups keeps unrestricted candidates equally likely without +// allocating a combined list or maintaining a positional boundary between the groups. +macro_rules! choose_core_type { + ( + $generator:expr, + $fully_static:expr, + dynamic_types: [$($dynamic:expr),+ $(,)?], + fully_static_types: [$($static:expr),+ $(,)?] $(,)? + ) => {{ + if $fully_static { + $generator.choose(&[$($static),+]).unwrap().clone() + } else { + $generator + .choose(&[$($dynamic),+, $($static),+]) + .unwrap() + .clone() + } + }}; +} + fn arbitrary_core_type(g: &mut Gen, fully_static: bool) -> Ty { // We could select a random integer here, but this would make it much less // likely to explore interesting edge cases: let int_lit = Ty::IntLiteral(*g.choose(&[-2, -1, 0, 1, 2]).unwrap()); let bool_lit = Ty::BooleanLiteral(bool::arbitrary(g)); - // Update this if new non-fully-static types are added below. - let fully_static_index = 8; - let types = &[ - Ty::Any, - Ty::Unknown, - Ty::Divergent, - Ty::TopDivergent, - Ty::BottomDivergent, - Ty::SubclassOfAny, - Ty::UnittestMockLiteral, - Ty::UnittestMockInstance, - // Add fully static types below, dynamic types above. - // Update `fully_static_index` above if adding new dynamic types! - Ty::Never, - Ty::None, - int_lit, - bool_lit, - Ty::StringLiteral(""), - Ty::StringLiteral("a"), - Ty::LiteralString, - Ty::BytesLiteral(""), - Ty::BytesLiteral("\x00"), - Ty::EnumLiteral("safe"), - Ty::EnumLiteral("unsafe"), - Ty::EnumLiteral("unknown"), - Ty::SingleMemberEnumLiteral, - Ty::KnownClassInstance(KnownClass::Object), - Ty::KnownClassInstance(KnownClass::Str), - Ty::KnownClassInstance(KnownClass::Int), - Ty::KnownClassInstance(KnownClass::Float), - Ty::KnownClassInstance(KnownClass::Complex), - Ty::KnownClassInstance(KnownClass::Bool), - Ty::KnownClassInstance(KnownClass::FunctionType), - Ty::KnownClassInstance(KnownClass::SpecialForm), - Ty::KnownClassInstance(KnownClass::TypeVar), - Ty::KnownClassInstance(KnownClass::TypeAliasType), - Ty::KnownClassInstance(KnownClass::NoDefaultType), - Ty::TypingLiteral, - Ty::BuiltinClassLiteral("str"), - Ty::BuiltinClassLiteral("int"), - Ty::BuiltinClassLiteral("bool"), - Ty::BuiltinClassLiteral("object"), - Ty::BuiltinInstance("type"), - Ty::AbcInstance("ABC"), - Ty::AbcInstance("ABCMeta"), - Ty::SubclassOfBuiltinClass("object"), - Ty::SubclassOfBuiltinClass("str"), - Ty::SubclassOfBuiltinClass("type"), - Ty::AbcClassLiteral("ABC"), - Ty::AbcClassLiteral("ABCMeta"), - Ty::SubclassOfAbcClass("ABC"), - Ty::SubclassOfAbcClass("ABCMeta"), - Ty::AlwaysTruthy, - Ty::AlwaysFalsy, - Ty::BuiltinsFunction("chr"), - Ty::BuiltinsFunction("ascii"), - Ty::BuiltinsBoundMethod { - class: "str", - method: "isascii", - }, - Ty::BuiltinsBoundMethod { - class: "int", - method: "bit_length", - }, - Ty::IntNewtypeInstance, - Ty::StrNewtypeInstance, - Ty::FloatNewtypeInstance, - Ty::ComplexNewtypeInstance, - Ty::SubNewTypeOfIntInstance, - Ty::SubSubNewTypeOfIntInstance, - Ty::SubNewTypeOfFloatInstance, - ]; - let types = if fully_static { - &types[fully_static_index..] - } else { - types - }; - g.choose(types).unwrap().clone() + choose_core_type!( + g, + fully_static, + dynamic_types: [ + Ty::Any, + Ty::Unknown, + Ty::Divergent, + Ty::TopDivergent, + Ty::BottomDivergent, + Ty::SubclassOfAny, + Ty::UnittestMockLiteral, + Ty::UnittestMockInstance, + ], + fully_static_types: [ + Ty::Never, + Ty::None, + int_lit, + bool_lit, + Ty::StringLiteral(""), + Ty::StringLiteral("a"), + Ty::LiteralString, + Ty::BytesLiteral(""), + Ty::BytesLiteral("\x00"), + Ty::EnumLiteral("safe"), + Ty::EnumLiteral("unsafe"), + Ty::EnumLiteral("unknown"), + Ty::SingleMemberEnumLiteral, + Ty::KnownClassInstance(KnownClass::Object), + Ty::KnownClassInstance(KnownClass::Str), + Ty::KnownClassInstance(KnownClass::Int), + Ty::KnownClassInstance(KnownClass::Float), + Ty::KnownClassInstance(KnownClass::Complex), + Ty::KnownClassInstance(KnownClass::Bool), + Ty::KnownClassInstance(KnownClass::FunctionType), + Ty::KnownClassInstance(KnownClass::SpecialForm), + Ty::KnownClassInstance(KnownClass::TypeVar), + Ty::KnownClassInstance(KnownClass::TypeAliasType), + Ty::KnownClassInstance(KnownClass::NoDefaultType), + Ty::TypingLiteral, + Ty::BuiltinClassLiteral("str"), + Ty::BuiltinClassLiteral("int"), + Ty::BuiltinClassLiteral("bool"), + Ty::BuiltinClassLiteral("object"), + Ty::BuiltinInstance("type"), + Ty::AbcInstance("ABC"), + Ty::AbcInstance("ABCMeta"), + Ty::SubclassOfBuiltinClass("object"), + Ty::SubclassOfBuiltinClass("str"), + Ty::SubclassOfBuiltinClass("type"), + Ty::AbcClassLiteral("ABC"), + Ty::AbcClassLiteral("ABCMeta"), + Ty::SubclassOfAbcClass("ABC"), + Ty::SubclassOfAbcClass("ABCMeta"), + Ty::AlwaysTruthy, + Ty::AlwaysFalsy, + Ty::BuiltinsFunction("chr"), + Ty::BuiltinsFunction("ascii"), + Ty::BuiltinsBoundMethod { + class: "str", + method: "isascii", + }, + Ty::BuiltinsBoundMethod { + class: "int", + method: "bit_length", + }, + Ty::IntNewtypeInstance, + Ty::StrNewtypeInstance, + Ty::FloatNewtypeInstance, + Ty::ComplexNewtypeInstance, + Ty::SubNewTypeOfIntInstance, + Ty::SubSubNewTypeOfIntInstance, + Ty::SubNewTypeOfFloatInstance, + ], + ) } /// Constructs an arbitrary type. From 083ec1e307e676d17409c78c7cf4651b199d4bd1 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Fri, 14 Aug 2026 04:04:11 -0700 Subject: [PATCH 031/371] [ty] Fix overload argument expansion with unpacked positional arguments (#27744) ## Summary Fixes astral-sh/ty#4258. The overload-expansion optimization compared an unpacked iterable against an individual parameter's type, causing valid calls with `*args` to be rejected before an unrelated union-typed argument could be expanded. - Check each unpacked element against its actual matched parameter instead of comparing the entire iterable. - Treat empty unpacking as imposing no positional constraint. - Preserve existing behavior for unresolved variadic generics and retain early pruning for incompatible arguments. ## Test plan Added overload-resolution mdtests covering empty unpacking, fixed-length and variable-length tuples, lists, invalid unpacked elements, heterogeneous positional parameters, unpacked tuple annotations, unresolved variadic generic prefixes, and early pruning of invalid nonempty unpacking. --- .../resources/mdtest/call/overloads.md | 124 +++++++++++++++++- .../ty_python_semantic/src/types/call/bind.rs | 40 ++++-- 2 files changed, 150 insertions(+), 14 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/call/overloads.md b/crates/ty_python_semantic/resources/mdtest/call/overloads.md index ba3f3d55ee..7aba6f4bb5 100644 --- a/crates/ty_python_semantic/resources/mdtest/call/overloads.md +++ b/crates/ty_python_semantic/resources/mdtest/call/overloads.md @@ -264,6 +264,109 @@ def _(a: A, bc: B | C, cd: C | D): reveal_type(f(*(a, cd))) # revealed: Unknown ``` +### Expanding a keyword argument after unpacking into a variadic parameter + +Valid unpacked positional arguments must not prevent expansion of an unrelated union-typed keyword +argument. The positional arguments may come from an empty tuple, a fixed-length tuple, a +variable-length tuple, or a list. + +`overloaded.pyi`: + +```pyi +from typing import overload + +@overload +def f(*values: str, kind: int) -> int: ... +@overload +def f(*values: str, kind: None) -> str: ... +``` + +Expanding `kind` matches one overload when its value is an `int` and the other when its value is +`None`, independently of how the positional arguments are provided. An incompatible positional +argument must still fail to match either overload. + +```py +from overloaded import f + +def _(one: tuple[str], many: tuple[str, ...], items: list[str], kind: int | None) -> None: + reveal_type(f("a", kind=kind)) # revealed: int | str + reveal_type(f(*(), kind=kind)) # revealed: int | str + reveal_type(f(*one, kind=kind)) # revealed: int | str + reveal_type(f(*many, kind=kind)) # revealed: int | str + reveal_type(f(*items, kind=kind)) # revealed: int | str + +def _(invalid: tuple[int], kind: int | None) -> None: + # error: [no-matching-overload] + reveal_type(f(*invalid, kind=kind)) # revealed: Unknown +``` + +### Expanding a keyword argument with an unpacked variadic annotation + +An unpacked variadic annotation can specify a different expected type for each positional argument. +Unpacked arguments must be checked against their corresponding element types while an unrelated +union-typed keyword argument is expanded. + +```toml +[environment] +python-version = "3.13" +``` + +`overloaded.pyi`: + +```pyi +from typing import overload + +@overload +def f(*values: *tuple[str, int], kind: int) -> int: ... +@overload +def f(*values: *tuple[str, int], kind: None) -> str: ... +@overload +def suffix[T: str, *Parts](*values: *tuple[*Parts, T], kind: int) -> int: ... +@overload +def suffix[T: str, *Parts](*values: *tuple[*Parts, T], kind: None) -> str: ... +``` + +Both directly supplied arguments and an unpacked tuple satisfy the heterogeneous annotation. A +generic variadic prefix also permits expansion when the fixed suffix has a compatible type. + +```py +from overloaded import f, suffix + +def _(values: tuple[str, int], kind: int | None) -> None: + reveal_type(f("a", 1, kind=kind)) # revealed: int | str + reveal_type(f(*values, kind=kind)) # revealed: int | str + +def _(pair: tuple[int, str], kind: int | None) -> None: + reveal_type(suffix(*pair, kind=kind)) # revealed: int | str +``` + +### Expanding a keyword argument after unpacking into positional parameters + +Expanding a union-typed keyword argument must also work when a fixed-length tuple supplies ordinary +positional parameters with different types instead of a variadic parameter. + +`overloaded.pyi`: + +```pyi +from typing import overload + +@overload +def f(value: str, count: int, *, kind: int) -> int: ... +@overload +def f(value: str, count: int, *, kind: None) -> str: ... +``` + +Both the direct positional arguments and the unpacked tuple select the same overloads after `kind` +is expanded. + +```py +from overloaded import f + +def _(values: tuple[str, int], kind: int | None) -> None: + reveal_type(f("a", 1, kind=kind)) # revealed: int | str + reveal_type(f(*values, kind=kind)) # revealed: int | str +``` + ### Generics (legacy) `overloaded.pyi`: @@ -755,7 +858,7 @@ class Foo: from overloaded import A, B, C, Foo, f from typing_extensions import Any, reveal_type -def _(ab: A | B, a: int | Any): +def _(ab: A | B, a: int | Any, invalid: tuple[C]): reveal_type(f(a1=a, a2=a, a3=a)) # revealed: C reveal_type(f(A(), a1=a, a2=a, a3=a)) # revealed: A reveal_type(f(B(), a1=a, a2=a, a3=a)) # revealed: B @@ -803,6 +906,25 @@ def _(ab: A | B, a: int | Any): ) ) + # An incompatible element in a definitely nonempty splat must also prevent expansion of the + # nine union-typed keyword arguments. + reveal_type( + # error: [no-matching-overload] + # revealed: Unknown + f( + *invalid, + a1=a, + a2=a, + a3=a, + a4=a, + a5=a, + a6=a, + a7=a, + a8=a, + a9=a, + ) + ) + # Here, the heuristics won't come into play because all arguments can be expanded but expanding # the first argument results in a successful evaluation of the call, so there's no exponential # growth of the number of argument lists. diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index cd935720b4..a61a083c94 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -3814,13 +3814,31 @@ impl<'db> CallableBinding<'db> { if is_expandable_type(db, env, argument_type) { continue; } - let mut is_argument_assignable_to_any_overload = false; - 'overload: for overload in &self.overloads { - for matched_parameter in &overload.argument_matches[argument_index].parameters { - let parameter_type = - overload.signature.parameters()[matched_parameter.index].annotated_type(); - let argument_type = argument_types.get_for_declared_type(parameter_type); - if argument_type + let is_argument_assignable_to_any_overload = self.overloads.iter().any(|overload| { + let matched_parameters = &overload.argument_matches[argument_index].parameters; + if matched_parameters.is_empty() { + return matches!(argument, Argument::Variadic) + && argument_type.iterate(db, env).len().minimum() == 0; + } + + // A starred argument contributes its individual element types, not the type of + // the iterable itself, and each element must match its corresponding parameter. + matched_parameters.iter().all(|matched_parameter| { + let parameter = &overload.signature.parameters()[matched_parameter.index]; + if parameter.has_starred_annotation() + && matched_parameter.expected_type.is_none() + { + return true; + } + + let parameter_type = matched_parameter + .expected_type + .unwrap_or_else(|| parameter.annotated_type()); + let argument_type = matched_parameter + .argument_type + .unwrap_or_else(|| argument_types.get_for_declared_type(parameter_type)); + + argument_type .when_assignable_to( db, env, @@ -3829,12 +3847,8 @@ impl<'db> CallableBinding<'db> { overload.inferable_typevars, ) .is_always_satisfied(db, env) - { - is_argument_assignable_to_any_overload = true; - break 'overload; - } - } - } + }) + }); if !is_argument_assignable_to_any_overload { tracing::debug!( "Argument at {argument_index} (`{}`) is not assignable to any of the \ From bf15c8cfcc24e4527c87c006b1a7b6225bd80af3 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Fri, 14 Aug 2026 04:04:37 -0700 Subject: [PATCH 032/371] [ty] Avoid exponential narrowing of gradual string-literal unions (#27742) ## Summary Fix the exponential memory regression when equality narrowing encounters a union of `Any & Literal["..."]` alternatives. `evaluate_target_union` intersected each surviving alternative with the complement of every rejected alternative, even when the survivor and rejected union were already disjoint. For gradual string literals, expanding those redundant complements created exponentially many equivalent intersections. Skip the exclusions when disjointness is proven; preserve them for alternatives that can overlap, including `Any` and `TypeVar` cases. On the original 20-member reproducer, peak memory drops from **304.4 MiB to 38.8 MiB**; the reduced intersection-only reproducer drops from **301.4 MiB to 38.5 MiB**. The original 67-member case, previously exponential/OOM, now completes in **0.061 s using 42.0 MiB**. Closes astral-sh/ty#4256. ## Test plan - Added mdtests covering both branches of `==` and `!=` for gradual string-literal unions, plus a 20-member regression that exercises the original exponential expansion. - Added the dedicated `ty_micro[gradual_literal_union_equality]` CodSpeed microbenchmark for the 20-member case. - Ran all 481 ty semantic mdtests and the repository hooks. --- crates/ruff_benchmark/benches/ty.rs | 34 ++++++++++++ .../mdtest/narrow/conditionals/eq.md | 53 +++++++++++++++++++ .../ty_python_semantic/src/types/equality.rs | 8 ++- 3 files changed, 94 insertions(+), 1 deletion(-) diff --git a/crates/ruff_benchmark/benches/ty.rs b/crates/ruff_benchmark/benches/ty.rs index faae0f88b5..f36a145da2 100644 --- a/crates/ruff_benchmark/benches/ty.rs +++ b/crates/ruff_benchmark/benches/ty.rs @@ -1254,6 +1254,39 @@ fn benchmark_literal_equality_fallthrough_guarded_any(criterion: &mut Criterion) ); } +/// Regression benchmark for . +/// +/// Excluding rejected gradual string literals must not expand the complement of each intersection +/// into exponentially many equivalent alternatives. +fn benchmark_gradual_literal_union_equality(criterion: &mut Criterion) { + setup_rayon(); + + let mut code = String::from( + "from typing import Any, Literal\nfrom ty_extensions import Intersection\n\ndef check(value: (\n", + ); + for index in 0..20 { + writeln!( + &mut code, + " {}Intersection[Any, Literal[\"{index}\"]]", + if index == 0 { "" } else { "| " }, + ) + .ok(); + } + code.push_str(")) -> None:\n assert value == \"0\"\n repr(value)\n"); + + criterion.bench_function("ty_micro[gradual_literal_union_equality]", |b| { + b.iter_batched_ref( + || setup_micro_case(&code), + |case| { + let Case { db } = case; + let result = db.check(); + assert_eq!(result.len(), 0); + }, + BatchSize::SmallInput, + ); + }); +} + /// Regression benchmark for . /// /// Reachability analysis for a large literal OR pattern on `Any` used to rebuild the remaining @@ -1582,6 +1615,7 @@ criterion_group!( benchmark_literal_match_fallthrough, benchmark_literal_match_fallthrough_guarded_any, benchmark_literal_equality_fallthrough_guarded_any, + benchmark_gradual_literal_union_equality, benchmark_literal_or_pattern_reachability, benchmark_typeis_narrowing, benchmark_repeated_statement_calls, diff --git a/crates/ty_python_semantic/resources/mdtest/narrow/conditionals/eq.md b/crates/ty_python_semantic/resources/mdtest/narrow/conditionals/eq.md index 95cd22a1f8..0f15bf57d1 100644 --- a/crates/ty_python_semantic/resources/mdtest/narrow/conditionals/eq.md +++ b/crates/ty_python_semantic/resources/mdtest/narrow/conditionals/eq.md @@ -2184,6 +2184,59 @@ def gradual_enum_union_inequality(value: Color | Any, other: Color): reveal_type(value) # revealed: Color | Any ``` +## Unions of gradual string literals + +Comparing a union of string literals intersected with `Any` keeps the matching alternative for +equality and removes it for inequality: + +```py +from typing import Any, Literal +from ty_extensions import Intersection + +def equality(value: Intersection[Any, Literal["a"]] | Intersection[Any, Literal["b"]]): + if value == "a": + reveal_type(value) # revealed: Any & Literal["a"] + else: + reveal_type(value) # revealed: Any & Literal["b"] + + if value != "a": + reveal_type(value) # revealed: Any & Literal["b"] + else: + reveal_type(value) # revealed: Any & Literal["a"] +``` + +Larger unions must narrow without expanding the complement of every rejected alternative, which +would make memory use grow exponentially: + +```py +def larger_union( + value: ( + Intersection[Any, Literal["a"]] + | Intersection[Any, Literal["b"]] + | Intersection[Any, Literal["c"]] + | Intersection[Any, Literal["d"]] + | Intersection[Any, Literal["e"]] + | Intersection[Any, Literal["f"]] + | Intersection[Any, Literal["g"]] + | Intersection[Any, Literal["h"]] + | Intersection[Any, Literal["i"]] + | Intersection[Any, Literal["j"]] + | Intersection[Any, Literal["k"]] + | Intersection[Any, Literal["l"]] + | Intersection[Any, Literal["m"]] + | Intersection[Any, Literal["n"]] + | Intersection[Any, Literal["o"]] + | Intersection[Any, Literal["p"]] + | Intersection[Any, Literal["q"]] + | Intersection[Any, Literal["r"]] + | Intersection[Any, Literal["s"]] + | Intersection[Any, Literal["t"]] + ), +): + if value == "a": + reveal_type(value) # revealed: Any & Literal["a"] +``` + ## Booleans and integers ```py diff --git a/crates/ty_python_semantic/src/types/equality.rs b/crates/ty_python_semantic/src/types/equality.rs index 5400a61036..92a7aeab69 100644 --- a/crates/ty_python_semantic/src/types/equality.rs +++ b/crates/ty_python_semantic/src/types/equality.rs @@ -1166,7 +1166,13 @@ fn evaluate_target_union<'db>( let Some(mut narrowed) = narrowed else { continue; }; - if let Some(removed) = removed { + // A surviving alternative that is disjoint from every rejected alternative already + // satisfies their exclusions. Constructing those redundant exclusions can exponentially + // expand intersections such as `Any & Literal["a"]` when the rejected alternatives are + // similarly shaped intersections with other string literals. + if let Some(removed) = removed + && !narrowed.is_disjoint_from(db, env, removed) + { narrowed = IntersectionBuilder::new(db, env) .add_positive(narrowed) .add_negative(removed) From a103832c28d5a8624d0d3527eeb3b5d8ed09d58f Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Fri, 14 Aug 2026 13:24:27 +0200 Subject: [PATCH 033/371] [ty] Check PEP 723 scripts in isolation (#27462) ## Summary This PR extends the PEP 723 configuration support to `ty.environment` settings. That is: * Scripts are checked with a separate virtual environment and search paths * Scripts can configure a different Python version * Scripts can configure a different Python platform Our long-term plan is to use uv to create the script's virtual environment. We don't do this in this PR. Instead, we: * Respect `VIRTUAL_ENV`, LSP fallback environment (the selected environment), global available Python installation. Unlike for projects, ty does not search for a local `.venv` folder * Unlike for projects, scripts default to an empty `environment.root` because scripts are considered to work in isolation. Users can customize `environment.root`. Out of scope for this PR but things we should look into next: * Report a diagnostic if a script's metadata fails to parse instead of falling back to the project's configuration. * Add support for the Python environment extension's file-specific virtual env (so that you can select a virtual env for a specific file) * Preserve source ranges for configuration options from script metadata (shouldn't be too hard but the PR is already large as it is) * Integrating with uv. I expect that this will require significant changes to the implementation to provide good UX, but I like to tackle problems one step at a time (it also makes it easier to explain why we changed something). * We currently don't watch for changes in script search paths. That means, results in the LSP or when using `ty check --watch` could get outdated ## Test Plan Testing: Added integration and language-server coverage and verified the relevant unit, CLI, server, and Markdown suites. --- crates/ruff_linter/src/rule_selector.rs | 1 + crates/ruff_ranged_value/src/lib.rs | 26 +- crates/ty/docs/configuration.md | 10 + crates/ty/tests/cli/scripts.rs | 823 +++++++++++++++++- crates/ty_project/src/db.rs | 39 +- crates/ty_project/src/lib.rs | 17 +- crates/ty_project/src/metadata.rs | 55 +- crates/ty_project/src/metadata/options.rs | 168 ++-- crates/ty_project/src/metadata/pyproject.rs | 112 ++- crates/ty_project/src/metadata/settings.rs | 44 +- crates/ty_project/src/metadata/value.rs | 8 +- crates/ty_project/src/script.rs | 292 ++++++- crates/ty_python_core/src/db.rs | 2 +- crates/ty_python_core/src/program.rs | 4 +- .../resources/mdtest/scripts.md | 14 +- .../ty_python_semantic/src/diagnostic/mod.rs | 20 +- crates/ty_python_semantic/src/lint.rs | 3 + .../src/types/class/known.rs | 2 +- .../ty_python_semantic/src/types/context.rs | 3 + .../src/types/infer/tests.rs | 8 +- crates/ty_server/tests/e2e/goto_definition.rs | 58 ++ crates/ty_server/tests/e2e/hover.rs | 93 ++ crates/ty_server/tests/e2e/main.rs | 44 +- .../tests/e2e/publish_diagnostics.rs | 147 +++- ..._did_change_script_python_requirement.snap | 72 ++ ...ting_file_workspace_with_untitled_uri.snap | 4 +- ...orts_inline_configuration_diagnostics.snap | 52 ++ crates/ty_site_packages/src/lib.rs | 44 +- crates/ty_site_packages/src/version.rs | 3 + ty.schema.json | 8 +- 30 files changed, 1881 insertions(+), 295 deletions(-) create mode 100644 crates/ty_server/tests/e2e/goto_definition.rs create mode 100644 crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_change_script_python_requirement.snap create mode 100644 crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_open_virtual_script_reports_inline_configuration_diagnostics.snap diff --git a/crates/ruff_linter/src/rule_selector.rs b/crates/ruff_linter/src/rule_selector.rs index 1205af1a6b..a23ee90797 100644 --- a/crates/ruff_linter/src/rule_selector.rs +++ b/crates/ruff_linter/src/rule_selector.rs @@ -104,6 +104,7 @@ impl std::fmt::Display for RuleResolutionError { }; let source = match &source { ValueSource::File(path) => format_args!("`{}`", path.as_path()), + ValueSource::ScriptMetadata(_) => format_args!("script metadata"), ValueSource::Cli => format_args!("the CLI"), ValueSource::Editor => format_args!("the editor configuration"), ValueSource::UvWorkspace => format_args!("uv workspace metadata"), diff --git a/crates/ruff_ranged_value/src/lib.rs b/crates/ruff_ranged_value/src/lib.rs index f1305e61fd..e8a4a2ccda 100644 --- a/crates/ruff_ranged_value/src/lib.rs +++ b/crates/ruff_ranged_value/src/lib.rs @@ -8,7 +8,9 @@ use std::sync::Arc; use serde::{Deserialize, Deserializer}; use toml::Spanned; -use ruff_db::system::{SystemPath, SystemPathBuf}; +use ruff_db::Db; +use ruff_db::files::{File, system_path_to_file}; +use ruff_db::system::SystemPathBuf; use ruff_text_size::{TextRange, TextSize}; #[derive(Clone, Debug, PartialEq)] @@ -20,6 +22,12 @@ pub enum ValueSource { /// created when loading the configuration. File(Arc), + /// Value loaded from inline metadata in a standalone script. + /// + /// Unlike project configuration, scripts are parsed after the database exists, so their + /// existing Salsa file can be retained directly, including for virtual files. + ScriptMetadata(File), + /// The value comes from a CLI argument, while it's left open if specified using a short argument, /// long argument (`--extra-paths`) or `--config key=value`. Cli, @@ -35,9 +43,11 @@ pub enum ValueSource { } impl ValueSource { - pub fn file(&self) -> Option<&SystemPath> { + /// Resolves the file containing this setting, if its source is file-backed. + pub fn file(&self, db: &dyn Db) -> Option { match self { - ValueSource::File(path) => Some(&**path), + ValueSource::File(path) => system_path_to_file(db, &**path).ok(), + ValueSource::ScriptMetadata(file) => Some(*file), ValueSource::Cli => None, ValueSource::Editor => None, ValueSource::UvWorkspace => None, @@ -140,15 +150,19 @@ where impl RangedValue { pub fn new(value: T, source: ValueSource) -> Self { - Self::with_range(value, source, TextRange::default()) + Self { + value, + source, + range: None, + } } pub fn cli(value: T) -> Self { - Self::with_range(value, ValueSource::Cli, TextRange::default()) + Self::new(value, ValueSource::Cli) } pub fn python_extension(value: T) -> Self { - Self::with_range(value, ValueSource::Editor, TextRange::default()) + Self::new(value, ValueSource::Editor) } fn with_range(value: T, source: ValueSource, range: TextRange) -> Self { diff --git a/crates/ty/docs/configuration.md b/crates/ty/docs/configuration.md index d60457fc28..34b53b0ae1 100644 --- a/crates/ty/docs/configuration.md +++ b/crates/ty/docs/configuration.md @@ -364,6 +364,10 @@ your environment from an activated Conda environment, and will look for a `.venv in the project root if none of the above apply. Failing that, ty will look for a `python3` or `python` binary available in `PATH`. +Scripts with inline metadata use their own Python environment. They can use an explicitly +configured environment, an activated environment, or an environment selected by the editor. +Unlike projects, they do not automatically use a `.venv` directory. + [`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix **Default value**: `null` @@ -446,6 +450,9 @@ to determine a value: and attempt to infer the Python version of that environment 3. Fall back to the default value (see below) +Scripts with inline metadata use their `requires-python` field instead of +`project.requires-python`. They do not inherit the Python version of the enclosing project. + For some language features, ty can also understand conditionals based on comparisons with `sys.version_info`. These are commonly found in typeshed, for example, to reflect the differing contents of the standard library across Python versions. @@ -486,6 +493,9 @@ if they exist and are not packages (i.e. they do not contain `__init__.py` or `_ * `./` (if a `.//` directory exists) * `./python` +Scripts with inline metadata have no first-party roots by default because they are +single-file programs. Set `root = ["."]` to allow importing local modules. + **Default value**: `null` **Type**: `list[str]` diff --git a/crates/ty/tests/cli/scripts.rs b/crates/ty/tests/cli/scripts.rs index dc44976430..725c9e839e 100644 --- a/crates/ty/tests/cli/scripts.rs +++ b/crates/ty/tests/cli/scripts.rs @@ -49,6 +49,82 @@ fn project_settings_and_overrides_do_not_apply() -> anyhow::Result<()> { Ok(()) } +#[test] +fn verbose_rule_diagnostics_identify_script_metadata() -> anyhow::Result<()> { + let case = CliTest::with_file( + "script.py", + r#" + # /// script + # [tool.ty.rules] + # unresolved-reference = "warn" + # /// + + print(missing) + "#, + )?; + + assert_cmd_snapshot!(case.command().arg("--verbose"), @" + success: false + exit_code: 1 + ----- stdout ----- + warning[unresolved-reference]: Name `missing` used when not defined + --> script.py:7:7 + | + 7 | print(missing) + | ^^^^^^^ + info: rule `unresolved-reference` was selected in script metadata + + Found 1 diagnostic + + ----- stderr ----- + INFO Indexed 1 file(s) in 0.000s + "); + + Ok(()) +} + +#[test] +fn python_version_diagnostics_identify_script_metadata() -> anyhow::Result<()> { + let case = CliTest::with_file( + "script.py", + r#" + # /// script + # requires-python = ">=3.12" + # /// + + PythonFinalizationError + "#, + )?; + + assert_cmd_snapshot!(case.command(), @" + success: false + exit_code: 1 + ----- stdout ----- + error[unresolved-reference]: Name `PythonFinalizationError` used when not defined + --> script.py:6:1 + | + 6 | PythonFinalizationError + | ^^^^^^^^^^^^^^^^^^^^^^^ + info: `PythonFinalizationError` was added as a builtin in Python 3.13 + info: Python 3.12 was assumed when resolving types because it was specified in script metadata + + Found 1 diagnostic + + ----- stderr ----- + "); + assert_cmd_snapshot!(case.command().arg("--output-format").arg("concise"), @" + success: false + exit_code: 1 + ----- stdout ----- + script.py:6:1: error[unresolved-reference] Name `PythonFinalizationError` used when not defined + Found 1 diagnostic + + ----- stderr ----- + "); + + Ok(()) +} + #[test] fn metadata_without_tool_ty_uses_default_settings() -> anyhow::Result<()> { let case = CliTest::with_files([ @@ -97,32 +173,22 @@ fn metadata_without_tool_ty_uses_default_settings() -> anyhow::Result<()> { #[test] fn environment_options() -> anyhow::Result<()> { - // TODO: This is not yet supported, but we should support this. - let case = CliTest::with_files([ - ( - "pyproject.toml", - r#" - [tool.ty.environment] - python-version = "3.12" - "#, - ), - ( - "script.py", - r#" - # /// script - # requires-python = ">=3.7" - # - # [tool.ty.environment] - # python-version = "3.7" - # /// + let case = CliTest::with_file( + "script.py", + r#" + # /// script + # requires-python = ">=3.13" + # + # [tool.ty.environment] + # python-version = "3.11" + # /// - import sys - from typing import reveal_type + import sys + from typing import reveal_type - reveal_type(sys.version_info[:2] == (3, 12)) - "#, - ), - ])?; + reveal_type(sys.version_info[:2]) + "#, + )?; assert_cmd_snapshot!(case.command(), @" success: true @@ -131,8 +197,8 @@ fn environment_options() -> anyhow::Result<()> { info[revealed-type]: Revealed type --> script.py:12:13 | - 12 | reveal_type(sys.version_info[:2] == (3, 12)) - | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `Literal[True]` + 12 | reveal_type(sys.version_info[:2]) + | ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[11]]` Found 1 diagnostic @@ -403,3 +469,708 @@ fn explicit_config_replaces_inline_metadata() -> anyhow::Result<()> { Ok(()) } + +#[test] +fn explicit_config_replaces_the_script_environment() -> anyhow::Result<()> { + let case = CliTest::with_files([ + ( + "explicit.toml", + r#" + [environment] + python-version = "3.12" + python-platform = "linux" + "#, + ), + ( + "script.py", + r#" + # /// script + # requires-python = ">=3.13" + # [tool.ty.environment] + # python-platform = "win32" + # /// + + import sys + from typing import reveal_type + + reveal_type(sys.version_info[:2]) + reveal_type(sys.platform) + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command().arg("--config-file").arg("explicit.toml"), @r#" + success: true + exit_code: 0 + ----- stdout ----- + info[revealed-type]: Revealed type + --> script.py:11:13 + | + 11 | reveal_type(sys.version_info[:2]) + | ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[12]]` + + info[revealed-type]: Revealed type + --> script.py:12:13 + | + 12 | reveal_type(sys.platform) + | ^^^^^^^^^^^^ `Literal["linux"]` + + Found 2 diagnostics + + ----- stderr ----- + "#); + + Ok(()) +} + +#[test] +fn cli_arguments_override_script_environment() -> anyhow::Result<()> { + let case = CliTest::with_file( + "script.py", + r#" + # /// script + # requires-python = ">=3.13" + # [tool.ty.environment] + # python-platform = "win32" + # /// + + import sys + from typing import reveal_type + + reveal_type(sys.version_info[:2]) + reveal_type(sys.platform) + "#, + )?; + + assert_cmd_snapshot!( + case.command() + .arg("--python-version") + .arg("3.12") + .arg("--python-platform") + .arg("linux"), + @r#" + success: true + exit_code: 0 + ----- stdout ----- + info[revealed-type]: Revealed type + --> script.py:11:13 + | + 11 | reveal_type(sys.version_info[:2]) + | ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[12]]` + + info[revealed-type]: Revealed type + --> script.py:12:13 + | + 12 | reveal_type(sys.platform) + | ^^^^^^^^^^^^ `Literal["linux"]` + + Found 2 diagnostics + + ----- stderr ----- + "# + ); + + Ok(()) +} + +#[test] +fn script_version_and_platform_are_isolated_from_project_configuration() -> anyhow::Result<()> { + let case = CliTest::with_files([ + ( + "pyproject.toml", + r#" + [tool.ty.environment] + python-version = "3.12" + python-platform = "linux" + "#, + ), + ( + "script.py", + r#" + # /// script + # requires-python = ">=3.13" + # [tool.ty.environment] + # python-version = "3.11" + # python-platform = "win32" + # /// + + import sys + from typing import reveal_type + + reveal_type(sys.version_info[:2]) + reveal_type(sys.platform) + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @r#" + success: true + exit_code: 0 + ----- stdout ----- + info[revealed-type]: Revealed type + --> script.py:12:13 + | + 12 | reveal_type(sys.version_info[:2]) + | ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[11]]` + + info[revealed-type]: Revealed type + --> script.py:13:13 + | + 13 | reveal_type(sys.platform) + | ^^^^^^^^^^^^ `Literal["win32"]` + + Found 2 diagnostics + + ----- stderr ----- + "#); + + Ok(()) +} + +#[test] +fn python_requirement_overrides_user_configuration() -> anyhow::Result<()> { + let case = CliTest::with_file( + "script.py", + r#" + # /// script + # requires-python = ">=3.13" + # /// + + import sys + from typing import reveal_type + + reveal_type(sys.version_info[:2]) + "#, + )?; + case.write_file( + case.user_config_directory().join("ty/ty.toml"), + r#" + [environment] + python-version = "3.12" + "#, + )?; + + assert_cmd_snapshot!(case.command(), @" + success: true + exit_code: 0 + ----- stdout ----- + info[revealed-type]: Revealed type + --> script.py:9:13 + | + 9 | reveal_type(sys.version_info[:2]) + | ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[13]]` + + Found 1 diagnostic + + ----- stderr ----- + "); + + Ok(()) +} + +#[test] +fn scripts_have_no_implicit_first_party_roots() -> anyhow::Result<()> { + let case = CliTest::with_files([ + ("shared.py", "value = 1\n"), + ("src/layout_dependency.py", "value = 1\n"), + ("scripts/local_dependency.py", "value = 1\n"), + ( + "scripts/script.py", + r#" + # /// script + # dependencies = [] + # /// + + from layout_dependency import value as layout_value + from local_dependency import value as local_value + from shared import value + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @" + success: false + exit_code: 1 + ----- stdout ----- + error[unresolved-import]: Cannot resolve imported module `layout_dependency` + --> scripts/script.py:6:6 + | + 6 | from layout_dependency import value as layout_value + | ^^^^^^^^^^^^^^^^^ + info: Searched in the following paths during module resolution: + info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty) + info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment + + error[unresolved-import]: Cannot resolve imported module `local_dependency` + --> scripts/script.py:7:6 + | + 7 | from local_dependency import value as local_value + | ^^^^^^^^^^^^^^^^ + info: Searched in the following paths during module resolution: + info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty) + info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment + + error[unresolved-import]: Cannot resolve imported module `shared` + --> scripts/script.py:8:6 + | + 8 | from shared import value + | ^^^^^^ + info: Searched in the following paths during module resolution: + info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty) + info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment + + Found 3 diagnostics + + ----- stderr ----- + "); + + Ok(()) +} + +#[test] +fn configured_source_roots_and_extra_paths_are_relative_to_the_script() -> anyhow::Result<()> { + let case = CliTest::with_files([ + ("scripts/source/first_party.py", "value = 1\n"), + ("scripts/extra/dependency.py", "value = 1\n"), + ( + "scripts/script.py", + r#" + # /// script + # [tool.ty.environment] + # root = ["./source"] + # extra-paths = ["./extra"] + # /// + + from dependency import value as dependency + from first_party import value as first_party + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @" + success: true + exit_code: 0 + ----- stdout ----- + All checks passed! + + ----- stderr ----- + "); + + Ok(()) +} + +#[test] +fn project_search_paths_do_not_apply_to_scripts() -> anyhow::Result<()> { + let case = CliTest::with_files([ + ( + "pyproject.toml", + r#" + [tool.ty.environment] + root = ["./project-source"] + extra-paths = ["./project-extra"] + "#, + ), + ("project-source/project_only.py", "value = 1\n"), + ("project-extra/extra_only.py", "value = 1\n"), + ( + "ordinary.py", + "from extra_only import value as extra\nfrom project_only import value as project\n", + ), + ( + "scripts/script.py", + r#" + # /// script + # dependencies = [] + # /// + + from extra_only import value as extra + from project_only import value as project + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @" + success: false + exit_code: 1 + ----- stdout ----- + error[unresolved-import]: Cannot resolve imported module `extra_only` + --> scripts/script.py:6:6 + | + 6 | from extra_only import value as extra + | ^^^^^^^^^^ + info: Searched in the following paths during module resolution: + info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty) + info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment + + error[unresolved-import]: Cannot resolve imported module `project_only` + --> scripts/script.py:7:6 + | + 7 | from project_only import value as project + | ^^^^^^^^^^^^ + info: Searched in the following paths during module resolution: + info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty) + info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment + + Found 2 diagnostics + + ----- stderr ----- + "); + + Ok(()) +} + +#[test] +fn shared_imports_use_each_scripts_platform() -> anyhow::Result<()> { + let case = CliTest::with_files([ + ( + "shared.py", + r#" + import sys + + if sys.platform == "win32": + value = "windows" + else: + value = "other" + "#, + ), + ( + "windows.py", + r#" + # /// script + # [tool.ty.environment] + # extra-paths = ["."] + # python-platform = "win32" + # /// + + from shared import value + from typing import reveal_type + + reveal_type(value) + "#, + ), + ( + "linux.py", + r#" + # /// script + # [tool.ty.environment] + # extra-paths = ["."] + # python-platform = "linux" + # /// + + from shared import value + from typing import reveal_type + + reveal_type(value) + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @r#" + success: true + exit_code: 0 + ----- stdout ----- + info[revealed-type]: Revealed type + --> linux.py:11:13 + | + 11 | reveal_type(value) + | ^^^^^ `Literal["other"]` + + info[revealed-type]: Revealed type + --> windows.py:11:13 + | + 11 | reveal_type(value) + | ^^^^^ `Literal["windows"]` + + Found 2 diagnostics + + ----- stderr ----- + "#); + + Ok(()) +} + +#[test] +fn inherited_file_settings_are_relative_to_the_script() -> anyhow::Result<()> { + let case = CliTest::with_files([ + ("user-extra/project_dependency.py", "value = 1\n"), + ("scripts/user-extra/user_dependency.py", "value = 1\n"), + ("cli-extra/cli_dependency.py", "value = 1\n"), + ( + "scripts/script.py", + r#" + # /// script + # dependencies = [] + # /// + + from user_dependency import value as user_value + from cli_dependency import value as cli_value + "#, + ), + ])?; + case.write_file( + case.user_config_directory().join("ty/ty.toml"), + r#" + [environment] + extra-paths = ["./user-extra"] + "#, + )?; + + assert_cmd_snapshot!(case.command().arg("--extra-search-path").arg("./cli-extra"), @" + success: true + exit_code: 0 + ----- stdout ----- + All checks passed! + + ----- stderr ----- + "); + + Ok(()) +} + +#[test] +fn scripts_do_not_use_an_inactive_project_environment() -> anyhow::Result<()> { + let dependency = if cfg!(windows) { + ".venv/Lib/site-packages/project_dependency.py" + } else { + ".venv/lib/python3.13/site-packages/project_dependency.py" + }; + + let case = CliTest::with_files([ + (".venv/pyvenv.cfg", "home = ./\nversion = 3.13\n"), + (dependency, "value = 1\n"), + ( + "scripts/script.py", + r#" + # /// script + # dependencies = [] + # /// + + from project_dependency import value + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @" + success: false + exit_code: 1 + ----- stdout ----- + error[unresolved-import]: Cannot resolve imported module `project_dependency` + --> scripts/script.py:6:6 + | + 6 | from project_dependency import value + | ^^^^^^^^^^^^^^^^^^ + info: Searched in the following paths during module resolution: + info: 1. vendored://stdlib (stdlib typeshed stubs vendored by ty) + info: make sure your Python environment is properly configured: https://docs.astral.sh/ty/modules/#python-environment + + Found 1 diagnostic + + ----- stderr ----- + "); + + Ok(()) +} + +#[test] +fn scripts_use_an_activated_virtual_environment() -> anyhow::Result<()> { + let dependency = if cfg!(windows) { + ".venv/Lib/site-packages/project_dependency.py" + } else { + ".venv/lib/python3.13/site-packages/project_dependency.py" + }; + + let case = CliTest::with_files([ + (".venv/pyvenv.cfg", "home = ./\nversion = 3.13\n"), + (dependency, "value = 1\n"), + ( + "scripts/script.py", + r#" + # /// script + # dependencies = [] + # /// + + from project_dependency import value + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command().env("VIRTUAL_ENV", case.root().join(".venv")), @" + success: true + exit_code: 0 + ----- stdout ----- + All checks passed! + + ----- stderr ----- + "); + + Ok(()) +} + +#[test] +fn invalid_python_requirement_falls_back_to_project_configuration() -> anyhow::Result<()> { + let case = CliTest::with_files([ + ( + "pyproject.toml", + r#" + [tool.ty.environment] + python-platform = "linux" + + [tool.ty.rules] + unresolved-reference = "error" + "#, + ), + ( + "script.py", + r#" + # /// script + # requires-python = "<3.12" + # [tool.ty.environment] + # python-platform = "win32" + # [tool.ty.rules] + # unresolved-reference = "warn" + # /// + + import sys + from typing import reveal_type + + reveal_type(sys.platform) + print(missing) + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @r#" + success: false + exit_code: 1 + ----- stdout ----- + info[revealed-type]: Revealed type + --> script.py:13:13 + | + 13 | reveal_type(sys.platform) + | ^^^^^^^^^^^^ `Literal["linux"]` + + error[unresolved-reference]: Name `missing` used when not defined + --> script.py:14:7 + | + 14 | print(missing) + | ^^^^^^^ + + Found 2 diagnostics + + ----- stderr ----- + "#); + assert_cmd_snapshot!(case.command().arg("--output-format").arg("concise"), @r#" + success: false + exit_code: 1 + ----- stdout ----- + script.py:13:13: info[revealed-type] Revealed type: `Literal["linux"]` + script.py:14:7: error[unresolved-reference] Name `missing` used when not defined + Found 2 diagnostics + + ----- stderr ----- + "#); + + Ok(()) +} + +#[test] +fn invalid_script_settings_fall_back_to_project_configuration() -> anyhow::Result<()> { + // FIXME: Scripts with invalid settings should not be checked. + let case = CliTest::with_files([ + ( + "pyproject.toml", + r#" + [tool.ty.environment] + python-platform = "linux" + "#, + ), + ( + "script.py", + r#" + # /// script + # [tool.ty.src] + # include = ["src/**test/"] + # [tool.ty.environment] + # python-platform = "win32" + # /// + + import sys + from typing import reveal_type + + reveal_type(sys.platform) + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @r#" + success: true + exit_code: 0 + ----- stdout ----- + info[revealed-type]: Revealed type + --> script.py:12:13 + | + 12 | reveal_type(sys.platform) + | ^^^^^^^^^^^^ `Literal["linux"]` + + Found 1 diagnostic + + ----- stderr ----- + "#); + + Ok(()) +} + +#[test] +fn invalid_script_environment_falls_back_to_project_configuration() -> anyhow::Result<()> { + // FIXME: Scripts with invalid environments should not be checked. + let case = CliTest::with_files([ + ( + "pyproject.toml", + r#" + [tool.ty.environment] + python-version = "3.13" + python-platform = "linux" + "#, + ), + ( + "script.py", + r#" + # /// script + # [tool.ty.environment] + # python = "./missing-environment" + # python-version = "3.12" + # python-platform = "win32" + # /// + + import sys + from typing import reveal_type + + reveal_type(sys.version_info[:2]) + reveal_type(sys.platform) + "#, + ), + ])?; + + assert_cmd_snapshot!(case.command(), @r#" + success: true + exit_code: 0 + ----- stdout ----- + info[revealed-type]: Revealed type + --> script.py:12:13 + | + 12 | reveal_type(sys.version_info[:2]) + | ^^^^^^^^^^^^^^^^^^^^ `tuple[Literal[3], Literal[13]]` + + info[revealed-type]: Revealed type + --> script.py:13:13 + | + 13 | reveal_type(sys.platform) + | ^^^^^^^^^^^^ `Literal["linux"]` + + Found 2 diagnostics + + ----- stderr ----- + "#); + + Ok(()) +} diff --git a/crates/ty_project/src/db.rs b/crates/ty_project/src/db.rs index 50f23c2390..c7e56d1475 100644 --- a/crates/ty_project/src/db.rs +++ b/crates/ty_project/src/db.rs @@ -6,6 +6,7 @@ use std::{cmp, fmt}; pub use self::changes::ChangeResult; use crate::CollectReporter; use crate::metadata::settings::file_settings; +use crate::script::Script; use crate::{ProgressReporter, Project, ProjectMetadata}; use get_size2::StandardTracker; use ruff_db::Db as SourceDb; @@ -527,11 +528,19 @@ impl SemanticDb for ProjectDatabase { } fn program_file(&self, file: File) -> ProgramFile<'_> { - self.project().program(self).program_file(self, file) + let program = match Script::for_file(self, file) { + None => self.project().program(self), + Some(script) => script.program(self), + }; + + program.program_file(self, file) } - fn python_version_with_source(&self, _file: File) -> &PythonVersionWithSource { - &self.project().program_settings(self).python_version + fn python_version_with_source(&self, file: File) -> &PythonVersionWithSource { + match Script::for_file(self, file) { + None => &self.project().program_settings(self).python_version, + Some(script) => script.python_version_with_source(self), + } } fn rule_selection(&self, file: File) -> &RuleSelection { @@ -642,6 +651,8 @@ pub(crate) mod testing { use ty_python_semantic::{AnalysisSettings, PythonVersionWithSource}; use crate::db::Db; + use crate::metadata::settings::file_settings; + use crate::script::Script; use crate::{Project, ProjectMetadata}; type Events = Arc>>; @@ -773,11 +784,19 @@ pub(crate) mod testing { #[salsa::db] impl ty_python_semantic::Db for TestDb { fn program_file(&self, file: File) -> ProgramFile<'_> { - self.project().program(self).program_file(self, file) + let program = match Script::for_file(self, file) { + None => self.project().program(self), + Some(script) => script.program(self), + }; + + program.program_file(self, file) } - fn python_version_with_source(&self, _file: File) -> &PythonVersionWithSource { - &self.project().program_settings(self).python_version + fn python_version_with_source(&self, file: File) -> &PythonVersionWithSource { + match Script::for_file(self, file) { + None => &self.project().program_settings(self).python_version, + Some(script) => script.python_version_with_source(self), + } } #[inline] @@ -785,16 +804,16 @@ pub(crate) mod testing { crate::check_file(self, file) } - fn rule_selection(&self, _file: ruff_db::files::File) -> &RuleSelection { - self.project().rules(self) + fn rule_selection(&self, file: ruff_db::files::File) -> &RuleSelection { + file_settings(self, file).rules(self) } fn lint_registry(&self) -> &LintRegistry { ty_python_semantic::default_lint_registry() } - fn analysis_settings(&self, _file: ruff_db::files::File) -> &AnalysisSettings { - self.project().settings(self).analysis() + fn analysis_settings(&self, file: ruff_db::files::File) -> &AnalysisSettings { + file_settings(self, file).analysis(self) } fn verbose(&self) -> bool { diff --git a/crates/ty_project/src/lib.rs b/crates/ty_project/src/lib.rs index d80f5b631c..6bc4424122 100644 --- a/crates/ty_project/src/lib.rs +++ b/crates/ty_project/src/lib.rs @@ -5,6 +5,7 @@ use crate::glob::{GlobFilterCheckMode, IncludeResult}; use crate::metadata::options::OptionDiagnostic; use crate::parallel::ParallelIteratorExt; +use crate::script::Script; use crate::walk::{ProjectFilesFilter, ProjectFilesWalker}; #[cfg(feature = "testing")] pub use db::testing::TestDb; @@ -243,7 +244,7 @@ impl Project { #[salsa::tracked(returns(copy), heap_size=ruff_memory_usage::heap_size)] pub fn program(self, db: &dyn Db) -> Program<'_> { - Program::from_settings(db, self.program_settings(db).clone()) + Program::from_settings(db, self.program_settings(db)) } pub fn update_program(self, db: &mut dyn Db, settings: ProgramSettings) { @@ -742,7 +743,19 @@ pub(crate) fn check_file_impl( { let db = AssertUnwindSafe(db); match catch(&**db, source_file, || { - ty_python_semantic::check_file(*db, file) + let diagnostics = ty_python_semantic::check_file(*db, file)?; + let Some(script) = Script::for_file(*db, source_file) else { + return Ok(diagnostics); + }; + + let script_diagnostics = script.diagnostics(*db); + if script_diagnostics.is_empty() { + return Ok(diagnostics); + } + + let mut diagnostics = diagnostics.into_vec(); + diagnostics.extend(script_diagnostics.iter().cloned()); + Ok(diagnostics.into_boxed_slice()) }) { Ok(result) => result, Err(diagnostic) => Ok(Box::new([diagnostic])), diff --git a/crates/ty_project/src/metadata.rs b/crates/ty_project/src/metadata.rs index 04afe3fb72..5ee4433fb6 100644 --- a/crates/ty_project/src/metadata.rs +++ b/crates/ty_project/src/metadata.rs @@ -12,7 +12,8 @@ use ty_static::EnvVars; use crate::Db; use crate::metadata::options::{ - EnvironmentOptions, OptionDiagnostic, ProgramSettingsDiagnostic, ToSettingsError, + EnvironmentOptions, OptionDiagnostic, OptionsContext, ProgramSettingsDiagnostic, + ToSettingsError, }; use crate::metadata::pyproject::{Project, PyProject, PyProjectError, ResolveRequiresPythonError}; use crate::metadata::settings::Settings; @@ -143,26 +144,13 @@ impl ProjectMetadata { .map(|name| ProjectName::new(&**name)) .unwrap_or_else(|| ProjectName::new(root.file_name().unwrap_or("root"))); - // If the `options` don't specify a python version but the `project.requires-python` field is set, - // use that as a lower bound instead. if let Some(project) = project { - if options - .environment - .as_ref() - .is_none_or(|env| env.python_version.is_none()) - { - let requires_python = strategy.fallback_opt( - project.resolve_requires_python_lower_bound(), - |err| { - tracing::debug!("skipping invalid requires_python lower bound: {err}"); - }, - )?; - if let Some(requires_python) = requires_python.flatten() { - let mut environment = options.environment.unwrap_or_default(); - environment.python_version = Some(requires_python); - options.environment = Some(environment); - } - } + // If the `options` don't specify a python version but the `project.requires-python` field is set, + // use that as a lower bound instead. + strategy.fallback( + options.apply_requires_python(project.requires_python.as_ref()), + |error| tracing::debug!("skipping invalid requires_python lower bound: {error}"), + )?; } Ok(Self { @@ -412,7 +400,7 @@ impl ProjectMetadata { self.name.as_str() } - fn options(&self) -> &Options { + pub(crate) fn options(&self) -> &Options { &self.options } @@ -496,6 +484,26 @@ impl ProjectMetadata { .chain(self.fallback_options.as_deref()) } + /// Returns the option layers applicable to a standalone script. + /// + /// Scripts inherit invocation and user settings, but not the enclosing project's options or + /// Python-version settings derived from its uv workspace. + pub(crate) fn script_options_in_precedence_order<'a>( + &'a self, + options: &'a Options, + ) -> impl Iterator { + self.override_options + .as_deref() + .into_iter() + .chain(std::iter::once(options)) + .chain( + self.user_configuration + .as_deref() + .map(|(_, options)| options), + ) + .chain(self.fallback_options.as_deref()) + } + /// Loads the lower-precedence options from configuration files. /// /// This includes: @@ -567,7 +575,7 @@ impl MergedOptions<'_> { ) -> Result<(ProgramSettings, Vec), Strategy::Error> { self.options.to_program_settings( - self.metadata.root(), + OptionsContext::Project(self.metadata.root()), self.metadata.name(), system, vendored, @@ -580,7 +588,8 @@ impl MergedOptions<'_> { db: &dyn Db, strategy: &Strategy, ) -> Result<(Settings, Vec), Strategy::Error> { - self.options.to_settings(db, self.metadata.root(), strategy) + self.options + .to_settings(db, OptionsContext::Project(self.metadata.root()), strategy) } } diff --git a/crates/ty_project/src/metadata/options.rs b/crates/ty_project/src/metadata/options.rs index a004813337..fa98bd5087 100644 --- a/crates/ty_project/src/metadata/options.rs +++ b/crates/ty_project/src/metadata/options.rs @@ -1,5 +1,6 @@ use crate::Db; use crate::glob::{ExcludeFilter, IncludeExcludeFilter, IncludeFilter, PortableGlobKind}; +use crate::metadata::pyproject::{ResolveRequiresPythonError, resolve_requires_python_lower_bound}; use crate::metadata::python_version::SupportedPythonVersion; use crate::metadata::settings::{OverrideSettings, SrcSettings}; @@ -7,12 +8,12 @@ use super::settings::{Override, Settings, TerminalSettings}; use crate::metadata::value::{RelativeGlobPattern, RelativePathBuf}; use anyhow::Context; use ordermap::OrderMap; +use pep440_rs::VersionSpecifiers; use ruff_db::RustDoc; use ruff_db::diagnostic::{ Annotation, Diagnostic, DiagnosticFormat, DiagnosticId, DisplayDiagnosticConfig, Severity, Span, SubDiagnostic, SubDiagnosticSeverity, }; -use ruff_db::files::system_path_to_file; use ruff_db::system::{System, SystemPath, SystemPathBuf}; use ruff_db::vendored::VendoredFileSystem; use ruff_macros::{Combine, OptionsMetadata, RustDoc}; @@ -111,13 +112,6 @@ pub struct Options { } impl Options { - pub(super) fn file_options(&self) -> FileOptions { - FileOptions { - rules: self.rules.clone(), - analysis: self.analysis.clone(), - } - } - pub fn from_toml_str(content: &str, source: ValueSource) -> Result { let _guard = ValueSourceGuard::new(source, true); let mut options: Self = toml::from_str(content)?; @@ -125,6 +119,28 @@ impl Options { Ok(options) } + /// Infers the Python version from `requires-python` unless it was configured explicitly. + pub(crate) fn apply_requires_python( + &mut self, + requires_python: Option<&RangedValue>, + ) -> Result<(), ResolveRequiresPythonError> { + if self + .environment + .as_ref() + .is_some_and(|environment| environment.python_version.is_some()) + { + return Ok(()); + } + + if let Some(requires_python) = requires_python + && let Some(python_version) = resolve_requires_python_lower_bound(requires_python)? + { + self.environment.get_or_insert_default().python_version = Some(python_version); + } + + Ok(()) + } + /// Ensures that the `all` selector is applied before per-rule selectors /// in all rule tables (top-level and overrides). /// @@ -164,9 +180,10 @@ impl Options { Self::deserialize(deserializer) } + /// Resolve configured paths and discover defaults according to the project or script context. pub(crate) fn to_program_settings( &self, - project_root: &SystemPath, + context: OptionsContext<'_>, project_name: &str, system: &dyn System, vendored: &VendoredFileSystem, @@ -196,15 +213,20 @@ impl Options { ValueSource::File(path) => { SysPrefixPathOrigin::ConfigFileSetting(path.clone(), python_path.range()) } + ValueSource::ScriptMetadata(_) => SysPrefixPathOrigin::ScriptMetadataSetting, ValueSource::Editor => SysPrefixPathOrigin::Editor, ValueSource::UvWorkspace => SysPrefixPathOrigin::UvWorkspace, }; - PythonEnvironment::new(python_path.absolute(project_root, system), origin, system) - .map_err(anyhow::Error::from) - .map(Some) + PythonEnvironment::new( + python_path.absolute(context.configuration_root(), system), + origin, + system, + ) + .map_err(anyhow::Error::from) + .map(Some) } else { - PythonEnvironment::discover(project_root, system) + PythonEnvironment::discover(context.project_root(), system) .context("Failed to discover local Python environment") }; @@ -277,7 +299,7 @@ impl Options { // Safe mode is handled inside this function, so we just assume this can't fail let search_paths = strategy.to_anyhow(self.to_search_paths( - project_root, + context, project_name, site_packages_paths, real_stdlib_path, @@ -304,7 +326,7 @@ impl Options { #[expect(clippy::too_many_arguments)] fn to_search_paths( &self, - project_root: &SystemPath, + context: OptionsContext<'_>, project_name: &str, site_packages_paths: SitePackagesPaths, real_stdlib_path: Option, @@ -317,9 +339,10 @@ impl Options { let environment_roots = if let Some(roots) = environment.root.as_deref() { roots .iter() - .map(|root| root.absolute(project_root, system)) + .map(|root| root.absolute(context.configuration_root(), system)) .collect() } else { + let project_root = context.configuration_root(); let mut roots = vec![]; let is_package = |dir: &SystemPath| { system.is_file(&dir.join("__init__.py")) @@ -375,7 +398,7 @@ impl Options { .as_deref() .unwrap_or_default() .iter() - .map(|path| path.absolute(project_root, system)) + .map(|path| path.absolute(context.configuration_root(), system)) .collect(); // read all the paths off the PYTHONPATH environment variable, check @@ -421,7 +444,7 @@ impl Options { custom_typeshed: environment .typeshed .as_ref() - .map(|path| path.absolute(project_root, system)), + .map(|path| path.absolute(context.configuration_root(), system)), site_packages_paths: site_packages_paths.into_vec(), real_stdlib_path, }; @@ -432,7 +455,7 @@ impl Options { pub(crate) fn to_settings( &self, db: &dyn Db, - project_root: &SystemPath, + context: OptionsContext<'_>, strategy: &Strategy, ) -> Result<(Settings, Vec), Strategy::Error> { let mut diagnostics = Vec::new(); @@ -451,7 +474,7 @@ impl Options { let src_options = self.src.or_default(); let src = src_options - .to_settings(db, project_root, &mut diagnostics) + .to_settings(db, context.configuration_root(), &mut diagnostics) .map_err(|err| ToSettingsError { diagnostic: err, output_format: terminal.output_format, @@ -478,7 +501,7 @@ impl Options { let analysis = strategy.fallback(analysis_result, |_| AnalysisSettings::default())?; let overrides = self - .to_overrides_settings(db, project_root, &mut diagnostics) + .to_overrides_settings(db, context.configuration_root(), &mut diagnostics) .map_err(|err| ToSettingsError { diagnostic: err, output_format: terminal.output_format, @@ -534,6 +557,29 @@ impl Options { } } +/// The project or standalone script whose options are being resolved. +#[derive(Clone, Copy, Debug)] +pub(crate) enum OptionsContext<'a> { + Project(&'a SystemPath), + /// The directory containing a standalone script, or the working directory for a virtual script. + Script(&'a SystemPath), +} + +impl<'a> OptionsContext<'a> { + fn configuration_root(self) -> &'a SystemPath { + match self { + Self::Project(root) | Self::Script(root) => root, + } + } + + fn project_root(self) -> Option<&'a SystemPath> { + match self { + Self::Project(root) => Some(root), + Self::Script(_) => None, + } + } +} + fn python_version_from_config( ranged_version: &RangedValue, ) -> PythonVersionWithSource { @@ -544,6 +590,9 @@ fn python_version_from_config( ValueSource::File(path) => PythonVersionSource::ConfigFile( PythonVersionFileSource::new(path.clone(), ranged_version.range()), ), + ValueSource::ScriptMetadata(file) => PythonVersionSource::ScriptMetadata( + Span::from(*file).with_optional_range(ranged_version.range()), + ), ValueSource::Editor => PythonVersionSource::Editor, ValueSource::UvWorkspace => PythonVersionSource::UvWorkspace, }, @@ -637,6 +686,12 @@ fn unsupported_inferred_python_version_diagnostic( SubDiagnosticSeverity::Info, "The version was inferred from a configuration file.", )), + source @ PythonVersionSource::ScriptMetadata(_) => diagnostic + .with_annotation(inferred_python_version_source_annotation(db, source)) + .sub(SubDiagnostic::new( + SubDiagnosticSeverity::Info, + "The version was inferred from script metadata.", + )), source @ PythonVersionSource::PyvenvCfgFile(_) => diagnostic .with_annotation(inferred_python_version_source_annotation(db, source)) .sub(SubDiagnostic::new( @@ -751,6 +806,9 @@ pub struct EnvironmentOptions { /// * `./src` /// * `./` (if a `.//` directory exists) /// * `./python` + /// + /// Scripts with inline metadata have no first-party roots by default because they are + /// single-file programs. Set `root = ["."]` to allow importing local modules. #[serde(skip_serializing_if = "Option::is_none")] #[option( default = r#"null"#, @@ -780,6 +838,9 @@ pub struct EnvironmentOptions { /// and attempt to infer the Python version of that environment /// 3. Fall back to the default value (see below) /// + /// Scripts with inline metadata use their `requires-python` field instead of + /// `project.requires-python`. They do not inherit the Python version of the enclosing project. + /// /// For some language features, ty can also understand conditionals based on comparisons /// with `sys.version_info`. These are commonly found in typeshed, for example, /// to reflect the differing contents of the standard library across Python versions. @@ -864,6 +925,10 @@ pub struct EnvironmentOptions { /// in the project root if none of the above apply. Failing that, ty will look for a `python3` /// or `python` binary available in `PATH`. /// + /// Scripts with inline metadata use their own Python environment. They can use an explicitly + /// configured environment, an activated environment, or an environment selected by the editor. + /// Unlike projects, they do not automatically use a `.venv` directory. + /// /// [`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix #[serde(skip_serializing_if = "Option::is_none")] #[option( @@ -1079,6 +1144,7 @@ impl Rules { let source = rule_name.source(); let lint_source = match source { ValueSource::File(_) => LintSource::File, + ValueSource::ScriptMetadata(_) => LintSource::ScriptMetadata, ValueSource::Cli => LintSource::Cli, ValueSource::Editor => LintSource::Editor, ValueSource::UvWorkspace => LintSource::UvWorkspace, @@ -1105,12 +1171,9 @@ impl Rules { set_lint_level(lint); } Err(error) => { - // `system_path_to_file` can return `Err` if the file was deleted since the configuration - // was read. This should be rare and it should be okay to default to not showing a configuration - // file in that case. - let file = source - .file() - .and_then(|path| system_path_to_file(db, path).ok()); + // The file may have been deleted since its configuration was read. In that + // case, report the diagnostic without a configuration-file annotation. + let file = source.file(db); // TODO: Add a note if the value was configured on the CLI let diagnostic = OptionDiagnostic::new( @@ -1187,14 +1250,12 @@ fn build_include_filter( )); // Add source annotation if we have source information - if let Some(source_file) = include_patterns.source().file() { - if let Ok(file) = system_path_to_file(db, source_file) { - let annotation = Annotation::primary( - Span::from(file).with_optional_range(include_patterns.range()), - ) - .message("This `include` list is empty"); - diagnostic = diagnostic.with_annotation(Some(annotation)); - } + if let Some(file) = include_patterns.source().file(db) { + let annotation = Annotation::primary( + Span::from(file).with_optional_range(include_patterns.range()), + ) + .message("This `include` list is empty"); + diagnostic = diagnostic.with_annotation(Some(annotation)); } diagnostics.push(diagnostic); @@ -1951,13 +2012,11 @@ impl ToOverride for RangedValue { )); // Add source annotation if we have source information - if let Some(source_file) = self.source().file() { - if let Ok(file) = system_path_to_file(db, source_file) { - let annotation = - Annotation::primary(Span::from(file).with_optional_range(self.range())) - .message("This overrides section overrides no settings"); - diagnostic = diagnostic.with_annotation(Some(annotation)); - } + if let Some(file) = self.source().file(db) { + let annotation = + Annotation::primary(Span::from(file).with_optional_range(self.range())) + .message("This overrides section overrides no settings"); + diagnostic = diagnostic.with_annotation(Some(annotation)); } diagnostics.push(diagnostic); @@ -2004,13 +2063,11 @@ impl ToOverride for RangedValue { )); // Add source annotation if we have source information - if let Some(source_file) = self.source().file() { - if let Ok(file) = system_path_to_file(db, source_file) { - let annotation = - Annotation::primary(Span::from(file).with_optional_range(self.range())) - .message("This overrides section applies to all files"); - diagnostic = diagnostic.with_annotation(Some(annotation)); - } + if let Some(file) = self.source().file(db) { + let annotation = + Annotation::primary(Span::from(file).with_optional_range(self.range())) + .message("This overrides section applies to all files"); + diagnostic = diagnostic.with_annotation(Some(annotation)); } diagnostics.push(diagnostic); @@ -2080,15 +2137,6 @@ pub(super) struct InnerOverrideOptions { pub(super) analysis: Option, } -/// The settings that can vary between individual files. -#[derive(Debug, Default, Clone, PartialEq, Eq, Hash, Combine, get_size2::GetSize)] -pub(super) struct FileOptions { - /// Raw rule options, preserved so multiple configuration layers can be merged. - pub(super) rules: Option, - - pub(super) analysis: Option, -} - /// Error returned when the settings can't be resolved because of a hard error. #[derive(Debug)] pub struct ToSettingsError { @@ -2263,8 +2311,8 @@ impl OptionDiagnostic { err: impl Display, ) -> Self { match value.source() { - ValueSource::File(file_path) => { - if let Ok(file) = system_path_to_file(db, &**file_path) { + ValueSource::File(_) | ValueSource::ScriptMetadata(_) => { + if let Some(file) = value.source().file(db) { let concise_message = std::mem::take(&mut self.message); self.with_concise_message(concise_message) .with_message(format_args!("Invalid {value_label}")) diff --git a/crates/ty_project/src/metadata/pyproject.rs b/crates/ty_project/src/metadata/pyproject.rs index 91e1a90272..c3e02842ad 100644 --- a/crates/ty_project/src/metadata/pyproject.rs +++ b/crates/ty_project/src/metadata/pyproject.rs @@ -40,14 +40,6 @@ impl PyProject { Self::deserialize_toml(content) } - pub(crate) fn from_toml_str_without_spans( - content: &str, - source: ValueSource, - ) -> Result { - let _guard = ValueSourceGuard::new(source, false); - Self::deserialize_toml(content) - } - fn deserialize_toml(content: &str) -> Result { let mut pyproject: Self = toml::from_str(content).map_err(PyProjectError::TomlSyntax)?; // TOML tables are unordered and the `toml` crate sorts keys @@ -79,69 +71,63 @@ pub struct Project { pub(crate) requires_python: Option>, } -impl Project { - pub(super) fn resolve_requires_python_lower_bound( - &self, - ) -> Result>, ResolveRequiresPythonError> { - let Some(requires_python) = self.requires_python.as_ref() else { - return Ok(None); - }; - - tracing::debug!("Resolving requires-python constraint: `{requires_python}`"); - - let ranges = release_specifiers_to_ranges((**requires_python).clone()); - let Some((lower, _)) = ranges.bounding_range() else { - return Ok(None); - }; - - let version = match lower { - // Ex) `>=3.10.1` -> `>=3.10` - Bound::Included(version) => version, - - // Ex) `>3.10.1` -> `>=3.10` or `>3.10` -> `>=3.10` - // The second example looks obscure at first but it is required because - // `3.10.1 > 3.10` is true but we only have two digits here. So including 3.10 is the - // right move. Overall, using `>` without a patch release is most likely bogus. - Bound::Excluded(version) => version, - - // Ex) `<3.10` or `` - Bound::Unbounded => { - return Err(ResolveRequiresPythonError::NoLowerBound( - requires_python.to_string(), - )); - } - }; +pub(super) fn resolve_requires_python_lower_bound( + requires_python: &RangedValue, +) -> Result>, ResolveRequiresPythonError> { + tracing::debug!("Resolving requires-python constraint: `{requires_python}`"); + + let ranges = release_specifiers_to_ranges((**requires_python).clone()); + let Some((lower, _)) = ranges.bounding_range() else { + return Ok(None); + }; + + let version = match lower { + // Ex) `>=3.10.1` -> `>=3.10` + Bound::Included(version) => version, + + // Ex) `>3.10.1` -> `>=3.10` or `>3.10` -> `>=3.10` + // The second example looks obscure at first but it is required because + // `3.10.1 > 3.10` is true but we only have two digits here. So including 3.10 is the + // right move. Overall, using `>` without a patch release is most likely bogus. + Bound::Excluded(version) => version, + + // Ex) `<3.10` or `` + Bound::Unbounded => { + return Err(ResolveRequiresPythonError::NoLowerBound( + requires_python.to_string(), + )); + } + }; - // Take the major and minor version - let mut versions = version.release().iter().take(2); + // Take the major and minor version + let mut versions = version.release().iter().take(2); - let Some(major) = versions.next().copied() else { - return Ok(None); - }; + let Some(major) = versions.next().copied() else { + return Ok(None); + }; - let minor = versions.next().copied().unwrap_or_default(); + let minor = versions.next().copied().unwrap_or_default(); - tracing::debug!("Resolved requires-python constraint to: {major}.{minor}"); + tracing::debug!("Resolved requires-python constraint to: {major}.{minor}"); - let major = - u8::try_from(major).map_err(|_| ResolveRequiresPythonError::TooLargeMajor(major))?; - let minor = - u8::try_from(minor).map_err(|_| ResolveRequiresPythonError::TooLargeMinor(minor))?; + let major = + u8::try_from(major).map_err(|_| ResolveRequiresPythonError::TooLargeMajor(major))?; + let minor = + u8::try_from(minor).map_err(|_| ResolveRequiresPythonError::TooLargeMinor(minor))?; - let lower_bound = PythonVersion::from((major, minor)); - let supported_version = SupportedPythonVersion::iter() - .find(|supported_version| supported_version.to_python_version() >= lower_bound); + let lower_bound = PythonVersion::from((major, minor)); + let supported_version = SupportedPythonVersion::iter() + .find(|supported_version| supported_version.to_python_version() >= lower_bound); - let Some(supported_version) = supported_version else { - return Err(ResolveRequiresPythonError::NoSupportedVersion( - requires_python.to_string(), - )); - }; + let Some(supported_version) = supported_version else { + return Err(ResolveRequiresPythonError::NoSupportedVersion( + requires_python.to_string(), + )); + }; - Ok(Some( - requires_python.clone().map_value(|_| supported_version), - )) - } + Ok(Some( + requires_python.clone().map_value(|_| supported_version), + )) } #[derive(Debug, Error)] diff --git a/crates/ty_project/src/metadata/settings.rs b/crates/ty_project/src/metadata/settings.rs index 5a0b6cd69e..c183d7ce9f 100644 --- a/crates/ty_project/src/metadata/settings.rs +++ b/crates/ty_project/src/metadata/settings.rs @@ -5,8 +5,8 @@ use ty_combine::Combine; use ty_python_semantic::AnalysisSettings; use ty_python_semantic::lint::RuleSelection; -use crate::metadata::options::{InnerOverrideOptions, Options, OutputFormat}; -use crate::script::script_metadata; +use crate::metadata::options::{InnerOverrideOptions, OutputFormat}; +use crate::script::Script; use crate::{Db, glob::IncludeExcludeFilter}; /// The resolved [`super::Options`] for the project. @@ -125,38 +125,16 @@ impl Override { /// Resolves the settings for a given file. #[salsa::tracked(returns(ref), heap_size=ruff_memory_usage::heap_size)] pub(crate) fn file_settings(db: &dyn Db, file: File) -> FileSettings { - let project = db.project(); - - // Ignore script settings for files that aren't checked as part of the project. Check for - // metadata first so files without metadata don't depend on the low-durability open-file set. - if let Some(script) = script_metadata(db, file) - && crate::should_check_file(db, file) - { - let inline = script.ty().cloned().unwrap_or_default(); - let metadata = project.metadata(db); - let primary = if metadata.config_file_override().is_some() { - metadata.options() - } else { - &inline - }; - let mut options = metadata - .options_in_precedence_order(primary) - .map(Options::file_options); - let mut merged = options.next().unwrap_or_default(); - - for option in options { - merged.combine_with(option); - } - - let rules = merged.rules.unwrap_or_default(); - let analysis = merged.analysis.unwrap_or_default(); - - let rules = rules.to_rule_selection(db, &mut Vec::new()); - let analysis = analysis.to_settings(db, &mut Vec::new()); - - return FileSettings::File(Arc::new(OverrideSettings { rules, analysis })); + if let Some(script) = Script::for_file(db, file) { + let settings = script.settings(db); + return FileSettings::File(Arc::new(OverrideSettings { + rules: settings.rules().clone(), + analysis: settings.analysis().clone(), + })); } + let project = db.project(); + let settings = project.settings(db); let path = match file.path(db) { @@ -249,7 +227,7 @@ fn merge_overrides(db: &dyn Db, overrides: Vec>, _: () /// The resolved settings for a file. #[derive(Debug, Eq, PartialEq, Clone, get_size2::GetSize)] -pub enum FileSettings { +pub(crate) enum FileSettings { /// The file uses the global settings. Global, diff --git a/crates/ty_project/src/metadata/value.rs b/crates/ty_project/src/metadata/value.rs index 616a118080..09d597214c 100644 --- a/crates/ty_project/src/metadata/value.rs +++ b/crates/ty_project/src/metadata/value.rs @@ -17,7 +17,7 @@ use crate::glob::{ /// require different anchoring: /// /// * CLI: The path is relative to the current working directory -/// * Configuration file: The path is relative to the project's root. +/// * Configuration file: The path is relative to the project's or script's configuration root. #[derive( Debug, Clone, @@ -62,9 +62,9 @@ impl RelativePathBuf { } /// Resolves the absolute path for `self` based on its origin. - pub fn absolute(&self, project_root: &SystemPath, system: &dyn System) -> SystemPathBuf { + pub fn absolute(&self, configuration_root: &SystemPath, system: &dyn System) -> SystemPathBuf { let relative_to = match self.0.source() { - ValueSource::File(_) => project_root, + ValueSource::File(_) | ValueSource::ScriptMetadata(_) => configuration_root, ValueSource::Cli | ValueSource::Editor | ValueSource::UvWorkspace => { system.current_directory() } @@ -136,7 +136,7 @@ impl RelativeGlobPattern { kind: PortableGlobKind, ) -> Result { let relative_to = match self.0.source() { - ValueSource::File(_) => project_root, + ValueSource::File(_) | ValueSource::ScriptMetadata(_) => project_root, ValueSource::Cli | ValueSource::Editor | ValueSource::UvWorkspace => { system.current_directory() } diff --git a/crates/ty_project/src/script.rs b/crates/ty_project/src/script.rs index 1800ddc9b5..19822dac86 100644 --- a/crates/ty_project/src/script.rs +++ b/crates/ty_project/src/script.rs @@ -1,17 +1,175 @@ -use std::sync::Arc; - -use ruff_db::Db; +use pep440_rs::VersionSpecifiers; +use ruff_db::Db as SourceDb; +use ruff_db::diagnostic::Diagnostic; use ruff_db::files::File; use ruff_db::source::source_text; -use ruff_db::system::SystemPathBuf; use ruff_python_ast::script::ScriptTag; -use ruff_ranged_value::ValueSource; +use ruff_ranged_value::{RangedValue, ValueSource, ValueSourceGuard}; +use serde::Deserialize; +use ty_combine::Combine; +use ty_python_core::program::{FallibleStrategy, Program, ProgramSettings}; +use ty_python_semantic::PythonVersionWithSource; + +use crate::metadata::options::{Options, OptionsContext}; +use crate::metadata::pyproject::Tool; +use crate::metadata::settings::Settings; +use crate::{Db, ProjectMetadata}; + +/// A standalone PEP 723 script and its resolved settings. +#[salsa::tracked(debug, heap_size=ruff_memory_usage::heap_size)] +pub(crate) struct Script<'db> { + #[returns(copy)] + pub(crate) file: File, + + #[tracked] + #[returns(ref)] + pub(crate) settings: Settings, + + #[tracked] + #[returns(copy)] + pub(crate) program: Program<'db>, + + #[tracked] + #[returns(ref)] + pub(crate) python_version_with_source: PythonVersionWithSource, + + #[tracked] + #[returns(deref)] + pub(crate) diagnostics: Box<[Diagnostic]>, +} -use crate::metadata::pyproject::PyProject; +impl<'db> Script<'db> { + /// Returns the script for `file` without creating a second Salsa memo for ordinary files. + pub(crate) fn for_file(db: &'db dyn Db, file: File) -> Option { + // Most files are not scripts. Check the existing metadata query first so ordinary files + // do not also allocate a tracked `script` memo just to cache another `None`. + script_metadata(db, file)?; + script(db, file) + } +} + +impl get_size2::GetSize for Script<'_> {} + +/// Resolve the `Script` for `file` if it has a PEP 723 metadata block or `None` otherwise. +#[salsa::tracked(returns(copy), heap_size=ruff_memory_usage::heap_size)] +pub(crate) fn script(db: &dyn Db, file: File) -> Option> { + // Files without script metadata must not depend on the low-durability open-file set. + let metadata = script_metadata(db, file)?; + + // Never treat third-party files as scripts. + if !crate::should_check_file(db, file) { + return None; + } + + let configuration_root = file + .path(db) + .as_system_path() + .and_then(|path| path.parent()) + .unwrap_or_else(|| db.system().current_directory()); + let context = OptionsContext::Script(configuration_root); + + let project_metadata = db.project().metadata(db); + + let mut diagnostics = Vec::new(); + // FIXME: Report configuration errors as diagnostics and skip checking the script entirely so + // that fixes cannot be applied using the enclosing project's configuration. + let options = resolve_script_options(project_metadata, metadata)?; + let settings = resolve_script_settings(db, &options, context, &mut diagnostics)?; + let program_settings = resolve_script_program_settings( + db, + &options, + context, + project_metadata.name(), + &mut diagnostics, + )?; + + program_settings.search_paths.try_register_static_roots(db); + + let program = Program::from_settings(db, &program_settings); + + Some(Script::new( + db, + file, + settings, + program, + program_settings.python_version, + diagnostics.into_boxed_slice(), + )) +} + +fn resolve_script_options( + project_metadata: &ProjectMetadata, + metadata: &ScriptMetadata, +) -> Option { + // When using `--config-file `, use the settings from `` + let inline = if project_metadata.config_file_override().is_some() { + project_metadata.options().clone() + } else { + // Otherwise use the script's settings. + metadata.to_options()? + }; + + let mut options = Options::default(); + // Merge the options with CLI, LSP, user configuration, and fallback options + for layer in project_metadata.script_options_in_precedence_order(&inline) { + options.combine_with(layer.clone()); + } + + // Unlike Project's, default to `[]` for scripts (unless explicitly specified). + options + .environment + .get_or_insert_default() + .root + .get_or_insert_default(); + + Some(options) +} + +fn resolve_script_settings( + db: &dyn Db, + options: &Options, + context: OptionsContext<'_>, + diagnostics: &mut Vec, +) -> Option { + let (settings, settings_diagnostics) = + options.to_settings(db, context, &FallibleStrategy).ok()?; + diagnostics.extend( + settings_diagnostics + .into_iter() + .map(|diagnostic| diagnostic.to_diagnostic()), + ); + Some(settings) +} + +fn resolve_script_program_settings( + db: &dyn Db, + options: &Options, + context: OptionsContext<'_>, + project_name: &str, + diagnostics: &mut Vec, +) -> Option { + let (settings, settings_diagnostics) = options + .to_program_settings( + context, + project_name, + db.system(), + db.vendored(), + &FallibleStrategy, + ) + .ok()?; + diagnostics.extend( + settings_diagnostics + .into_iter() + .map(|diagnostic| diagnostic.into_diagnostic(db).to_diagnostic()), + ); + Some(settings) +} /// Returns the PEP 723 metadata embedded in `file`. -#[salsa::tracked(returns(ref))] -pub(crate) fn script_metadata(db: &dyn Db, file: File) -> Option> { +/// +/// Most files have no script metadata. Boxing keeps the cached result compact when it is `None`. +#[salsa::tracked(returns(as_deref))] +pub(crate) fn script_metadata(db: &dyn SourceDb, file: File) -> Option> { let path = file.path(db); if path.is_vendored_path() { return None; @@ -23,9 +181,119 @@ pub(crate) fn script_metadata(db: &dyn Db, file: File) -> Option> } let tag = ScriptTag::parse(source.as_bytes())?; - let value_source = ValueSource::File(Arc::new(SystemPathBuf::from(path.as_str()))); + let _guard = ValueSourceGuard::new(ValueSource::ScriptMetadata(file), false); + // FIXME: Report invalid TOML in script metadata instead of silently ignoring it. + let mut metadata: ScriptMetadata = toml::from_str(tag.metadata()).ok()?; + + if let Some(options) = metadata.tool.as_mut().and_then(|tool| tool.ty.as_mut()) { + options.prioritize_all_selectors(); + } + + Some(Box::new(metadata)) +} + +/// PEP 723 metadata, whose Python requirement belongs at the top level rather than in `project`. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize)] +#[serde(rename_all = "kebab-case")] +pub(crate) struct ScriptMetadata { + requires_python: Option>, + tool: Option, +} + +impl ScriptMetadata { + fn to_options(&self) -> Option { + let mut options = self.ty().cloned().unwrap_or_default(); + options + .apply_requires_python(self.requires_python.as_ref()) + .ok()?; + Some(options) + } + + fn ty(&self) -> Option<&Options> { + self.tool.as_ref().and_then(|tool| tool.ty.as_ref()) + } +} - PyProject::from_toml_str_without_spans(tag.metadata(), value_source) - .map(Box::new) - .ok() +#[cfg(test)] +mod tests { + use ruff_db::files::system_path_to_file; + use ruff_db::system::{DbWithWritableSystem as _, SystemPath, SystemPathBuf}; + use ruff_db::testing::assert_function_query_was_not_run; + use ty_python_semantic::Db as _; + + use crate::db::testing::TestDb; + use crate::{Db as _, ProjectMetadata}; + + use super::{Script, script}; + + #[test] + fn ordinary_files_do_not_depend_on_open_files() -> anyhow::Result<()> { + let mut db = TestDb::new(ProjectMetadata::new( + "test", + SystemPathBuf::from("/project"), + )); + db.write_files([ + ("/project/ordinary.py", "value = 1\n"), + ("/project/opened.py", "value = 2\n"), + ])?; + let ordinary = system_path_to_file(&db, SystemPath::new("/project/ordinary.py"))?; + let opened = system_path_to_file(&db, SystemPath::new("/project/opened.py"))?; + + assert!(Script::for_file(&db, ordinary).is_none()); + let events = db.take_salsa_events(); + assert_function_query_was_not_run(&db, script, ordinary, &events); + + assert!(script(&db, ordinary).is_none()); + db.take_salsa_events(); + + db.project().open_file(&mut db, opened); + db.take_salsa_events(); + + assert!(script(&db, ordinary).is_none()); + let events = db.take_salsa_events(); + assert_function_query_was_not_run(&db, crate::should_check_file, ordinary, &events); + assert_function_query_was_not_run(&db, script, ordinary, &events); + + Ok(()) + } + + #[test] + fn equivalent_script_settings_share_programs() -> anyhow::Result<()> { + let mut db = TestDb::new(ProjectMetadata::new( + "test", + SystemPathBuf::from("/project"), + )); + db.write_dedented( + "/project/requirement.py", + r#" + # /// script + # requires-python = ">=3.12" + # /// + "#, + )?; + db.write_dedented( + "/project/nested/configured.py", + r#" + # /// script + # [tool.ty.environment] + # python-version = "3.12" + # /// + "#, + )?; + + let requirement = system_path_to_file(&db, SystemPath::new("/project/requirement.py"))?; + let configured = + system_path_to_file(&db, SystemPath::new("/project/nested/configured.py"))?; + + assert_eq!( + db.program_file(requirement).program(&db), + db.program_file(configured).program(&db) + ); + assert_ne!( + db.python_version_with_source(requirement), + db.python_version_with_source(configured) + ); + + Ok(()) + } } diff --git a/crates/ty_python_core/src/db.rs b/crates/ty_python_core/src/db.rs index 98bd1ae087..93ba90c44f 100644 --- a/crates/ty_python_core/src/db.rs +++ b/crates/ty_python_core/src/db.rs @@ -23,7 +23,7 @@ pub trait TestProgramDb: Db { { #[salsa::tracked(returns(copy), heap_size=ruff_memory_usage::heap_size)] fn program_inner(db: &dyn TestProgramDb) -> Program<'_> { - Program::from_settings(db, db.program_settings().clone()) + Program::from_settings(db, db.program_settings()) } program_inner(self) diff --git a/crates/ty_python_core/src/program.rs b/crates/ty_python_core/src/program.rs index 613bd8d634..f46480e1e6 100644 --- a/crates/ty_python_core/src/program.rs +++ b/crates/ty_python_core/src/program.rs @@ -25,7 +25,7 @@ impl get_size2::GetSize for Program<'_> {} impl<'db> Program<'db> { /// Creates a program from settings whose search roots have already been registered. - pub fn from_settings(db: &'db dyn Db, settings: ProgramSettings) -> Self { + pub fn from_settings(db: &'db dyn Db, settings: &ProgramSettings) -> Self { let ProgramSettings { python_version, python_platform, @@ -33,7 +33,7 @@ impl<'db> Program<'db> { } = settings; let resolver_environment = - ResolverEnvironment::new(db, python_version.version, &search_paths); + ResolverEnvironment::new(db, python_version.version, search_paths); Program::new(db, python_platform, resolver_environment) } diff --git a/crates/ty_python_semantic/resources/mdtest/scripts.md b/crates/ty_python_semantic/resources/mdtest/scripts.md index abffa906eb..64c3ca9173 100644 --- a/crates/ty_python_semantic/resources/mdtest/scripts.md +++ b/crates/ty_python_semantic/resources/mdtest/scripts.md @@ -1,6 +1,6 @@ -Scripts with PEP 723 metadata are considered single-file projects. For now, they can configure -`rules` and `analysis`, but we plan to also support dependencies and changing `environment` -settings. +Scripts with PEP 723 metadata are considered single-file projects. They can configure `rules`, +`analysis`, and their Python environment independently of the enclosing project. Dependencies are +resolved from existing Python environments; ty does not install them. ```toml [environment] @@ -15,10 +15,10 @@ respect-type-ignore-comments = false # Inline settings -A script can change its `rules` and `analysis` settings. In the future, it can also change its -`environment` settings. A script is standalone, it does not inherit any settings from the project -(that's not entirely true today, because scripts still inherit `environment` settings but it's our -end goal). +A script can change its `rules`, `analysis`, and `environment` settings. A script does not inherit +the enclosing project's configuration or Python environment, but can use an activated or explicitly +configured environment. First-party imports require explicitly configured source roots or extra +search paths. ```py # /// script diff --git a/crates/ty_python_semantic/src/diagnostic/mod.rs b/crates/ty_python_semantic/src/diagnostic/mod.rs index f8346f25a0..09a6e5dc2d 100644 --- a/crates/ty_python_semantic/src/diagnostic/mod.rs +++ b/crates/ty_python_semantic/src/diagnostic/mod.rs @@ -27,8 +27,12 @@ pub fn inferred_python_version_source_annotation( source: &PythonVersionSource, ) -> Option { match source { - PythonVersionSource::ConfigFile(source) => source.span(db).map(Annotation::primary), - PythonVersionSource::PyvenvCfgFile(source) => source.span(db).map(Annotation::primary), + PythonVersionSource::ConfigFile(source) | PythonVersionSource::PyvenvCfgFile(source) => { + source.span(db).map(Annotation::primary) + } + PythonVersionSource::ScriptMetadata(span) => { + span.range().map(|_| Annotation::primary(span.clone())) + } PythonVersionSource::InstallationDirectoryLayout { source, .. } => source .as_ref() .and_then(|source| source.span(db)) @@ -72,6 +76,18 @@ pub(crate) fn add_inferred_python_version_hint_to_diagnostic( )); } } + source @ crate::PythonVersionSource::ScriptMetadata(_) => { + let mut sub_diagnostic = SubDiagnostic::new( + SubDiagnosticSeverity::Info, + format_args!( + "Python {version} was assumed when {action} because it was specified in script metadata" + ), + ); + if let Some(annotation) = inferred_python_version_source_annotation(db, source) { + sub_diagnostic.annotate(annotation.message("Python version configured here")); + } + diagnostic.sub(sub_diagnostic); + } source @ crate::PythonVersionSource::PyvenvCfgFile(_) => { if let Some(annotation) = inferred_python_version_source_annotation(db, source) { let mut sub_diagnostic = SubDiagnostic::new( diff --git a/crates/ty_python_semantic/src/lint.rs b/crates/ty_python_semantic/src/lint.rs index 4f2487eebf..215eef0e9d 100644 --- a/crates/ty_python_semantic/src/lint.rs +++ b/crates/ty_python_semantic/src/lint.rs @@ -660,6 +660,9 @@ pub enum LintSource { /// The rule was enabled in a configuration file. File, + /// The rule was enabled in a standalone script's inline metadata. + ScriptMetadata, + /// The rule was enabled from the configuration in the editor. Editor, diff --git a/crates/ty_python_semantic/src/types/class/known.rs b/crates/ty_python_semantic/src/types/class/known.rs index a75792d696..480a792b8c 100644 --- a/crates/ty_python_semantic/src/types/class/known.rs +++ b/crates/ty_python_semantic/src/types/class/known.rs @@ -2148,7 +2148,7 @@ mod tests { python_platform: python_platform.clone(), search_paths: search_paths.clone(), }; - program = Program::from_settings(&db, settings); + program = Program::from_settings(&db, &settings); current_version = version_added; } diff --git a/crates/ty_python_semantic/src/types/context.rs b/crates/ty_python_semantic/src/types/context.rs index c47d4297f6..bf80396a76 100644 --- a/crates/ty_python_semantic/src/types/context.rs +++ b/crates/ty_python_semantic/src/types/context.rs @@ -527,6 +527,9 @@ impl Drop for LintDiagnosticGuard<'_, '_> { LintSource::File => { format!("rule `{rule}` was selected in the configuration file") } + LintSource::ScriptMetadata => { + format!("rule `{rule}` was selected in script metadata") + } LintSource::Editor => { format!("rule `{rule}` was selected in the editor settings") } diff --git a/crates/ty_python_semantic/src/types/infer/tests.rs b/crates/ty_python_semantic/src/types/infer/tests.rs index 72960282d9..223deb1cb4 100644 --- a/crates/ty_python_semantic/src/types/infer/tests.rs +++ b/crates/ty_python_semantic/src/types/infer/tests.rs @@ -122,7 +122,7 @@ fn same_file_at_different_python_versions() -> anyhow::Result<()> { file, Program::from_settings( &db, - ProgramSettings { + &ProgramSettings { python_version: PythonVersionWithSource { version: PythonVersion::PY311, source: PythonVersionSource::Default, @@ -137,7 +137,7 @@ fn same_file_at_different_python_versions() -> anyhow::Result<()> { file, Program::from_settings( &db, - ProgramSettings { + &ProgramSettings { python_version: PythonVersionWithSource { version: PythonVersion::PY312, source: PythonVersionSource::Default, @@ -201,7 +201,7 @@ fn program_file_changes_with_python_version() -> anyhow::Result<()> { let equivalent_program = Program::from_settings( &db, - ProgramSettings { + &ProgramSettings { python_version: db.program_settings().python_version.clone(), python_platform: program.python_platform(&db).clone(), search_paths: program.search_paths(&db).clone(), @@ -215,7 +215,7 @@ fn program_file_changes_with_python_version() -> anyhow::Result<()> { let py312_program = Program::from_settings( &db, - ProgramSettings { + &ProgramSettings { python_version: PythonVersionWithSource { version: PythonVersion::PY312, source: PythonVersionSource::Default, diff --git a/crates/ty_server/tests/e2e/goto_definition.rs b/crates/ty_server/tests/e2e/goto_definition.rs new file mode 100644 index 0000000000..ceee6520eb --- /dev/null +++ b/crates/ty_server/tests/e2e/goto_definition.rs @@ -0,0 +1,58 @@ +use anyhow::Result; +use lsp_types::Position; +use ruff_db::system::SystemPath; + +use crate::TestServerBuilder; + +#[test] +fn script_search_paths_resolve_imported_symbols() -> Result<()> { + let workspace_root = SystemPath::new("src"); + let dependency = SystemPath::new("src/dependencies/dependency.py"); + + let script = SystemPath::new("src/script.py"); + let script_content = r#"# /// script +# [tool.ty.environment] +# extra-paths = ["./dependencies"] +# /// + +from dependency import script_only +"#; + + let ordinary = SystemPath::new("src/ordinary.py"); + let ordinary_content = "from dependency import script_only\n"; + + let mut server = TestServerBuilder::new()? + .with_workspace(workspace_root, None)? + .with_file(dependency, "def script_only() -> None: ...\n")? + .with_file(script, script_content)? + .with_file(ordinary, ordinary_content)? + .build() + .wait_until_workspaces_are_initialized(); + + server.open_text_document(script, script_content, 1); + server.open_text_document(ordinary, ordinary_content, 1); + + let script_definition = server.goto_definition_request(script, Position::new(5, 24)); + insta::assert_json_snapshot!(script_definition, @r#" + [ + { + "uri": "file:///src/dependencies/dependency.py", + "range": { + "start": { + "line": 0, + "character": 4 + }, + "end": { + "line": 0, + "character": 15 + } + } + } + ] + "#); + + let ordinary_definition = server.goto_definition_request(ordinary, Position::new(0, 24)); + insta::assert_json_snapshot!(ordinary_definition, @"null"); + + Ok(()) +} diff --git a/crates/ty_server/tests/e2e/hover.rs b/crates/ty_server/tests/e2e/hover.rs index 65669cf9a3..db91b51246 100644 --- a/crates/ty_server/tests/e2e/hover.rs +++ b/crates/ty_server/tests/e2e/hover.rs @@ -40,6 +40,99 @@ fn supports_only_plain_text() -> Result<()> { Ok(()) } +#[test] +fn shared_import_hover_uses_each_script_python_version() -> Result<()> { + let workspace_root = SystemPath::new("src"); + let shared = SystemPath::new("src/shared.py"); + let older = SystemPath::new("src/older.py"); + let newer = SystemPath::new("src/newer.py"); + let shared_content = "\ +import sys + +if sys.version_info >= (3, 13): + value = 13 +else: + value = 12 +"; + let older_content = r#"# /// script +# requires-python = ">=3.12" +# [tool.ty.environment] +# extra-paths = ["."] +# /// + +from shared import value +value +"#; + let newer_content = r#"# /// script +# requires-python = ">=3.13" +# [tool.ty.environment] +# extra-paths = ["."] +# /// + +from shared import value +value +"#; + + let mut server = TestServerBuilder::new()? + .with_workspace(workspace_root, None)? + .with_file( + "src/pyproject.toml", + r#"[tool.ty.environment] +python-version = "3.12" +"#, + )? + .with_file(shared, shared_content)? + .with_file(older, older_content)? + .with_file(newer, newer_content)? + .build() + .wait_until_workspaces_are_initialized(); + + server.open_text_document(older, older_content, 1); + server.open_text_document(newer, newer_content, 1); + + let older_hover = server.hover_request(older, Position::new(7, 1)); + insta::assert_json_snapshot!(older_hover, @r#" + { + "contents": { + "kind": "plaintext", + "value": "Literal[12]" + }, + "range": { + "start": { + "line": 7, + "character": 0 + }, + "end": { + "line": 7, + "character": 5 + } + } + } + "#); + + let newer_hover = server.hover_request(newer, Position::new(7, 1)); + insta::assert_json_snapshot!(newer_hover, @r#" + { + "contents": { + "kind": "plaintext", + "value": "Literal[13]" + }, + "range": { + "start": { + "line": 7, + "character": 0 + }, + "end": { + "line": 7, + "character": 5 + } + } + } + "#); + + Ok(()) +} + fn hover_content_format(formats: Vec) -> Result { let workspace_root = SystemPath::new("src"); let document_path = SystemPath::new("src/foo.py"); diff --git a/crates/ty_server/tests/e2e/main.rs b/crates/ty_server/tests/e2e/main.rs index 996b3d62da..50b46fddf3 100644 --- a/crates/ty_server/tests/e2e/main.rs +++ b/crates/ty_server/tests/e2e/main.rs @@ -33,6 +33,7 @@ mod commands; mod completions; mod configuration; mod folding_range; +mod goto_definition; mod hover; mod implementation; mod initialize; @@ -59,7 +60,8 @@ use insta::internals::SettingsBindDropGuard; use lsp_server::{Connection, Message, RequestId, Response, ResponseError}; use lsp_types::{ ClientCapabilities, CompletionItem, CompletionParams, CompletionRequest, CompletionResponse, - CompletionTriggerKind, ConfigurationParams, ConfigurationRequest, DiagnosticClientCapabilities, + CompletionTriggerKind, ConfigurationParams, ConfigurationRequest, DefinitionParams, + DefinitionRequest, DefinitionResponse, DiagnosticClientCapabilities, DidChangeTextDocumentNotification, DidChangeTextDocumentParams, DidChangeWatchedFilesClientCapabilities, DidChangeWatchedFilesNotification, DidChangeWatchedFilesParams, DidChangeWorkspaceFoldersNotification, @@ -78,7 +80,7 @@ use lsp_types::{ WorkspaceDiagnosticParams, WorkspaceDiagnosticReport, WorkspaceDiagnosticRequest, WorkspaceEdit, WorkspaceFolder, WorkspaceFoldersChangeEvent, WorkspaceFoldersInitializeParams, }; -use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf, TestSystem}; +use ruff_db::system::{OsSystem, SystemPath, SystemPathBuf, SystemVirtualPath, TestSystem}; use rustc_hash::FxHashMap; use tempfile::TempDir; use ty_server::{ClientOptions, LogLevel, Server, init_logging}; @@ -798,9 +800,25 @@ impl TestServer { content: impl AsRef, version: i32, ) { + self.open_text_document_with_uri(self.file_uri(path), content, version); + } + + /// Send a `textDocument/didOpen` notification for an unsaved virtual document. + pub(crate) fn open_virtual_text_document( + &mut self, + path: impl AsRef, + content: impl AsRef, + version: i32, + ) -> Result<()> { + let uri = Uri::parse(path.as_ref().as_str())?; + self.open_text_document_with_uri(uri, content, version); + Ok(()) + } + + fn open_text_document_with_uri(&mut self, uri: Uri, content: impl AsRef, version: i32) { let params = DidOpenTextDocumentParams { text_document: TextDocumentItem { - uri: self.file_uri(path), + uri, language_id: LanguageKind::Python, version, text: content.as_ref().to_string(), @@ -955,6 +973,26 @@ impl TestServer { self.await_response::(&id) } + /// Send a `textDocument/definition` request for the document at the given path and position. + pub(crate) fn goto_definition_request( + &mut self, + path: impl AsRef, + position: Position, + ) -> Option { + let params = DefinitionParams { + text_document_position_params: TextDocumentPositionParams { + text_document: TextDocumentIdentifier { + uri: self.file_uri(path), + }, + position, + }, + work_done_progress_params: WorkDoneProgressParams::default(), + partial_result_params: PartialResultParams::default(), + }; + let id = self.send_request::(params); + self.await_response::(&id) + } + /// Send a `textDocument/hover` request for the document at the given path and position. pub(crate) fn hover_request( &mut self, diff --git a/crates/ty_server/tests/e2e/publish_diagnostics.rs b/crates/ty_server/tests/e2e/publish_diagnostics.rs index 9b0ac72ffa..f53e4fa1be 100644 --- a/crates/ty_server/tests/e2e/publish_diagnostics.rs +++ b/crates/ty_server/tests/e2e/publish_diagnostics.rs @@ -8,7 +8,7 @@ use lsp_types::{ TextDocumentContentChangePartial, TextDocumentContentChangeWholeDocument, TextDocumentItem, Uri, }; -use ruff_db::system::SystemPath; +use ruff_db::system::{SystemPath, SystemVirtualPath}; use ty_server::ClientOptions; use crate::notebook::NotebookBuilder; @@ -142,7 +142,7 @@ def foo() -> str: #[test] fn on_did_open_non_existing_file_workspace_with_untitled_uri() -> Result<()> { let workspace_root = SystemPath::new("src"); - let foo = SystemPath::new("src/foo.py"); + let foo = SystemVirtualPath::new("untitled:foo.py"); let foo_content = "\ def foo() -> str: return 42 @@ -159,17 +159,7 @@ def foo() -> str: .build() .wait_until_workspaces_are_initialized(); - server.send_notification::(DidOpenTextDocumentParams { - text_document: TextDocumentItem { - uri: { - let uri = server.file_uri(foo); - Uri::parse(&format!("untitled://{}", uri.path())).unwrap() - }, - language_id: LanguageKind::Python, - version: 1, - text: foo_content.to_string(), - }, - }); + server.open_virtual_text_document(foo, foo_content, 1)?; let diagnostics = server.await_notification::(); insta::assert_debug_snapshot!(diagnostics); @@ -245,6 +235,137 @@ def foo() -> str: Ok(()) } +#[test] +fn on_did_change_script_python_requirement() -> Result<()> { + let workspace_root = SystemPath::new("src"); + let script = SystemPath::new("src/script.py"); + let initial = r#"# /// script +# requires-python = ">=3.12" +# /// + +PythonFinalizationError +"#; + let updated = r#"# /// script +# requires-python = ">=3.13" +# /// + +PythonFinalizationError +"#; + + let mut server = TestServerBuilder::new()? + .with_workspace(workspace_root, None)? + .with_file(script, initial)? + .enable_pull_diagnostics(false) + .build() + .wait_until_workspaces_are_initialized(); + + server.open_text_document(script, initial, 1); + let initial_diagnostics = server.await_notification::(); + insta::assert_debug_snapshot!(initial_diagnostics); + + server.change_text_document( + script, + vec![ + lsp_types::TextDocumentContentChangeEvent::TextDocumentContentChangeWholeDocument( + TextDocumentContentChangeWholeDocument { + text: updated.to_string(), + }, + ), + ], + 2, + ); + + let updated_diagnostics = server.await_notification::(); + insta::assert_debug_snapshot!(updated_diagnostics, @r#" + PublishDiagnosticsParams { + uri: Url { + scheme: "file", + cannot_be_a_base: false, + username: "", + password: None, + host: None, + port: None, + path: "/src/script.py", + query: None, + fragment: None, + }, + version: Some( + 2, + ), + diagnostics: [], + } + "#); + + Ok(()) +} + +#[test] +fn on_did_open_virtual_script_uses_its_python_requirement() -> Result<()> { + let workspace_root = SystemPath::new("src"); + let script = SystemVirtualPath::new("untitled:script.py"); + let content = r#"# /// script +# requires-python = ">=3.13" +# /// + +PythonFinalizationError +"#; + + let mut server = TestServerBuilder::new()? + .with_workspace(workspace_root, None)? + .enable_pull_diagnostics(false) + .build() + .wait_until_workspaces_are_initialized(); + + server.open_virtual_text_document(script, content, 1)?; + + let diagnostics = server.await_notification::(); + insta::assert_debug_snapshot!(diagnostics, @r#" + PublishDiagnosticsParams { + uri: Url { + scheme: "untitled", + cannot_be_a_base: true, + username: "", + password: None, + host: None, + port: None, + path: "script.py", + query: None, + fragment: None, + }, + version: Some( + 1, + ), + diagnostics: [], + } + "#); + + Ok(()) +} + +#[test] +fn on_did_open_virtual_script_reports_inline_configuration_diagnostics() -> Result<()> { + let workspace_root = SystemPath::new("src"); + let script = SystemVirtualPath::new("untitled:script.py"); + let content = r#"# /// script +# [tool.ty.rules] +# unknown-rule = "warn" +# /// +"#; + + let mut server = TestServerBuilder::new()? + .with_workspace(workspace_root, None)? + .enable_pull_diagnostics(false) + .build() + .wait_until_workspaces_are_initialized(); + + server.open_virtual_text_document(script, content, 1)?; + + let diagnostics = server.await_notification::(); + insta::assert_debug_snapshot!(diagnostics); + + Ok(()) +} + #[test] fn on_did_save_publishes_open_file_documents() -> Result<()> { let workspace_root = SystemPath::new("src"); diff --git a/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_change_script_python_requirement.snap b/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_change_script_python_requirement.snap new file mode 100644 index 0000000000..11db5553f1 --- /dev/null +++ b/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_change_script_python_requirement.snap @@ -0,0 +1,72 @@ +--- +source: crates/ty_server/tests/e2e/publish_diagnostics.rs +expression: initial_diagnostics +--- +PublishDiagnosticsParams { + uri: Url { + scheme: "file", + cannot_be_a_base: false, + username: "", + password: None, + host: None, + port: None, + path: "/src/script.py", + query: None, + fragment: None, + }, + version: Some( + 1, + ), + diagnostics: [ + Diagnostic { + range: Range { + start: Position { + line: 4, + character: 0, + }, + end: Position { + line: 4, + character: 23, + }, + }, + severity: Some( + Error, + ), + code: Some( + String( + "unresolved-reference", + ), + ), + code_description: Some( + CodeDescription { + href: Url { + scheme: "https", + cannot_be_a_base: false, + username: "", + password: None, + host: Some( + Domain( + "ty.dev", + ), + ), + port: None, + path: "/rules", + query: None, + fragment: Some( + "unresolved-reference", + ), + }, + }, + ), + source: Some( + "ty", + ), + message: String( + "Name `PythonFinalizationError` used when not defined\n\ninfo: `PythonFinalizationError` was added as a builtin in Python 3.13\ninfo: Python 3.12 was assumed when resolving types because it was specified in script metadata", + ), + tags: None, + related_information: None, + data: None, + }, + ], +} diff --git a/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_open_non_existing_file_workspace_with_untitled_uri.snap b/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_open_non_existing_file_workspace_with_untitled_uri.snap index 54183854d2..355bde4d9f 100644 --- a/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_open_non_existing_file_workspace_with_untitled_uri.snap +++ b/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_open_non_existing_file_workspace_with_untitled_uri.snap @@ -5,12 +5,12 @@ expression: diagnostics PublishDiagnosticsParams { uri: Url { scheme: "untitled", - cannot_be_a_base: false, + cannot_be_a_base: true, username: "", password: None, host: None, port: None, - path: "/src/foo.py", + path: "foo.py", query: None, fragment: None, }, diff --git a/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_open_virtual_script_reports_inline_configuration_diagnostics.snap b/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_open_virtual_script_reports_inline_configuration_diagnostics.snap new file mode 100644 index 0000000000..3c46b8f4c6 --- /dev/null +++ b/crates/ty_server/tests/e2e/snapshots/e2e__publish_diagnostics__on_did_open_virtual_script_reports_inline_configuration_diagnostics.snap @@ -0,0 +1,52 @@ +--- +source: crates/ty_server/tests/e2e/publish_diagnostics.rs +expression: diagnostics +--- +PublishDiagnosticsParams { + uri: Url { + scheme: "untitled", + cannot_be_a_base: true, + username: "", + password: None, + host: None, + port: None, + path: "script.py", + query: None, + fragment: None, + }, + version: Some( + 1, + ), + diagnostics: [ + Diagnostic { + range: Range { + start: Position { + line: 0, + character: 0, + }, + end: Position { + line: 0, + character: 0, + }, + }, + severity: Some( + Warning, + ), + code: Some( + String( + "unknown-rule", + ), + ), + code_description: None, + source: Some( + "ty", + ), + message: String( + "Unknown rule `unknown-rule`. Did you mean `unknown-argument`?", + ), + tags: None, + related_information: None, + data: None, + }, + ], +} diff --git a/crates/ty_site_packages/src/lib.rs b/crates/ty_site_packages/src/lib.rs index 7649164acf..a60b6f6cbd 100644 --- a/crates/ty_site_packages/src/lib.rs +++ b/crates/ty_site_packages/src/lib.rs @@ -279,10 +279,10 @@ impl PythonEnvironment { /// /// 1. activated virtual environment /// 2. conda (child) - /// 3. working dir virtual environment + /// 3. project virtual environment, when a project root is provided /// 4. conda (base) pub fn discover( - project_root: &SystemPath, + project_root: Option<&SystemPath>, system: &dyn System, ) -> Result, SitePackagesDiscoveryError> { fn resolve_environment( @@ -308,22 +308,24 @@ impl PythonEnvironment { .map(Some); } - tracing::debug!("Discovering virtual environment in `{project_root}`"); - let virtual_env_directory = project_root.join(".venv"); + if let Some(project_root) = project_root { + tracing::debug!("Discovering virtual environment in `{project_root}`"); + let virtual_env_directory = project_root.join(".venv"); - match PythonEnvironment::new( - &virtual_env_directory, - SysPrefixPathOrigin::LocalVenv, - system, - ) { - Ok(environment) => return Ok(Some(environment)), - Err(err) => { - if system.is_directory(&virtual_env_directory) { - tracing::debug!( - "Ignoring automatically detected virtual environment at `{}`: {}", - &virtual_env_directory, - err - ); + match PythonEnvironment::new( + &virtual_env_directory, + SysPrefixPathOrigin::LocalVenv, + system, + ) { + Ok(environment) => return Ok(Some(environment)), + Err(err) => { + if system.is_directory(&virtual_env_directory) { + tracing::debug!( + "Ignoring automatically detected virtual environment at `{}`: {}", + &virtual_env_directory, + err + ); + } } } } @@ -2119,6 +2121,8 @@ impl Deref for SysPrefixPath { pub enum SysPrefixPathOrigin { /// The `sys.prefix` path came from a configuration file setting: `pyproject.toml` or `ty.toml` ConfigFileSetting(Arc, Option), + /// The `sys.prefix` path came from a standalone script's inline metadata. + ScriptMetadataSetting, /// The `sys.prefix` path came from a `--python` CLI flag PythonCliFlag, /// The selected interpreter in the user's editor. @@ -2149,6 +2153,7 @@ impl SysPrefixPathOrigin { match self { Self::LocalVenv | Self::VirtualEnvVar => true, Self::ConfigFileSetting(..) + | Self::ScriptMetadataSetting | Self::PythonCliFlag | Self::Editor | Self::DerivedFromPyvenvCfg @@ -2167,6 +2172,7 @@ impl SysPrefixPathOrigin { match self { Self::PythonCliFlag | Self::ConfigFileSetting(..) + | Self::ScriptMetadataSetting | Self::Editor | Self::SelfEnvironment | Self::PythonBinary => false, @@ -2188,6 +2194,7 @@ impl SysPrefixPathOrigin { | Self::Editor | Self::DerivedFromPyvenvCfg | Self::ConfigFileSetting(..) + | Self::ScriptMetadataSetting | Self::PythonCliFlag | Self::PythonBinary | Self::UvWorkspace => false, @@ -2201,6 +2208,9 @@ impl std::fmt::Display for SysPrefixPathOrigin { match self { Self::PythonCliFlag => f.write_str("`--python` argument"), Self::ConfigFileSetting(_, _) => f.write_str("`environment.python` setting"), + Self::ScriptMetadataSetting => { + f.write_str("`environment.python` setting in script metadata") + } Self::VirtualEnvVar => f.write_str("`VIRTUAL_ENV` environment variable"), Self::CondaPrefixVar => f.write_str("`CONDA_PREFIX` environment variable"), Self::DerivedFromPyvenvCfg => f.write_str("derived `sys.prefix` path"), diff --git a/crates/ty_site_packages/src/version.rs b/crates/ty_site_packages/src/version.rs index 6832650350..87aae364e2 100644 --- a/crates/ty_site_packages/src/version.rs +++ b/crates/ty_site_packages/src/version.rs @@ -15,6 +15,9 @@ pub enum PythonVersionSource { /// Value loaded from a project's configuration file. ConfigFile(PythonVersionFileSource), + /// Value configured in a standalone script's inline metadata. + ScriptMetadata(Span), + /// Value loaded from the `pyvenv.cfg` file of the virtual environment. /// The virtual environment might have been configured, activated or inferred. PyvenvCfgFile(PythonVersionFileSource), diff --git a/ty.schema.json b/ty.schema.json index dc45c5b18a..4958a43be5 100644 --- a/ty.schema.json +++ b/ty.schema.json @@ -136,7 +136,7 @@ } }, "python": { - "description": "Path to your project's Python environment or interpreter.\n\nty uses the `site-packages` directory of your project's Python environment\nto resolve third-party (and, in some cases, first-party) imports in your code.\n\nThis can be a path to:\n\n- A Python interpreter, e.g. `.venv/bin/python3`\n- A virtual environment directory, e.g. `.venv`\n- A system Python [`sys.prefix`] directory, e.g. `/usr`\n\nIf you're using a project management tool such as uv, you should not generally need to\nspecify this option, as commands such as `uv run` will set the `VIRTUAL_ENV` environment\nvariable to point to your project's virtual environment. ty can also infer the location of\nyour environment from an activated Conda environment, and will look for a `.venv` directory\nin the project root if none of the above apply. Failing that, ty will look for a `python3`\nor `python` binary available in `PATH`.\n\n[`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix", + "description": "Path to your project's Python environment or interpreter.\n\nty uses the `site-packages` directory of your project's Python environment\nto resolve third-party (and, in some cases, first-party) imports in your code.\n\nThis can be a path to:\n\n- A Python interpreter, e.g. `.venv/bin/python3`\n- A virtual environment directory, e.g. `.venv`\n- A system Python [`sys.prefix`] directory, e.g. `/usr`\n\nIf you're using a project management tool such as uv, you should not generally need to\nspecify this option, as commands such as `uv run` will set the `VIRTUAL_ENV` environment\nvariable to point to your project's virtual environment. ty can also infer the location of\nyour environment from an activated Conda environment, and will look for a `.venv` directory\nin the project root if none of the above apply. Failing that, ty will look for a `python3`\nor `python` binary available in `PATH`.\n\nScripts with inline metadata use their own Python environment. They can use an explicitly\nconfigured environment, an activated environment, or an environment selected by the editor.\nUnlike projects, they do not automatically use a `.venv` directory.\n\n[`sys.prefix`]: https://docs.python.org/3/library/sys.html#sys.prefix", "anyOf": [ { "$ref": "#/definitions/RelativePathBuf" @@ -158,7 +158,7 @@ ] }, "python-version": { - "description": "Specifies the version of Python that will be used to analyze the source code.\nThe version should be specified as a string in the format `M.m` where `M` is the major version\nand `m` is the minor (e.g. `\"3.7\"` or `\"3.12\"`).\nIf a version is provided, ty will generate errors if the source code makes use of language features\nthat are not supported in that version.\n\nty officially supports type checking code that targets Python 3.10 and later. Python 3.7\nthrough 3.9 can still be selected, but ty may produce false positives or false negatives for\nstandard-library APIs because its bundled stubs do not fully describe those versions.\n\nIf a version is not specified, ty will try the following techniques in order of preference\nto determine a value:\n1. Check for the `project.requires-python` setting in a `pyproject.toml` file\n and use the minimum version from the specified range\n2. Check for an activated or configured Python environment\n and attempt to infer the Python version of that environment\n3. Fall back to the default value (see below)\n\nFor some language features, ty can also understand conditionals based on comparisons\nwith `sys.version_info`. These are commonly found in typeshed, for example,\nto reflect the differing contents of the standard library across Python versions.", + "description": "Specifies the version of Python that will be used to analyze the source code.\nThe version should be specified as a string in the format `M.m` where `M` is the major version\nand `m` is the minor (e.g. `\"3.7\"` or `\"3.12\"`).\nIf a version is provided, ty will generate errors if the source code makes use of language features\nthat are not supported in that version.\n\nty officially supports type checking code that targets Python 3.10 and later. Python 3.7\nthrough 3.9 can still be selected, but ty may produce false positives or false negatives for\nstandard-library APIs because its bundled stubs do not fully describe those versions.\n\nIf a version is not specified, ty will try the following techniques in order of preference\nto determine a value:\n1. Check for the `project.requires-python` setting in a `pyproject.toml` file\n and use the minimum version from the specified range\n2. Check for an activated or configured Python environment\n and attempt to infer the Python version of that environment\n3. Fall back to the default value (see below)\n\nScripts with inline metadata use their `requires-python` field instead of\n`project.requires-python`. They do not inherit the Python version of the enclosing project.\n\nFor some language features, ty can also understand conditionals based on comparisons\nwith `sys.version_info`. These are commonly found in typeshed, for example,\nto reflect the differing contents of the standard library across Python versions.", "anyOf": [ { "$ref": "#/definitions/SupportedPythonVersion" @@ -169,7 +169,7 @@ ] }, "root": { - "description": "The root paths of the project, used for finding first-party modules.\n\nAccepts a list of directory paths searched in priority order (first has highest priority).\n\nIf left unspecified, ty will try to detect common project layouts and initialize `root` accordingly.\nThe project root (`.`) is always included. Additionally, the following directories are included\nif they exist and are not packages (i.e. they do not contain `__init__.py` or `__init__.pyi` files):\n\n* `./src`\n* `./` (if a `.//` directory exists)\n* `./python`", + "description": "The root paths of the project, used for finding first-party modules.\n\nAccepts a list of directory paths searched in priority order (first has highest priority).\n\nIf left unspecified, ty will try to detect common project layouts and initialize `root` accordingly.\nThe project root (`.`) is always included. Additionally, the following directories are included\nif they exist and are not packages (i.e. they do not contain `__init__.py` or `__init__.pyi` files):\n\n* `./src`\n* `./` (if a `.//` directory exists)\n* `./python`\n\nScripts with inline metadata have no first-party roots by default because they are\nsingle-file programs. Set `root = [\".\"]` to allow importing local modules.", "type": [ "array", "null" @@ -325,7 +325,7 @@ ] }, "RelativePathBuf": { - "description": "A possibly relative path in a configuration file.\n\nRelative paths in configuration files or from CLI options\nrequire different anchoring:\n\n* CLI: The path is relative to the current working directory\n* Configuration file: The path is relative to the project's root.", + "description": "A possibly relative path in a configuration file.\n\nRelative paths in configuration files or from CLI options\nrequire different anchoring:\n\n* CLI: The path is relative to the current working directory\n* Configuration file: The path is relative to the project's or script's configuration root.", "allOf": [ { "$ref": "#/definitions/SystemPathBuf" From 7a9aed24ffa150677657f9dde0eb252a8377c09d Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Fri, 14 Aug 2026 14:01:30 +0200 Subject: [PATCH 034/371] Guarantee minimum stack size when parsing a module, standalone expression, and suites (#25464) --- Cargo.lock | 58 ++++- Cargo.toml | 1 + crates/ruff_python_parser/Cargo.toml | 1 + crates/ruff_python_parser/src/error.rs | 4 - crates/ruff_python_parser/src/lexer.rs | 6 - .../src/parser/expression.rs | 127 ++--------- crates/ruff_python_parser/src/parser/mod.rs | 58 +++-- .../ruff_python_parser/src/parser/options.rs | 32 --- .../ruff_python_parser/src/parser/pattern.rs | 72 +++---- .../src/parser/statement.rs | 26 +-- crates/ruff_python_parser/src/parser/tests.rs | 203 +++++++----------- crates/ruff_python_parser/src/string.rs | 30 +-- crates/ruff_python_parser/src/token_source.rs | 6 - 13 files changed, 206 insertions(+), 418 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index c4d507d8c8..65d26aed2e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -168,6 +168,15 @@ dependencies = [ "num-traits", ] +[[package]] +name = "ar_archive_writer" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73cd58deff2140a0a8eae87e417bd01db68a33e148aa93d1e8cd837e55e312b6" +dependencies = [ + "object", +] + [[package]] name = "arc-swap" version = "1.9.2" @@ -566,7 +575,7 @@ dependencies = [ "terminfo", "thiserror 2.0.19", "which", - "windows-sys 0.59.0", + "windows-sys 0.61.0", ] [[package]] @@ -685,7 +694,7 @@ version = "3.1.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "faf9468729b8cbcea668e36183cb69d317348c2e08e994829fb56ebfdfbaac34" dependencies = [ - "windows-sys 0.52.0", + "windows-sys 0.61.0", ] [[package]] @@ -1035,7 +1044,7 @@ dependencies = [ "libc", "option-ext", "redox_users", - "windows-sys 0.59.0", + "windows-sys 0.61.0", ] [[package]] @@ -1115,7 +1124,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" dependencies = [ "libc", - "windows-sys 0.52.0", + "windows-sys 0.61.0", ] [[package]] @@ -2300,6 +2309,15 @@ version = "4.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "ef25abbcd74fb2609453eb695bd2f860d389e457f67dc17cafc8b8cbc89d0c33" +[[package]] +name = "object" +version = "0.39.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2e5a6c098c7a3b6547378093f5cc30bc54fd361ce711e05293a5cc589562739b" +dependencies = [ + "memchr", +] + [[package]] name = "once_cell" version = "1.21.4" @@ -2732,6 +2750,16 @@ dependencies = [ "unicode-ident", ] +[[package]] +name = "psm" +version = "0.1.32" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4dcd034599e63b970727f70d79e02d62390a4a84f7c6b827c27c46d5ac3fa622" +dependencies = [ + "ar_archive_writer", + "cc", +] + [[package]] name = "ptr_meta" version = "0.3.1" @@ -3603,6 +3631,7 @@ dependencies = [ "rustc-hash", "serde", "serde_json", + "stacker", "static_assertions", "thin-vec", "unicode-ident", @@ -3839,7 +3868,7 @@ dependencies = [ "errno", "libc", "linux-raw-sys", - "windows-sys 0.52.0", + "windows-sys 0.61.0", ] [[package]] @@ -4142,6 +4171,19 @@ version = "1.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "a8f112729512f8e442d81f95a8a7ddf2b7c6b8a1a6f509a95864142b30cab2d3" +[[package]] +name = "stacker" +version = "0.1.25" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "707f49d46706bacf8a2b00d51dace3f9de527c13eec3778f570c411f89e69967" +dependencies = [ + "cc", + "cfg-if", + "libc", + "psm", + "windows-sys 0.61.0", +] + [[package]] name = "static_assertions" version = "1.1.0" @@ -4249,7 +4291,7 @@ dependencies = [ "getrandom 0.4.2", "once_cell", "rustix", - "windows-sys 0.52.0", + "windows-sys 0.61.0", ] [[package]] @@ -4259,7 +4301,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "230a1b821ccbd75b185820a1f1ff7b14d21da1e442e22c0863ea5f08771a8874" dependencies = [ "rustix", - "windows-sys 0.59.0", + "windows-sys 0.61.0", ] [[package]] @@ -5431,7 +5473,7 @@ version = "0.1.11" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" dependencies = [ - "windows-sys 0.52.0", + "windows-sys 0.61.0", ] [[package]] diff --git a/Cargo.toml b/Cargo.toml index b11a50e23c..930cee4ce2 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -181,6 +181,7 @@ snapbox = { version = "1.0.0", features = [ "examples", ] } static_assertions = "1.1.0" +stacker = "0.1.24" strum = { version = "0.28.0", features = ["strum_macros"] } strum_macros = { version = "0.28.0" } supports-hyperlinks = { version = "3.1.0" } diff --git a/crates/ruff_python_parser/Cargo.toml b/crates/ruff_python_parser/Cargo.toml index df30c68c67..9802c67f64 100644 --- a/crates/ruff_python_parser/Cargo.toml +++ b/crates/ruff_python_parser/Cargo.toml @@ -24,6 +24,7 @@ get-size2 = { workspace = true } memchr = { workspace = true } rustc-hash = { workspace = true } static_assertions = { workspace = true } +stacker = { workspace = true } thin-vec = { workspace = true } unicode-ident = { workspace = true } unicode-normalization = { workspace = true } diff --git a/crates/ruff_python_parser/src/error.rs b/crates/ruff_python_parser/src/error.rs index 0b2b41e8c4..13e4b5a628 100644 --- a/crates/ruff_python_parser/src/error.rs +++ b/crates/ruff_python_parser/src/error.rs @@ -205,9 +205,6 @@ pub enum ParseErrorType { TStringError(InterpolatedStringErrorType), /// Parser encountered an error during lexing. Lexical(LexicalErrorType), - - /// Parser aborted because [`crate::ParseOptions::max_recursion_depth`] was exceeded. - RecursionLimitExceeded, } impl ParseErrorType { @@ -339,7 +336,6 @@ impl std::fmt::Display for ParseErrorType { ParseErrorType::UnexpectedExpressionToken => { write!(f, "Unexpected token at the end of an expression") } - ParseErrorType::RecursionLimitExceeded => f.write_str("Source is too deeply nested"), } } } diff --git a/crates/ruff_python_parser/src/lexer.rs b/crates/ruff_python_parser/src/lexer.rs index 5d0124a7fa..cdf3452ae4 100644 --- a/crates/ruff_python_parser/src/lexer.rs +++ b/crates/ruff_python_parser/src/lexer.rs @@ -123,12 +123,6 @@ impl<'src> Lexer<'src> { self.current_range } - /// Returns the current parenthesis, bracket, and brace nesting level. - #[inline] - pub(crate) const fn nesting(&self) -> u32 { - self.nesting - } - /// Returns the flags for the current token. pub(crate) const fn current_flags(&self) -> TokenFlags { self.current_flags diff --git a/crates/ruff_python_parser/src/parser/expression.rs b/crates/ruff_python_parser/src/parser/expression.rs index 7b7adc5e54..b07d7aa884 100644 --- a/crates/ruff_python_parser/src/parser/expression.rs +++ b/crates/ruff_python_parser/src/parser/expression.rs @@ -248,9 +248,11 @@ impl<'src> Parser<'src> { left_precedence: OperatorPrecedence, context: ExpressionContext, ) -> ParsedExpr { - let start = self.node_start(); - let lhs = self.parse_lhs_expression(left_precedence, context); - self.parse_binary_expression_or_higher_recursive(lhs, left_precedence, context, start) + self.with_recursion(|parser| { + let start = parser.node_start(); + let lhs = parser.parse_lhs_expression(left_precedence, context); + parser.parse_binary_expression_or_higher_recursive(lhs, left_precedence, context, start) + }) } fn parse_binary_expression_or_higher_recursive( @@ -303,22 +305,7 @@ impl<'src> Parser<'src> { BinaryLikeOperator::Binary(bin_op) => { self.bump(TokenKind::from(bin_op)); - let right = if new_precedence.is_right_associative() { - // For right-associative operators (`**`), the right - // operand recursion is unbounded in `a**a**a**...`, - // and it bypasses the guard in `parse_lhs_expression` - // (that scope is exited once the atom is parsed). - if let Some(right) = self.with_recursion(|parser| { - parser.parse_binary_expression_or_higher(new_precedence, context) - }) { - right - } else { - self.report_recursion_limit_exceeded(self.current_token_range()); - self.recursion_recovery_expr() - } - } else { - self.parse_binary_expression_or_higher(new_precedence, context) - }; + let right = self.parse_binary_expression_or_higher(new_precedence, context); Expr::BinOp(ast::ExprBinOp { left: Box::new(left.expr), @@ -349,59 +336,6 @@ impl<'src> Parser<'src> { context: ExpressionContext, ) -> ParsedExpr { let token = self.current_token_kind(); - if !Self::token_starts_recursive_lhs(token) { - return self.parse_lhs_expression_inner(left_precedence, context, token); - } - - if let Some(result) = self.with_recursion(|parser| { - parser.parse_lhs_expression_inner(left_precedence, context, token) - }) { - result - } else { - self.report_recursion_limit_exceeded(self.current_token_range()); - self.recursion_recovery_expr() - } - } - - /// Returns whether parsing an expression that starts with `token` can - /// immediately recurse through another expression parse. - #[inline] - fn token_starts_recursive_lhs(token: TokenKind) -> bool { - token.as_unary_operator().is_some() - || matches!( - token, - TokenKind::Star - | TokenKind::Await - | TokenKind::Lambda - | TokenKind::Yield - | TokenKind::FStringStart - | TokenKind::TStringStart - | TokenKind::Lpar - | TokenKind::Lsqb - | TokenKind::Lbrace - ) - } - - /// The standard expression-recovery node returned when the recursion - /// limit is exceeded: an empty `Name` with the `Invalid` context. - fn recursion_recovery_expr(&mut self) -> ParsedExpr { - ParsedExpr { - expr: Expr::Name(ast::ExprName { - range: self.missing_node_range(), - id: Name::empty(), - ctx: ExprContext::Invalid, - node_index: AtomicNodeIndex::NONE, - }), - is_parenthesized: false, - } - } - - fn parse_lhs_expression_inner( - &mut self, - left_precedence: OperatorPrecedence, - context: ExpressionContext, - token: TokenKind, - ) -> ParsedExpr { let start = self.node_start(); if let Some(unary_op) = token.as_unary_operator() { @@ -754,20 +688,8 @@ impl<'src> Parser<'src> { ) -> Expr { loop { lhs = match self.current_token_kind() { - TokenKind::Lpar => { - if self.tokens.nesting() > self.max_nesting_depth { - self.report_recursion_limit_exceeded(self.current_token_range()); - break lhs; - } - Expr::Call(self.parse_call_expression(lhs, start)) - } - TokenKind::Lsqb => { - if self.tokens.nesting() > self.max_nesting_depth { - self.report_recursion_limit_exceeded(self.current_token_range()); - break lhs; - } - Expr::Subscript(self.parse_subscript_expression(lhs, start)) - } + TokenKind::Lpar => Expr::Call(self.parse_call_expression(lhs, start)), + TokenKind::Lsqb => Expr::Subscript(self.parse_subscript_expression(lhs, start)), TokenKind::Dot => { Expr::Attribute(self.parse_attribute_expression(lhs, start, context)) } @@ -1899,18 +1821,13 @@ impl<'src> Parser<'src> { let format_spec = if self.eat(TokenKind::Colon) { let spec_start = self.node_start(); - let elements = if let Some(elements) = self.with_recursion(|parser| { + let elements = self.with_recursion(|parser| { parser.parse_interpolated_string_elements( flags, InterpolatedStringElementsKind::FormatSpec(string_kind), string_kind, ) - }) { - elements - } else { - self.report_recursion_limit_exceeded(self.current_token_range()); - ast::InterpolatedStringElements::from(vec![]) - }; + }); Some(Box::new(ast::InterpolatedStringFormatSpec { range: self.node_range(spec_start), elements, @@ -2989,15 +2906,8 @@ impl<'src> Parser<'src> { // lambda x: yield y // lambda x: yield from y - // `lambda: lambda: lambda: ...` recurses through the lambda body at - // the conditional layer, bypassing the `parse_lhs_expression` guard. - let body = - if let Some(body) = self.with_recursion(Self::parse_conditional_expression_or_higher) { - body - } else { - self.report_recursion_limit_exceeded(self.current_token_range()); - self.recursion_recovery_expr() - }; + // Lambda bodies recurse through the conditional layer without entering the binary parser. + let body = self.with_recursion(Self::parse_conditional_expression_or_higher); ast::ExprLambda { body: Box::new(body.expr), @@ -3021,17 +2931,8 @@ impl<'src> Parser<'src> { self.expect(TokenKind::Else); - // `a if b else a if b else ...` recurses through `orelse` at the - // conditional layer, which is not covered by the `parse_lhs_expression` - // guard (that scope is released once each atom is parsed). Guard here. - let orelse = if let Some(orelse) = - self.with_recursion(Self::parse_conditional_expression_or_higher) - { - orelse - } else { - self.report_recursion_limit_exceeded(self.current_token_range()); - self.recursion_recovery_expr() - }; + // The binary-expression guard has already returned before parsing the `else` branch. + let orelse = self.with_recursion(Self::parse_conditional_expression_or_higher); ast::ExprIf { body: Box::new(body), diff --git a/crates/ruff_python_parser/src/parser/mod.rs b/crates/ruff_python_parser/src/parser/mod.rs index 31a8d29414..7f59aa1ab8 100644 --- a/crates/ruff_python_parser/src/parser/mod.rs +++ b/crates/ruff_python_parser/src/parser/mod.rs @@ -60,6 +60,12 @@ impl NameInterner { } } +// Stack probes access thread-local state, so avoid them while recursive parser calls remain +// shallow. `STACK_RED_ZONE` must cover the stack used before the first deferred probe. +const STACK_RED_ZONE: usize = 100 * 1024; +const STACK_SIZE: usize = 1024 * 1024; +const MAX_UNCHECKED_RECURSION_DEPTH: usize = 20; + #[derive(Debug)] pub(crate) struct Parser<'src> { source: &'src str, @@ -95,11 +101,8 @@ pub(crate) struct Parser<'src> { /// The start offset in the source code from which to start parsing at. start_offset: TextSize, - /// Current parser recursion depth remaining before the depth limit is exceeded. - depth_remaining: u16, - - /// Maximum lexer nesting depth before postfix calls and subscripts should stop recursing. - max_nesting_depth: u32, + /// Number of active recursive statement, expression, and pattern parsing operations. + recursion_depth: usize, /// Reusable, nesting-safe scratch storage for expression lists. expr_scratch: ScratchBuffer, @@ -133,8 +136,6 @@ impl<'src> Parser<'src> { options: ParseOptions, ) -> Self { let tokens = TokenSource::from_source(source, options.mode, start_offset); - let depth_remaining = options.max_recursion_depth; - let max_nesting_depth = u32::from(options.max_recursion_depth.saturating_sub(2)); Parser { options, @@ -147,9 +148,8 @@ impl<'src> Parser<'src> { recovery_context: RecoveryContext::empty(), prev_token_end: TextSize::new(0), start_offset, + recursion_depth: 0, current_token_id: TokenId::default(), - depth_remaining, - max_nesting_depth, expr_scratch: ScratchBuffer::with_capacity(16), keyword_scratch: ScratchBuffer::new(), parameter_scratch: ScratchBuffer::new(), @@ -159,44 +159,34 @@ impl<'src> Parser<'src> { } } - /// Runs `f` if the recursive parser depth limit has not been hit. - /// - /// # Note - /// - /// This recursion guard is a temporary fix for #22930. - #[must_use] + /// Grows the stack for recursive parser calls only after shallow nesting is exceeded. #[inline] - fn with_recursion(&mut self, f: impl FnOnce(&mut Self) -> T) -> Option { - if self.depth_remaining == 0 { - return None; - } + fn with_recursion(&mut self, f: impl FnOnce(&mut Self) -> T) -> T { + self.recursion_depth += 1; + + let result = if self.recursion_depth > MAX_UNCHECKED_RECURSION_DEPTH { + self.grow_stack(f) + } else { + f(self) + }; - self.depth_remaining -= 1; - let result = f(self); - self.depth_remaining += 1; - Some(result) + self.recursion_depth -= 1; + result } #[cold] - #[inline(never)] - fn report_recursion_limit_exceeded(&mut self, ranged: R) { - self.add_error(ParseErrorType::RecursionLimitExceeded, ranged); - // Skip to end-of-file so outer parser frames unwind quickly and our - // `ParserProgress` infinite-loop guards don't fire when they see the - // same `(` / `[` etc. that this frame failed to consume. - while self.current_token_kind() != TokenKind::EndOfFile { - self.bump_any(); - } + fn grow_stack(&mut self, f: impl FnOnce(&mut Self) -> T) -> T { + stacker::maybe_grow(STACK_RED_ZONE, STACK_SIZE, || f(self)) } /// Consumes the [`Parser`] and returns the parsed [`Parsed`]. pub(crate) fn parse(mut self) -> Parsed { - let syntax = match self.options.mode { + let syntax = stacker::maybe_grow(STACK_RED_ZONE, STACK_SIZE, || match self.options.mode { Mode::Expression | Mode::ParenthesizedExpression => { Mod::Expression(self.parse_single_expression()) } Mode::Module | Mode::Ipython => Mod::Module(self.parse_module()), - }; + }); self.finish(syntax) } diff --git a/crates/ruff_python_parser/src/parser/options.rs b/crates/ruff_python_parser/src/parser/options.rs index 9d3d1ce74e..ec87a72d1e 100644 --- a/crates/ruff_python_parser/src/parser/options.rs +++ b/crates/ruff_python_parser/src/parser/options.rs @@ -2,20 +2,6 @@ use ruff_python_ast::{PySourceType, PythonVersion}; use crate::{AsMode, Mode}; -/// The default maximum recursion depth used by the parser. -/// -/// Real-world Python rarely nests more than a handful of levels deep; this cap -/// exists to keep the parser from overflowing the stack on adversarial or -/// machine-generated input. -/// -/// The default value mirrors CPython's `MAXSTACK` of 200 nested parentheses -/// (`Parser/parser.c`): a one-statement module of the form `((((1))))` at -/// depth 200 must parse, and one at depth 201 must fail. Each nesting level -/// costs one `with_recursion` call, plus two framing calls (one for the -/// surrounding statement and one for the innermost atom), so the cap is set -/// to `200 + 2`. -const DEFAULT_MAX_RECURSION_DEPTH: u16 = 202; - /// Options for controlling how a source file is parsed. /// /// You can construct a [`ParseOptions`] directly from a [`Mode`]: @@ -40,11 +26,6 @@ pub struct ParseOptions { pub(crate) mode: Mode, /// Target version for detecting version-related syntax errors. pub(crate) target_version: PythonVersion, - /// Maximum recursion depth for the parser. The parser aborts with a - /// [`crate::ParseErrorType::RecursionLimitExceeded`] error once this many - /// nested expression / statement / pattern nodes are on the parser's call - /// stack. Defaults to [`DEFAULT_MAX_RECURSION_DEPTH`]. - pub(crate) max_recursion_depth: u16, } impl ParseOptions { @@ -57,17 +38,6 @@ impl ParseOptions { pub fn target_version(&self) -> PythonVersion { self.target_version } - - /// Set the maximum recursion depth for the parser. - #[must_use] - pub fn with_max_recursion_depth(mut self, depth: u16) -> Self { - self.max_recursion_depth = depth; - self - } - - pub fn max_recursion_depth(&self) -> u16 { - self.max_recursion_depth - } } impl From for ParseOptions { @@ -75,7 +45,6 @@ impl From for ParseOptions { Self { mode, target_version: PythonVersion::default(), - max_recursion_depth: DEFAULT_MAX_RECURSION_DEPTH, } } } @@ -85,7 +54,6 @@ impl From for ParseOptions { Self { mode: source_type.as_mode(), target_version: PythonVersion::default(), - max_recursion_depth: DEFAULT_MAX_RECURSION_DEPTH, } } } diff --git a/crates/ruff_python_parser/src/parser/pattern.rs b/crates/ruff_python_parser/src/parser/pattern.rs index f6a82d38d9..bde2fd478a 100644 --- a/crates/ruff_python_parser/src/parser/pattern.rs +++ b/crates/ruff_python_parser/src/parser/pattern.rs @@ -88,28 +88,6 @@ impl Parser<'_> { /// /// See: fn parse_match_pattern(&mut self, allow_star_pattern: AllowStarPattern) -> Pattern { - if let Some(result) = - self.with_recursion(|parser| parser.parse_match_pattern_inner(allow_star_pattern)) - { - result - } else { - let range = self.missing_node_range(); - self.report_recursion_limit_exceeded(self.current_token_range()); - let invalid_node = Expr::Name(ast::ExprName { - range, - id: Name::empty(), - ctx: ExprContext::Invalid, - node_index: AtomicNodeIndex::NONE, - }); - Pattern::MatchValue(ast::PatternMatchValue { - range: invalid_node.range(), - value: Box::new(invalid_node), - node_index: AtomicNodeIndex::NONE, - }) - } - } - - fn parse_match_pattern_inner(&mut self, allow_star_pattern: AllowStarPattern) -> Pattern { let start = self.node_start(); // We don't yet know if it's an or pattern or an as pattern, so use whatever @@ -162,33 +140,37 @@ impl Parser<'_> { /// /// See: fn parse_match_pattern_lhs(&mut self, allow_star_pattern: AllowStarPattern) -> Pattern { - let start = self.node_start(); - - let mut lhs = match self.current_token_kind() { - TokenKind::Lbrace => Pattern::MatchMapping(self.parse_match_pattern_mapping()), - TokenKind::Star => { - let star_pattern = self.parse_match_pattern_star(); - if allow_star_pattern.is_no() { - self.add_error(ParseErrorType::InvalidStarPatternUsage, &star_pattern); + self.with_recursion(|parser| { + let start = parser.node_start(); + + let mut lhs = match parser.current_token_kind() { + TokenKind::Lbrace => Pattern::MatchMapping(parser.parse_match_pattern_mapping()), + TokenKind::Star => { + let star_pattern = parser.parse_match_pattern_star(); + if allow_star_pattern.is_no() { + parser.add_error(ParseErrorType::InvalidStarPatternUsage, &star_pattern); + } + Pattern::MatchStar(star_pattern) } - Pattern::MatchStar(star_pattern) - } - TokenKind::Lpar | TokenKind::Lsqb => self.parse_parenthesized_or_sequence_pattern(), - _ => self.parse_match_pattern_literal(), - }; + TokenKind::Lpar | TokenKind::Lsqb => { + parser.parse_parenthesized_or_sequence_pattern() + } + _ => parser.parse_match_pattern_literal(), + }; - if self.at(TokenKind::Lpar) { - lhs = Pattern::MatchClass(self.parse_match_pattern_class(lhs, start)); - } + if parser.at(TokenKind::Lpar) { + lhs = Pattern::MatchClass(parser.parse_match_pattern_class(lhs, start)); + } - if matches!( - self.current_token_kind(), - TokenKind::Plus | TokenKind::Minus - ) { - lhs = Pattern::MatchValue(self.parse_complex_literal_pattern(lhs, start)); - } + if matches!( + parser.current_token_kind(), + TokenKind::Plus | TokenKind::Minus + ) { + lhs = Pattern::MatchValue(parser.parse_complex_literal_pattern(lhs, start)); + } - lhs + lhs + }) } /// Parses a mapping pattern. diff --git a/crates/ruff_python_parser/src/parser/statement.rs b/crates/ruff_python_parser/src/parser/statement.rs index a13865afc3..8fd22dec69 100644 --- a/crates/ruff_python_parser/src/parser/statement.rs +++ b/crates/ruff_python_parser/src/parser/statement.rs @@ -2883,22 +2883,7 @@ impl<'src> Parser<'src> { // Although this statement is not a valid `async` statement, // we still parse it. Guard the recursive recovery path so // `async async async ...` cannot overflow the parser stack. - if let Some(stmt) = self.with_recursion(Self::parse_statement) { - stmt - } else { - let range = self.node_range(async_start); - self.add_error(ParseErrorType::RecursionLimitExceeded, range); - Stmt::Expr(ast::StmtExpr { - range, - value: Box::new(Expr::Name(ast::ExprName { - range, - id: Name::new_static("async"), - ctx: ExprContext::Invalid, - node_index: AtomicNodeIndex::NONE, - })), - node_index: AtomicNodeIndex::NONE, - }) - } + self.with_recursion(Self::parse_statement) } } } @@ -3139,7 +3124,7 @@ impl<'src> Parser<'src> { fn parse_block(&mut self) -> Suite { self.bump(TokenKind::Indent); - let statements = if let Some(statements) = self.with_recursion(|parser| { + let statements = self.with_recursion(|parser| { let snapshot = parser.stmt_scratch.snapshot(); parser.parse_list(RecoveryContextKind::BlockStatements, |parser| { let statement = parser.parse_statement(); @@ -3147,12 +3132,7 @@ impl<'src> Parser<'src> { }); parser.stmt_scratch.take_thin_vec(snapshot) - }) { - statements - } else { - self.report_recursion_limit_exceeded(self.current_token_range()); - Suite::new() - }; + }); self.expect(TokenKind::Dedent); diff --git a/crates/ruff_python_parser/src/parser/tests.rs b/crates/ruff_python_parser/src/parser/tests.rs index 2eafcda9b1..86b8cc95bf 100644 --- a/crates/ruff_python_parser/src/parser/tests.rs +++ b/crates/ruff_python_parser/src/parser/tests.rs @@ -1,6 +1,9 @@ use ruff_python_ast::{Expr, InterpolatedStringElement, IpyEscapeKind, Number, Stmt}; -use crate::{Mode, ParseErrorType, ParseOptions, parse, parse_expression, parse_module}; +use crate::{Mode, ParseOptions, parse, parse_expression, parse_module}; + +// Keep recursive ASTs shallow enough for Windows's 1 MiB test-thread stacks. +const RECURSIVE_AST_TEST_DEPTH: usize = 1_000; #[test] fn test_modes() { @@ -338,51 +341,39 @@ fn test_tstring_fstring_middle_fuzzer() { insta::assert_debug_snapshot!(error); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_nested_parens() { +fn nested_parens_grow_stack() { let src = format!("{}1{}", "(".repeat(1_000), ")".repeat(1_000)); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); + let parsed = stacker::grow(32 * 1024, || parse_module(&src)); + assert!(parsed.is_ok()); } #[test] -fn recursion_limit_normal_python_unaffected() { - // 50 levels is well above what real-world Python ever produces and well - // below the default cap — the point is to confirm the default doesn't - // reject ordinary input. +fn normal_python_unaffected() { let src = format!("x = {}1{}", "(".repeat(50), ")".repeat(50)); parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_preserves_prior_statements() { - // Recursion-limit recovery is limited for now: we drain the rest of the file but keep the - // statements parsed before the overflowing statement. - // TODO: Recover at the next newline so the trailing statement is preserved too. +fn deep_nesting_preserves_surrounding_statements() { let src = format!( "before = 1\n{}1{}\nafter = 2\n", "(".repeat(1_000), ")".repeat(1_000), ); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let parsed = crate::parse_unchecked(&src, opts) - .try_into_module() - .unwrap(); - - assert!(matches!( - parsed.errors().first().map(|error| &error.error), - Some(ParseErrorType::RecursionLimitExceeded) - )); + let parsed = parse_module(&src).unwrap(); + assert!(matches!(parsed.suite().first(), Some(Stmt::Assign(_)))); + assert!(matches!(parsed.suite().last(), Some(Stmt::Assign(_)))); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_nested_def_blocks() { - // Nested function definitions exercise instrumentation on - // `parse_statement` rather than `parse_lhs_expression`. Each level - // needs one more leading tab to make indentation valid. - let depth = 400; +fn nested_def_blocks_grow_stack() { + // Each nested function crosses the suite boundary where the parser rechecks the stack. + let depth = RECURSIVE_AST_TEST_DEPTH; let mut src = String::new(); for i in 0..depth { src.push_str(&"\t".repeat(i)); @@ -390,37 +381,33 @@ fn recursion_limit_nested_def_blocks() { } src.push_str(&"\t".repeat(depth)); src.push_str("pass\n"); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); + parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_nested_lists() { +fn nested_lists_grow_stack() { let src = format!("{}1{}", "[".repeat(1_000), "]".repeat(1_000)); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); + parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_nested_calls() { +fn nested_calls_grow_stack() { let src = format!("x = {}1{}", "f(".repeat(1_000), ")".repeat(1_000)); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); + parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_nested_subscripts() { +fn nested_subscripts_grow_stack() { let src = format!("x = {}1{}", "a[".repeat(1_000), "]".repeat(1_000)); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); + parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_nested_match_patterns() { +fn nested_match_patterns_grow_stack() { // Deeply parenthesised match patterns — exercises pattern-parsing // instrumentation in addition to statement / expression paths. let mut src = String::from("match x:\n case "); @@ -432,17 +419,29 @@ fn recursion_limit_nested_match_patterns() { src.push(')'); } src.push_str(": pass\n"); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); + parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_binary_paren_interplay() { +fn nested_invalid_mapping_pattern_keys_grow_stack() { + let depth = 512; + let src = format!( + "match value:\n case {}0{}:\n pass\n", + "{".repeat(depth), + ": 0}".repeat(depth) + ); + let parsed = crate::parse_unchecked(&src, ParseOptions::from(Mode::Module)); + assert!(!parsed.errors().is_empty()); +} + +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] +#[test] +fn binary_paren_interplay_grows_stack() { // `1+(1+(1+(1+...)))` — each level alternates a binary operator and a // parenthesised sub-expression, exactly like the pattern described in // the tracking issue. - let depth = 2_000; + let depth = RECURSIVE_AST_TEST_DEPTH; let mut src = String::new(); for _ in 0..depth { src.push_str("1+("); @@ -451,112 +450,72 @@ fn recursion_limit_binary_paren_interplay() { for _ in 0..depth { src.push(')'); } - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); -} - -#[test] -fn recursion_limit_first_error_is_recursion_not_noise() { - // When the limit is hit the outer parser frames will emit secondary - // errors as they unwind. Callers read the first error via `into_result` - // / `Parsed::errors()`, so `RecursionLimitExceeded` must come first, and - // the drain-to-EOF after reporting the recursion limit should keep the total count - // small rather than producing one noisy error per unwound frame. - let src = format!("{}1{}", "(".repeat(2_000), ")".repeat(2_000)); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(50); - let parsed = crate::parse_unchecked(&src, opts); - let errors = parsed.errors(); - let first = errors.first().expect("expected at least one error"); - assert!(matches!( - first.error, - ParseErrorType::RecursionLimitExceeded - )); - // Exactly one `RecursionLimitExceeded` — guards against a regression - // where the unwind loops and re-triggers the limit check. - let recursion_errors = errors - .iter() - .filter(|e| matches!(e.error, ParseErrorType::RecursionLimitExceeded)) - .count(); - assert_eq!(recursion_errors, 1); - // Small, bounded tail of follow-up errors from the unwinding frames. - // Today this is 0; the generous cap is a regression gate, not a spec. - assert!( - errors.len() <= 8, - "expected a small number of errors, got {}: {errors:?}", - errors.len(), - ); -} - -#[test] -fn recursion_limit_default_set() { - let opts = ParseOptions::from(Mode::Module); - // Guards against someone accidentally unsetting the default. Real-world - // Python never approaches this depth, and the value must stay within the - // threading stack's capacity — see the const's docs in `options.rs`. - assert!(opts.max_recursion_depth() >= 200); - assert!(opts.max_recursion_depth() <= 2000); + parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_right_assoc_pow_chain() { +fn right_assoc_pow_chain_grows_stack() { // `1**1**1**...**1` — `**` is right-associative, so the right operand // is parsed by a recursive `parse_binary_expression_or_higher` call // *without* any intervening parentheses or atom nesting. This exercises // the binary-expression recursion path directly, unlike the // `1+(1+(...))` interplay test which recurses through parenthesised // atoms. - let depth = 2_000; + let depth = RECURSIVE_AST_TEST_DEPTH; let mut src = String::with_capacity(depth * 3 + 1); for _ in 0..depth { src.push_str("1**"); } src.push('1'); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!( - matches!(err.error, ParseErrorType::RecursionLimitExceeded), - "expected RecursionLimitExceeded, got {:?}", - err.error - ); + parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_ternary_else_chain() { +fn ternary_else_chain_grows_stack() { // `1 if 1 else 1 if 1 else ...` — the `else` operand recurses at the // conditional layer (`parse_if_expression` -> `orelse`), which is not - // covered by the `parse_lhs_expression` guard. - let depth = 2_000; + // covered by the binary-expression guard. + let depth = RECURSIVE_AST_TEST_DEPTH; let mut src = String::with_capacity(depth * 12 + 1); for _ in 0..depth { src.push_str("1 if 1 else "); } src.push('1'); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!( - matches!(err.error, ParseErrorType::RecursionLimitExceeded), - "expected RecursionLimitExceeded, got {:?}", - err.error - ); + parse_module(&src).unwrap(); } +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] #[test] -fn recursion_limit_nested_lambda_chain() { +fn nested_lambda_chain_grows_stack() { // `lambda: lambda: lambda: ...` — the lambda body recurses at the // conditional layer (`parse_lambda_expr` -> body), bypassing the - // `parse_lhs_expression` guard entirely. - let depth = 2_000; + // binary-expression guard entirely. + let depth = RECURSIVE_AST_TEST_DEPTH; let mut src = String::from("x = "); for _ in 0..depth { src.push_str("lambda: "); } src.push('1'); - let opts = ParseOptions::from(Mode::Module).with_max_recursion_depth(100); - let err = parse(&src, opts).unwrap_err(); - assert!( - matches!(err.error, ParseErrorType::RecursionLimitExceeded), - "expected RecursionLimitExceeded, got {:?}", - err.error - ); + parse_module(&src).unwrap(); +} + +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] +#[test] +fn invalid_async_chain_grows_stack() { + let source = format!("{}x = 1\n", "async ".repeat(5_000)); + let parsed = crate::parse_unchecked(&source, ParseOptions::from(Mode::Module)); + assert!(!parsed.errors().is_empty()); +} + +#[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] +#[test] +fn nested_unary_chains_grow_stack() { + let depth = 300; + let source = format!("{}1\n", "-~+".repeat(depth)); + parse_module(&source).unwrap(); + + let source = format!("{}True\n", "not ".repeat(depth)); + parse_module(&source).unwrap(); } diff --git a/crates/ruff_python_parser/src/string.rs b/crates/ruff_python_parser/src/string.rs index 8855c0e9a0..bd7e9961ab 100644 --- a/crates/ruff_python_parser/src/string.rs +++ b/crates/ruff_python_parser/src/string.rs @@ -532,10 +532,7 @@ mod tests { use ruff_python_ast::Suite; use crate::error::LexicalErrorType; - use crate::{ - InterpolatedStringErrorType, Mode, ParseError, ParseErrorType, ParseOptions, Parsed, parse, - parse_module, - }; + use crate::{InterpolatedStringErrorType, ParseError, ParseErrorType, Parsed, parse_module}; const WINDOWS_EOL: &str = "\r\n"; const MAC_EOL: &str = "\r"; @@ -545,17 +542,6 @@ mod tests { parse_module(source).map(Parsed::into_suite) } - fn parse_suite_with_recursion_limit( - source: &str, - max_recursion_depth: u16, - ) -> Result { - parse( - source, - ParseOptions::from(Mode::Module).with_max_recursion_depth(max_recursion_depth), - ) - .map(|parsed| parsed.try_into_module().unwrap().into_suite()) - } - fn nested_format_spec(prefix: char, depth: usize) -> String { let mut replacement_field = String::from("{spec}"); for _ in 0..depth { @@ -602,11 +588,8 @@ mod tests { } #[test] - fn test_parse_fstring_nested_spec_recursion_limit() { - assert!(parse_suite_with_recursion_limit(r#"f"{foo:{spec}}""#, 8).is_ok()); - - let err = parse_suite_with_recursion_limit(&nested_format_spec('f', 200), 8).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); + fn parse_fstring_nested_spec_grows_stack() { + assert!(parse_suite(&nested_format_spec('f', 200)).is_ok()); } #[test] @@ -722,11 +705,8 @@ mod tests { } #[test] - fn test_parse_tstring_nested_spec_recursion_limit() { - assert!(parse_suite_with_recursion_limit(r#"t"{foo:{spec}}""#, 8).is_ok()); - - let err = parse_suite_with_recursion_limit(&nested_format_spec('t', 200), 8).unwrap_err(); - assert!(matches!(err.error, ParseErrorType::RecursionLimitExceeded)); + fn parse_tstring_nested_spec_grows_stack() { + assert!(parse_suite(&nested_format_spec('t', 200)).is_ok()); } #[test] diff --git a/crates/ruff_python_parser/src/token_source.rs b/crates/ruff_python_parser/src/token_source.rs index ea1e482484..98d8a03965 100644 --- a/crates/ruff_python_parser/src/token_source.rs +++ b/crates/ruff_python_parser/src/token_source.rs @@ -47,12 +47,6 @@ impl<'src> TokenSource<'src> { self.lexer.current_range() } - /// Returns the current parenthesis, bracket, and brace nesting level. - #[inline] - pub(crate) const fn nesting(&self) -> u32 { - self.lexer.nesting() - } - /// Returns the flags for the current token. pub(crate) const fn current_flags(&self) -> TokenFlags { self.lexer.current_flags() From 4fa13b3a3c1561567a3c534b5303655da5bf3234 Mon Sep 17 00:00:00 2001 From: Dhruv Manilawala Date: Fri, 14 Aug 2026 18:21:15 +0530 Subject: [PATCH 035/371] [ty] Support TypeVarTuple in call binding (#26886) ## Summary Infer `TypeVarTuple` from arguments passed to `*args`, including unpacked tuples and iterables, fixed prefixes and suffixes, and nested type variable tuples. This PR contains only the core call-binding change. Callback forwarding is handled separately in #27371 to keep this PR focused and avoid adding more work to the old constraint solver. Moving `TypeVarTuple` and `ParamSpec` to the new solver is separate follow-up work. Continues #25240. ## Test plan - Pack inference for direct arguments, unpacked arguments, fixed boundaries, nested packs, and legacy `Unpack`. - Invalid starred arguments, bounded type variables, and diagnostic locations. --- .../resources/mdtest/annotations/starred.md | 20 +- .../mdtest/generics/legacy/typevartuple.md | 7 +- .../mdtest/generics/legacy/unpack.md | 17 +- .../mdtest/generics/pep695/typevartuple.md | 407 ++++++++++++++++-- .../ty_python_semantic/src/types/call/bind.rs | 353 ++++++++++++++- 5 files changed, 748 insertions(+), 56 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/annotations/starred.md b/crates/ty_python_semantic/resources/mdtest/annotations/starred.md index 2d5e4dd108..6bd1b1290d 100644 --- a/crates/ty_python_semantic/resources/mdtest/annotations/starred.md +++ b/crates/ty_python_semantic/resources/mdtest/annotations/starred.md @@ -5,7 +5,7 @@ python-version = "3.11" ``` -Type annotations for `*args` can be starred expressions themselves: +An unpacked type variable tuple keeps the types of positional arguments passed to `*args`. ```py from typing_extensions import TypeVarTuple @@ -17,14 +17,20 @@ def append_int(*args: *Ts) -> tuple[*Ts, int]: return (*args, 1) -# TODO should be tuple[Literal[True], Literal["a"], int] -reveal_type(append_int(True, "a")) # revealed: tuple[*tuple[Unknown, ...], int] -# TODO should be tuple[int] -reveal_type(append_int()) # revealed: tuple[*tuple[Unknown, ...], int] +reveal_type(append_int(True, "a")) # revealed: tuple[Literal[True], Literal["a"], int] +reveal_type(append_int()) # revealed: tuple[int] +``` + +A concrete starred tuple checks its fixed first argument, remaining argument types, and arity. +```py def first_arg_int(*args: *tuple[int, *tuple[str, ...]]): ... first_arg_int(42, "42", "42") # fine -first_arg_int("not an int", "42", "42") # error: [invalid-argument-type] -first_arg_int(56, "42", 56) # error: [invalid-argument-type] +# error: [invalid-argument-type] "Argument to function `first_arg_int` is incorrect: Expected `int`" +first_arg_int("not an int", "42", "42") +# error: [invalid-argument-type] "Argument to function `first_arg_int` is incorrect: Expected `str`, found `Literal[56]`" +first_arg_int(56, "42", 56) +# error: [missing-argument] "No argument provided for required parameter `*args` of function `first_arg_int`" +first_arg_int() ``` diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/typevartuple.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/typevartuple.md index 9755910ddd..55cf73ace8 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/typevartuple.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/typevartuple.md @@ -414,13 +414,12 @@ class Variadic(Generic[*Ts]): reveal_type(Positional(())) # revealed: Positional[()] reveal_type(Positional((1, "a"))) # revealed: Positional[int, str] -# TODO: Infer the `TypeVarTuple` from arguments matched to the variadic parameter. -reveal_type(Variadic()) # revealed: Variadic[*tuple[Unknown, ...]] -reveal_type(Variadic(1, "a")) # revealed: Variadic[*tuple[Unknown, ...]] +reveal_type(Variadic()) # revealed: Variadic[()] +reveal_type(Variadic(1, "a")) # revealed: Variadic[int, str] def _(i: int, s: str) -> None: reveal_type(Positional((i, s))) # revealed: Positional[int, str] - reveal_type(Variadic(i, s)) # revealed: Variadic[*tuple[Unknown, ...]] + reveal_type(Variadic(i, s)) # revealed: Variadic[int, str] ``` ### Unspecified type arguments diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/unpack.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/unpack.md index 435ca08fe7..64b26e03fc 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/unpack.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/unpack.md @@ -40,9 +40,20 @@ def collect(*args: Unpack[Ts]) -> tuple[Unpack[Ts]]: reveal_type(args) # revealed: tuple[*Ts@collect] raise NotImplementedError -# TODO: Infer the `TypeVarTuple` from arguments matched to the variadic parameter. -reveal_type(collect()) # revealed: tuple[Unknown, ...] -reveal_type(collect(1, "a")) # revealed: tuple[Unknown, ...] +reveal_type(collect()) # revealed: tuple[()] +reveal_type(collect(1, "a")) # revealed: tuple[Literal[1], Literal["a"]] +``` + +The legacy spelling must also preserve argument-derived types when a surrounding assignment expects +an incompatible return type. + +```py +inferred = collect(1) +reveal_type(inferred) # revealed: tuple[Literal[1]] +# error: [invalid-assignment] +indirect: tuple[str] = inferred +# error: [invalid-assignment] +direct: tuple[str] = collect(1) ``` ## Callable parameters diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md index f414064d7d..857a44a5fe 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/typevartuple.md @@ -151,13 +151,26 @@ class Variadic[*Ts]: reveal_type(Positional(())) # revealed: Positional[()] reveal_type(Positional((1, "a"))) # revealed: Positional[int, str] -# TODO: Infer the `TypeVarTuple` from arguments matched to the variadic parameter. -reveal_type(Variadic()) # revealed: Variadic[*tuple[Unknown, ...]] -reveal_type(Variadic(1, "a")) # revealed: Variadic[*tuple[Unknown, ...]] +reveal_type(Variadic()) # revealed: Variadic[()] +reveal_type(Variadic(1, "a")) # revealed: Variadic[int, str] def _(i: int, s: str) -> None: reveal_type(Positional((i, s))) # revealed: Positional[int, str] - reveal_type(Variadic(i, s)) # revealed: Variadic[*tuple[Unknown, ...]] + reveal_type(Variadic(i, s)) # revealed: Variadic[int, str] +``` + +Constructor arguments determine the class specialization even when the assignment expects a +different specialization. + +```py +valid: Variadic[int] = Variadic(1) + +inferred = Variadic(1) +reveal_type(inferred) # revealed: Variadic[int] +# error: [invalid-assignment] +indirect: Variadic[str] = inferred +# error: [invalid-assignment] +direct: Variadic[str] = Variadic(1) ``` ### Unspecified type arguments @@ -211,10 +224,31 @@ class Array[*Ts]: ### Multiple type variable tuples Generic functions can declare multiple type variable tuples because their type parameters are -inferred from arguments; functions cannot be explicitly specialized. +inferred from arguments; functions cannot be explicitly specialized. Separate tuple arguments infer +their type variable tuples independently. + +```py +def pair[*Ts, *Us]( + first: tuple[*Ts], + second: tuple[*Us], +) -> tuple[tuple[*Ts], tuple[*Us]]: + return first, second + +def check_pair(first: int, second: str, third: bool, fourth: bytes) -> None: + reveal_type(pair((first, second), (third, fourth))) # revealed: tuple[tuple[int, str], tuple[bool, bytes]] +``` + +A variadic parameter can also infer one type variable tuple from a fixed nested tuple and another +from its remaining arguments. ```py -def pair[*Ts1, *Ts2](first: tuple[*Ts1], second: tuple[*Ts2]) -> None: ... +def nested[*Ts, *Us]( + *args: *tuple[tuple[*Us], *Ts], +) -> tuple[tuple[*Us], tuple[*Ts]]: + raise NotImplementedError + +def check_nested(first: int, second: str, third: bool, fourth: bytes) -> None: + reveal_type(nested((first, second), third, fourth)) # revealed: tuple[tuple[int, str], tuple[bool, bytes]] ``` ### Tuple arguments and returns @@ -317,8 +351,9 @@ def materialized_default[*Ts = *tuple[Any, ...]]() -> None: ### Starred variadic parameters -An unpacked `TypeVarTuple` can annotate `*args`. Inferring the `TypeVarTuple` from arguments matched -to the variadic parameter is not yet supported, so these calls use a gradual specialization. +An unpacked `TypeVarTuple` can annotate `*args`. Call binding infers the pack from direct arguments +and from the residual tuple shape of splatted arguments, while generic function bodies retain the +symbolic pack declared by the function. ```py def simple[*Ts](*args: *Ts) -> tuple[*Ts]: @@ -328,33 +363,230 @@ def simple[*Ts](*args: *Ts) -> tuple[*Ts]: def with_prefix[T, *Ts](prefix: T, *args: *Ts) -> tuple[T, *Ts]: raise NotImplementedError +def bounded[*Ts](head: int, *rest: *tuple[*Ts, str]) -> tuple[*Ts]: + raise NotImplementedError + def with_kw_only[T, *Ts](*args: *Ts, kw: T) -> tuple[*Ts, T]: raise NotImplementedError -def f(i: int, s: str, b: bool, t: tuple[int, str], vt: tuple[int, ...]) -> None: - reveal_type(simple()) # revealed: tuple[Unknown, ...] - reveal_type(simple(i, s)) # revealed: tuple[Unknown, ...] - reveal_type(simple(*(i, s))) # revealed: tuple[Unknown, ...] - reveal_type(simple(t)) # revealed: tuple[Unknown, ...] - reveal_type(simple(*t)) # revealed: tuple[Unknown, ...] - reveal_type(simple(*vt)) # revealed: tuple[Unknown, ...] - - reveal_type(with_prefix(i)) # revealed: tuple[int, *tuple[Unknown, ...]] - reveal_type(with_prefix(i, s, b)) # revealed: tuple[int, *tuple[Unknown, ...]] - reveal_type(with_prefix(*t)) # revealed: tuple[int, *tuple[Unknown, ...]] - reveal_type(with_prefix(i, *t)) # revealed: tuple[int, *tuple[Unknown, ...]] - reveal_type(with_prefix(*vt)) # revealed: tuple[int, *tuple[Unknown, ...]] - reveal_type(with_prefix(i, *vt)) # revealed: tuple[int, *tuple[Unknown, ...]] - - reveal_type(with_kw_only(kw=b)) # revealed: tuple[*tuple[Unknown, ...], bool] - reveal_type(with_kw_only(i, s, kw=b)) # revealed: tuple[*tuple[Unknown, ...], bool] - reveal_type(with_kw_only(t, kw=b)) # revealed: tuple[*tuple[Unknown, ...], bool] - reveal_type(with_kw_only(*t, kw=b)) # revealed: tuple[*tuple[Unknown, ...], bool] - reveal_type(with_kw_only(vt, kw=b)) # revealed: tuple[*tuple[Unknown, ...], bool] - reveal_type(with_kw_only(*vt, kw=b)) # revealed: tuple[*tuple[Unknown, ...], bool] +def forward[*Us](*args: *Us) -> tuple[*Us]: + reveal_type(simple(*args)) # revealed: tuple[*Us@forward] + return simple(*args) + +def f( + i: int, + s: str, + b: bool, + empty: tuple[()], + one: tuple[int], + fixed: tuple[int, str], + suffix: tuple[bool, str], + unbounded: tuple[int, ...], + mixed: tuple[int, *tuple[str, ...], bytes], + xs: list[int], +) -> None: + reveal_type(simple()) # revealed: tuple[()] + reveal_type(simple(i)) # revealed: tuple[int] + reveal_type(simple(i, s)) # revealed: tuple[int, str] + reveal_type(simple(*(i, s))) # revealed: tuple[int, str] + reveal_type(simple(i, s, b)) # revealed: tuple[int, str, bool] + reveal_type(simple(fixed)) # revealed: tuple[tuple[int, str]] + reveal_type(simple(*empty)) # revealed: tuple[()] + reveal_type(simple(*one)) # revealed: tuple[int] + reveal_type(simple(*fixed)) # revealed: tuple[int, str] + reveal_type(simple(*unbounded)) # revealed: tuple[int, ...] + reveal_type(simple(*mixed)) # revealed: tuple[int, *tuple[str, ...], bytes] + reveal_type(simple(*xs)) # revealed: tuple[int, ...] + + reveal_type(with_prefix(i)) # revealed: tuple[int] + reveal_type(with_prefix(i, s, b)) # revealed: tuple[int, str, bool] + reveal_type(with_prefix(*fixed)) # revealed: tuple[int, str] + reveal_type(with_prefix(i, *fixed)) # revealed: tuple[int, int, str] + reveal_type(with_prefix(*unbounded)) # revealed: tuple[int, *tuple[int, ...]] + reveal_type(with_prefix(i, *unbounded)) # revealed: tuple[int, *tuple[int, ...]] + reveal_type(with_prefix(*xs)) # revealed: tuple[int, *tuple[int, ...]] + + reveal_type(bounded(i, *suffix)) # revealed: tuple[bool] + + reveal_type(with_kw_only(kw=b)) # revealed: tuple[bool] + reveal_type(with_kw_only(i, s, kw=b)) # revealed: tuple[int, str, bool] + reveal_type(with_kw_only(fixed, kw=b)) # revealed: tuple[tuple[int, str], bool] + reveal_type(with_kw_only(*fixed, kw=b)) # revealed: tuple[int, str, bool] + reveal_type(with_kw_only(unbounded, kw=b)) # revealed: tuple[tuple[int, ...], bool] + reveal_type(with_kw_only(*unbounded, kw=b)) # revealed: tuple[*tuple[int, ...], bool] + reveal_type(with_kw_only(*xs, kw=b)) # revealed: tuple[*tuple[int, ...], bool] # error: [missing-argument] "No argument provided for required parameter `kw` of function `with_kw_only`" - reveal_type(with_kw_only(i, s, b)) # revealed: tuple[*tuple[Unknown, ...], Unknown] + reveal_type(with_kw_only(i, s, b)) # revealed: tuple[int, str, bool, Unknown] +``` + +Variadic inference preserves contextual argument types, including an outer type variable. + +```py +from typing import TypedDict + +class Payload(TypedDict): + value: int + +def contextual[T](value: T) -> None: + concrete: tuple[Payload, list[int]] = simple({"value": 1}, []) + generic: tuple[Payload, T] = simple({"value": 1}, value) + # error: [invalid-assignment] + # error: [invalid-argument-type] + invalid: tuple[Payload] = simple({"value": "wrong"}) +``` + +Fixed values next to a type variable tuple keep their normal bound diagnostics. + +```py +def bounded_arguments[U: bytes, T: str, *Ts](first: U, *args: *tuple[*Ts, T]) -> tuple[*Ts, T]: + raise NotImplementedError + +bounded_arguments( + 1, # error: [invalid-argument-type] "upper bound `bytes`" + "ok", + 2, # error: [invalid-argument-type] "upper bound `str`" +) + +def check_splat_error(values: list[int]) -> None: + bounded_arguments( + b"valid", + *values, # snapshot: invalid-argument-type + ) +``` + +```snapshot +error[invalid-argument-type]: Argument to function `bounded_arguments` is incorrect + --> src/mdtest_snippet.py:86:9 + | +86 | *values, # snapshot: invalid-argument-type + | ^^^^^^^ Argument type `int` does not satisfy upper bound `str` of type variable `T` +info: Type variable defined here + --> src/mdtest_snippet.py:74:33 + | +74 | def bounded_arguments[U: bytes, T: str, *Ts](first: U, *args: *tuple[*Ts, T]) -> tuple[*Ts, T]: + | ^^^^^^ +``` + +### Union splatted arguments + +Equal-length tuple unions preserve their length and combine the types at each position. Different +lengths produce an open tuple, while direct arguments around the splat keep their known positions. + +```py +def collect[*Ts](*args: *Ts) -> tuple[*Ts]: + return args + +def check( + same_length: tuple[int] | tuple[str], + paired: tuple[int, str] | tuple[bytes, bool], + different_lengths: tuple[int] | tuple[str, bytes], + prefix: bool, + suffix: bytes, +) -> None: + reveal_type(collect(*same_length)) # revealed: tuple[int | str] + reveal_type(collect(*paired)) # revealed: tuple[int | bytes, str | bool] + reveal_type(collect(*different_lengths)) # revealed: tuple[int | str | bytes, ...] + reveal_type(collect(prefix, *same_length, suffix)) # revealed: tuple[bool, int | str, bytes] + + # error: [invalid-assignment] + wrong: tuple[bytes] = collect(*same_length) +``` + +### Starred variadic arguments without a variadic return + +A bounded or constrained element is checked even when the return type does not contain its pack. + +```py +def bounded_prefix[T: str, *Ts](*args: *tuple[T, *Ts]) -> None: ... +def constrained_suffix[T: (str, bytes), *Ts](*args: *tuple[*Ts, T]) -> None: ... +def check(values: list[int], valid: list[str]) -> None: + bounded_prefix(*valid) + constrained_suffix(*valid) + + # error: [invalid-argument-type] + bounded_prefix(*values) + # error: [invalid-argument-type] + constrained_suffix(*values) +``` + +### Argument types override incompatible contextual return types + +A contextual return type can guide compatible arguments, but it must not override the argument types +or the number of arguments in a call. + +```py +def collect[*Ts](*args: *Ts) -> tuple[*Ts]: + return args + +valid: tuple[int] = collect(1) + +inferred = collect(1) +reveal_type(inferred) # revealed: tuple[Literal[1]] +# error: [invalid-assignment] +indirect: tuple[str] = inferred +# error: [invalid-assignment] +direct: tuple[str] = collect(1) + +valid_empty: tuple[()] = collect() +# error: [invalid-assignment] +invalid_empty: tuple[str] = collect() +``` + +Return statements and arguments to other functions also provide contextual return types. + +```py +def invalid_return() -> tuple[str]: + # error: [invalid-return-type] + return collect(1) + +def accept_strings(values: tuple[str]) -> None: ... + +accept_strings(collect("valid")) +# error: [invalid-argument-type] +accept_strings(collect(1)) +``` + +### Fixed boundaries around variadic type variable tuples + +Fixed values before or after a type variable tuple do not become part of its inferred shape. Open +splats can provide those boundaries while preserving fixed values already present on the other side. + +```py +def prefixed[*Ts](*args: *tuple[int, *Ts]) -> tuple[*Ts]: + raise NotImplementedError + +def suffixed[*Ts](*args: *tuple[*Ts, str]) -> tuple[*Ts]: + raise NotImplementedError + +def bounded[*Ts](*args: *tuple[int, *Ts, int]) -> tuple[*Ts]: + raise NotImplementedError + +def check( + ints: list[int], + strings: list[str], + extra_prefix: tuple[int, bool, *tuple[str, ...], bytes], + extra_suffix: tuple[bool, *tuple[int, ...], bytes, str], + extra_boundaries: tuple[int, bool, *tuple[str, ...], bytes, int], + missing_prefix: tuple[*tuple[int, ...], bytes], + missing_suffix: tuple[bool, *tuple[str, ...]], +) -> None: + reveal_type(prefixed(1)) # revealed: tuple[()] + reveal_type(prefixed(1, True)) # revealed: tuple[Literal[True]] + reveal_type(prefixed(*ints)) # revealed: tuple[int, ...] + reveal_type(prefixed(*extra_prefix)) # revealed: tuple[bool, *tuple[str, ...], bytes] + reveal_type(prefixed(*missing_prefix)) # revealed: tuple[*tuple[int, ...], bytes] + + reveal_type(suffixed("last")) # revealed: tuple[()] + reveal_type(suffixed(True, "last")) # revealed: tuple[Literal[True]] + reveal_type(suffixed(*strings)) # revealed: tuple[str, ...] + reveal_type(suffixed(*extra_suffix)) # revealed: tuple[bool, *tuple[int, ...], bytes] + reveal_type(suffixed(*missing_suffix)) # revealed: tuple[bool, *tuple[str, ...]] + + reveal_type(bounded(1, 1)) # revealed: tuple[()] + reveal_type(bounded(1, True, 1)) # revealed: tuple[Literal[True]] + reveal_type(bounded(*ints)) # revealed: tuple[int, ...] + reveal_type(bounded(*extra_boundaries)) # revealed: tuple[bool, *tuple[str, ...], bytes] ``` ### Callable inference @@ -597,12 +829,55 @@ def forward_mixed[*Ts]( accept_mixed_forwarded(callback, args) ``` +### Callable inference through nested callable parameters + +Nested callable parameters make the pack covariant, but inference currently loses its fixed length. + +```py +from typing import Callable + +def nested[*Ts]( + callback: Callable[[Callable[[*Ts], None]], None], + *args: *Ts, +) -> tuple[*Ts]: + return args + +def accepts_int_callback(callback: Callable[[int], None]) -> None: ... +def check(value: int, other: str) -> None: + # TODO: Should reveal `tuple[int]`. + reveal_type(nested(accepts_int_callback, value)) # revealed: tuple[int, ...] + # TODO: Should reveal `tuple[int | str]`. + reveal_type(nested(accepts_int_callback, other)) # revealed: tuple[int, ...] + + # TODO: Should report an error because the callback accepts only one argument. + nested(accepts_int_callback, value, other) +``` + +### Starred variadic tuple normalization + +A fixed provided tuple containing `Never` keeps its shape during tuple-level constraint inference. +Its `Never` element must not be discarded or replaced by an unknown-length tuple. + +```py +from typing import Never + +def collect[*Ts](*args: *Ts) -> tuple[*Ts]: + raise NotImplementedError + +def collect_prefixed[*Ts](*args: *tuple[int, *Ts]) -> tuple[*Ts]: + raise NotImplementedError + +def check_never(value: Never) -> None: + reveal_type(collect(value)) # revealed: tuple[Never] + reveal_type(collect_prefixed(1, value)) # revealed: tuple[Never] +``` + ### Unsupported callable checks are deferred -Until call binding can infer a `TypeVarTuple` from `*args`, a generic callback can leave the -expected callable with a gradual positional parameter list. Similarly, inferring each position from -an overload independently loses the correlation between overload branches. Avoid reporting these -cases until the missing inference is implemented. +A generic callback can leave the expected callable with a gradual positional parameter list until +callback constraints are combined with the inferred arguments. Similarly, inferring each position +from an overload independently loses the correlation between overload branches. Avoid reporting +these cases until callback forwarding is supported. ```py from collections.abc import Awaitable, Callable @@ -813,6 +1088,67 @@ def f(i: int, s: str, b: bool) -> None: reveal_type(foo((i,), (s, b))) # revealed: tuple[int] ``` +A positional tuple and `*args` using the same type variable tuple must have the same length. When +their lengths match, their element types are combined. + +```py +def repeat[*Ts](expected: tuple[*Ts], *args: *Ts) -> tuple[*Ts]: + return expected + +def check_repeated(i: int, s: str) -> None: + reveal_type(repeat(())) # revealed: tuple[()] + reveal_type(repeat((i, s), i, s)) # revealed: tuple[int, str] + reveal_type(repeat((i, s), i, i)) # revealed: tuple[int, str | int] + + # error: 5 [invalid-argument-type] "Argument to function `repeat` is incorrect: Expected `tuple[int]`, found `tuple[()]`" + repeat((i,)) + # error: 20 [invalid-argument-type] "Argument to function `repeat` is incorrect: Expected `tuple[int, str]`, found `tuple[int]`" + repeat((i, s), i) + # snapshot: invalid-argument-type + repeat((i,), i, s) +``` + +```snapshot +error[invalid-argument-type]: Argument to function `repeat` is incorrect + --> src/mdtest_snippet.py:21:18 + | +21 | repeat((i,), i, s) + | ^^^^ Expected `tuple[int]`, found `tuple[int, str]` +info: a tuple of length 2 is not assignable to a tuple of length 1 +info: Function defined here + --> src/mdtest_snippet.py:8:5 + | +8 | def repeat[*Ts](expected: tuple[*Ts], *args: *Ts) -> tuple[*Ts]: + | ^^^^^^ ---------- Parameter declared here +``` + +The same length and element-type rules apply when the tuple is passed as a keyword-only argument. + +```py +def repeat_keyword[*Ts](*args: *Ts, expected: tuple[*Ts]) -> tuple[*Ts]: + return expected + +def check_repeated_keyword(i: int, s: str) -> None: + reveal_type(repeat_keyword(expected=())) # revealed: tuple[()] + reveal_type(repeat_keyword(i, s, expected=(i, s))) # revealed: tuple[int, str] + reveal_type(repeat_keyword(i, i, expected=(i, s))) # revealed: tuple[int, str | int] + + # error: 20 [invalid-argument-type] "Argument to function `repeat_keyword` is incorrect: Expected `tuple[int, str]`, found `tuple[int]`" + repeat_keyword(i, expected=(i, s)) + # error: 20 [invalid-argument-type] "Argument to function `repeat_keyword` is incorrect: Expected `tuple[int]`, found `tuple[int, str]`" + repeat_keyword(i, s, expected=(i,)) +``` + +Matching lengths are also required when the return type does not contain the type variable tuple. + +```py +def repeat_without_return[*Ts](expected: tuple[*Ts], *args: *Ts) -> None: ... + +repeat_without_return((1, "value"), 1, "value") +# error: [invalid-argument-type] +repeat_without_return((1, "value"), 1) +``` + ## Type concatenation A type variable tuple can be combined with fixed leading or trailing types. @@ -908,8 +1244,7 @@ accept_str_in_between(True, "phase", "status", b"ok") accept_str_in_between(True, b"ok") accept_str_in_between(True, 1, b"bad") # error: [invalid-argument-type] -# TODO: Infer the `TypeVarTuple` from arguments matched to the variadic parameter. -reveal_type(remove_bytes(1, "record", b"sum")) # revealed: tuple[Unknown, ...] +reveal_type(remove_bytes(1, "record", b"sum")) # revealed: tuple[Literal[1], Literal["record"]] ``` ## `@staticmethod` and `@classmethod` diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index a61a083c94..01b1990940 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -58,9 +58,10 @@ use crate::types::signatures::{ CallableSignature, Parameter, ParameterDisplayName, ParameterKind, Parameters, ParametersKind, PartialApplication, PartialSignatureApplication, }; -use crate::types::tuple::{TupleLength, TupleSpec, TupleType, VariableSegment}; +use crate::types::tuple::{TupleLength, TupleSpec, TupleSpecBuilder, TupleType, VariableSegment}; use crate::types::typed_dict::{TypedDictOpenness, extract_unpacked_typed_dict_from_value_type}; use crate::types::typevar::{BoundTypeVarIdentity, TypeVarNonceGenerator, TypeVarSet}; +use crate::types::variance::VarianceInferable; use crate::types::visitor::{ TypeCollector, TypeKind, TypeVisitor, any_over_type, walk_non_atomic_type, walk_type_with_recursion_guard, @@ -6035,6 +6036,239 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { self.inference = Some(inference); } + /// Infers a variadic type variable tuple from every argument matched to `*args`. + /// + /// Comparing the complete argument tuple with the declared tuple preserves fixed elements, + /// nested type variable tuples, and the unbounded shape of splatted arguments. + /// + /// ```py + /// def collect[*Ts](*args: *Ts) -> tuple[*Ts]: ... + /// def nested[*Ts, *Us](*args: *tuple[tuple[*Us], *Ts]) -> tuple[tuple[*Us], tuple[*Ts]]: ... + /// + /// collect(1, "value") + /// nested((1, "value"), True, b"last") + /// ``` + fn infer_typevartuple_argument_constraints<'c>( + &self, + builder: &mut SpecializationBuilder<'db, 'c>, + specialization_errors: &mut Vec>, + ) -> Result<(), (SpecializationError<'db>, Option)> { + let db = self.db; + let Some((parameter_index, parameter)) = self.signature.parameters().variadic() else { + return Ok(()); + }; + if !parameter.has_starred_annotation() { + return Ok(()); + } + + let (formal, typevartuple) = match parameter.annotated_type() { + Type::TypeVar(typevar) if typevar.is_typevartuple(db) => ( + Type::tuple(TupleType::unpacked_typevartuple(db, self.env, typevar)), + typevar, + ), + annotation => { + let Some(typevartuple) = + annotation.exact_tuple_instance_spec(db).and_then(|tuple| { + match tuple.as_ref() { + TupleSpec::Variable(variable) => variable.variable().typevartuple(), + TupleSpec::Fixed(_) => None, + } + }) + else { + return Ok(()); + }; + (annotation, typevartuple) + } + }; + + if !self.can_infer_typevartuple_arguments(parameter_index, typevartuple) { + return Ok(()); + } + + let Some((actual, argument_indices)) = + self.collect_typevartuple_arguments(parameter_index, parameter, formal) + else { + return Ok(()); + }; + + builder.infer(formal, actual).map_err(|error| { + let argument_index = + argument_indices.and_then(|(first, last)| (first == last).then_some(first)); + (error, argument_index) + })?; + + if let Some(generic_context) = self.signature.generic_context { + let specialization = builder.build_with(generic_context, |_, _| None); + let expected_ty = formal.apply_specialization(db, specialization); + + // The legacy solver keeps the first pack when another occurrence has a different length. + if let (Some(expected_tuple), Some(actual_tuple)) = ( + expected_ty.exact_tuple_instance_spec(db), + actual.exact_tuple_instance_spec(db), + ) && let (TupleLength::Fixed(expected), TupleLength::Fixed(provided)) = + (expected_tuple.len(), actual_tuple.len()) + && expected != provided + { + specialization_errors.push(BindingError::InvalidArgumentType { + parameter: ParameterContext::new(parameter, parameter_index, false), + argument_index: argument_indices.map(|(first, _)| first), + last_argument_index: argument_indices.map(|(_, last)| last), + expected_ty, + provided_ty: actual, + provenance: InvalidArgumentTypeProvenance::Argument, + parameter_source: None, + }); + } + } + + Ok(()) + } + + /// Returns whether a type variable tuple can be inferred from its variadic arguments. + /// + /// The old solver stores one specialization per pack, so it can merge covariant occurrences + /// but cannot combine their lower bounds with the upper bounds from a callable parameter. + /// + /// ```py + /// from collections.abc import Callable + /// + /// def repeat[*Ts](expected: tuple[*Ts], *args: *Ts) -> tuple[*Ts]: ... + /// def invoke[*Ts](callback: Callable[[*Ts], None], *args: *Ts) -> None: ... + /// def accepts_str(value: str) -> None: ... + /// + /// repeat((1, "value"), 1, 2) # safe to infer `Ts = (int, str | int)` + /// invoke(accepts_str, 1) # do not widen the callback's `str` parameter + /// ``` + /// + /// TODO: Remove this guard when the new constraint solver can represent and solve both bounds. + fn can_infer_typevartuple_arguments( + &self, + parameter_index: usize, + typevartuple: BoundTypeVarInstance<'db>, + ) -> bool { + let db = self.db; + !self + .enumerate_argument_types() + .any(|(argument_index, _, argument, _)| { + !matches!(argument, Argument::Synthetic) + && self.argument_matches[argument_index].iter().any(|matched| { + matched.index != parameter_index + && !self.signature.parameters()[matched.index] + .annotated_type() + .variance_of(db, self.env, typevartuple.identity(db)) + .is_covariant() + }) + }) + } + + /// Collects arguments matched to a starred parameter into their complete tuple shape. + /// + /// Direct arguments and splatted values are collected in call order. Values already consumed + /// by earlier parameters are removed from splats, and open tuples are resized to expose any + /// required fixed prefix or suffix. The returned indices cover all contributing source + /// arguments and are used for diagnostics. + /// + /// ```py + /// def tail[*Ts](head: int, *args: *Ts) -> tuple[*Ts]: ... + /// def prefixed[*Ts](*args: *tuple[int, *Ts]) -> tuple[*Ts]: ... + /// + /// def example(values: tuple[int, str, bytes], numbers: list[int]) -> None: + /// tail(*values) # collected `*args`: tuple[str, bytes] + /// prefixed(*numbers) # collected `*args`: tuple[int, *tuple[int, ...]] + /// ``` + fn collect_typevartuple_arguments( + &self, + parameter_index: usize, + parameter: &Parameter<'db>, + formal: Type<'db>, + ) -> Option<(Type<'db>, Option<(usize, usize)>)> { + let db = self.db; + let mut actual = TupleSpecBuilder::with_capacity(self.arguments.len()); + // Source indices of the first and last arguments matched to the variadic parameter. + let mut argument_indices: Option<(usize, usize)> = None; + for (argument_index, adjusted_argument_index, argument, argument_types) in + self.enumerate_argument_types() + { + let matches = &self.argument_matches[argument_index]; + if !matches + .iter() + .any(|matched| matched.index == parameter_index) + { + continue; + } + + if let Some(index) = adjusted_argument_index { + argument_indices = + Some((argument_indices.map_or(index, |(first, _)| first), index)); + } + + if matches!(argument, Argument::Variadic) { + let argument_type = argument_types.get_default()?; + let mut argument_tuple = argument_type.iterate(db, self.env); + let consumed_prefix = matches + .parameters + .iter() + .take_while(|matched| matched.index != parameter_index) + .count(); + if consumed_prefix != 0 { + let consumed_prefix = i32::try_from(consumed_prefix).ok()?; + let sliced = argument_tuple + .py_slice_type(db, self.env, Some(consumed_prefix), None, None) + .ok()?; + argument_tuple = sliced.exact_tuple_instance_spec(db)?; + } + actual = actual.concat(db, self.env, &argument_tuple); + continue; + } + + for matched in matches + .iter() + .filter(|matched| matched.index == parameter_index) + { + let declared_type = matched + .expected_type + .unwrap_or_else(|| parameter.annotated_type()); + actual.push( + matched + .argument_type + .unwrap_or_else(|| argument_types.get_for_declared_type(declared_type)), + ); + } + } + + let mut actual = actual.build(); + if let Some(formal_tuple) = formal.exact_tuple_instance_spec(db) + && let (TupleSpec::Variable(formal), TupleSpec::Variable(provided)) = + (formal_tuple.as_ref(), &actual) + && provided + .variable() + .homogeneous_type() + .is_some_and(|element| !element.resolve_type_alias(db).is_never()) + { + // Expose required boundaries without discarding fixed values the splat already has. + let target_length = TupleLength::Variable( + formal + .prefix_elements() + .len() + .max(provided.prefix_elements().len()), + formal + .suffix_elements() + .len() + .max(provided.suffix_elements().len()), + ); + if actual.len() != target_length + && let Ok(resized) = actual.resize(db, self.env, target_length) + { + actual = resized; + } + } + + Some(( + Type::tuple(TupleType::new(db, self.env, &actual)), + argument_indices, + )) + } + fn infer_argument_constraints<'c>( &mut self, builder: &mut SpecializationBuilder<'db, 'c>, @@ -6044,8 +6278,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { ) -> bool { let db = self.db; for relation in self.argument_relations() { - // TODO: Infer a `TypeVarTuple` from all matched positional arguments as a single - // tuple. Fixed elements beside that pack can still infer ordinary type variables. + // Fixed elements can infer normally; the complete variadic pack is inferred below. if relation.has_starred_annotation && relation.matched_parameter.expected_type.is_none() && (matches!( @@ -6072,6 +6305,24 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { } } + if let Err((error, argument_index)) = + self.infer_typevartuple_argument_constraints(builder, specialization_errors) + && !specialization_errors.iter().any(|existing| { + matches!( + existing, + BindingError::SpecializationError { + error: existing, + .. + } if existing == &error + ) + }) + { + specialization_errors.push(BindingError::SpecializationError { + error, + argument_index, + }); + } + preferred_type_mappings .iter() .all(|(&identity, &preferred_ty)| { @@ -6204,6 +6455,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { self.errors.push(BindingError::InvalidArgumentType { parameter: ParameterContext::new(parameter, parameter_index, positional), argument_index: adjusted_argument_index, + last_argument_index: None, expected_ty, provided_ty: argument_type, provenance: matched_parameter.provenance, @@ -7188,6 +7440,69 @@ impl<'db> Binding<'db> { .then_some(parameter_type) } + /// Returns the expected tuple element for an argument matched to a `TypeVarTuple`. + /// + /// For `result: tuple[Payload, list[int]] = collect({"value": 1}, [])`, the arguments + /// receive `Payload` and `list[int]` as context without overriding their inferred types. + fn typevartuple_argument_context( + &self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + binding: &CallableBinding<'db>, + arguments_types: &CallArguments<'_, 'db>, + argument_index: usize, + expected_return_ty: Type<'db>, + ) -> Option> { + let [matched_parameter] = self + .matched_argument_for_call_argument(binding, argument_index)? + .parameters + .as_slice() + else { + return None; + }; + let parameter = &self.signature.parameters()[matched_parameter.index]; + let Type::TypeVar(typevartuple) = parameter.annotated_type() else { + return None; + }; + if !parameter.is_variadic() + || !parameter.has_starred_annotation() + || !typevartuple.is_typevartuple(db) + { + return None; + } + + let return_tuple = self.signature.return_ty.exact_tuple_instance_spec(db)?; + let TupleSpec::Variable(return_tuple) = return_tuple.as_ref() else { + return None; + }; + if return_tuple.variable().typevartuple()?.identity(db) != typevartuple.identity(db) + || arguments_types + .iter() + .take(argument_index) + .any(|(argument, types)| { + matches!(argument, Argument::Variadic) + && types + .get_default() + .is_none_or(|ty| ty.iterate(db, env).len().maximum().is_none()) + }) + { + return None; + } + + let tuple_index = return_tuple.prefix_elements().len().checked_add( + (0..argument_index) + .filter_map(|index| self.matched_argument_for_call_argument(binding, index)) + .flat_map(MatchedArgument::iter) + .filter(|matched| matched.index == matched_parameter.index) + .count(), + )?; + expected_return_ty + .exact_tuple_instance_spec(db)? + .as_ref() + .py_index(db, env, i32::try_from(tuple_index).ok()?) + .ok() + } + /// Returns the type context to use for bidirectional inference of a source call argument, /// using the provided argument specialization. /// @@ -7284,6 +7599,18 @@ impl<'db> Binding<'db> { } parameter_type = parameter_type.apply_optional_specialization(db, specialization); + if let Some(expected_return_ty) = call_expression_tcx.annotation + && let Some(expected) = self.typevartuple_argument_context( + db, + env, + binding, + arguments_types, + argument_index, + expected_return_ty, + ) + { + parameter_type = expected; + } } Some(ArgumentTypeContext::standard( @@ -8173,6 +8500,8 @@ pub(crate) enum BindingError<'db> { InvalidArgumentType { parameter: ParameterContext, argument_index: Option, + /// Last argument when this error describes all arguments matched to a variadic parameter. + last_argument_index: Option, expected_ty: Type<'db>, provided_ty: Type<'db>, provenance: InvalidArgumentTypeProvenance, @@ -8330,8 +8659,16 @@ impl BindingError<'_> { *argument_index = map(*argument_index); }; match self { - BindingError::InvalidArgumentType { argument_index, .. } - | BindingError::InvalidKeyType { argument_index, .. } + BindingError::InvalidArgumentType { + argument_index, + last_argument_index, + .. + } => { + remap(argument_index); + remap(last_argument_index); + } + + BindingError::InvalidKeyType { argument_index, .. } | BindingError::UnknownArgument { argument_index, .. } | BindingError::UnknownKeywordVariadicArgument { argument_index } | BindingError::PositionalOnlyParameterAsKwarg { argument_index, .. } @@ -8460,6 +8797,7 @@ impl<'db> BindingError<'db> { Self::InvalidArgumentType { parameter, argument_index, + last_argument_index, expected_ty, provided_ty, provenance, @@ -8470,7 +8808,10 @@ impl<'db> BindingError<'db> { // silenced diagnostics during overload evaluation, and rely on the assignability // diagnostic being emitted here. - let range = context.get_range(node, *argument_index); + let mut range = context.get_range(node, *argument_index); + if let Some(last) = last_argument_index { + range = range.cover(context.get_range(node, Some(*last))); + } let Some(builder) = context.report_lint(&INVALID_ARGUMENT_TYPE, range) else { return; }; From 815ea22670c36892136094ea9b9f97ef80f191b2 Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Fri, 14 Aug 2026 10:03:20 -0400 Subject: [PATCH 036/371] [ty] Accept enum members for lax Pydantic string and integer fields (#27751) ## Summary Previously, we rejected enum members passed to Pydantic string and integer fields, even though Pydantic accepts them in lax mode: ```python from enum import Enum from pydantic import BaseModel class Color(Enum): RED = "red" class Number(Enum): SEVEN = 7 class Model(BaseModel): name: str count: int Model(name=Color.RED, count=Number.SEVEN) Model(name=Number.SEVEN, count=Number.SEVEN) ``` We now include `Enum` in the accepted input types for both lax string and integer fields. Strict models and strict fields continue to reject ordinary enum members, while fields that opt out of model-wide strictness accept them. Closes https://github.com/astral-sh/ty/issues/4259. --- .../resources/mdtest/external/pydantic.md | 95 +++++++++++++++++++ crates/ty_vendored/ty_extensions/pydantic.pyi | 5 +- 2 files changed, 98 insertions(+), 2 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/external/pydantic.md b/crates/ty_python_semantic/resources/mdtest/external/pydantic.md index c93f6f4413..c8c3d90419 100644 --- a/crates/ty_python_semantic/resources/mdtest/external/pydantic.md +++ b/crates/ty_python_semantic/resources/mdtest/external/pydantic.md @@ -738,6 +738,101 @@ JsonValueModel(value=SomethingElse()) # error: [invalid-argument-type] JsonValueModel(value={"outer": [1, {"inner": SomethingElse()}]}) ``` +### Enum values for string fields + +In lax mode, Pydantic converts enum members to strings regardless of the member's underlying value. + +```py +from enum import Enum + +from pydantic import BaseModel, ConfigDict, Field + +class StringEnum(Enum): + VALUE = "value" + +class IntegerEnum(Enum): + VALUE = 1 + +class LaxModel(BaseModel): + value: str + +LaxModel(value=StringEnum.VALUE) +LaxModel(value=IntegerEnum.VALUE) +``` + +Strict models and fields reject ordinary enum members because they are not strings. + +```py +class StrictModel(BaseModel): + model_config = ConfigDict(strict=True) + + value: str + +class StrictFieldModel(BaseModel): + value: str = Field(strict=True) + +StrictModel(value=StringEnum.VALUE) # error: [invalid-argument-type] +StrictModel(value=IntegerEnum.VALUE) # error: [invalid-argument-type] +StrictFieldModel(value=StringEnum.VALUE) # error: [invalid-argument-type] +StrictFieldModel(value=IntegerEnum.VALUE) # error: [invalid-argument-type] +``` + +A field that opts out of model-wide strict mode accepts enum members again. + +```py +class LaxFieldModel(BaseModel): + model_config = ConfigDict(strict=True) + + value: str = Field(strict=False) + +LaxFieldModel(value=StringEnum.VALUE) +LaxFieldModel(value=IntegerEnum.VALUE) +``` + +### Enum values for integer fields + +In lax mode, Pydantic accepts enum members as integers by using their underlying values. + +```py +from enum import Enum + +from pydantic import BaseModel, ConfigDict, Field + +class IntegerEnum(Enum): + VALUE = 1 + +class LaxModel(BaseModel): + value: int + +LaxModel(value=IntegerEnum.VALUE) +``` + +Strict models and fields reject ordinary enum members because they are not integers. + +```py +class StrictModel(BaseModel): + model_config = ConfigDict(strict=True) + + value: int + +class StrictFieldModel(BaseModel): + value: int = Field(strict=True) + +StrictModel(value=IntegerEnum.VALUE) # error: [invalid-argument-type] +StrictFieldModel(value=IntegerEnum.VALUE) # error: [invalid-argument-type] +``` + +A field that opts out of model-wide strict mode accepts enum members again. + +```py +class LaxFieldModel(BaseModel): + model_config = ConfigDict(strict=True) + + value: int = Field(strict=False) + +LaxFieldModel(value=IntegerEnum.VALUE) +``` + ### Changing a specific field Strict mode can also be activated for a specific field only: diff --git a/crates/ty_vendored/ty_extensions/pydantic.pyi b/crates/ty_vendored/ty_extensions/pydantic.pyi index af7b9c6350..7057990e6d 100644 --- a/crates/ty_vendored/ty_extensions/pydantic.pyi +++ b/crates/ty_vendored/ty_extensions/pydantic.pyi @@ -3,6 +3,7 @@ from datetime import date, datetime, time, timedelta from decimal import Decimal +from enum import Enum from ipaddress import ( IPv4Address, IPv4Interface, @@ -22,7 +23,7 @@ type LaxDate = bytes | date | datetime | float | int | str | Decimal type LaxDatetime = bytes | date | datetime | float | int | str | Decimal type LaxDecimal = float | int | str | Decimal type LaxFloat = bool | bytes | float | int | str | Decimal -type LaxInt = bool | bytes | float | int | str | Decimal +type LaxInt = bool | bytes | float | int | str | Decimal | Enum type LaxIPv4Address = bytes | int | str | IPv4Address | IPv4Interface type LaxIPv4Interface = ( bytes | int | str | tuple[object, object] | IPv4Address | IPv4Interface @@ -36,7 +37,7 @@ type LaxIPv6Network = bytes | int | str | IPv6Address | IPv6Interface | IPv6Netw type LaxPath = str | Path type LaxStrPattern = str | Pattern[str] type LaxBytesPattern = bytes | Pattern[bytes] -type LaxStr = bytearray | bytes | str +type LaxStr = bytearray | bytes | str | Enum type LaxTime = bytes | float | int | str | time | Decimal type LaxTimedelta = bytes | float | int | str | timedelta | Decimal type LaxUUID = str | UUID From 71e87c8fcbb9851074e436afdf64ae2416a58440 Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Fri, 14 Aug 2026 10:23:02 -0400 Subject: [PATCH 037/371] [ty] Accept documented boolean and fractional Pydantic inputs (#27754) ## Summary Previously, we rejected byte strings for Pydantic boolean fields and `Fraction` instances for integer and floating-point fields, even though Pydantic explicitly documents these conversions: ```python from fractions import Fraction from pydantic import BaseModel class Model(BaseModel): enabled: bool count: int ratio: float Model(enabled=b"true", count=Fraction(2, 1), ratio=Fraction(1, 2)) ``` We now include `bytes` in `LaxBool` and `Fraction` in `LaxInt` and `LaxFloat`. These inputs are explicitly described in Pydantic's [boolean](https://pydantic.dev/docs/validation/latest/api/pydantic/standard_library_types/#booleans), [integer](https://pydantic.dev/docs/validation/latest/api/pydantic/standard_library_types/#integers), and [float](https://pydantic.dev/docs/validation/latest/api/pydantic/standard_library_types/#floats) documentation, even though they are missing from its published [conversion table](https://pydantic.dev/docs/validation/latest/concepts/conversion_table/). --- .../resources/mdtest/external/pydantic.md | 4 ++++ crates/ty_vendored/ty_extensions/pydantic.pyi | 7 ++++--- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/external/pydantic.md b/crates/ty_python_semantic/resources/mdtest/external/pydantic.md index c8c3d90419..b43334416c 100644 --- a/crates/ty_python_semantic/resources/mdtest/external/pydantic.md +++ b/crates/ty_python_semantic/resources/mdtest/external/pydantic.md @@ -152,6 +152,7 @@ Scalar types follow the Python-input conversions in Pydantic's [conversion table import re from datetime import date, datetime, time, timedelta from decimal import Decimal +from fractions import Fraction from ipaddress import ( IPv4Address, IPv4Interface, @@ -174,6 +175,7 @@ LaxBool(value=1.0) LaxBool(value=1) LaxBool(value=Decimal(1)) LaxBool(value="true") +LaxBool(value=b"true") LaxBool(value=[True]) # error: [invalid-argument-type] class LaxBytes(BaseModel): @@ -217,6 +219,7 @@ LaxFloat(value=True) LaxFloat(value=b"1.0") LaxFloat(value="1.0") LaxFloat(value=Decimal("1.0")) +LaxFloat(value=Fraction(1, 2)) LaxFloat(value=(1, 0)) # error: [invalid-argument-type] class LaxInt(BaseModel): @@ -228,6 +231,7 @@ LaxInt(value=b"1") LaxInt(value=1.0) LaxInt(value="1") LaxInt(value=Decimal(1)) +LaxInt(value=Fraction(2, 1)) LaxInt(value=(1,)) # error: [invalid-argument-type] class LaxStr(BaseModel): diff --git a/crates/ty_vendored/ty_extensions/pydantic.pyi b/crates/ty_vendored/ty_extensions/pydantic.pyi index 7057990e6d..4b629e4239 100644 --- a/crates/ty_vendored/ty_extensions/pydantic.pyi +++ b/crates/ty_vendored/ty_extensions/pydantic.pyi @@ -4,6 +4,7 @@ from datetime import date, datetime, time, timedelta from decimal import Decimal from enum import Enum +from fractions import Fraction from ipaddress import ( IPv4Address, IPv4Interface, @@ -16,14 +17,14 @@ from pathlib import Path from re import Pattern from uuid import UUID -type LaxBool = bool | float | int | str | Decimal +type LaxBool = bool | bytes | float | int | str | Decimal type LaxBytes = bytearray | bytes | str type LaxByteSize = float | int | str | Decimal type LaxDate = bytes | date | datetime | float | int | str | Decimal type LaxDatetime = bytes | date | datetime | float | int | str | Decimal type LaxDecimal = float | int | str | Decimal -type LaxFloat = bool | bytes | float | int | str | Decimal -type LaxInt = bool | bytes | float | int | str | Decimal | Enum +type LaxFloat = bool | bytes | float | int | str | Decimal | Fraction +type LaxInt = bool | bytes | float | int | str | Decimal | Enum | Fraction type LaxIPv4Address = bytes | int | str | IPv4Address | IPv4Interface type LaxIPv4Interface = ( bytes | int | str | tuple[object, object] | IPv4Address | IPv4Interface From 4d813d53cdedbfef2c284278b349d9883c01ffb5 Mon Sep 17 00:00:00 2001 From: Ross Titmarsh <23349806+rosstitmarsh@users.noreply.github.com> Date: Fri, 14 Aug 2026 17:08:37 +0100 Subject: [PATCH 038/371] Fix broken link to Python docs (#27757) Fixed a link to `frozen.discard` in the Python docs this was a valid link in older Python docs but has been broken. While I was there I noticed a few other uncanonical links so replaced them with more canonical versions and removed some links to specific Python versions that are unnecessary. --- .../ruff_linter/src/rules/pylint/rules/bad_str_strip_call.rs | 2 +- .../src/rules/refurb/rules/check_and_remove_from_set.rs | 2 +- .../ruff_linter/src/rules/refurb/rules/delete_full_slice.rs | 4 ++-- crates/ruff_linter/src/rules/refurb/rules/if_expr_min_max.rs | 4 ++-- crates/ruff_linter/src/rules/ruff/helpers.rs | 2 +- .../src/rules/ruff/rules/falsy_dict_get_fallback.rs | 2 +- crates/ruff_python_parser/src/parser/expression.rs | 2 +- crates/ruff_python_stdlib/src/identifiers.rs | 2 +- 8 files changed, 10 insertions(+), 10 deletions(-) diff --git a/crates/ruff_linter/src/rules/pylint/rules/bad_str_strip_call.rs b/crates/ruff_linter/src/rules/pylint/rules/bad_str_strip_call.rs index 71958f972d..c82c6420ee 100644 --- a/crates/ruff_linter/src/rules/pylint/rules/bad_str_strip_call.rs +++ b/crates/ruff_linter/src/rules/pylint/rules/bad_str_strip_call.rs @@ -46,7 +46,7 @@ use ruff_python_ast::PythonVersion; /// - `target-version` /// /// ## References -/// - [Python documentation: `str.strip`](https://docs.python.org/3/library/stdtypes.html?highlight=strip#str.strip) +/// - [Python documentation: `str.strip`](https://docs.python.org/3/library/stdtypes.html#str.strip) #[derive(ViolationMetadata)] #[violation_metadata(stable_since = "v0.0.242")] pub(crate) struct BadStrStripCall { diff --git a/crates/ruff_linter/src/rules/refurb/rules/check_and_remove_from_set.rs b/crates/ruff_linter/src/rules/refurb/rules/check_and_remove_from_set.rs index b0dfb6dd76..c544e17490 100644 --- a/crates/ruff_linter/src/rules/refurb/rules/check_and_remove_from_set.rs +++ b/crates/ruff_linter/src/rules/refurb/rules/check_and_remove_from_set.rs @@ -38,7 +38,7 @@ use crate::{AlwaysFixableViolation, Edit, Fix}; /// ``` /// /// ## References -/// - [Python documentation: `set.discard()`](https://docs.python.org/3/library/stdtypes.html?highlight=list#frozenset.discard) +/// - [Python documentation: `set.discard()`](https://docs.python.org/3/library/stdtypes.html#set.discard) #[derive(ViolationMetadata)] #[violation_metadata(stable_since = "0.12.0")] pub(crate) struct CheckAndRemoveFromSet { diff --git a/crates/ruff_linter/src/rules/refurb/rules/delete_full_slice.rs b/crates/ruff_linter/src/rules/refurb/rules/delete_full_slice.rs index 08ffd45463..16a66f8a40 100644 --- a/crates/ruff_linter/src/rules/refurb/rules/delete_full_slice.rs +++ b/crates/ruff_linter/src/rules/refurb/rules/delete_full_slice.rs @@ -41,8 +41,8 @@ use crate::rules::refurb::helpers::generate_method_call; /// ``` /// /// ## References -/// - [Python documentation: Mutable Sequence Types](https://docs.python.org/3/library/stdtypes.html?highlight=list#mutable-sequence-types) -/// - [Python documentation: `dict.clear()`](https://docs.python.org/3/library/stdtypes.html?highlight=list#dict.clear) +/// - [Python documentation: Mutable Sequence Types](https://docs.python.org/3/library/stdtypes.html#typesseq-mutable) +/// - [Python documentation: `dict.clear()`](https://docs.python.org/3/library/stdtypes.html#dict.clear) #[derive(ViolationMetadata)] #[violation_metadata(preview_since = "v0.0.287")] pub(crate) struct DeleteFullSlice; diff --git a/crates/ruff_linter/src/rules/refurb/rules/if_expr_min_max.rs b/crates/ruff_linter/src/rules/refurb/rules/if_expr_min_max.rs index 577efa94da..6c2f6b4771 100644 --- a/crates/ruff_linter/src/rules/refurb/rules/if_expr_min_max.rs +++ b/crates/ruff_linter/src/rules/refurb/rules/if_expr_min_max.rs @@ -36,8 +36,8 @@ use crate::{Edit, Fix, FixAvailability, Violation}; /// This rule's fix is marked as safe, unless the expression contains comments. /// /// ## References -/// - [Python documentation: `min`](https://docs.python.org/3.11/library/functions.html#min) -/// - [Python documentation: `max`](https://docs.python.org/3.11/library/functions.html#max) +/// - [Python documentation: `min`](https://docs.python.org/3/library/functions.html#min) +/// - [Python documentation: `max`](https://docs.python.org/3/library/functions.html#max) #[derive(ViolationMetadata)] #[violation_metadata(stable_since = "0.5.0")] pub(crate) struct IfExprMinMax { diff --git a/crates/ruff_linter/src/rules/ruff/helpers.rs b/crates/ruff_linter/src/rules/ruff/helpers.rs index ba28950a92..002ff28b18 100644 --- a/crates/ruff_linter/src/rules/ruff/helpers.rs +++ b/crates/ruff_linter/src/rules/ruff/helpers.rs @@ -227,7 +227,7 @@ pub(super) fn has_default_copy_semantics( /// Returns `true` if the given function is an instantiation of a class that implements the /// descriptor protocol. /// -/// See: +/// See: pub(super) fn is_descriptor_class(func: &Expr, semantic: &SemanticModel) -> bool { semantic.lookup_attribute(func).is_some_and(|id| { let BindingKind::ClassDefinition(scope_id) = semantic.binding(id).kind else { diff --git a/crates/ruff_linter/src/rules/ruff/rules/falsy_dict_get_fallback.rs b/crates/ruff_linter/src/rules/ruff/rules/falsy_dict_get_fallback.rs index c4a708cd69..82aa1d714e 100644 --- a/crates/ruff_linter/src/rules/ruff/rules/falsy_dict_get_fallback.rs +++ b/crates/ruff_linter/src/rules/ruff/rules/falsy_dict_get_fallback.rs @@ -38,7 +38,7 @@ use crate::{Applicability, Fix, FixAvailability, Violation}; /// shown in the [documentation], `dict.get` takes two positional-only arguments, so invalid cases /// are identified by the presence of more than two arguments or any keyword arguments. /// -/// [documentation]: https://docs.python.org/3.13/library/stdtypes.html#dict.get +/// [documentation]: https://docs.python.org/3/library/stdtypes.html#dict.get #[derive(ViolationMetadata)] #[violation_metadata(preview_since = "0.8.5")] pub(crate) struct FalsyDictGetFallback; diff --git a/crates/ruff_python_parser/src/parser/expression.rs b/crates/ruff_python_parser/src/parser/expression.rs index b07d7aa884..c74d55e782 100644 --- a/crates/ruff_python_parser/src/parser/expression.rs +++ b/crates/ruff_python_parser/src/parser/expression.rs @@ -1515,7 +1515,7 @@ impl<'src> Parser<'src> { /// /// If the parser isn't positioned at a `String` token. /// - /// See: + /// See: fn parse_string_or_byte_literal(&mut self) -> StringType { let range = self.current_token_range(); let flags = self.tokens.current_flags().as_any_string_flags(); diff --git a/crates/ruff_python_stdlib/src/identifiers.rs b/crates/ruff_python_stdlib/src/identifiers.rs index 950b128c98..b6407e1127 100644 --- a/crates/ruff_python_stdlib/src/identifiers.rs +++ b/crates/ruff_python_stdlib/src/identifiers.rs @@ -46,7 +46,7 @@ fn is_identifier_continuation(c: char) -> bool { /// identifier is defined in a class definition, it will be mangled prior to /// code generation. /// -/// See: . +/// See: . pub fn is_mangled_private(id: &str) -> bool { id.starts_with("__") && !id.ends_with("__") } From e986989cea9ca740d7b7ca2e298533ee64dafa5f Mon Sep 17 00:00:00 2001 From: Roy Buitenhuis Date: Fri, 14 Aug 2026 18:14:09 +0200 Subject: [PATCH 039/371] [`ruff`] Add `ctypes.LittleEndianStructure` and others alike to existing exception (`RUF012`) (#27753) ## Summary #22559 Added an exception to the _fields_ Class Variable not needing an annotation. I have the same issue as what the fix is supposed to fix for certain variants of ctypes.Structure. ## Test Plan I extended the ruff012.py file with extra base types and then I added the extra base types to the helper.rs function that detected the original case. Then I updated the snapshot, as the output is what I now expect. --- .../resources/test/fixtures/ruff/RUF012.py | 39 +++++++++++++ crates/ruff_linter/src/rules/ruff/helpers.rs | 13 ++++- ..._rules__ruff__tests__RUF012_RUF012.py.snap | 56 +++++++++++++++++++ 3 files changed, 107 insertions(+), 1 deletion(-) diff --git a/crates/ruff_linter/resources/test/fixtures/ruff/RUF012.py b/crates/ruff_linter/resources/test/fixtures/ruff/RUF012.py index e5ec4d610b..af37af2333 100644 --- a/crates/ruff_linter/resources/test/fixtures/ruff/RUF012.py +++ b/crates/ruff_linter/resources/test/fixtures/ruff/RUF012.py @@ -150,3 +150,42 @@ class S(ctypes.Structure): ("propagation", ctypes.c_uint64), ("userns_fd", ctypes.c_uint64), ] + +class LES(ctypes.LittleEndianStructure): + test = [""] + _fields_ = [ + ("attr_set", ctypes.c_uint64), + ("attr_clr", ctypes.c_uint64), + ("propagation", ctypes.c_uint64), + ("userns_fd", ctypes.c_uint64), + ] + +class BES(ctypes.BigEndianStructure): + test = [""] + _fields_ = [ + ("attr_set", ctypes.c_uint64), + ("attr_clr", ctypes.c_uint64), + ("propagation", ctypes.c_uint64), + ("userns_fd", ctypes.c_uint64), + ] + +class U(ctypes.Union): + test = [""] + _fields_ = [ + ("a", LES), + ("b", BES), + ] + +class LEU(ctypes.LittleEndianUnion): + test = [""] + _fields_ = [ + ("a", LES), + ("b", BES), + ] + +class BEU(ctypes.BigEndianUnion): + test = [""] + _fields_ = [ + ("a", LES), + ("b", BES), + ] diff --git a/crates/ruff_linter/src/rules/ruff/helpers.rs b/crates/ruff_linter/src/rules/ruff/helpers.rs index 002ff28b18..d37fd21197 100644 --- a/crates/ruff_linter/src/rules/ruff/helpers.rs +++ b/crates/ruff_linter/src/rules/ruff/helpers.rs @@ -252,7 +252,18 @@ pub(super) fn is_ctypes_structure_fields( ) -> bool { let is_ctypes_structure = analyze::class::any_qualified_base_class(class_def, semantic, |qualified_name| { - matches!(qualified_name.segments(), ["ctypes", "Structure"]) + matches!( + qualified_name.segments(), + [ + "ctypes", + "Structure" + | "BigEndianStructure" + | "LittleEndianStructure" + | "Union" + | "BigEndianUnion" + | "LittleEndianUnion" + ] + ) }); let is_fields = matches!( diff --git a/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF012_RUF012.py.snap b/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF012_RUF012.py.snap index 5cc2fd8c41..fdd09af79c 100644 --- a/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF012_RUF012.py.snap +++ b/crates/ruff_linter/src/rules/ruff/snapshots/ruff_linter__rules__ruff__tests__RUF012_RUF012.py.snap @@ -1,5 +1,6 @@ --- source: crates/ruff_linter/src/rules/ruff/mod.rs +assertion_line: 133 --- RUF012 Mutable default value for class attribute --> RUF012.py:9:34 @@ -139,3 +140,58 @@ RUF012 Mutable default value for class attribute 148 | ("attr_set", ctypes.c_uint64), | help: Consider initializing in `__init__` or annotating with `typing.ClassVar` + +RUF012 Mutable default value for class attribute + --> RUF012.py:155:12 + | +154 | class LES(ctypes.LittleEndianStructure): +155 | test = [""] + | ^^^^ +156 | _fields_ = [ +157 | ("attr_set", ctypes.c_uint64), + | +help: Consider initializing in `__init__` or annotating with `typing.ClassVar` + +RUF012 Mutable default value for class attribute + --> RUF012.py:164:12 + | +163 | class BES(ctypes.BigEndianStructure): +164 | test = [""] + | ^^^^ +165 | _fields_ = [ +166 | ("attr_set", ctypes.c_uint64), + | +help: Consider initializing in `__init__` or annotating with `typing.ClassVar` + +RUF012 Mutable default value for class attribute + --> RUF012.py:173:12 + | +172 | class U(ctypes.Union): +173 | test = [""] + | ^^^^ +174 | _fields_ = [ +175 | ("a", LES), + | +help: Consider initializing in `__init__` or annotating with `typing.ClassVar` + +RUF012 Mutable default value for class attribute + --> RUF012.py:180:12 + | +179 | class LEU(ctypes.LittleEndianUnion): +180 | test = [""] + | ^^^^ +181 | _fields_ = [ +182 | ("a", LES), + | +help: Consider initializing in `__init__` or annotating with `typing.ClassVar` + +RUF012 Mutable default value for class attribute + --> RUF012.py:187:12 + | +186 | class BEU(ctypes.BigEndianUnion): +187 | test = [""] + | ^^^^ +188 | _fields_ = [ +189 | ("a", LES), + | +help: Consider initializing in `__init__` or annotating with `typing.ClassVar` From a8ba9e0458e60f8c406216778f0ea1cc3759c9b2 Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Fri, 14 Aug 2026 12:28:04 -0400 Subject: [PATCH 040/371] Fix Git metadata tracking in linked worktrees (#27710) ## Summary Previously, we tracked Git metadata incorrectly in several cases: - Linked worktrees watched their entire worktree metadata directory, so staging an unrelated file invalidated the build even when the commit had not changed. - Branch references updated outside the worktree were not watched, potentially leaving embedded commit hashes, dates, tags, and tag distances stale. - Packed branch references and relative worktree paths registered nonexistent Cargo inputs, forcing a rebuild on every invocation. - A Ruff checkout nested directly inside an unrelated Git repository caused ty to embed the parent repository's commit instead of Ruff's. We now track each worktree's actual `HEAD`, resolve branch references through the shared Git directory, handle relative paths and packed references, and ignore unrelated parent repositories when selecting ty's commit metadata. When the current branch is packed, we also watch the nearest existing reference directory so a newly created loose reference becomes visible. This can invalidate a packed-reference worktree when another branch in the same repository changes. --- crates/ruff/build.rs | 76 ++++++++++++++++++++++++++++++-------- crates/ty/build.rs | 81 ++++++++++++++++++++++++++++++++--------- crates/ty_wasm/build.rs | 76 ++++++++++++++++++++++++++++++-------- 3 files changed, 186 insertions(+), 47 deletions(-) diff --git a/crates/ruff/build.rs b/crates/ruff/build.rs index befb62375c..0ed8220d13 100644 --- a/crates/ruff/build.rs +++ b/crates/ruff/build.rs @@ -27,18 +27,17 @@ fn commit_info(workspace_root: &Path) { if let Some(git_head_path) = git_head(&git_dir) { println!("cargo:rerun-if-changed={}", git_head_path.display()); - let git_head_contents = fs::read_to_string(git_head_path); + let git_head_contents = fs::read_to_string(&git_head_path); if let Ok(git_head_contents) = git_head_contents { // The contents are either a commit or a reference in the following formats // - "" when the head is detached - // - "ref " when working on a branch + // - "ref: " when working on a branch // If a commit, checking if the HEAD file has changed is sufficient - // If a ref, we need to add the head file for that ref to rebuild on commit + // If a ref, we also need to watch where Git stores its current commit let mut git_ref_parts = git_head_contents.split_whitespace(); git_ref_parts.next(); if let Some(git_ref) = git_ref_parts.next() { - let git_ref_path = git_dir.join(git_ref); - println!("cargo:rerun-if-changed={}", git_ref_path.display()); + watch_git_ref(&git_head_path, git_ref); } } } @@ -79,27 +78,74 @@ fn commit_info(workspace_root: &Path) { fn git_head(git_dir: &Path) -> Option { // The typical case is a standard git repository. - let git_head_path = git_dir.join("HEAD"); - if git_head_path.exists() { - return Some(git_head_path); + if git_dir.is_dir() { + return Some(git_dir.join("HEAD")); } if !git_dir.is_file() { return None; } - // If `.git/HEAD` doesn't exist and `.git` is actually a file, - // then let's try to attempt to read it as a worktree. If it's - // a worktree, then its contents will look like this, e.g.: + + // Watch the pointer in case the worktree's Git directory changes. + println!("cargo:rerun-if-changed={}", git_dir.display()); + // A linked worktree has a `.git` file instead of a `.git` directory. + // Its contents point to the worktree-specific Git directory, e.g.: // - // gitdir: /home/andrew/astral/uv/main/.git/worktrees/pr2 + // gitdir: /home/andrew/astral/ruff/main/.git/worktrees/pr2 // // And the HEAD file we want to watch will be at: // - // /home/andrew/astral/uv/main/.git/worktrees/pr2/HEAD + // /home/andrew/astral/ruff/main/.git/worktrees/pr2/HEAD let contents = fs::read_to_string(git_dir).ok()?; let (label, worktree_path) = contents.split_once(':')?; if label != "gitdir" { return None; } - let worktree_path = worktree_path.trim(); - Some(PathBuf::from(worktree_path)) + // Relative `gitdir:` paths are relative to the directory containing `.git`. + let worktree_path = PathBuf::from(worktree_path.trim()); + let worktree_path = if worktree_path.is_absolute() { + worktree_path + } else { + git_dir.parent()?.join(worktree_path) + }; + Some(worktree_path.join("HEAD")) +} + +/// Watch the loose or packed Git reference for the current branch. +fn watch_git_ref(git_head_path: &Path, git_ref: &str) { + let Some(worktree_git_dir) = git_head_path.parent() else { + return; + }; + + // Worktrees have their own HEAD, but branch refs live in the shared Git directory. Their + // `commondir` file points to that directory, either absolutely or relative to this Git directory. + let common_dir_path = worktree_git_dir.join("commondir"); + let common_git_dir = if let Ok(common_dir) = fs::read_to_string(&common_dir_path) { + println!("cargo:rerun-if-changed={}", common_dir_path.display()); + let common_dir = PathBuf::from(common_dir.trim()); + if common_dir.is_absolute() { + common_dir + } else { + worktree_git_dir.join(common_dir) + } + } else { + worktree_git_dir.to_path_buf() + }; + + let git_ref_path = common_git_dir.join(git_ref); + if git_ref_path.exists() { + println!("cargo:rerun-if-changed={}", git_ref_path.display()); + } else { + // A packed branch ref has no loose ref file. Watch `packed-refs` instead of the missing + // loose ref, since Cargo would rebuild on every invocation for a nonexistent watched path. + let packed_refs = common_git_dir.join("packed-refs"); + if packed_refs.exists() { + println!("cargo:rerun-if-changed={}", packed_refs.display()); + } + // A later commit can recreate the loose ref, even when its parent directories do not exist + // yet. Watch the nearest existing ancestor so Cargo notices that transition. This can + // also rebuild when another ref in that directory changes. + if let Some(parent) = git_ref_path.ancestors().find(|parent| parent.is_dir()) { + println!("cargo:rerun-if-changed={}", parent.display()); + } + } } diff --git a/crates/ty/build.rs b/crates/ty/build.rs index 69afef2fd8..26901b6783 100644 --- a/crates/ty/build.rs +++ b/crates/ty/build.rs @@ -14,9 +14,10 @@ fn main() { version_info(&ty_workspace_root); - // If not in a git repository, do not attempt to retrieve commit information + // An independent ty checkout has its own dist-workspace.toml. Without one, a parent Git + // repository is unrelated, so use the nested Ruff checkout's commit information instead. let git_dir = ty_workspace_root.join(".git"); - if git_dir.exists() { + if ty_workspace_root.join("dist-workspace.toml").is_file() && git_dir.exists() { commit_info(&git_dir, &ty_workspace_root, false); } else { // Try if we're inside the ruff repository and, if so, use that commit hash. @@ -61,18 +62,17 @@ fn commit_info(git_dir: &Path, workspace_root: &Path, is_ruff: bool) { if let Some(git_head_path) = git_head(git_dir) { println!("cargo:rerun-if-changed={}", git_head_path.display()); - let git_head_contents = fs::read_to_string(git_head_path); + let git_head_contents = fs::read_to_string(&git_head_path); if let Ok(git_head_contents) = git_head_contents { // The contents are either a commit or a reference in the following formats // - "" when the head is detached - // - "ref " when working on a branch + // - "ref: " when working on a branch // If a commit, checking if the HEAD file has changed is sufficient - // If a ref, we need to add the head file for that ref to rebuild on commit + // If a ref, we also need to watch where Git stores its current commit let mut git_ref_parts = git_head_contents.split_whitespace(); git_ref_parts.next(); if let Some(git_ref) = git_ref_parts.next() { - let git_ref_path = git_dir.join(git_ref); - println!("cargo:rerun-if-changed={}", git_ref_path.display()); + watch_git_ref(&git_head_path, git_ref); } } } @@ -117,27 +117,74 @@ fn commit_info(git_dir: &Path, workspace_root: &Path, is_ruff: bool) { fn git_head(git_dir: &Path) -> Option { // The typical case is a standard git repository. - let git_head_path = git_dir.join("HEAD"); - if git_head_path.exists() { - return Some(git_head_path); + if git_dir.is_dir() { + return Some(git_dir.join("HEAD")); } if !git_dir.is_file() { return None; } - // If `.git/HEAD` doesn't exist and `.git` is actually a file, - // then let's try to attempt to read it as a worktree. If it's - // a worktree, then its contents will look like this, e.g.: + + // Watch the pointer in case the worktree's Git directory changes. + println!("cargo:rerun-if-changed={}", git_dir.display()); + // A linked worktree has a `.git` file instead of a `.git` directory. + // Its contents point to the worktree-specific Git directory, e.g.: // - // gitdir: /home/andrew/astral/uv/main/.git/worktrees/pr2 + // gitdir: /home/andrew/astral/ruff/main/.git/worktrees/pr2 // // And the HEAD file we want to watch will be at: // - // /home/andrew/astral/uv/main/.git/worktrees/pr2/HEAD + // /home/andrew/astral/ruff/main/.git/worktrees/pr2/HEAD let contents = fs::read_to_string(git_dir).ok()?; let (label, worktree_path) = contents.split_once(':')?; if label != "gitdir" { return None; } - let worktree_path = worktree_path.trim(); - Some(PathBuf::from(worktree_path)) + // Relative `gitdir:` paths are relative to the directory containing `.git`. + let worktree_path = PathBuf::from(worktree_path.trim()); + let worktree_path = if worktree_path.is_absolute() { + worktree_path + } else { + git_dir.parent()?.join(worktree_path) + }; + Some(worktree_path.join("HEAD")) +} + +/// Watch the loose or packed Git reference for the current branch. +fn watch_git_ref(git_head_path: &Path, git_ref: &str) { + let Some(worktree_git_dir) = git_head_path.parent() else { + return; + }; + + // Worktrees have their own HEAD, but branch refs live in the shared Git directory. Their + // `commondir` file points to that directory, either absolutely or relative to this Git directory. + let common_dir_path = worktree_git_dir.join("commondir"); + let common_git_dir = if let Ok(common_dir) = fs::read_to_string(&common_dir_path) { + println!("cargo:rerun-if-changed={}", common_dir_path.display()); + let common_dir = PathBuf::from(common_dir.trim()); + if common_dir.is_absolute() { + common_dir + } else { + worktree_git_dir.join(common_dir) + } + } else { + worktree_git_dir.to_path_buf() + }; + + let git_ref_path = common_git_dir.join(git_ref); + if git_ref_path.exists() { + println!("cargo:rerun-if-changed={}", git_ref_path.display()); + } else { + // A packed branch ref has no loose ref file. Watch `packed-refs` instead of the missing + // loose ref, since Cargo would rebuild on every invocation for a nonexistent watched path. + let packed_refs = common_git_dir.join("packed-refs"); + if packed_refs.exists() { + println!("cargo:rerun-if-changed={}", packed_refs.display()); + } + // A later commit can recreate the loose ref, even when its parent directories do not exist + // yet. Watch the nearest existing ancestor so Cargo notices that transition. This can + // also rebuild when another ref in that directory changes. + if let Some(parent) = git_ref_path.ancestors().find(|parent| parent.is_dir()) { + println!("cargo:rerun-if-changed={}", parent.display()); + } + } } diff --git a/crates/ty_wasm/build.rs b/crates/ty_wasm/build.rs index 5355a7c669..57acd4db15 100644 --- a/crates/ty_wasm/build.rs +++ b/crates/ty_wasm/build.rs @@ -25,18 +25,17 @@ fn commit_info(workspace_root: &Path) { if let Some(git_head_path) = git_head(&git_dir) { println!("cargo:rerun-if-changed={}", git_head_path.display()); - let git_head_contents = fs::read_to_string(git_head_path); + let git_head_contents = fs::read_to_string(&git_head_path); if let Ok(git_head_contents) = git_head_contents { // The contents are either a commit or a reference in the following formats // - "" when the head is detached - // - "ref " when working on a branch + // - "ref: " when working on a branch // If a commit, checking if the HEAD file has changed is sufficient - // If a ref, we need to add the head file for that ref to rebuild on commit + // If a ref, we also need to watch where Git stores its current commit let mut git_ref_parts = git_head_contents.split_whitespace(); git_ref_parts.next(); if let Some(git_ref) = git_ref_parts.next() { - let git_ref_path = git_dir.join(git_ref); - println!("cargo:rerun-if-changed={}", git_ref_path.display()); + watch_git_ref(&git_head_path, git_ref); } } } @@ -64,27 +63,74 @@ fn commit_info(workspace_root: &Path) { fn git_head(git_dir: &Path) -> Option { // The typical case is a standard git repository. - let git_head_path = git_dir.join("HEAD"); - if git_head_path.exists() { - return Some(git_head_path); + if git_dir.is_dir() { + return Some(git_dir.join("HEAD")); } if !git_dir.is_file() { return None; } - // If `.git/HEAD` doesn't exist and `.git` is actually a file, - // then let's try to attempt to read it as a worktree. If it's - // a worktree, then its contents will look like this, e.g.: + + // Watch the pointer in case the worktree's Git directory changes. + println!("cargo:rerun-if-changed={}", git_dir.display()); + // A linked worktree has a `.git` file instead of a `.git` directory. + // Its contents point to the worktree-specific Git directory, e.g.: // - // gitdir: /home/andrew/astral/uv/main/.git/worktrees/pr2 + // gitdir: /home/andrew/astral/ruff/main/.git/worktrees/pr2 // // And the HEAD file we want to watch will be at: // - // /home/andrew/astral/uv/main/.git/worktrees/pr2/HEAD + // /home/andrew/astral/ruff/main/.git/worktrees/pr2/HEAD let contents = fs::read_to_string(git_dir).ok()?; let (label, worktree_path) = contents.split_once(':')?; if label != "gitdir" { return None; } - let worktree_path = worktree_path.trim(); - Some(PathBuf::from(worktree_path)) + // Relative `gitdir:` paths are relative to the directory containing `.git`. + let worktree_path = PathBuf::from(worktree_path.trim()); + let worktree_path = if worktree_path.is_absolute() { + worktree_path + } else { + git_dir.parent()?.join(worktree_path) + }; + Some(worktree_path.join("HEAD")) +} + +/// Watch the loose or packed Git reference for the current branch. +fn watch_git_ref(git_head_path: &Path, git_ref: &str) { + let Some(worktree_git_dir) = git_head_path.parent() else { + return; + }; + + // Worktrees have their own HEAD, but branch refs live in the shared Git directory. Their + // `commondir` file points to that directory, either absolutely or relative to this Git directory. + let common_dir_path = worktree_git_dir.join("commondir"); + let common_git_dir = if let Ok(common_dir) = fs::read_to_string(&common_dir_path) { + println!("cargo:rerun-if-changed={}", common_dir_path.display()); + let common_dir = PathBuf::from(common_dir.trim()); + if common_dir.is_absolute() { + common_dir + } else { + worktree_git_dir.join(common_dir) + } + } else { + worktree_git_dir.to_path_buf() + }; + + let git_ref_path = common_git_dir.join(git_ref); + if git_ref_path.exists() { + println!("cargo:rerun-if-changed={}", git_ref_path.display()); + } else { + // A packed branch ref has no loose ref file. Watch `packed-refs` instead of the missing + // loose ref, since Cargo would rebuild on every invocation for a nonexistent watched path. + let packed_refs = common_git_dir.join("packed-refs"); + if packed_refs.exists() { + println!("cargo:rerun-if-changed={}", packed_refs.display()); + } + // A later commit can recreate the loose ref, even when its parent directories do not exist + // yet. Watch the nearest existing ancestor so Cargo notices that transition. This can + // also rebuild when another ref in that directory changes. + if let Some(parent) = git_ref_path.ancestors().find(|parent| parent.is_dir()) { + println!("cargo:rerun-if-changed={}", parent.display()); + } + } } From 2741d28cd0e9fb3b792dd43af46da15759655db0 Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Fri, 14 Aug 2026 13:00:44 -0400 Subject: [PATCH 041/371] Compare stacked PRs against the preceding layer (#27756) ## Summary Previously, our ecosystem, typing-conformance, memory, and fuzzing comparisons calculated their baseline with `git merge-base "$GITHUB_SHA" "origin/$GITHUB_BASE_REF"`. For GitHub's native stacked pull requests, this compares every layer against the stack base instead of the immediately preceding pull request. We now use the first parent of GitHub's synthetic merge commit, `git rev-parse "${GITHUB_SHA}^1"`, as the comparison baseline, selecting the preceding merged layer for native stacks. CI reuses this baseline for change detection, Ruff's ecosystem comparison, and ty fuzzing; push and manually dispatched workflows retain their previous merge-base behavior. Ecosystem reports also label the comparison with the actual base branch instead of hardcoding `main`. For ordinary pull requests, the compared revisions do not change: the first parent is the same base-branch commit previously selected by `git merge-base`. Pull requests targeting branches other than `main` retain the same comparison but receive an accurate report label, and the baseline remains pinned to the exact revision GitHub merged even if the base branch changes afterward. On #27749, this reduced the ecosystem diff from `89 added / 676 removed / 8 changed` to `10 added / 0 removed / 0 changed`; conformance also stopped repeating improvements from the parent pull request. --- .github/workflows/ci.yaml | 12 +++++++++--- .github/workflows/memory_report.yaml | 3 ++- .github/workflows/ty-ecosystem-analyzer.yaml | 10 ++++++---- .github/workflows/typing_conformance.yaml | 3 ++- 4 files changed, 19 insertions(+), 9 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index cb27525daa..2a3953afb1 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -32,6 +32,7 @@ jobs: name: "Determine changes" runs-on: ubuntu-latest outputs: + comparison_base: ${{ steps.merge_base.outputs.sha }} # Flag that is raised when any code that affects parser is changed parser: ${{ steps.check_parser.outputs.changed }} # Flag that is raised when any code that affects linter is changed @@ -67,7 +68,12 @@ jobs: env: BASE_REF: ${{ github.event.pull_request.base.ref || 'main' }} run: | - sha=$(git merge-base HEAD "origin/${BASE_REF}") + if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then + # The first parent contains any preceding layers in a stacked pull request. + sha="$(git rev-parse "${GITHUB_SHA}^1")" + else + sha="$(git merge-base HEAD "origin/${BASE_REF}")" + fi echo "sha=${sha}" >> "$GITHUB_OUTPUT" - name: Check if release build inputs changed @@ -659,7 +665,7 @@ jobs: steps: - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 with: - ref: ${{ github.event.pull_request.base.ref }} + ref: ${{ needs.determine_changes.outputs.comparison_base }} persist-credentials: false - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 @@ -791,6 +797,7 @@ jobs: - name: Fuzz env: FORCE_COLOR: 1 + MERGE_BASE: ${{ needs.determine_changes.outputs.comparison_base }} # Line-tables-only debug info: faster builds, backtraces still work. CARGO_PROFILE_PROFILING_DEBUG: line-tables-only run: | @@ -799,7 +806,6 @@ jobs: cargo build --profile=profiling --bin=ty mv target/profiling/ty ty-new - MERGE_BASE="$(git merge-base "$GITHUB_SHA" "origin/$GITHUB_BASE_REF")" git checkout -b old_commit "$MERGE_BASE" echo "old commit (merge base)" git rev-list --format=%s --max-count=1 old_commit diff --git a/.github/workflows/memory_report.yaml b/.github/workflows/memory_report.yaml index 2583d2c704..da7a8fb546 100644 --- a/.github/workflows/memory_report.yaml +++ b/.github/workflows/memory_report.yaml @@ -88,7 +88,8 @@ jobs: cargo build --bin ty --profile profiling mv target/profiling/ty ty-new - MERGE_BASE="$(git merge-base "$GITHUB_SHA" "origin/$GITHUB_BASE_REF")" + # The first parent contains any preceding layers in a stacked pull request. + MERGE_BASE="$(git rev-parse "${GITHUB_SHA}^1")" git checkout -b old_commit "$MERGE_BASE" echo "old commit (merge base)" git rev-list --format=%s --max-count=1 old_commit diff --git a/.github/workflows/ty-ecosystem-analyzer.yaml b/.github/workflows/ty-ecosystem-analyzer.yaml index acd4b64879..8935ebac58 100644 --- a/.github/workflows/ty-ecosystem-analyzer.yaml +++ b/.github/workflows/ty-ecosystem-analyzer.yaml @@ -78,7 +78,8 @@ jobs: # Faster to do this separately than to use `fetch-depth: 0` with `actions/checkout` git fetch --no-tags --filter=blob:none --unshallow origin - MERGE_BASE="$(git merge-base "${GITHUB_SHA}" "origin/${GITHUB_BASE_REF}")" + # The first parent contains any preceding layers in a stacked pull request. + MERGE_BASE="$(git rev-parse "${GITHUB_SHA}^1")" echo "${MERGE_BASE}" > merge-base.txt echo "Merge base: ${MERGE_BASE}" echo "PR commit: ${GITHUB_SHA}" @@ -209,6 +210,7 @@ jobs: id: generate-reports env: REF_NAME: ${{ github.ref_name }} + BASE_REF_NAME: ${{ github.event.pull_request.base.ref }} run: | # Merge shard diagnostics jq -s '{ outputs: [.[].outputs[]] }' diagnostics-base-*.json > diagnostics-base.json @@ -222,7 +224,7 @@ jobs: generate-diff \ diagnostics-base.json \ diagnostics-PR.json \ - --old-name "main (merge base)" \ + --old-name "${BASE_REF_NAME} (merge base)" \ --new-name "$REF_NAME" \ --output-html dist/diff.html @@ -232,7 +234,7 @@ jobs: diagnostics-base.json \ diagnostics-PR.json \ --fail-on-new-abnormal-exits \ - --old-name "main (merge base)" \ + --old-name "${BASE_REF_NAME} (merge base)" \ --new-name "$REF_NAME" \ --output diff-statistics.md DIFF_STATISTICS_EXIT_CODE=$? @@ -242,7 +244,7 @@ jobs: generate-timing-diff \ diagnostics-base.json \ diagnostics-PR.json \ - --old-name "main (merge base)" \ + --old-name "${BASE_REF_NAME} (merge base)" \ --new-name "$REF_NAME" \ --output-html dist/timing.html diff --git a/.github/workflows/typing_conformance.yaml b/.github/workflows/typing_conformance.yaml index 15d122d566..bd276058ac 100644 --- a/.github/workflows/typing_conformance.yaml +++ b/.github/workflows/typing_conformance.yaml @@ -87,7 +87,8 @@ jobs: cargo build --bin ty mv target/debug/ty ty-new - MERGE_BASE="$(git merge-base "$GITHUB_SHA" "origin/$GITHUB_BASE_REF")" + # The first parent contains any preceding layers in a stacked pull request. + MERGE_BASE="$(git rev-parse "${GITHUB_SHA}^1")" git checkout -b old_commit "$MERGE_BASE" echo "old commit (merge base)" git rev-list --format=%s --max-count=1 old_commit From 0e8636cc4722cbd74200ca48aa90ffa5ad032c1c Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Fri, 14 Aug 2026 14:12:08 -0400 Subject: [PATCH 042/371] [ty] Diagnose invalid module-level __getattr__ calls (#27507) ## Summary Previously, an invalid module-level `__getattr__` function caused us to report that the requested attribute was missing, even though Python actually invokes the function and raises `TypeError`: ```python # example.py def __getattr__() -> str: return "fallback" # main.py import example example.missing # error: [invalid-attribute-access] from example import missing # error: [invalid-module-getattr-call] ``` We now propagate failed module-level `__getattr__` calls through member lookup, preserve the function's return type for error recovery, and point diagnostics at its definition. Direct attribute access uses `invalid-attribute-access`, while failed `from` imports use the dedicated `invalid-module-getattr-call` rule. Defined module attributes and real submodules continue to take precedence. --- crates/ty/docs/rules.md | 270 ++++++++++-------- .../lint_docs/invalid-module-getattr-call.md | 24 ++ .../resources/mdtest/import/module_getattr.md | 129 ++++++++- crates/ty_python_semantic/src/types.rs | 138 +++++++-- .../src/types/attribute_write.rs | 4 +- .../src/types/diagnostic.rs | 41 ++- .../src/types/infer/builder/imports.rs | 16 +- ty.schema.json | 10 + 8 files changed, 480 insertions(+), 152 deletions(-) create mode 100644 crates/ty_python_semantic/resources/lint_docs/invalid-module-getattr-call.md diff --git a/crates/ty/docs/rules.md b/crates/ty/docs/rules.md index 97d56a4d6b..34cc77f88f 100644 --- a/crates/ty/docs/rules.md +++ b/crates/ty/docs/rules.md @@ -8,7 +8,7 @@ Default level: error · Added in 0.0.64 · Related issues · -View source +View source @@ -44,7 +44,7 @@ class Base(ABC): Default level: error · Added in 0.0.13 · Related issues · -View source +View source @@ -90,7 +90,7 @@ class Derived(Base): # error Default level: warn · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -154,7 +154,7 @@ class SubProto(BaseProto, Protocol): Default level: error · Added in 0.0.14 · Related issues · -View source +View source @@ -237,7 +237,7 @@ value = unknown # ty: ignore[unresolved-reference] Default level: error · Added in 0.0.16 · Related issues · -View source +View source @@ -292,7 +292,7 @@ Foo.method() # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -320,7 +320,7 @@ Calling a non-callable object will raise a `TypeError` at runtime. Default level: error · Added in 0.0.7 · Related issues · -View source +View source @@ -355,7 +355,7 @@ def f(x: object): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -389,7 +389,7 @@ a = 1 # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -424,7 +424,7 @@ class C(A, B): ... # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -460,7 +460,7 @@ class B(A): ... # error Default level: error · Added in 0.0.1-alpha.29 · Related issues · -View source +View source @@ -496,7 +496,7 @@ type B = A # error Default level: error · Added in 0.0.15 · Related issues · -View source +View source @@ -533,7 +533,7 @@ class Example: Default level: warn · Added in 0.0.1-alpha.16 · Related issues · -View source +View source @@ -572,7 +572,7 @@ old_func() # error: [deprecated] Default level: ignore · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -605,7 +605,7 @@ false positives it can produce. Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -636,7 +636,7 @@ class B(A, A): ... # error Default level: error · Added in 0.0.1-alpha.12 · Related issues · -View source +View source @@ -679,7 +679,7 @@ class A: # error Default level: error · Added in 0.0.14 · Related issues · -View source +View source @@ -756,7 +756,7 @@ def foo() -> "intt\b": ... # error Default level: warn · Added in 0.0.50 · Related issues · -View source +View source @@ -796,7 +796,7 @@ def g(value: ~A) -> None: ... # error: [experimental-syntax] Default level: error · Added in 0.0.20 · Related issues · -View source +View source @@ -831,7 +831,7 @@ def my_function() -> int: Default level: error · Added in 0.0.15 · Related issues · -View source +View source @@ -947,7 +947,7 @@ def test() -> "Literal[5]": Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -983,7 +983,7 @@ class C(A, B): ... # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1013,7 +1013,7 @@ t[3] # error Default level: warn · Added in 0.0.1-alpha.33 · Related issues · -View source +View source @@ -1050,7 +1050,7 @@ class MyClass: ... Default level: error · Added in 0.0.1-alpha.12 · Related issues · -View source +View source @@ -1151,7 +1151,7 @@ an atypical memory layout. Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1183,7 +1183,7 @@ func("foo") # error: [invalid-argument-type] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1214,7 +1214,7 @@ a: int = "" # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1272,7 +1272,7 @@ C.instance_only_var = 56 # error Default level: error · Added in 0.0.33 · Related issues · -View source +View source @@ -1318,7 +1318,7 @@ class Sub(Base): Default level: error · Added in 0.0.1-alpha.19 · Related issues · -View source +View source @@ -1360,7 +1360,7 @@ asyncio.run(main()) Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1387,7 +1387,7 @@ class A(42): ... # error: [invalid-base] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1417,7 +1417,7 @@ with 1: # error Default level: error · Added in 0.0.12 · Related issues · -View source +View source @@ -1470,7 +1470,7 @@ See: Default level: error · Added in 0.0.13 · Related issues · -View source +View source @@ -1506,7 +1506,7 @@ class A: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1538,7 +1538,7 @@ a: str # error Default level: warn · Added in 0.0.20 · Related issues · -View source +View source @@ -1595,7 +1595,7 @@ class Pet(Enum): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1659,7 +1659,7 @@ This rule corresponds to Ruff's [`except-with-non-exception-classes` (`B030`)](h Default level: error · Added in 0.0.1-alpha.28 · Related issues · -View source +View source @@ -1712,7 +1712,7 @@ class D(A): Default level: error · Added in 0.0.1-alpha.35 · Related issues · -View source +View source @@ -1763,7 +1763,7 @@ class NonFrozenChild(FrozenBase): # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1812,7 +1812,7 @@ class D(Generic[U, T]): ... # error Default level: error · Added in 0.0.12 · Related issues · -View source +View source @@ -1908,7 +1908,7 @@ a = 20 / 0 # type: ignore Default level: error · Added in 0.0.1-alpha.17 · Related issues · -View source +View source @@ -1956,7 +1956,7 @@ carol = Person(name="Carol", aeg=25) # typo! Default level: warn · Added in 0.0.15 · Related issues · -View source +View source @@ -2018,7 +2018,7 @@ def f(x, y, /): # Python 3.8+ syntax Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2058,7 +2058,7 @@ def f(t: TypeVar("U")): ... # ty: ignore[invalid-type-form] Default level: error · Added in 0.0.18 · Related issues · -View source +View source @@ -2108,7 +2108,7 @@ match object(): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2143,7 +2143,7 @@ class B(metaclass=42): ... # error Default level: error · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -2255,13 +2255,51 @@ Correct use of `@override` is enforced by ty's [`invalid-explicit-override`](#in [liskov-substitution-principle]: https://en.wikipedia.org/wiki/Liskov_substitution_principle [override]: https://docs.python.org/3/library/typing.html#typing.override +## `invalid-module-getattr-call` + + +Default level: error · +Added in 0.0.72 · +Related issues · +View source + + + +**What it does** + + +Checks for imports that fail when calling a module-level `__getattr__` function. + +**Why is this bad?** + + +If a module defines `__getattr__`, Python calls it when a `from` import requests a name that is not +otherwise defined. The import raises an exception if `__getattr__` cannot accept the requested name. + +**Examples** + + +`module.py`: + +```python +def __getattr__() -> str: + return "fallback" +``` + +`main.py`: + +```python +# TypeError: __getattr__() takes 0 positional arguments but 1 was given +from module import missing # error +``` + ## `invalid-named-tuple` Default level: error · Added in 0.0.1-alpha.19 · Related issues · -View source +View source @@ -2328,7 +2366,7 @@ TypeError: typing.ClassVar[int] is not valid as type argument Default level: warn · Added in 0.0.31 · Related issues · -View source +View source @@ -2376,7 +2414,7 @@ admin[0] # "Alice" Default level: error · Added in 0.0.1-alpha.27 · Related issues · -View source +View source @@ -2414,7 +2452,7 @@ Baz = NewType("Baz", int | str) # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2471,7 +2509,7 @@ def foo(x: int) -> int: ... Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2500,7 +2538,7 @@ def f(a: int = ""): ... # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2536,7 +2574,7 @@ P2 = ParamSpec() # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2572,7 +2610,7 @@ TypeError: Protocols can only inherit from other protocols, got Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2643,7 +2681,7 @@ def g(): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2675,7 +2713,7 @@ def func() -> int: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2786,7 +2824,7 @@ class C: ... Default level: error · Added in 0.0.10 · Related issues · -View source +View source @@ -2837,7 +2875,7 @@ class MyClass: Default level: error · Added in 0.0.1-alpha.6 · Related issues · -View source +View source @@ -2883,7 +2921,7 @@ InvalidAlias = TypeAliasType("InvalidAlias", list[T], type_params=(list[T],)) # Default level: error · Added in 0.0.1-alpha.29 · Related issues · -View source +View source @@ -2950,7 +2988,7 @@ Bar[int] # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2983,7 +3021,7 @@ TYPE_CHECKING = "" # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3019,7 +3057,7 @@ b: Annotated[int] # error Default level: error · Added in 0.0.1-alpha.11 · Related issues · -View source +View source @@ -3076,7 +3114,7 @@ class C: Default level: error · Added in 0.0.15 · Related issues · -View source +View source @@ -3120,7 +3158,7 @@ def g[U, T: U](): ... # error: [invalid-type-variable-bound] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3177,7 +3215,7 @@ V = TypeVar("V", list[int], int) # valid constrained Type Default level: error · Added in 0.0.16 · Related issues · -View source +View source @@ -3219,7 +3257,7 @@ U = TypeVar("U", int, str, default=bytes) # error: [invalid-type-variable-defau Default level: error · Added in 0.0.28 · Related issues · -View source +View source @@ -3255,7 +3293,7 @@ class Child(Base): Default level: error · Added in 0.0.14 · Related issues · -View source +View source @@ -3298,7 +3336,7 @@ def f(options: dict[str, object]): Default level: error · Added in 0.0.9 · Related issues · -View source +View source @@ -3333,7 +3371,7 @@ class Foo(TypedDict): Default level: error · Added in 0.0.25 · Related issues · -View source +View source @@ -3368,7 +3406,7 @@ def gen() -> Iterator[int]: Default level: error · Added in 0.0.14 · Related issues · -View source +View source @@ -3435,7 +3473,7 @@ def h(arg2: type): Default level: error · Added in 0.0.15 · Related issues · -View source +View source @@ -3485,7 +3523,7 @@ def g(arg: object): Default level: warn · Added in 0.0.30 · Related issues · -View source +View source @@ -3528,7 +3566,7 @@ Movie = TypedDict("Film", {"title": str}) # error: [mismatched-type-name] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3559,7 +3597,7 @@ func() # error Default level: ignore · Added in 0.0.41 · Related issues · -View source +View source @@ -3618,7 +3656,7 @@ class ExplicitChild(Parent): Default level: ignore · Added in 0.0.45 · Related issues · -View source +View source @@ -3657,7 +3695,7 @@ def handle(m: re.Match[str]) -> str: Default level: error · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -3696,7 +3734,7 @@ alice["age"] # KeyError Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3734,7 +3772,7 @@ func("string") # error: [no-matching-overload] Default level: error · Added in 0.0.30 · Related issues · -View source +View source @@ -3772,7 +3810,7 @@ class Sub(Super): ... # error: [non-callable-init-subclass] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3801,7 +3839,7 @@ for i in 34: # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3829,7 +3867,7 @@ Subscripting an object that does not support it will raise a `TypeError` at runt Default level: error · Added in 0.0.1-alpha.29 · Related issues · -View source +View source @@ -3866,7 +3904,7 @@ class B(A): Default level: error · Added in 0.0.16 · Related issues · -View source +View source @@ -3903,7 +3941,7 @@ class B(A): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3934,7 +3972,7 @@ f(1, x=2) # error Default level: error · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -3965,7 +4003,7 @@ f(x=1) # error Default level: ignore · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -4004,7 +4042,7 @@ A.c # error Default level: warn · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -4043,7 +4081,7 @@ A()[0] # error Default level: ignore · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -4089,7 +4127,7 @@ from module import a # error Default level: warn · Added in 0.0.23 · Related issues · -View source +View source @@ -4121,7 +4159,7 @@ html.parser # error Default level: ignore · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4158,7 +4196,7 @@ print(x) # error Default level: warn · Added in 0.0.60 · Related issues · -View source +View source @@ -4233,7 +4271,7 @@ def test() -> "int": Default level: warn · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4268,7 +4306,7 @@ cast(int, f()) # error Default level: warn · Added in 0.0.18 · Related issues · -View source +View source @@ -4306,7 +4344,7 @@ class C: Default level: error · Added in 0.0.20 · Related issues · -View source +View source @@ -4350,7 +4388,7 @@ class Outer[T]: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4385,7 +4423,7 @@ static_assert(int(2.0 * 3.0) == 6) # error Default level: warn · Added in 0.0.39 · Related issues · -View source +View source @@ -4436,7 +4474,7 @@ Consider using [`functools.total_ordering`][total_ordering] instead, which does Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4470,7 +4508,7 @@ class B(A): ... # error Default level: error · Added in 0.0.1-alpha.30 · Related issues · -View source +View source @@ -4510,7 +4548,7 @@ class F(NamedTuple): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4540,7 +4578,7 @@ f("foo") # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4579,7 +4617,7 @@ def _(x: int): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4637,7 +4675,7 @@ class A: Default level: error · Added in 0.0.20 · Related issues · -View source +View source @@ -4681,7 +4719,7 @@ class C(Generic[T]): Default level: warn · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4710,7 +4748,7 @@ reveal_type(1) # revealed: Literal[1] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4741,7 +4779,7 @@ f(x=1, y=2) # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4774,7 +4812,7 @@ A().foo # error Default level: warn · Added in 0.0.1-alpha.15 · Related issues · -View source +View source @@ -4849,7 +4887,7 @@ def g(): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4878,7 +4916,7 @@ import foo # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4906,7 +4944,7 @@ print(x) # error Default level: ignore · Added in 0.0.70 · Related issues · -View source +View source @@ -5049,7 +5087,7 @@ Python code. Default level: ignore · Added in 0.0.70 · Related issues · -View source +View source @@ -5192,7 +5230,7 @@ generator boundaries. Default level: warn · Added in 0.0.1-alpha.7 · Related issues · -View source +View source @@ -5239,7 +5277,7 @@ class D(C): ... # error: [unsupported-base] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -5288,7 +5326,7 @@ b1 < b2 < b1 # error Default level: ignore · Added in 0.0.12 · Related issues · -View source +View source @@ -5335,7 +5373,7 @@ def factory(base: type[Base]) -> type: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -5368,7 +5406,7 @@ A() + A() # error Default level: warn · Added in 0.0.21 · Related issues · -View source +View source @@ -5488,7 +5526,7 @@ to `false`. Default level: warn · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -5567,7 +5605,7 @@ def foo(x: int | str) -> int | str: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source diff --git a/crates/ty_python_semantic/resources/lint_docs/invalid-module-getattr-call.md b/crates/ty_python_semantic/resources/lint_docs/invalid-module-getattr-call.md new file mode 100644 index 0000000000..8f55d56d53 --- /dev/null +++ b/crates/ty_python_semantic/resources/lint_docs/invalid-module-getattr-call.md @@ -0,0 +1,24 @@ +## What it does + +Checks for imports that fail when calling a module-level `__getattr__` function. + +## Why is this bad? + +If a module defines `__getattr__`, Python calls it when a `from` import requests a name that is not +otherwise defined. The import raises an exception if `__getattr__` cannot accept the requested name. + +## Examples + +`module.py`: + +```python +def __getattr__() -> str: + return "fallback" +``` + +`main.py`: + +```python +# TypeError: __getattr__() takes 0 positional arguments but 1 was given +from module import missing # error +``` diff --git a/crates/ty_python_semantic/resources/mdtest/import/module_getattr.md b/crates/ty_python_semantic/resources/mdtest/import/module_getattr.md index 79cce812fe..9207cfd74d 100644 --- a/crates/ty_python_semantic/resources/mdtest/import/module_getattr.md +++ b/crates/ty_python_semantic/resources/mdtest/import/module_getattr.md @@ -16,6 +16,63 @@ def __getattr__(name: str) -> str: return "hi" ``` +## Invalid `__getattr__` calls + +A module-level `__getattr__` must accept the attribute name passed by Python. If the call fails, the +access is invalid, but the function's return type remains available for error recovery. + +```py +import invalid_getattr_module + +invalid_getattr_module.missing # snapshot: invalid-attribute-access + +# error: [invalid-attribute-access] "Invalid access to attribute `missing` on type ``" +reveal_type(invalid_getattr_module.missing) # revealed: str + +reveal_type(invalid_getattr_module.defined) # revealed: Literal[1] +``` + +```snapshot +error[invalid-attribute-access]: Invalid access to attribute `missing` on type `` + --> src/mdtest_snippet.py:3:1 + | +3 | invalid_getattr_module.missing # snapshot: invalid-attribute-access + | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Too many positional arguments to function `__getattr__`: expected 0, got 1 +info: This access implicitly calls `__getattr__` +info: Function signature here + --> src/invalid_getattr_module.py:3:5 + | +3 | def __getattr__() -> str: + | ^^^^^^^^^^^^^^^^^^^^ +``` + +`invalid_getattr_module.py`: + +```py +defined = 1 + +def __getattr__() -> str: + return "fallback" +``` + +## Invalid `__getattr__` attribute-name types + +An incompatible attribute-name parameter also makes a module-level fallback call invalid. + +```py +import invalid_getattr_name + +# error: [invalid-attribute-access] "Invalid access to attribute `missing` on type ``" +reveal_type(invalid_getattr_name.missing) # revealed: bytes +``` + +`invalid_getattr_name.py`: + +```py +def __getattr__(name: int) -> bytes: + return b"fallback" +``` + ## `from import` with `__getattr__` At runtime, if `module` has a `__getattr__` implementation, you can do `from module import whatever` @@ -34,6 +91,40 @@ def __getattr__(name: str) -> int: return 42 ``` +## Invalid `__getattr__` calls in `from` imports + +An invalid module-level `__getattr__` call is reported on `from ... import` statements while +retaining the function's return type for recovery. Since the failed operation is an import, it +receives an `invalid-module-getattr-call` diagnostic instead of an `invalid-attribute-access` +diagnostic. + +```py +from invalid_getattr_module import missing # snapshot: invalid-module-getattr-call + +reveal_type(missing) # revealed: str +``` + +```snapshot +error[invalid-module-getattr-call]: Cannot import `missing` from module `invalid_getattr_module` + --> src/mdtest_snippet.py:1:36 + | +1 | from invalid_getattr_module import missing # snapshot: invalid-module-getattr-call + | ^^^^^^^ Too many positional arguments to function `__getattr__`: expected 0, got 1 +info: This import implicitly calls a module-level `__getattr__` function +info: Function signature here + --> src/invalid_getattr_module.py:1:5 + | +1 | def __getattr__() -> str: + | ^^^^^^^^^^^^^^^^^^^^ +``` + +`invalid_getattr_module.py`: + +```py +def __getattr__() -> str: + return "fallback" +``` + ## Precedence: explicit attributes take priority over `__getattr__` ```py @@ -110,20 +201,48 @@ from mod import sub reveal_type(sub) # revealed: ``` +## Precedence: submodules vs invalid `__getattr__` + +A real submodule takes precedence even when the package's `__getattr__` would reject its name. + +`invalid_mod/__init__.py`: + +```py +def __getattr__() -> str: + return "fallback" +``` + +`invalid_mod/sub.py`: + +```py +value = 42 +``` + +```py +from invalid_mod import sub + +reveal_type(sub) # revealed: +``` + ## Limiting names handled by `__getattr__` -If a module `__getattr__` is annotated to only accept certain string literals, then the module -`__getattr__` will be ignored for other names. (In principle this could be a more explicit way to -handle the precedence issues discussed above, but it's not currently used in the ecosystem.) +If a module `__getattr__` is annotated to accept only certain string literals, unsupported names +produce an import or attribute-access diagnostic, respectively, while preserving the recovered +return type. ```py from limited_getattr_module import known_attr -# error: [unresolved-import] +# error: [invalid-module-getattr-call] "Cannot import `unknown_attr` from module `limited_getattr_module`" from limited_getattr_module import unknown_attr reveal_type(known_attr) # revealed: int -reveal_type(unknown_attr) # revealed: Unknown +reveal_type(unknown_attr) # revealed: int + +import limited_getattr_module + +# error: [invalid-attribute-access] "Invalid access to attribute `unknown_attr` on type ``" +reveal_type(limited_getattr_module.unknown_attr) # revealed: int ``` `limited_getattr_module.py`: diff --git a/crates/ty_python_semantic/src/types.rs b/crates/ty_python_semantic/src/types.rs index 83e9107602..6c2c51354e 100644 --- a/crates/ty_python_semantic/src/types.rs +++ b/crates/ty_python_semantic/src/types.rs @@ -69,7 +69,7 @@ use crate::types::constraints::ConstraintSetBuilder; use crate::types::context::{LintDiagnosticGuard, LintDiagnosticGuardBuilder}; use crate::types::diagnostic::{ AttributeAccessMethod, INVALID_AWAIT, INVALID_TYPE_FORM, report_bad_attribute_access_call, - report_bad_dunder_get_call, + report_bad_dunder_get_call, report_bad_import_call, }; pub use crate::types::display::{DisplaySettings, TypeDetail, TypeDisplayDetails}; pub(crate) use crate::types::enums::{EnumClassLiteral, EnumComplementType, enum_metadata}; @@ -625,6 +625,12 @@ enum MemberLookupErrorKind<'db> { name: Type<'db>, }, + /// An invalid module-level `__getattr__` call, stored without its call bindings. + ModuleGetAttr { + callable: Type<'db>, + name: Type<'db>, + }, + /// An invalid attribute-interception call, represented by its receiver and attribute name. GetAttribute { receiver: Type<'db>, @@ -704,9 +710,58 @@ impl<'db> MemberLookupError<'db> { ); } } - MemberLookupErrorKind::DescriptorGet(_) => {} + MemberLookupErrorKind::ModuleGetAttr { .. } + if assigned_type.is_none() + && let Some(failure) = self.module_getattr_call_failure(db, env) => + { + report_bad_attribute_access_call( + context, + &failure, + object_type, + target, + AttributeAccessMethod::GetAttr, + ); + } + MemberLookupErrorKind::DescriptorGet(_) + | MemberLookupErrorKind::ModuleGetAttr { .. } => {} + } + } + + /// Reports a failed module `__getattr__` call on a `from` import. + /// + /// Imports defer this diagnostic until they have ruled out a real submodule: + /// + /// ```python + /// from package import missing # Calls package.__getattr__("missing"). + /// ``` + fn report_module_getattr_import_diagnostic( + self, + context: &InferContext<'db, '_>, + module: ModuleLiteralType<'db>, + target: &ast::Alias, + name: &str, + ) { + if let Some(failure) = + self.module_getattr_call_failure(context.db(), context.program_environment()) + { + report_bad_import_call(context, &failure, module, target, name); } } + + /// Recreates a failed module `__getattr__` call without caching its call bindings. + fn module_getattr_call_failure( + self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + ) -> Option> { + let MemberLookupErrorKind::ModuleGetAttr { callable, name } = self.kind(db) else { + return None; + }; + + callable + .try_call(db, env, &CallArguments::positional([name])) + .err() + } } /// A resolved member or an implicit-call error that retains its recovery value. @@ -3717,7 +3772,9 @@ impl<'db> Type<'db> { name: &str, ) -> Place<'db> { if let Type::ModuleLiteral(module) = self { - module.static_member(db, env, name).place + module + .static_member(db, env, name) + .map_or(Place::Undefined, |member| member.place) } else if let place @ Place::Defined(_) = self.class_member(db, env, name).place { place } else if let Some(place @ Place::Defined(_)) = self @@ -4926,7 +4983,7 @@ impl<'db> Type<'db> { Place::bound(Type::int_literal(i64::from(bool_value))).into() } - Type::ModuleLiteral(module) => module.static_member(db, env, name_str).into(), + Type::ModuleLiteral(module) => module.static_member(db, env, name_str), // If a protocol does not include a member and the policy disables falling back to // `object`, we return `Place::Undefined` here. This short-circuits attribute lookup @@ -9963,48 +10020,75 @@ impl<'db> ModuleLiteralType<'db> { Some(Type::module_literal(db, importing_file, submodule)) } + /// Resolves a missing member through the module's `__getattr__` function. + /// + /// Invalid calls retain their declared return type for recovery while deferring the diagnostic + /// until the caller determines whether the fallback actually takes precedence. + /// + /// ```python + /// # example.py + /// def __getattr__() -> str: ... + /// + /// # Another module: + /// import example + /// example.missing # Invalid call; the recovery type is str. + /// ``` fn try_module_getattr( self, db: &'db dyn Db, env: &ProgramEnvironment<'db>, name: &str, - ) -> PlaceAndQualifiers<'db> { - // For module literals, we want to try calling the module's own `__getattr__` function - // if it exists. First, we need to look up the `__getattr__` function in the module's scope. - let module = self.module(db); - if let Some(file) = module + ) -> MemberLookupResult<'db> { + if let Some(file) = self + .module(db) .file(db) .map(|file| ProgramFile::new(db, file, env.program(db))) + && let Place::Defined(place) = + imported_symbol(db, env, Some(file), "__getattr__", None).place { - let getattr_symbol = imported_symbol(db, env, Some(file), "__getattr__", None); - // If we found a __getattr__ function, try to call it with the name argument - if let Place::Defined(place) = getattr_symbol.place - && let Ok(outcome) = place.ty.try_call( - db, - env, - &CallArguments::positional([Type::string_literal(db, name)]), - ) - { - return PlaceAndQualifiers { + let name_type = Type::string_literal(db, name); + let (return_type, error) = + match place + .ty + .try_call(db, env, &CallArguments::positional([name_type])) + { + Ok(outcome) => (outcome.return_type(db, env), None), + Err(CallError(_, bindings)) => ( + bindings.return_type(db, env), + Some(MemberLookupErrorKind::ModuleGetAttr { + callable: place.ty, + name: name_type, + }), + ), + }; + + return member_lookup_result( + db, + PlaceAndQualifiers { place: Place::Defined(DefinedPlace { - ty: outcome.return_type(db, env), + ty: return_type, provenance: Provenance::Unknown, ..place }), qualifiers: TypeQualifiers::FROM_MODULE_GETATTR, - }; - } + }, + error, + ); } Place::Undefined.into() } + /// Looks up a module member while preserving failed module-level `__getattr__` calls. + /// + /// The failed call and its recovery type are retained so direct attribute access and `from` + /// imports can report the error after resolving lookup precedence. fn static_member( self, db: &'db dyn Db, env: &ProgramEnvironment<'db>, name: &str, - ) -> PlaceAndQualifiers<'db> { + ) -> MemberLookupResult<'db> { let module = self.module(db); // `__dict__` is a very special member that is never overridden by module globals; // we should always look it up directly as an attribute on `types.ModuleType`, @@ -10012,7 +10096,8 @@ impl<'db> ModuleLiteralType<'db> { if name == "__dict__" { return KnownClass::ModuleType .to_instance(db, env) - .member(db, env, "__dict__"); + .member(db, env, "__dict__") + .into(); } // If the file that originally imported the module has also imported a submodule @@ -10056,11 +10141,12 @@ impl<'db> ModuleLiteralType<'db> { ..defined }), qualifiers: place_and_qualifiers.qualifiers, - }; + } + .into(); } } - place_and_qualifiers + place_and_qualifiers.into() } } diff --git a/crates/ty_python_semantic/src/types/attribute_write.rs b/crates/ty_python_semantic/src/types/attribute_write.rs index 10eccd9433..ac4e416505 100644 --- a/crates/ty_python_semantic/src/types/attribute_write.rs +++ b/crates/ty_python_semantic/src/types/attribute_write.rs @@ -322,7 +322,9 @@ pub(super) fn attribute_write_requirement<'db>( { builtins_symbol(db, env, attribute) } else { - module.static_member(db, env, attribute) + module + .static_member(db, env, attribute) + .unwrap_or_else(|_| Place::Undefined.into()) }; AttributeWriteRequirement::Module(match symbol.place { Place::Defined(DefinedPlace { ty, .. }) => Some(ty), diff --git a/crates/ty_python_semantic/src/types/diagnostic.rs b/crates/ty_python_semantic/src/types/diagnostic.rs index 1fca7b39bb..6ec1d917aa 100644 --- a/crates/ty_python_semantic/src/types/diagnostic.rs +++ b/crates/ty_python_semantic/src/types/diagnostic.rs @@ -3,7 +3,7 @@ use super::context::InferContext; use super::mro::DuplicateBaseError; use super::{ CallArguments, CallDunderError, ClassBase, ClassLiteral, GenericAlias, KnownClass, - StaticClassLiteral, add_inferred_python_version_hint_to_diagnostic, + ModuleLiteralType, StaticClassLiteral, add_inferred_python_version_hint_to_diagnostic, }; use crate::diagnostic::{did_you_mean, format_enumeration}; use crate::lint::{Level, LintRegistryBuilder, LintStatus}; @@ -94,6 +94,7 @@ pub(crate) fn register_lints(registry: &mut LintRegistryBuilder) { registry.register_lint(&INVALID_ENUM_MEMBER_ANNOTATION); registry.register_lint(&INVALID_GENERIC_ENUM); registry.register_lint(&INVALID_GENERIC_CLASS); + registry.register_lint(&INVALID_MODULE_GETATTR_CALL); registry.register_lint(&INVALID_LEGACY_TYPE_VARIABLE); registry.register_lint(&INVALID_PARAMSPEC); registry.register_lint(&INVALID_TYPE_ALIAS_TYPE); @@ -564,6 +565,15 @@ declare_lint! { } } +declare_lint! { + #[doc = include_str!("../../resources/lint_docs/invalid-module-getattr-call.md")] + pub(crate) static INVALID_MODULE_GETATTR_CALL = { + summary: "detects imports that fail while calling module-level `__getattr__`", + status: LintStatus::stable("0.0.72"), + default_level: Level::Error, + } +} + declare_lint! { #[doc = include_str!("../../resources/lint_docs/non-callable-init-subclass.md")] pub(crate) static NON_CALLABLE_INIT_SUBCLASS = { @@ -1873,6 +1883,35 @@ pub(super) fn report_bad_attribute_access_call<'db>( ); } +/// Reports an import that fails while implicitly calling module-level `__getattr__`. +/// +/// ```python +/// from package import missing # Calls package.__getattr__("missing"). +/// ``` +pub(super) fn report_bad_import_call<'db>( + context: &InferContext<'db, '_>, + failure: &CallError<'db>, + module: ModuleLiteralType<'db>, + target: &ast::Alias, + name: &str, +) { + let db = context.db(); + + failure.report_diagnostics_with_override( + context, + target.into(), + &CallDiagnosticOverride { + lint: &INVALID_MODULE_GETATTR_CALL, + message: format!( + "Cannot import `{name}` from module `{}`", + module.module(db).name(db), + ), + info: "This import implicitly calls a module-level `__getattr__` function", + argument_ranges: &[target.range()], + }, + ); +} + pub(super) fn report_bad_dunder_set_call<'db>( context: &InferContext<'db, '_>, dunder_set_failure: &CallError<'db>, diff --git a/crates/ty_python_semantic/src/types/infer/builder/imports.rs b/crates/ty_python_semantic/src/types/infer/builder/imports.rs index 2814ffd535..32a29e68b5 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/imports.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/imports.rs @@ -385,6 +385,8 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // First try loading the requested attribute from the module. if !skip_self_referential_member_lookup { + let result = module_literal.static_member(db, self.program_environment(), name); + let error = result.err(); if let PlaceAndQualifiers { place: Place::Defined(DefinedPlace { @@ -394,7 +396,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { .. }), qualifiers, - } = module_literal.static_member(db, self.program_environment(), name) + } = result.unwrap_or_else(|error| error.fallback_member(db)) { if &alias.name != "*" && boundness == Definedness::PossiblyUndefined { // TODO: Consider loading _both_ the attribute and any submodule and unioning them @@ -409,7 +411,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { } } if qualifiers.contains(TypeQualifiers::FROM_MODULE_GETATTR) { - from_module_getattr = Some((ty, qualifiers, source_provenance)); + from_module_getattr = Some((ty, qualifiers, source_provenance, error)); } else { self.add_declaration_with_binding( alias.into(), @@ -467,7 +469,15 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // We've checked for a submodule, so now we can go ahead and use a type from module // `__getattr__`. - if let Some((ty, qualifiers, source_provenance)) = from_module_getattr { + if let Some((ty, qualifiers, source_provenance, error)) = from_module_getattr { + if let Some(error) = error { + error.report_module_getattr_import_diagnostic( + &self.context, + module_literal, + alias, + name, + ); + } self.add_declaration_with_binding( alias.into(), definition, diff --git a/ty.schema.json b/ty.schema.json index 4958a43be5..3f95750842 100644 --- a/ty.schema.json +++ b/ty.schema.json @@ -854,6 +854,16 @@ } ] }, + "invalid-module-getattr-call": { + "title": "detects imports that fail while calling module-level `__getattr__`", + "description": "## What it does\n\nChecks for imports that fail when calling a module-level `__getattr__` function.\n\n## Why is this bad?\n\nIf a module defines `__getattr__`, Python calls it when a `from` import requests a name that is not\notherwise defined. The import raises an exception if `__getattr__` cannot accept the requested name.\n\n## Examples\n\n`module.py`:\n\n```python\ndef __getattr__() -> str:\n return \"fallback\"\n```\n\n`main.py`:\n\n```python\n# TypeError: __getattr__() takes 0 positional arguments but 1 was given\nfrom module import missing # error\n```", + "default": "error", + "oneOf": [ + { + "$ref": "#/definitions/Level" + } + ] + }, "invalid-named-tuple": { "title": "detects invalid `NamedTuple` class definitions", "description": "## What it does\n\nChecks for invalidly defined `NamedTuple` classes.\n\n## Why is this bad?\n\nAn invalidly defined `NamedTuple` class may lead to the type checker\ndrawing incorrect conclusions. It may also lead to `TypeError`s or\n`AttributeError`s at runtime.\n\n## Examples\n\nA class definition cannot combine `NamedTuple` with other base classes\nin multiple inheritance; doing so raises a `TypeError` at runtime. The sole\nexception to this rule is `Generic[]`, which can be used alongside `NamedTuple`\nin a class's bases list.\n\n```pycon\n>>> from typing import NamedTuple\n>>> class Foo(NamedTuple, object): ...\nTypeError: can only inherit from a NamedTuple type and Generic\n```\n\nFurther, `NamedTuple` field names cannot start with an underscore:\n\n```pycon\n>>> from typing import NamedTuple\n>>> class Foo(NamedTuple):\n... _bar: int\nValueError: Field names cannot start with an underscore: '_bar'\n```\n\n`NamedTuple` classes also have certain synthesized attributes (like `_asdict`, `_make`,\n`_replace`, etc.) that cannot be overwritten. Attempting to assign to these attributes\nwithout a type annotation will raise an `AttributeError` at runtime.\n\n```pycon\n>>> from typing import NamedTuple\n>>> class Foo(NamedTuple):\n... x: int\n... _asdict = 42\nAttributeError: Cannot overwrite NamedTuple attribute _asdict\n```\n\nFinally, `NamedTuple` field annotations cannot use the `ClassVar` or `Final` type\nqualifiers. These qualifiers also cause a runtime error when annotations are evaluated eagerly:\n\n```pycon\n>>> from typing import ClassVar, NamedTuple\n>>> class Foo(NamedTuple):\n... x: ClassVar[int]\nTypeError: typing.ClassVar[int] is not valid as type argument\n```", From 99d68da2f17361121e684f0791706e62a8bfa727 Mon Sep 17 00:00:00 2001 From: Swayam Mhaskar Date: Sat, 15 Aug 2026 00:00:18 +0530 Subject: [PATCH 043/371] [`flake8-use-pathlib`] Add autofixes for `PTH116` (#26460) ## Summary part of #2331 ## Test Plan update snapshots for preview mode --------- Co-authored-by: Brent Westbrook --- .../mdtest/flake8-use-pathlib/os-stat.md | 172 ++++++++++++++ .../src/checkers/ast/analyze/expression.rs | 4 +- crates/ruff_linter/src/codes.rs | 2 +- crates/ruff_linter/src/preview.rs | 5 + .../src/rules/flake8_use_pathlib/rules/mod.rs | 2 + .../rules/flake8_use_pathlib/rules/os_stat.rs | 210 ++++++++++++++++++ .../rules/replaceable_by_pathlib.rs | 20 +- ...ake8_use_pathlib__tests__full_name.py.snap | 1 + ...ake8_use_pathlib__tests__import_as.py.snap | 1 + ...e8_use_pathlib__tests__import_from.py.snap | 1 + ...use_pathlib__tests__import_from_as.py.snap | 1 + ..._pathlib__tests__preview_full_name.py.snap | 14 +- ..._pathlib__tests__preview_import_as.py.snap | 14 +- ...athlib__tests__preview_import_from.py.snap | 14 +- ...lib__tests__preview_import_from_as.py.snap | 14 +- .../rules/flake8_use_pathlib/violations.rs | 56 ----- 16 files changed, 450 insertions(+), 81 deletions(-) create mode 100644 crates/ruff_linter/resources/mdtest/flake8-use-pathlib/os-stat.md create mode 100644 crates/ruff_linter/src/rules/flake8_use_pathlib/rules/os_stat.rs diff --git a/crates/ruff_linter/resources/mdtest/flake8-use-pathlib/os-stat.md b/crates/ruff_linter/resources/mdtest/flake8-use-pathlib/os-stat.md new file mode 100644 index 0000000000..2c897d48ef --- /dev/null +++ b/crates/ruff_linter/resources/mdtest/flake8-use-pathlib/os-stat.md @@ -0,0 +1,172 @@ +# `os-stat` (`PTH116`) + +## Python 3.9 + +```toml +preview = true +target-version = "py39" +lint.select = ["PTH116"] +``` + +`Path.stat` doesn't support the `follow_symlinks` keyword argument before 3.10, so the suggested +fixes have to use either `stat` or `lstat` depending on its value, when it's present. + +### `follow_symlinks=True` uses `stat` + +```py +import os + +os.stat("foo", follow_symlinks=True) # snapshot: os-stat +``` + +```snapshot +error[PTH116]: `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` + --> src/mdtest_snippet.py:3:1 + | +3 | os.stat("foo", follow_symlinks=True) # snapshot: os-stat + | ^^^^^^^ +help: Replace with `Path(...).stat()` + | +1 | import os +2 + import pathlib +3 | + - os.stat("foo", follow_symlinks=True) # snapshot: os-stat +4 + pathlib.Path("foo").stat() # snapshot: os-stat + | +note: This is an unsafe fix and may change runtime behavior +``` + +### No `follow_symlinks` also uses `stat` + +The default value is `True`, as above: + +```py +import os + +os.stat("foo") # snapshot: os-stat +``` + +```snapshot +error[PTH116]: `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` + --> src/mdtest_snippet.py:3:1 + | +3 | os.stat("foo") # snapshot: os-stat + | ^^^^^^^ +help: Replace with `Path(...).stat()` + | +1 | import os +2 + import pathlib +3 | + - os.stat("foo") # snapshot: os-stat +4 + pathlib.Path("foo").stat() # snapshot: os-stat + | +note: This is an unsafe fix and may change runtime behavior +``` + +### `follow_symlinks=False` uses `lstat` + +```py +import os + +os.stat("foo", follow_symlinks=False) # snapshot: os-stat +``` + +```snapshot +error[PTH116]: `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` + --> src/mdtest_snippet.py:3:1 + | +3 | os.stat("foo", follow_symlinks=False) # snapshot: os-stat + | ^^^^^^^ +help: Replace with `Path(...).lstat()` + | +1 | import os +2 + import pathlib +3 | + - os.stat("foo", follow_symlinks=False) # snapshot: os-stat +4 + pathlib.Path("foo").lstat() # snapshot: os-stat + | +note: This is an unsafe fix and may change runtime behavior +``` + +### Dynamic `follow_symlinks` suppresses the fix + +If we can't resolve the value of `follow_symlinks`, we still emit a diagnostic but can't reliably +suggest one of the `stat` methods in a fix. + +```py +import os + +follow = True + +os.stat("foo", follow_symlinks=follow) # snapshot: os-stat +``` + +```snapshot +error[PTH116]: `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` + --> src/mdtest_snippet.py:5:1 + | +5 | os.stat("foo", follow_symlinks=follow) # snapshot: os-stat + | ^^^^^^^ +``` + +## Python 3.10+ + +```toml +preview = true +target-version = "py310" +lint.select = ["PTH116"] +``` + +After 3.10, the fixes can always use `stat` and pass along the `follow_symlinks` argument. + +```py +import os + +os.stat("foo", follow_symlinks=False) # snapshot: os-stat +``` + +```snapshot +error[PTH116]: `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` + --> src/mdtest_snippet.py:3:1 + | +3 | os.stat("foo", follow_symlinks=False) # snapshot: os-stat + | ^^^^^^^ +help: Replace with `Path(...).stat()` + | +1 | import os +2 + import pathlib +3 | + - os.stat("foo", follow_symlinks=False) # snapshot: os-stat +4 + pathlib.Path("foo").stat(follow_symlinks=False) # snapshot: os-stat +5 | follow = True + | +note: This is an unsafe fix and may change runtime behavior +``` + +This is also the case for dynamic values: + +```py +follow = True + +os.stat("foo", follow_symlinks=follow) # snapshot: os-stat +``` + +```snapshot +error[PTH116]: `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` + --> src/mdtest_snippet.py:6:1 + | +6 | os.stat("foo", follow_symlinks=follow) # snapshot: os-stat + | ^^^^^^^ +help: Replace with `Path(...).stat()` + | +1 | import os +2 + import pathlib +3 | +4 | os.stat("foo", follow_symlinks=False) # snapshot: os-stat +5 | follow = True +6 | + - os.stat("foo", follow_symlinks=follow) # snapshot: os-stat +7 + pathlib.Path("foo").stat(follow_symlinks=follow) # snapshot: os-stat + | +note: This is an unsafe fix and may change runtime behavior +``` diff --git a/crates/ruff_linter/src/checkers/ast/analyze/expression.rs b/crates/ruff_linter/src/checkers/ast/analyze/expression.rs index feedfa8373..d30f5911b2 100644 --- a/crates/ruff_linter/src/checkers/ast/analyze/expression.rs +++ b/crates/ruff_linter/src/checkers/ast/analyze/expression.rs @@ -1102,7 +1102,6 @@ pub(crate) fn expression(expr: &Expr, checker: &Checker) { flake8_simplify::rules::zip_dict_keys_and_values(checker, call); } if checker.any_rule_enabled(&[ - Rule::OsStat, Rule::OsPathJoin, Rule::OsPathSplitext, Rule::PyPath, @@ -1185,6 +1184,9 @@ pub(crate) fn expression(expr: &Expr, checker: &Checker) { if checker.is_rule_enabled(Rule::OsMakedirs) { flake8_use_pathlib::rules::os_makedirs(checker, call, segments); } + if checker.is_rule_enabled(Rule::OsStat) { + flake8_use_pathlib::rules::os_stat(checker, call, segments); + } if checker.is_rule_enabled(Rule::OsSymlink) { flake8_use_pathlib::rules::os_symlink(checker, call, segments); } diff --git a/crates/ruff_linter/src/codes.rs b/crates/ruff_linter/src/codes.rs index e78302f81e..e750d2be52 100644 --- a/crates/ruff_linter/src/codes.rs +++ b/crates/ruff_linter/src/codes.rs @@ -956,7 +956,7 @@ pub fn code_to_rule(linter: Linter, code: &str) -> Option<(RuleGroup, Rule)> { (Flake8UsePathlib, "113") => rules::flake8_use_pathlib::rules::OsPathIsfile, (Flake8UsePathlib, "114") => rules::flake8_use_pathlib::rules::OsPathIslink, (Flake8UsePathlib, "115") => rules::flake8_use_pathlib::rules::OsReadlink, - (Flake8UsePathlib, "116") => rules::flake8_use_pathlib::violations::OsStat, + (Flake8UsePathlib, "116") => rules::flake8_use_pathlib::rules::OsStat, (Flake8UsePathlib, "117") => rules::flake8_use_pathlib::rules::OsPathIsabs, (Flake8UsePathlib, "118") => rules::flake8_use_pathlib::violations::OsPathJoin, (Flake8UsePathlib, "119") => rules::flake8_use_pathlib::rules::OsPathBasename, diff --git a/crates/ruff_linter/src/preview.rs b/crates/ruff_linter/src/preview.rs index 853063ec23..4c9da42e6f 100644 --- a/crates/ruff_linter/src/preview.rs +++ b/crates/ruff_linter/src/preview.rs @@ -190,6 +190,11 @@ pub(crate) const fn is_fix_os_makedirs_enabled(settings: &LinterSettings) -> boo settings.preview.is_enabled() } +// https://github.com/astral-sh/ruff/pull/26460 +pub(crate) const fn is_fix_os_stat_enabled(settings: &LinterSettings) -> bool { + settings.preview.is_enabled() +} + // https://github.com/astral-sh/ruff/pull/20009 pub(crate) const fn is_fix_os_symlink_enabled(settings: &LinterSettings) -> bool { settings.preview.is_enabled() diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/mod.rs b/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/mod.rs index 339d2c6a9e..2a262807a2 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/mod.rs +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/mod.rs @@ -25,6 +25,7 @@ pub(crate) use os_rename::*; pub(crate) use os_replace::*; pub(crate) use os_rmdir::*; pub(crate) use os_sep_split::*; +pub(crate) use os_stat::*; pub(crate) use os_symlink::*; pub(crate) use os_unlink::*; pub(crate) use path_constructor_current_directory::*; @@ -57,6 +58,7 @@ mod os_rename; mod os_replace; mod os_rmdir; mod os_sep_split; +mod os_stat; mod os_symlink; mod os_unlink; mod path_constructor_current_directory; diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/os_stat.rs b/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/os_stat.rs new file mode 100644 index 0000000000..c4229d2502 --- /dev/null +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/os_stat.rs @@ -0,0 +1,210 @@ +use std::fmt; + +use ruff_diagnostics::{Edit, Fix}; +use ruff_macros::{ViolationMetadata, derive_message_formats}; +use ruff_python_ast::{self as ast, ArgOrKeyword, Arguments, Expr, ExprCall, PythonVersion}; +use ruff_text_size::Ranged; + +use crate::{ + FixAvailability, Violation, + checkers::ast::Checker, + importer::ImportRequest, + preview::is_fix_os_stat_enabled, + rules::flake8_use_pathlib::helpers::{ + has_unknown_keywords_or_starred_expr, is_file_descriptor, + is_keyword_only_argument_non_default, is_pathlib_path_call, + }, +}; + +/// ## What it does +/// Checks for uses of `os.stat`. +/// +/// ## Why is this bad? +/// `pathlib` offers a high-level API for path manipulation, as compared to +/// the lower-level API offered by `os`. When possible, using `Path` object +/// methods such as `Path.stat()` can improve readability over the `os` +/// module's counterparts (e.g., `os.path.stat()`). +/// +/// ## Examples +/// ```python +/// import os +/// from pwd import getpwuid +/// from grp import getgrgid +/// +/// stat = os.stat(file_name) +/// owner_name = getpwuid(stat.st_uid).pw_name +/// group_name = getgrgid(stat.st_gid).gr_name +/// ``` +/// +/// Use instead: +/// ```python +/// from pathlib import Path +/// +/// file_path = Path(file_name) +/// stat = file_path.stat() +/// owner_name = file_path.owner() +/// group_name = file_path.group() +/// ``` +/// +/// ## Known issues +/// While using `pathlib` can improve the readability and type safety of your code, +/// it can be less performant than the lower-level alternatives that work directly with strings, +/// especially on older versions of Python. +/// +/// ## Fix Safety +/// This rule's fix is always marked as unsafe because `pathlib.Path` and `os.stat` differ in their +/// handling of `bytes` paths and file descriptors. +/// +/// ## References +/// - [Python documentation: `Path.stat`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.stat) +/// - [Python documentation: `Path.group`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.group) +/// - [Python documentation: `Path.owner`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.owner) +/// - [Python documentation: `os.stat`](https://docs.python.org/3/library/os.html#os.stat) +/// - [PEP 428 – The pathlib module – object-oriented filesystem paths](https://peps.python.org/pep-0428/) +/// - [Correspondence between `os` and `pathlib`](https://docs.python.org/3/library/pathlib.html#corresponding-tools) +/// - [Why you should be using pathlib](https://treyhunner.com/2018/12/why-you-should-be-using-pathlib/) +/// - [No really, pathlib is great](https://treyhunner.com/2019/01/no-really-pathlib-is-great/) +#[derive(ViolationMetadata)] +#[violation_metadata(stable_since = "v0.0.231")] +pub(crate) struct OsStat { + method: Option, +} + +impl Violation for OsStat { + const FIX_AVAILABILITY: FixAvailability = FixAvailability::Sometimes; + + #[derive_message_formats] + fn message(&self) -> String { + "`os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()`" + .to_string() + } + + fn fix_title(&self) -> Option { + self.method + .map(|method| format!("Replace with `Path(...).{method}()`")) + } +} + +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum StatMethod { + Stat, + LStat, +} + +impl StatMethod { + fn as_str(self) -> &'static str { + match self { + StatMethod::Stat => "stat", + StatMethod::LStat => "lstat", + } + } +} + +impl fmt::Display for StatMethod { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(self.as_str()) + } +} + +// PTH116 +pub(crate) fn os_stat(checker: &Checker, call: &ExprCall, segment: &[&str]) { + if segment != ["os", "stat"] { + return; + } + + // `dir_fd` is not supported by pathlib, so check if it's set to non-default values. + // Signature as of Python 3.13 (https://docs.python.org/3/library/os.html#os.stat) + // ```text + // 0 1 2 + // os.stat(path, *, dir_fd=None, follow_symlinks=True) + // ``` + if is_keyword_only_argument_non_default(&call.arguments, "dir_fd") { + return; + } + + let Some(path_args) = call.arguments.find_argument_value("path", 0) else { + return; + }; + + if is_file_descriptor(path_args, checker.semantic()) { + return; + } + + let method = if checker.target_version() >= PythonVersion::PY310 { + Some(StatMethod::Stat) + } else { + match is_boolean_literal_or_default(&call.arguments, "follow_symlinks") { + Some(true) => Some(StatMethod::Stat), + Some(false) => Some(StatMethod::LStat), + None => None, + } + }; + + let range = call.range(); + let mut diagnostic = checker.report_diagnostic(OsStat { method }, call.func.range()); + + if !is_fix_os_stat_enabled(checker.settings()) { + return; + } + + if has_unknown_keywords_or_starred_expr(&call.arguments, &["path", "dir_fd", "follow_symlinks"]) + { + return; + } + + let Some(method) = method else { + return; + }; + + diagnostic.try_set_fix(|| { + let (import_edit, binding) = checker.importer().get_or_import_symbol( + &ImportRequest::import("pathlib", "Path"), + call.start(), + checker.semantic(), + )?; + + let locator = checker.locator(); + let path_code = locator.slice(path_args.range()); + + let args = |arg: ArgOrKeyword| match arg { + ArgOrKeyword::Arg(expr) if expr.range() != path_args.range() => { + Some(locator.slice(expr.range())) + } + ArgOrKeyword::Keyword(kw) + if matches!(kw.arg.as_deref(), Some("follow_symlinks")) + && checker.target_version() >= PythonVersion::PY310 => + { + Some(locator.slice(kw.range())) + } + _ => None, + }; + + let stat_args = itertools::join(call.arguments.iter_source_order().filter_map(args), ", "); + + let replacement = if is_pathlib_path_call(checker, path_args) { + format!("{path_code}.{method}({stat_args})") + } else { + format!("{binding}({path_code}).{method}({stat_args})") + }; + + Ok(Fix::unsafe_edits( + Edit::range_replacement(replacement, range), + [import_edit], + )) + }); +} + +/// Returns the value of the given boolean keyword argument. +/// +/// If the keyword is omitted, returns `Some(true)` (its default value). +/// Returns `None` if the keyword argument is present but not a boolean literal. +fn is_boolean_literal_or_default(argument: &Arguments, name: &str) -> Option { + let Some(kw) = argument.find_keyword(name) else { + return Some(true); + }; + + match &kw.value { + Expr::BooleanLiteral(ast::ExprBooleanLiteral { value, .. }) => Some(*value), + _ => None, + } +} diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/replaceable_by_pathlib.rs b/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/replaceable_by_pathlib.rs index 5ecef9398c..15257a3bb1 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/replaceable_by_pathlib.rs +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/rules/replaceable_by_pathlib.rs @@ -7,7 +7,7 @@ use crate::rules::flake8_use_pathlib::helpers::{ }; use crate::rules::flake8_use_pathlib::{ rules::Glob, - violations::{Joiner, OsListdir, OsPathJoin, OsPathSplitext, OsStat, PyPath}, + violations::{Joiner, OsListdir, OsPathJoin, OsPathSplitext, PyPath}, }; pub(crate) fn replaceable_by_pathlib(checker: &Checker, call: &ExprCall) { @@ -17,24 +17,6 @@ pub(crate) fn replaceable_by_pathlib(checker: &Checker, call: &ExprCall) { let range = call.func.range(); match qualified_name.segments() { - // PTH116 - ["os", "stat"] => { - // `dir_fd` is not supported by pathlib, so check if it's set to non-default values. - // Signature as of Python 3.13 (https://docs.python.org/3/library/os.html#os.stat) - // ```text - // 0 1 2 - // os.stat(path, *, dir_fd=None, follow_symlinks=True) - // ``` - if call - .arguments - .find_argument_value("path", 0) - .is_some_and(|expr| is_file_descriptor(expr, checker.semantic())) - || is_keyword_only_argument_non_default(&call.arguments, "dir_fd") - { - return; - } - checker.report_diagnostic_if_enabled(OsStat, range) - } // PTH118 ["os", "path", "join"] => checker.report_diagnostic_if_enabled( OsPathJoin { diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__full_name.py.snap b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__full_name.py.snap index bc4e59683c..2b0cad24a1 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__full_name.py.snap +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__full_name.py.snap @@ -202,6 +202,7 @@ PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path 24 | os.path.isabs(p) 25 | os.path.join(p, q) | +help: Replace with `Path(...).stat()` PTH117 `os.path.isabs()` should be replaced by `Path.is_absolute()` --> full_name.py:24:1 diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_as.py.snap b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_as.py.snap index 89653f3915..5c6884fdd6 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_as.py.snap +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_as.py.snap @@ -202,6 +202,7 @@ PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path 24 | foo_p.isabs(p) 25 | foo_p.join(p, q) | +help: Replace with `Path(...).stat()` PTH117 `os.path.isabs()` should be replaced by `Path.is_absolute()` --> import_as.py:24:1 diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_from.py.snap b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_from.py.snap index 3482761ca3..3f1937c0f5 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_from.py.snap +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_from.py.snap @@ -202,6 +202,7 @@ PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path 26 | isabs(p) 27 | join(p, q) | +help: Replace with `Path(...).stat()` PTH117 `os.path.isabs()` should be replaced by `Path.is_absolute()` --> import_from.py:26:1 diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_from_as.py.snap b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_from_as.py.snap index 224ed57fba..383e6cf8c2 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_from_as.py.snap +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__import_from_as.py.snap @@ -202,6 +202,7 @@ PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path 31 | xisabs(p) 32 | xjoin(p, q) | +help: Replace with `Path(...).stat()` PTH117 `os.path.isabs()` should be replaced by `Path.is_absolute()` --> import_from_as.py:31:1 diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_full_name.py.snap b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_full_name.py.snap index 968f293563..b66c9d1543 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_full_name.py.snap +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_full_name.py.snap @@ -315,7 +315,7 @@ help: Replace with `Path(...).readlink()` 24 | os.stat(p) | -PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` +PTH116 [*] `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` --> full_name.py:23:1 | 21 | bbbbb = os.path.islink(p) @@ -325,6 +325,18 @@ PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path 24 | os.path.isabs(p) 25 | os.path.join(p, q) | +help: Replace with `Path(...).stat()` + | +2 | import os.path +3 + import pathlib +4 | +-------------------------------------------------------------------------------- +23 | os.readlink(p) + - os.stat(p) +24 + pathlib.Path(p).stat() +25 | os.path.isabs(p) + | +note: This is an unsafe fix and may change runtime behavior PTH117 [*] `os.path.isabs()` should be replaced by `Path.is_absolute()` --> full_name.py:24:1 diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_as.py.snap b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_as.py.snap index 327e1a1bc0..3e79328059 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_as.py.snap +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_as.py.snap @@ -315,7 +315,7 @@ help: Replace with `Path(...).readlink()` 24 | foo.stat(p) | -PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` +PTH116 [*] `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` --> import_as.py:23:1 | 21 | bbbbb = foo_p.islink(p) @@ -325,6 +325,18 @@ PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path 24 | foo_p.isabs(p) 25 | foo_p.join(p, q) | +help: Replace with `Path(...).stat()` + | +2 | import os.path as foo_p +3 + import pathlib +4 | +-------------------------------------------------------------------------------- +23 | foo.readlink(p) + - foo.stat(p) +24 + pathlib.Path(p).stat() +25 | foo_p.isabs(p) + | +note: This is an unsafe fix and may change runtime behavior PTH117 [*] `os.path.isabs()` should be replaced by `Path.is_absolute()` --> import_as.py:24:1 diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_from.py.snap b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_from.py.snap index 0f889292ca..8098596c97 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_from.py.snap +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_from.py.snap @@ -315,7 +315,7 @@ help: Replace with `Path(...).readlink()` 26 | stat(p) | -PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` +PTH116 [*] `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` --> import_from.py:25:1 | 23 | bbbbb = islink(p) @@ -325,6 +325,18 @@ PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path 26 | isabs(p) 27 | join(p, q) | +help: Replace with `Path(...).stat()` + | +4 | from os.path import isabs, join, basename, dirname, samefile, splitext +5 + import pathlib +6 | +-------------------------------------------------------------------------------- +25 | readlink(p) + - stat(p) +26 + pathlib.Path(p).stat() +27 | isabs(p) + | +note: This is an unsafe fix and may change runtime behavior PTH117 [*] `os.path.isabs()` should be replaced by `Path.is_absolute()` --> import_from.py:26:1 diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_from_as.py.snap b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_from_as.py.snap index 2e5f5bcf1a..bc50264039 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_from_as.py.snap +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/snapshots/ruff_linter__rules__flake8_use_pathlib__tests__preview_import_from_as.py.snap @@ -315,7 +315,7 @@ help: Replace with `Path(...).readlink()` 31 | xstat(p) | -PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` +PTH116 [*] `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()` --> import_from_as.py:30:1 | 28 | bbbbb = xislink(p) @@ -325,6 +325,18 @@ PTH116 `os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path 31 | xisabs(p) 32 | xjoin(p, q) | +help: Replace with `Path(...).stat()` + | +9 | from os.path import samefile as xsamefile, splitext as xsplitext +10 + import pathlib +11 | +-------------------------------------------------------------------------------- +30 | xreadlink(p) + - xstat(p) +31 + pathlib.Path(p).stat() +32 | xisabs(p) + | +note: This is an unsafe fix and may change runtime behavior PTH117 [*] `os.path.isabs()` should be replaced by `Path.is_absolute()` --> import_from_as.py:31:1 diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/violations.rs b/crates/ruff_linter/src/rules/flake8_use_pathlib/violations.rs index b5bcfdb1e3..1310bf2c86 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/violations.rs +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/violations.rs @@ -2,62 +2,6 @@ use ruff_macros::{ViolationMetadata, derive_message_formats}; use crate::Violation; -/// ## What it does -/// Checks for uses of `os.stat`. -/// -/// ## Why is this bad? -/// `pathlib` offers a high-level API for path manipulation, as compared to -/// the lower-level API offered by `os`. When possible, using `Path` object -/// methods such as `Path.stat()` can improve readability over the `os` -/// module's counterparts (e.g., `os.path.stat()`). -/// -/// ## Examples -/// ```python -/// import os -/// from pwd import getpwuid -/// from grp import getgrgid -/// -/// stat = os.stat(file_name) -/// owner_name = getpwuid(stat.st_uid).pw_name -/// group_name = getgrgid(stat.st_gid).gr_name -/// ``` -/// -/// Use instead: -/// ```python -/// from pathlib import Path -/// -/// file_path = Path(file_name) -/// stat = file_path.stat() -/// owner_name = file_path.owner() -/// group_name = file_path.group() -/// ``` -/// -/// ## Known issues -/// While using `pathlib` can improve the readability and type safety of your code, -/// it can be less performant than the lower-level alternatives that work directly with strings, -/// especially on older versions of Python. -/// -/// ## References -/// - [Python documentation: `Path.stat`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.stat) -/// - [Python documentation: `Path.group`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.group) -/// - [Python documentation: `Path.owner`](https://docs.python.org/3/library/pathlib.html#pathlib.Path.owner) -/// - [Python documentation: `os.stat`](https://docs.python.org/3/library/os.html#os.stat) -/// - [PEP 428 – The pathlib module – object-oriented filesystem paths](https://peps.python.org/pep-0428/) -/// - [Correspondence between `os` and `pathlib`](https://docs.python.org/3/library/pathlib.html#corresponding-tools) -/// - [Why you should be using pathlib](https://treyhunner.com/2018/12/why-you-should-be-using-pathlib/) -/// - [No really, pathlib is great](https://treyhunner.com/2019/01/no-really-pathlib-is-great/) -#[derive(ViolationMetadata)] -#[violation_metadata(stable_since = "v0.0.231")] -pub(crate) struct OsStat; - -impl Violation for OsStat { - #[derive_message_formats] - fn message(&self) -> String { - "`os.stat()` should be replaced by `Path.stat()`, `Path.owner()`, or `Path.group()`" - .to_string() - } -} - /// ## What it does /// Checks for uses of `os.path.join`. /// From 5226c063128b63ecb3f035fd9a0b0b4b007d615c Mon Sep 17 00:00:00 2001 From: Lakshay Saini <76612216+lakshayxi@users.noreply.github.com> Date: Sat, 15 Aug 2026 00:58:20 +0530 Subject: [PATCH 044/371] [`pylint`] Allow `os._exit` imports in `import-private-name` (`PLC2701`) (#27738) ## Summary `PLC2701` currently reports `from os import _exit` as a private import, even though `os._exit` is a documented public API. `SLF001` already handles `os._exit` as an exception to the usual leading-underscore rule. This change moves that exception into a shared helper so `PLC2701` and `SLF001` treat `os._exit` consistently. Fixes #18143. ## Test Plan - Added tests for direct and aliased `os._exit` imports. - Added negative cases to make sure `_exit` from other modules and other private `os` members are still reported. - Added coverage for mixed imports and aliased `SLF001` access. - Ran the relevant Ruff tests, Clippy, formatting checks, and `prek`. --- .../test/fixtures/flake8_self/SLF001.py | 4 +++ .../import_private_name/submodule/__main__.py | 7 ++++ .../rules/private_member_access.rs | 6 ++-- .../ruff_linter/src/rules/pylint/helpers.rs | 7 ++++ .../rules/pylint/rules/import_private_name.rs | 5 +++ ..._private_name__submodule____main__.py.snap | 33 ++++++++++++++++++- 6 files changed, 59 insertions(+), 3 deletions(-) diff --git a/crates/ruff_linter/resources/test/fixtures/flake8_self/SLF001.py b/crates/ruff_linter/resources/test/fixtures/flake8_self/SLF001.py index 96e178cac2..389dd927da 100644 --- a/crates/ruff_linter/resources/test/fixtures/flake8_self/SLF001.py +++ b/crates/ruff_linter/resources/test/fixtures/flake8_self/SLF001.py @@ -78,6 +78,10 @@ def __eq__(self, other): os._exit() +import os as operating_system + +operating_system._exit(1) + from enum import Enum diff --git a/crates/ruff_linter/resources/test/fixtures/pylint/import_private_name/submodule/__main__.py b/crates/ruff_linter/resources/test/fixtures/pylint/import_private_name/submodule/__main__.py index 17e5cc7d3d..c26c5376aa 100644 --- a/crates/ruff_linter/resources/test/fixtures/pylint/import_private_name/submodule/__main__.py +++ b/crates/ruff_linter/resources/test/fixtures/pylint/import_private_name/submodule/__main__.py @@ -50,3 +50,10 @@ def generic[T: _nn](arg: T) -> T: return arg from foo. _bar import baz + +# PLC2701 exceptions: `os._exit` is considered public despite leading underscore. +from os import _exit +from os import _exit as process_exit +from another_module import _exit as another_exit +from os import _private_member +from os import _exit as os_exit, _other_private_member diff --git a/crates/ruff_linter/src/rules/flake8_self/rules/private_member_access.rs b/crates/ruff_linter/src/rules/flake8_self/rules/private_member_access.rs index 162b9b8760..d2d1242bde 100644 --- a/crates/ruff_linter/src/rules/flake8_self/rules/private_member_access.rs +++ b/crates/ruff_linter/src/rules/flake8_self/rules/private_member_access.rs @@ -10,7 +10,9 @@ use ruff_text_size::Ranged; use crate::Violation; use crate::checkers::ast::Checker; -use crate::rules::pylint::helpers::is_dunder_operator_method; +use crate::rules::pylint::helpers::{ + is_dunder_operator_method, is_underscore_prefixed_public_member, +}; /// ## What it does /// Checks for accesses on "private" class members. @@ -104,7 +106,7 @@ pub(crate) fn private_member_access(checker: &Checker, expr: &Expr) { // Allow some public functions whose names start with an underscore, like `os._exit()`. if let Some(qualified_name) = semantic.resolve_qualified_name(expr) { - if matches!(qualified_name.segments(), ["os", "_exit"]) { + if is_underscore_prefixed_public_member(&qualified_name) { return; } } diff --git a/crates/ruff_linter/src/rules/pylint/helpers.rs b/crates/ruff_linter/src/rules/pylint/helpers.rs index e3ac9bb5e8..2ee6f0bd5b 100644 --- a/crates/ruff_linter/src/rules/pylint/helpers.rs +++ b/crates/ruff_linter/src/rules/pylint/helpers.rs @@ -1,5 +1,6 @@ use ruff_python_ast as ast; use ruff_python_ast::ExceptHandler; +use ruff_python_ast::name::QualifiedName; use ruff_python_ast::visitor::Visitor; use ruff_python_ast::{Arguments, Expr, Stmt, visitor}; use ruff_python_semantic::analyze::function_type; @@ -8,6 +9,12 @@ use ruff_text_size::TextRange; use crate::settings::LinterSettings; +/// Returns `true` if a module member is public despite having an +/// underscore-prefixed name. +pub(crate) fn is_underscore_prefixed_public_member(qualified_name: &QualifiedName) -> bool { + matches!(qualified_name.segments(), ["os", "_exit"]) +} + /// Returns the value of the `name` parameter to, e.g., a `TypeVar` constructor. pub(super) fn type_param_name(arguments: &Arguments) -> Option<&str> { // Handle both `TypeVar("T")` and `TypeVar(name="T")`. diff --git a/crates/ruff_linter/src/rules/pylint/rules/import_private_name.rs b/crates/ruff_linter/src/rules/pylint/rules/import_private_name.rs index 82c4da9d05..ab193a4a70 100644 --- a/crates/ruff_linter/src/rules/pylint/rules/import_private_name.rs +++ b/crates/ruff_linter/src/rules/pylint/rules/import_private_name.rs @@ -11,6 +11,7 @@ use ruff_text_size::Ranged; use crate::Violation; use crate::checkers::ast::Checker; use crate::package::PackageRoot; +use crate::rules::pylint::helpers::is_underscore_prefixed_public_member; /// ## What it does /// Checks for import statements that import a private name (a name starting @@ -122,6 +123,10 @@ pub(crate) fn import_private_name(checker: &Checker, scope: &Scope) { continue; }; + if is_underscore_prefixed_public_member(import_info.qualified_name) { + continue; + } + // Ignore private imports used exclusively for typing. if !binding.references.is_empty() && binding diff --git a/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLC2701_import_private_name__submodule____main__.py.snap b/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLC2701_import_private_name__submodule____main__.py.snap index 4b0c527088..7430839d66 100644 --- a/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLC2701_import_private_name__submodule____main__.py.snap +++ b/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLC2701_import_private_name__submodule____main__.py.snap @@ -1,6 +1,5 @@ --- source: crates/ruff_linter/src/rules/pylint/mod.rs -assertion_line: 256 --- PLC2701 Private name import `_a` --> __main__.py:2:6 @@ -84,3 +83,35 @@ PLC2701 Private name import `_bar` from external module `foo` 51 | 52 | from foo. _bar import baz | ^^^^ +53 | +54 | # PLC2701 exceptions: `os._exit` is considered public despite leading underscore. + | + +PLC2701 Private name import `_exit` from external module `another_module` + --> __main__.py:57:28 + | +55 | from os import _exit +56 | from os import _exit as process_exit +57 | from another_module import _exit as another_exit + | ^^^^^ +58 | from os import _private_member +59 | from os import _exit as os_exit, _other_private_member + | + +PLC2701 Private name import `_private_member` from external module `os` + --> __main__.py:58:16 + | +56 | from os import _exit as process_exit +57 | from another_module import _exit as another_exit +58 | from os import _private_member + | ^^^^^^^^^^^^^^^ +59 | from os import _exit as os_exit, _other_private_member + | + +PLC2701 Private name import `_other_private_member` from external module `os` + --> __main__.py:59:34 + | +57 | from another_module import _exit as another_exit +58 | from os import _private_member +59 | from os import _exit as os_exit, _other_private_member + | ^^^^^^^^^^^^^^^^^^^^^ From 89ac93b12650197c9e262780a34c6ebe66657999 Mon Sep 17 00:00:00 2001 From: august Date: Fri, 14 Aug 2026 12:37:57 -0700 Subject: [PATCH 045/371] [`refurb`] Restrict `delete-full-slice` to lists (`FURB131`) (#27711) Fixes #27216 Co-authored-by: Brent Westbrook <36778786+ntBre@users.noreply.github.com> --- .../rules/refurb/rules/delete_full_slice.rs | 21 ++---- ...es__refurb__tests__FURB131_FURB131.py.snap | 74 ------------------- 2 files changed, 8 insertions(+), 87 deletions(-) diff --git a/crates/ruff_linter/src/rules/refurb/rules/delete_full_slice.rs b/crates/ruff_linter/src/rules/refurb/rules/delete_full_slice.rs index 16a66f8a40..fd55449fe2 100644 --- a/crates/ruff_linter/src/rules/refurb/rules/delete_full_slice.rs +++ b/crates/ruff_linter/src/rules/refurb/rules/delete_full_slice.rs @@ -1,7 +1,7 @@ use ruff_macros::{ViolationMetadata, derive_message_formats}; use ruff_python_ast::{self as ast, Expr}; use ruff_python_semantic::SemanticModel; -use ruff_python_semantic::analyze::typing::{is_dict, is_list}; +use ruff_python_semantic::analyze::typing::is_list; use ruff_text_size::Ranged; use crate::checkers::ast::Checker; @@ -10,8 +10,7 @@ use crate::{Edit, Fix, FixAvailability, Violation}; use crate::rules::refurb::helpers::generate_method_call; /// ## What it does -/// Checks for `del` statements that delete the entire slice of a list or -/// dictionary. +/// Checks for `del` statements that delete the entire slice of a list. /// /// ## Why is this bad? /// It is faster and more succinct to remove all items via the `clear()` @@ -19,30 +18,26 @@ use crate::rules::refurb::helpers::generate_method_call; /// /// ## Known problems /// This rule is prone to false negatives due to type inference limitations, -/// as it will only detect lists and dictionaries that are instantiated as -/// literals or annotated with a type annotation. +/// as it will only detect lists that are instantiated as literals or annotated +/// with a type annotation. /// /// ## Example /// ```python -/// names = {"key": "value"} /// nums = [1, 2, 3] /// -/// del names[:] /// del nums[:] /// ``` /// /// Use instead: /// ```python -/// names = {"key": "value"} /// nums = [1, 2, 3] /// -/// names.clear() /// nums.clear() /// ``` /// /// ## References /// - [Python documentation: Mutable Sequence Types](https://docs.python.org/3/library/stdtypes.html#typesseq-mutable) -/// - [Python documentation: `dict.clear()`](https://docs.python.org/3/library/stdtypes.html#dict.clear) +/// - [Python documentation: `list.clear()`](https://docs.python.org/3/library/stdtypes.html#sequence.clear) #[derive(ViolationMetadata)] #[violation_metadata(preview_since = "v0.0.287")] pub(crate) struct DeleteFullSlice; @@ -81,7 +76,7 @@ pub(crate) fn delete_full_slice(checker: &Checker, delete: &ast::StmtDelete) { } } -/// Match `del expr[:]` where `expr` is a list or a dict. +/// Match `del expr[:]` where `expr` is a list. fn match_full_slice<'a>(expr: &'a Expr, semantic: &SemanticModel) -> Option<&'a ast::ExprName> { // Check that it is `del expr[...]`. let subscript = expr.as_subscript_expr()?; @@ -100,10 +95,10 @@ fn match_full_slice<'a>(expr: &'a Expr, semantic: &SemanticModel) -> Option<&'a return None; } - // It should only apply to variables that are known to be lists or dicts. + // It should only apply to variables that are known to be lists. let name = subscript.value.as_name_expr()?; let binding = semantic.binding(semantic.only_binding(name)?); - if !(is_dict(binding, semantic) || is_list(binding, semantic)) { + if !is_list(binding, semantic) { return None; } diff --git a/crates/ruff_linter/src/rules/refurb/snapshots/ruff_linter__rules__refurb__tests__FURB131_FURB131.py.snap b/crates/ruff_linter/src/rules/refurb/snapshots/ruff_linter__rules__refurb__tests__FURB131_FURB131.py.snap index eed82175cc..edcbd44630 100644 --- a/crates/ruff_linter/src/rules/refurb/snapshots/ruff_linter__rules__refurb__tests__FURB131_FURB131.py.snap +++ b/crates/ruff_linter/src/rules/refurb/snapshots/ruff_linter__rules__refurb__tests__FURB131_FURB131.py.snap @@ -16,21 +16,6 @@ help: Replace with `clear()` | note: This is an unsafe fix and may change runtime behavior -FURB131 [*] Prefer `clear` over deleting a full slice - --> FURB131.py:15:1 - | -14 | # FURB131 -15 | del names[:] - | ^^^^^^^^^^^^ -help: Replace with `clear()` - | -14 | # FURB131 - - del names[:] -15 + names.clear() -16 | - | -note: This is an unsafe fix and may change runtime behavior - FURB131 Prefer `clear` over deleting a full slice --> FURB131.py:19:1 | @@ -39,14 +24,6 @@ FURB131 Prefer `clear` over deleting a full slice | ^^^^^^^^^^^^^^ help: Replace with `clear()` -FURB131 Prefer `clear` over deleting a full slice - --> FURB131.py:23:1 - | -22 | # FURB131 -23 | del y, names[:], x - | ^^^^^^^^^^^^^^^^^^ -help: Replace with `clear()` - FURB131 [*] Prefer `clear` over deleting a full slice --> FURB131.py:28:5 | @@ -63,22 +40,6 @@ help: Replace with `clear()` | note: This is an unsafe fix and may change runtime behavior -FURB131 [*] Prefer `clear` over deleting a full slice - --> FURB131.py:33:5 - | -31 | def yes_two(x: dict[int, str]): -32 | # FURB131 -33 | del x[:] - | ^^^^^^^^ -help: Replace with `clear()` - | -32 | # FURB131 - - del x[:] -33 + x.clear() -34 | - | -note: This is an unsafe fix and may change runtime behavior - FURB131 [*] Prefer `clear` over deleting a full slice --> FURB131.py:38:5 | @@ -95,41 +56,6 @@ help: Replace with `clear()` | note: This is an unsafe fix and may change runtime behavior -FURB131 [*] Prefer `clear` over deleting a full slice - --> FURB131.py:43:5 - | -41 | def yes_four(x: Dict[int, str]): -42 | # FURB131 -43 | del x[:] - | ^^^^^^^^ -help: Replace with `clear()` - | -42 | # FURB131 - - del x[:] -43 + x.clear() -44 | - | -note: This is an unsafe fix and may change runtime behavior - -FURB131 [*] Prefer `clear` over deleting a full slice - --> FURB131.py:48:5 - | -46 | def yes_five(x: Dict[int, str]): -47 | # FURB131 -48 | del x[:] - | ^^^^^^^^ -49 | -50 | x = 1 - | -help: Replace with `clear()` - | -47 | # FURB131 - - del x[:] -48 + x.clear() -49 | - | -note: This is an unsafe fix and may change runtime behavior - FURB131 [*] Prefer `clear` over deleting a full slice --> FURB131.py:58:1 | From 19e03975cadc3014c7640740ecc275b055de691e Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Fri, 14 Aug 2026 12:52:11 -0700 Subject: [PATCH 046/371] [ty] Fix specialization of generic TypedDict aliases (#27760) Legacy generic aliases and functions failed to discover type variables that appeared only inside a generic `TypedDict`. Alias specialization could also incorrectly classify an alias as recursive when a `TypedDict` member contained an unrelated recursive field. Together, these bugs prevented valid imported tagged-union aliases from specializing and produced false-positive diagnostics. Collect legacy type variables from class-based `TypedDict` definitions, and restrict recursive-alias detection to the alias structure and its generic arguments. Update existing overload fixtures to preserve their diagnostic coverage now that `Self` inside a `TypedDict` is correctly discovered. Closes astral-sh/ty#4255. ## Test plan - Generic `TypedDict` aliases in unions and containers, including specialized fields, default specialization, type-variable ordering, and repeated variables. - Imported tagged-union aliases from stubs, including discriminator narrowing and concrete field access. - Recursive `TypedDict` fields that do not make their containing alias recursive. - Generic functions whose type variable appears only in a `TypedDict` parameter. - Existing overload diagnostic-context scenarios after correcting nested `Self` discovery. --- .../mdtest/diagnostics/error_context.md | 65 ++++----- .../mdtest/generics/legacy/functions.md | 21 +++ .../resources/mdtest/implicit_type_aliases.md | 126 ++++++++++++++++++ crates/ty_python_semantic/src/types.rs | 11 +- .../types/infer/builder/type_expression.rs | 5 +- 5 files changed, 193 insertions(+), 35 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/diagnostics/error_context.md b/crates/ty_python_semantic/resources/mdtest/diagnostics/error_context.md index 43e9a702fb..7fee42a91d 100644 --- a/crates/ty_python_semantic/resources/mdtest/diagnostics/error_context.md +++ b/crates/ty_python_semantic/resources/mdtest/diagnostics/error_context.md @@ -881,12 +881,12 @@ help: A subclass of `Empty` could validly add a new field of an arbitrary type, ## Generic `TypedDict` field conflicts in overload diagnostics -A generic `TypedDict` relation can be unsatisfiable without being the `never` terminal. The -resulting overload diagnostic should still explain which field introduced the conflicting -constraints. +A generic `TypedDict` relation can be unsatisfiable without being the `never` terminal. Capturing +its type variable from an enclosing function keeps the overload non-generic while retaining the +conflicting constraints. The resulting diagnostic should explain which field introduced them. ```py -from typing import Generic, Self, TypeVar, TypedDict, overload +from typing import Generic, TypeVar, TypedDict, overload T = TypeVar("T") @@ -898,38 +898,39 @@ class Fixed(TypedDict): first: int second: str -class OverloadedSelf: +def outer(value: T) -> None: @overload - def method(self, value: Fixed) -> None: ... # snapshot: invalid-overload + def inner(value: Fixed) -> None: ... # snapshot: invalid-overload @overload - def method(self, value: str) -> None: ... - def method(self, value: Pair[Self] | str) -> None: ... + def inner(value: str) -> None: ... + def inner(value: Pair[T] | str) -> None: ... ``` ```snapshot error[invalid-overload]: Implementation does not accept all arguments of this overload --> src/mdtest_snippet.py:15:9 | -15 | def method(self, value: Fixed) -> None: ... # snapshot: invalid-overload - | ^^^^^^ +15 | def inner(value: Fixed) -> None: ... # snapshot: invalid-overload + | ^^^^^ 16 | @overload -17 | def method(self, value: str) -> None: ... -18 | def method(self, value: Pair[Self] | str) -> None: ... - | ------ Implementation defined here -info: Implementation signature `(self, value: Pair[Self@method] | str) -> None` is not assignable to overload signature `(self, value: Fixed) -> None` -info: parameter `value` has an incompatible type: `Fixed` is not assignable to `Pair[Self@method] | str` -info: └── type `Fixed` is not assignable to any element of the union `Pair[Self@method] | str` -info: ├── field "second" on TypedDict `Fixed` has type `str` which is not assignable to type `Self@method` expected by TypedDict `Pair` +17 | def inner(value: str) -> None: ... +18 | def inner(value: Pair[T] | str) -> None: ... + | ----- Implementation defined here +info: Implementation signature `(value: Pair[T@outer] | str) -> None` is not assignable to overload signature `(value: Fixed) -> None` +info: parameter `value` has an incompatible type: `Fixed` is not assignable to `Pair[T@outer] | str` +info: └── type `Fixed` is not assignable to any element of the union `Pair[T@outer] | str` +info: ├── field "second" on TypedDict `Fixed` has type `str` which is not assignable to type `T@outer` expected by TypedDict `Pair` info: └── ... omitted 1 union element without additional context ``` ## Stop checking callable parameters after incompatible generic constraints Once earlier parameters produce an unsatisfiable nonterminal constraint set, continuing to a later -parameter must not replace the diagnostic context that explains the original incompatibility. +parameter must not replace the diagnostic context that explains the original incompatibility. The +type variable belongs to the enclosing function, so the overload itself remains non-generic. ```py -from typing import Generic, Self, TypeVar, TypedDict, overload +from typing import Generic, TypeVar, TypedDict, overload T = TypeVar("T") @@ -941,28 +942,28 @@ class Fixed(TypedDict): first: int second: str -class OverloadedSelf: +def outer(value: T) -> None: @overload - def method(self, value: Fixed, later: int) -> None: ... # snapshot: invalid-overload + def inner(value: Fixed, later: int) -> None: ... # snapshot: invalid-overload @overload - def method(self, value: str, later: str) -> None: ... - def method(self, value: Pair[Self] | str, later: str) -> None: ... + def inner(value: str, later: str) -> None: ... + def inner(value: Pair[T] | str, later: str) -> None: ... ``` ```snapshot error[invalid-overload]: Implementation does not accept all arguments of this overload --> src/mdtest_snippet.py:15:9 | -15 | def method(self, value: Fixed, later: int) -> None: ... # snapshot: invalid-overload - | ^^^^^^ +15 | def inner(value: Fixed, later: int) -> None: ... # snapshot: invalid-overload + | ^^^^^ 16 | @overload -17 | def method(self, value: str, later: str) -> None: ... -18 | def method(self, value: Pair[Self] | str, later: str) -> None: ... - | ------ Implementation defined here -info: Implementation signature `(self, value: Pair[Self@method] | str, later: str) -> None` is not assignable to overload signature `(self, value: Fixed, later: int) -> None` -info: parameter `value` has an incompatible type: `Fixed` is not assignable to `Pair[Self@method] | str` -info: └── type `Fixed` is not assignable to any element of the union `Pair[Self@method] | str` -info: ├── field "second" on TypedDict `Fixed` has type `str` which is not assignable to type `Self@method` expected by TypedDict `Pair` +17 | def inner(value: str, later: str) -> None: ... +18 | def inner(value: Pair[T] | str, later: str) -> None: ... + | ----- Implementation defined here +info: Implementation signature `(value: Pair[T@outer] | str, later: str) -> None` is not assignable to overload signature `(value: Fixed, later: int) -> None` +info: parameter `value` has an incompatible type: `Fixed` is not assignable to `Pair[T@outer] | str` +info: └── type `Fixed` is not assignable to any element of the union `Pair[T@outer] | str` +info: ├── field "second" on TypedDict `Fixed` has type `str` which is not assignable to type `T@outer` expected by TypedDict `Pair` info: └── ... omitted 1 union element without additional context ``` diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md index 7858e69fd5..9008b40c94 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md @@ -170,6 +170,27 @@ def pick(x: object) -> str | bool: reveal_type(pick([1])) # revealed: bool ``` +## Inferring generic typed-dictionary parameters + +A type variable that appears only inside a typed dictionary still makes the function generic, so +specialized typed dictionaries can be passed to it. + +```py +from typing import Generic, TypeVar, TypedDict + +T = TypeVar("T") + +class Item(TypedDict, Generic[T]): + value: T + +def accept(value: Item[T]) -> None: ... + +item: Item[int] = {"value": 1} + +reveal_type(accept) # revealed: def accept[T](value: Item[T]) -> None +accept(item) +``` + ## Inferring a class-object parameter through a generic factory A factory can infer its type arguments from a specialized subclass of its class-object parameter. diff --git a/crates/ty_python_semantic/resources/mdtest/implicit_type_aliases.md b/crates/ty_python_semantic/resources/mdtest/implicit_type_aliases.md index 601b9fa1f1..c169ed97af 100644 --- a/crates/ty_python_semantic/resources/mdtest/implicit_type_aliases.md +++ b/crates/ty_python_semantic/resources/mdtest/implicit_type_aliases.md @@ -632,6 +632,73 @@ class Derived2(GenericBaseAlias[int]): pass ``` +### Generic typed dictionaries in aliases + +First, define a generic typed dictionary whose field uses a legacy type variable: + +```py +from typing import Generic, TypeVar, TypedDict + +T = TypeVar("T") + +class Item(TypedDict, Generic[T]): + value: T +``` + +An implicit union alias remains generic when its only type variable appears in the typed dictionary, +and specialization preserves the dictionary's field type: + +```py +OptionalItem = Item[T] | None + +def _(item: OptionalItem[int]): + reveal_type(item) # revealed: Item[int] | None + + if item is not None: + reveal_type(item["value"]) # revealed: int +``` + +Without an explicit type argument, the alias uses the type variable's default specialization: + +```py +def _(item: OptionalItem): + reveal_type(item) # revealed: Item[Unknown] | None +``` + +The type variable is also discovered when the typed dictionary is nested inside a container: + +```py +Items = list[Item[T]] + +def _(items: Items[str]): + reveal_type(items) # revealed: list[Item[str]] + reveal_type(items[0]["value"]) # revealed: str +``` + +### Type-variable order in generic typed-dictionary aliases + +Type variables that appear inside a typed dictionary are collected in the same order as variables in +other union members: + +```py +from typing import Generic, TypeVar, TypedDict + +T = TypeVar("T") +U = TypeVar("U") + +class Item(TypedDict, Generic[T]): + value: T + +TypedDictFirst = Item[T] | list[U] +TypedDictLast = list[U] | Item[T] +Repeated = Item[T] | list[T] + +def _(first: TypedDictFirst[int, str], last: TypedDictLast[str, int], repeated: Repeated[int]): + reveal_type(first) # revealed: Item[int] | list[str] + reveal_type(last) # revealed: list[str] | Item[int] + reveal_type(repeated) # revealed: Item[int] | list[int] +``` + ### Imported aliases Generic implicit type aliases can be imported from other modules and specialized: @@ -664,6 +731,45 @@ def _( reveal_type(list_of_str_or_none) # revealed: list[str] | None ``` +### Imported tagged typed-dictionary aliases + +A stub can define a generic tagged union containing a typed dictionary and expose a concrete +specialization through a function signature: + +`events.pyi`: + +```pyi +from typing import Generic, Literal, TypeVar, TypedDict, Union + +T = TypeVar("T") + +class ObjectEvent(TypedDict, Generic[T]): + type: Literal["ADDED"] + object: T + +class BookmarkEvent(TypedDict): + type: Literal["BOOKMARK"] + object: object + +DecodedEvent = Union[ObjectEvent[T], BookmarkEvent, None] + +def get_event() -> DecodedEvent[int]: ... +``` + +After excluding the other union members, Python code sees the concrete typed-dictionary field type: + +`main.py`: + +```py +from events import get_event + +event = get_event() +if event is not None and event["type"] != "BOOKMARK": + reveal_type(event) # revealed: ObjectEvent[int] + reveal_type(event["object"]) # revealed: int + event["object"].bit_length() +``` + ### In stringified annotations Generic implicit type aliases can be specialized in stringified annotations: @@ -1974,6 +2080,26 @@ def _( reveal_type(recursive_dict4) # revealed: dict[Divergent, int] ``` +### Recursive typed-dictionary fields in generic aliases + +A recursive field on a non-generic typed dictionary does not make an unrelated enclosing generic +alias recursive: + +```py +from typing import TypeVar, TypedDict + +T = TypeVar("T") +RecursiveList = list["RecursiveList | None"] + +class Payload(TypedDict): + value: RecursiveList + +ListOrPayload = list[T] | Payload + +def _(value: ListOrPayload[int]): + reveal_type(value) # revealed: list[int] | Payload +``` + ### Self-referential generic implicit type aliases ```py diff --git a/crates/ty_python_semantic/src/types.rs b/crates/ty_python_semantic/src/types.rs index 6c2c51354e..6fb3645aae 100644 --- a/crates/ty_python_semantic/src/types.rs +++ b/crates/ty_python_semantic/src/types.rs @@ -8159,6 +8159,14 @@ impl<'db> Type<'db> { instance.find_legacy_typevars_impl(db, env, binding_context, typevars, visitor); } + Type::TypedDict(TypedDictType::Class(class)) => { + class.find_legacy_typevars_impl(db, env, binding_context, typevars, visitor); + } + + // Synthesized schemas can contain type variables, but their internal narrowing and + // update constraints inherit those variables from an existing generic context. + Type::TypedDict(TypedDictType::Synthesized(_)) => {} + Type::NewTypeInstance(_) => { // A newtype can never be constructed from an unspecialized generic class, so it is // impossible that we could ever find any legacy typevars in a newtype instance or @@ -8307,8 +8315,7 @@ impl<'db> Type<'db> { | Type::ClassLiteral(_) | Type::LiteralValue(_) | Type::BoundSuper(_) - | Type::SpecialForm(_) - | Type::TypedDict(_) => {} + | Type::SpecialForm(_) => {} } } diff --git a/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs b/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs index 33187fcbac..8a5bef5d47 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs @@ -1440,7 +1440,10 @@ impl<'db> TypeInferenceBuilder<'db, '_> { // instead of two. So until we properly support these, specialize all remaining type // variables with a `@Todo` type (since we don't know which of the type arguments // belongs to the remaining type variables). - if any_over_type(db, env, value_ty, true, |ty| ty.is_divergent()) { + // + // A lazily inferred class member can contain its own unrelated recursive type, so only + // inspect the alias structure and generic arguments when checking whether it is recursive. + if any_over_type(db, env, value_ty, false, |ty| ty.is_divergent()) { let value_ty = value_ty.apply_specialization( db, generic_context.specialize( From 8f703fcb76f06f5ca0a5aac7771f8735709ec965 Mon Sep 17 00:00:00 2001 From: Baltasar Blanco Date: Fri, 14 Aug 2026 17:48:10 -0300 Subject: [PATCH 047/371] [`refurb`] Skip `FURB101` and `FURB103` when the `open` argument is a file descriptor (#27643) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary The core issue is that `pathlib.Path` does not accept a file descriptor. This results in a safe fix that breaks at runtime. On top of that, `find_file_open` in refurb never received this check. `FURB101` and `FURB103` both go through `find_file_opens`. There is precedent for this cross-plugin import: `flake8_blind_except` imports from `flake8_logging`'s helpers, and `flake8_pyi` from `flake8_type_checking`'s. The scope is integer literals, names annotated as `int`, and class attributes annotated as `int`. `os.open(...)` is excluded because it requires type inference. One last thing: this is unrelated to the `FURB103` truncation issue (astral-sh/ruff#26920), which stays open. This PR covers only the file descriptor case. #26922 remains open to track the bytes case. ## Test Plan The most important data point is that 2812 tests pass. The cases are added at the end of `FURB101_0.py` and `FURB103_0.py` (the `str` control is still reported). Zero pre-existing diagnostics were altered — `git diff -U0 | grep '^-'` comes out **empty**. `cargo dev generate-all` is still up to date. --------- Co-authored-by: Brent Westbrook --- .../resources/test/fixtures/refurb/FURB101_0.py | 4 ++++ .../resources/test/fixtures/refurb/FURB103_0.py | 5 +++++ crates/ruff_linter/src/rules/flake8_use_pathlib/mod.rs | 2 +- crates/ruff_linter/src/rules/refurb/helpers.rs | 7 +++++++ 4 files changed, 17 insertions(+), 1 deletion(-) diff --git a/crates/ruff_linter/resources/test/fixtures/refurb/FURB101_0.py b/crates/ruff_linter/resources/test/fixtures/refurb/FURB101_0.py index 9d971d2c77..7e8920ea13 100644 --- a/crates/ruff_linter/resources/test/fixtures/refurb/FURB101_0.py +++ b/crates/ruff_linter/resources/test/fixtures/refurb/FURB101_0.py @@ -141,3 +141,7 @@ def bar(x): with open("file1.txt", encoding="utf-8") as f: contents: str = process_contents(f.read()) + +# `open` accepts a file descriptor, but `Path` does not +with open(3) as f: + x = f.read() diff --git a/crates/ruff_linter/resources/test/fixtures/refurb/FURB103_0.py b/crates/ruff_linter/resources/test/fixtures/refurb/FURB103_0.py index c6a4196fe3..782dffacba 100644 --- a/crates/ruff_linter/resources/test/fixtures/refurb/FURB103_0.py +++ b/crates/ruff_linter/resources/test/fixtures/refurb/FURB103_0.py @@ -162,3 +162,8 @@ def bar(x): other = 1.234 """, )) + + +# `open` accepts a file descriptor, but `Path` does not +with open(3, "w") as f: + f.write("test") diff --git a/crates/ruff_linter/src/rules/flake8_use_pathlib/mod.rs b/crates/ruff_linter/src/rules/flake8_use_pathlib/mod.rs index 23c0fd4498..6855b66c32 100644 --- a/crates/ruff_linter/src/rules/flake8_use_pathlib/mod.rs +++ b/crates/ruff_linter/src/rules/flake8_use_pathlib/mod.rs @@ -1,5 +1,5 @@ //! Rules from [flake8-use-pathlib](https://pypi.org/project/flake8-use-pathlib/). -mod helpers; +pub(crate) mod helpers; pub(crate) mod rules; pub(crate) mod violations; diff --git a/crates/ruff_linter/src/rules/refurb/helpers.rs b/crates/ruff_linter/src/rules/refurb/helpers.rs index 651bfb4ebe..52c27be5bc 100644 --- a/crates/ruff_linter/src/rules/refurb/helpers.rs +++ b/crates/ruff_linter/src/rules/refurb/helpers.rs @@ -8,6 +8,7 @@ use ruff_text_size::{Ranged, TextRange}; use crate::checkers::ast::Checker; use crate::rules::flake8_async::rules::blocking_open_call::is_open_call_from_pathlib; +use crate::rules::flake8_use_pathlib::helpers::is_file_descriptor; use crate::{Applicability, Edit, Fix}; /// Format a code snippet to call `name.method()`. @@ -281,6 +282,12 @@ fn find_file_open<'a>( // Match positional arguments, get filename and mode. let (filename, pos_mode) = match_open_args(args)?; + // `open` accepts a file descriptor, but `Path` does not, so a `pathlib` replacement + // would fail at runtime. `PTH123` skips these for the same reason. + if is_file_descriptor(filename, semantic) { + return None; + } + // Match keyword arguments, get keyword arguments to forward and possibly mode. let (keywords, kw_mode) = match_open_keywords(keywords, read_mode, python_version)?; From 4a4abe4ad35bd5d6acb715e2c00144277d59685b Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Fri, 14 Aug 2026 17:01:35 -0400 Subject: [PATCH 048/371] Align Renovate config with preset (#27762) Signed-off-by: William Woodruff --- .github/renovate.json5 | 36 ------------------------------------ 1 file changed, 36 deletions(-) diff --git a/.github/renovate.json5 b/.github/renovate.json5 index 16b96fdeb5..a2903fd85a 100644 --- a/.github/renovate.json5 +++ b/.github/renovate.json5 @@ -1,17 +1,8 @@ { $schema: "https://docs.renovatebot.com/renovate-schema.json", - dependencyDashboard: true, - suppressNotifications: ["prEditedNotification"], extends: ["github>astral-sh/renovate-config"], - labels: ["internal"], schedule: ["before 4am on Wednesday"], - semanticCommits: "disabled", - separateMajorMinor: false, enabledManagers: ["github-actions", "pre-commit", "cargo", "pep621", "pip_requirements", "npm", "custom.regex"], - cargo: { - // See https://docs.renovatebot.com/configuration-options/#rangestrategy - rangeStrategy: "update-lockfile", - }, pep621: { // The default for this package manager is to only search for `pyproject.toml` files // found at the repository root: https://docs.renovatebot.com/modules/manager/pep621/#file-matching @@ -35,29 +26,7 @@ // found at the repository root: https://docs.renovatebot.com/modules/manager/npm/#file-matching managerFilePatterns: ["^playground/.*package\\.json$"], }, - "pre-commit": { - enabled: true, - }, packageRules: [ - // Pin GitHub Actions to immutable SHAs. - { - matchDepTypes: ["action"], - pinDigests: true, - }, - // Annotate GitHub Actions SHAs with a SemVer version. - { - extends: ["helpers:pinGitHubActionDigests"], - extractVersion: "^(?v?\\d+\\.\\d+\\.\\d+)$", - versioning: "regex:^v?(?\\d+)(\\.(?\\d+)\\.(?\\d+))?$", - }, - { - // Group upload/download artifact updates, the versions are dependent - groupName: "Artifact GitHub Actions dependencies", - matchManagers: ["github-actions"], - matchDatasources: ["gitea-tags", "github-tags"], - matchPackageNames: ["actions/upload-artifact", "actions/download-artifact"], - description: "Weekly update of artifact-related GitHub Actions dependencies", - }, { // This package rule disables updates for GitHub runners: // we'd only pin them to a specific version @@ -104,7 +73,6 @@ // We have a rolling support policy for the MSRV // 2 releases back * 6 weeks per release * 7 days per week + 1 minimumReleaseAge: "85 days", - internalChecksFilter: "strict", groupName: "MSRV", }, { @@ -153,8 +121,4 @@ datasourceTemplate: "github-releases", }, ], - vulnerabilityAlerts: { - commitMessageSuffix: "", - labels: ["internal", "security"], - }, } From 3b067a163e58614fd022c24f1274404a0f386179 Mon Sep 17 00:00:00 2001 From: Ibraheem Ahmed Date: Fri, 14 Aug 2026 17:19:14 -0400 Subject: [PATCH 049/371] [ty] Avoid bypassing solver during literal promotion (#27763) Literal promotion currently bypasses the constraint solver validation pass, allowing unsatisfiable results to be inferred, e.g., ```py from typing import Callable def f[T](value: T, upper: Callable[[T], None]) -> list[T]: return [value] def _(upper: Callable[[int], None]): # error[invalid-argument-type] reveal_type(f("x", upper)) # revealed: list[str] ``` Notice that the chosen solution of `str` does not satisfy the upper bound of `int`. It is possible to construct examples where the diagnostics are currently avoided entirely: ```py class Base[T]: ... class Specialized(Base[str]): ... class Unspecialized(Base): ... class Mixed(Specialized, Unspecialized): ... def f[T](values: list[T], base: Base[T]) -> list[T]: ... f([1], Mixed()) ``` --- .../resources/mdtest/promotion.md | 30 +++++++++++++++++++ .../ty_python_semantic/src/types/call/bind.rs | 13 +++++--- 2 files changed, 39 insertions(+), 4 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/promotion.md b/crates/ty_python_semantic/resources/mdtest/promotion.md index 20f0e78b4a..3c2955ff73 100644 --- a/crates/ty_python_semantic/resources/mdtest/promotion.md +++ b/crates/ty_python_semantic/resources/mdtest/promotion.md @@ -637,6 +637,36 @@ reveal_type(i("a")) # revealed: list[str] reveal_type(i(1)) # revealed: list[Literal[1]] ``` +## Promotion respects inferred upper bounds + +Promotion must not select a solution that violates its inferred upper bound. + +```py +from typing import Callable + +def f[T](value: T, upper: Callable[[T], None]) -> list[T]: + return [value] + +def _(upper: Callable[[int], None]): + # error: [invalid-argument-type] + reveal_type(f("x", upper)) # revealed: list[str | int] +``` + +This also applies when multiple inheritance contributes both static and gradual specializations: + +```py +from typing import Any + +class Base[T]: ... +class Specialized(Base[str]): ... +class Mixed(Specialized, Base[Any]): ... + +def g[T](values: list[T], base: Base[T]) -> list[T]: + return values + +g([1], Mixed()) # error: [invalid-argument-type] +``` + ## Literal annotations from declaration are respected Literal types that are explicitly annotated when declared will not be promoted, even if they are diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index 01b1990940..490858c515 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -5996,8 +5996,12 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { return None; } - let lower = bounds.lower?; - let promoted = lower.promote(db, self.env); + // The promotion override must not select an unsatisfiable solution. + let Ok(Some(solution)) = PathBounds::default_solve(db, self.env, constraints, bounds) + else { + return None; + }; + let promoted = solution.promote(db, self.env); // If the TypeVar has an upper bound, only use the promoted type if it // still satisfies the bound. @@ -6012,8 +6016,9 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { let mut choose = |typevar: BoundTypeVarInstance<'db>, bounds: Option<&PathBound<'db>>| { let bounds = bounds?; - if let Some(lower) = bounds.lower - && let Some(&preferred_ty) = preferred_type_mappings.get(&typevar.identity(db)) + let lower = bounds.lower?; + + if let Some(&preferred_ty) = preferred_type_mappings.get(&typevar.identity(db)) && lower.is_assignable_to(db, self.env, preferred_ty) { return Some(preferred_ty); From 9de726fe287010ab1ca3ee256cdaab73441e3222 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Sat, 15 Aug 2026 02:17:43 -0700 Subject: [PATCH 050/371] [ty] Sync vendored typeshed and repair local patches (#27771) The automated typeshed sync failed because an upstream change added Pyrefly suppression comments to two signatures matched by our local patches. Update those patches to preserve the new comments and include the already-synchronized typeshed snapshot so the sync can land as one change. Updates vendored typeshed to [python/typeshed@6fba3ae73db5a9807780514b463126f1ee8ff216](https://github.com/python/typeshed/commit/6fba3ae73db5a9807780514b463126f1ee8ff216). ## Ecosystem impact The typeshed update specializes `subprocess.Popen` streams by text/binary mode (`IO[AnyStr]` instead of `IO[Any]`). The ecosystem run reports **5 added, 4 removed, and 43 changed diagnostics**: - Two added diagnostics in manticore catch genuine text/bytes bugs. - Three added diagnostics in paasta and pytest expose ty's existing tendency to select the first self-specialized constructor overload when keyword arguments are `Any`; this produces false positives and is tracked in astral-sh/ty#4272. - Four previous warnings disappear, and the remaining changes primarily refine existing diagnostics with more precise subprocess stream types. The upstream annotations are correct, and the existing constructor-overload inference bug will be addressed separately. Fixes astral-sh/ty#4270. ## Test plan - Existing semantic mdtests cover `Mapping.get` overloads accepting arbitrary keys and `is_dataclass` narrowing after all vendored patches are reapplied. - Existing vendored, semantic, and IDE test suites pass against the refreshed stubs. --- .../0002-mapping-get-object.patch | 4 +- .../0007-dataclasses-is-dataclass-top.patch | 4 +- .../vendor/typeshed/source_commit.txt | 2 +- .../vendor/typeshed/stdlib/_asyncio.pyi | 2 +- .../vendor/typeshed/stdlib/_ctypes.pyi | 6 ++ .../typeshed/stdlib/_typeshed/__init__.pyi | 14 ++++ .../vendor/typeshed/stdlib/_winapi.pyi | 3 +- .../vendor/typeshed/stdlib/argparse.pyi | 4 +- .../typeshed/stdlib/asyncio/base_events.pyi | 3 +- .../vendor/typeshed/stdlib/asyncio/events.pyi | 3 +- .../typeshed/stdlib/asyncio/protocols.pyi | 8 +-- .../vendor/typeshed/stdlib/base64.pyi | 2 +- .../typeshed/stdlib/collections/__init__.pyi | 9 ++- .../stdlib/compression/zstd/_zstdfile.pyi | 2 +- .../stdlib/concurrent/futures/_base.pyi | 2 +- .../stdlib/concurrent/futures/process.pyi | 3 +- .../stdlib/concurrent/futures/thread.pyi | 3 +- .../vendor/typeshed/stdlib/contextlib.pyi | 4 +- .../vendor/typeshed/stdlib/copy.pyi | 22 ++++-- .../vendor/typeshed/stdlib/csv.pyi | 4 ++ .../vendor/typeshed/stdlib/ctypes/util.pyi | 2 +- .../vendor/typeshed/stdlib/dataclasses.pyi | 2 +- .../vendor/typeshed/stdlib/decimal.pyi | 2 +- .../vendor/typeshed/stdlib/doctest.pyi | 6 ++ .../vendor/typeshed/stdlib/encodings/big5.pyi | 2 +- .../typeshed/stdlib/encodings/big5hkscs.pyi | 2 +- .../typeshed/stdlib/encodings/cp932.pyi | 2 +- .../typeshed/stdlib/encodings/cp949.pyi | 2 +- .../typeshed/stdlib/encodings/cp950.pyi | 2 +- .../stdlib/encodings/euc_jis_2004.pyi | 2 +- .../stdlib/encodings/euc_jisx0213.pyi | 2 +- .../typeshed/stdlib/encodings/euc_jp.pyi | 2 +- .../typeshed/stdlib/encodings/euc_kr.pyi | 2 +- .../typeshed/stdlib/encodings/gb18030.pyi | 2 +- .../typeshed/stdlib/encodings/gb2312.pyi | 2 +- .../vendor/typeshed/stdlib/encodings/gbk.pyi | 2 +- .../vendor/typeshed/stdlib/encodings/hz.pyi | 2 +- .../typeshed/stdlib/encodings/iso2022_jp.pyi | 2 +- .../stdlib/encodings/iso2022_jp_1.pyi | 2 +- .../stdlib/encodings/iso2022_jp_2.pyi | 2 +- .../stdlib/encodings/iso2022_jp_2004.pyi | 2 +- .../stdlib/encodings/iso2022_jp_3.pyi | 2 +- .../stdlib/encodings/iso2022_jp_ext.pyi | 2 +- .../typeshed/stdlib/encodings/iso2022_kr.pyi | 2 +- .../typeshed/stdlib/encodings/johab.pyi | 2 +- .../typeshed/stdlib/encodings/shift_jis.pyi | 2 +- .../stdlib/encodings/shift_jis_2004.pyi | 2 +- .../stdlib/encodings/shift_jisx0213.pyi | 2 +- .../vendor/typeshed/stdlib/functools.pyi | 2 +- .../vendor/typeshed/stdlib/imaplib.pyi | 9 ++- .../vendor/typeshed/stdlib/importlib/abc.pyi | 31 +++++--- .../vendor/typeshed/stdlib/inspect.pyi | 25 ++++--- .../vendor/typeshed/stdlib/mailbox.pyi | 8 ++- .../vendor/typeshed/stdlib/os/__init__.pyi | 2 +- .../vendor/typeshed/stdlib/pydoc.pyi | 6 +- .../typeshed/stdlib/pyexpat/__init__.pyi | 70 +++++++++---------- .../vendor/typeshed/stdlib/subprocess.pyi | 6 +- .../typeshed/stdlib/tkinter/__init__.pyi | 6 ++ .../vendor/typeshed/stdlib/tkinter/font.pyi | 3 + .../vendor/typeshed/stdlib/turtle.pyi | 4 +- .../vendor/typeshed/stdlib/types.pyi | 10 +-- .../vendor/typeshed/stdlib/typing.pyi | 2 +- .../typeshed/stdlib/typing_extensions.pyi | 2 +- .../vendor/typeshed/stdlib/webbrowser.pyi | 2 + 64 files changed, 216 insertions(+), 136 deletions(-) diff --git a/crates/ty_vendored/typeshed_patches/0002-mapping-get-object.patch b/crates/ty_vendored/typeshed_patches/0002-mapping-get-object.patch index 120378ea8e..22ba3e2655 100644 --- a/crates/ty_vendored/typeshed_patches/0002-mapping-get-object.patch +++ b/crates/ty_vendored/typeshed_patches/0002-mapping-get-object.patch @@ -6,8 +6,8 @@ + def get(self, key: object, /) -> _VT_co | None: """D.get(k[,d]) -> D[k] if k in D, else d. d defaults to None.""" @overload -- def get(self, key: _KT, default: _VT_co, /) -> _VT_co: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter -+ def get(self, key: object, default: _VT_co, /) -> _VT_co: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter +- def get(self, key: _KT, default: _VT_co, /) -> _VT_co: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter # pyrefly: ignore [invalid-variance] ++ def get(self, key: object, default: _VT_co, /) -> _VT_co: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter # pyrefly: ignore [invalid-variance] @overload - def get(self, key: _KT, default: _T, /) -> _VT_co | _T: ... + def get(self, key: object, default: _T, /) -> _VT_co | _T: ... diff --git a/crates/ty_vendored/typeshed_patches/0007-dataclasses-is-dataclass-top.patch b/crates/ty_vendored/typeshed_patches/0007-dataclasses-is-dataclass-top.patch index c0f47888b2..8b700d2250 100644 --- a/crates/ty_vendored/typeshed_patches/0007-dataclasses-is-dataclass-top.patch +++ b/crates/ty_vendored/typeshed_patches/0007-dataclasses-is-dataclass-top.patch @@ -14,8 +14,8 @@ index d46b694a7e..1db97b1893 100644 # HACK: `obj: Never` typing matches if object argument is using `Any` type. @overload --def is_dataclass(obj: Never) -> TypeIs[DataclassInstance | type[DataclassInstance]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] -+def is_dataclass(obj: Never) -> TypeIs[Top[DataclassInstance | type[DataclassInstance]]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] +-def is_dataclass(obj: Never) -> TypeIs[DataclassInstance | type[DataclassInstance]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] # pyrefly: ignore [bad-function-definition] ++def is_dataclass(obj: Never) -> TypeIs[Top[DataclassInstance | type[DataclassInstance]]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] # pyrefly: ignore [bad-function-definition] """Returns True if obj is a dataclass or an instance of a dataclass. """ diff --git a/crates/ty_vendored/vendor/typeshed/source_commit.txt b/crates/ty_vendored/vendor/typeshed/source_commit.txt index 5ba2971442..6849262268 100644 --- a/crates/ty_vendored/vendor/typeshed/source_commit.txt +++ b/crates/ty_vendored/vendor/typeshed/source_commit.txt @@ -1 +1 @@ -1b116673774d062a4af7b0a0b3d05533a6be55d0 +6fba3ae73db5a9807780514b463126f1ee8ff216 diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/_asyncio.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/_asyncio.pyi index 9d05cede8f..36111d0634 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/_asyncio.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/_asyncio.pyi @@ -133,7 +133,7 @@ else: # since the only reason why `asyncio.Future` is invariant is the `set_result()` method, # and `asyncio.Task.set_result()` always raises. @disjoint_base -class Task(Future[_T_co]): # type: ignore[type-var] # pyright: ignore[reportInvalidTypeArguments] # ty:ignore[invalid-generic-class] +class Task(Future[_T_co]): # type: ignore[type-var] # pyright: ignore[reportInvalidTypeArguments] # ty:ignore[invalid-generic-class] # pyrefly: ignore [invalid-variance] """A coroutine wrapped in a Future.""" if sys.version_info >= (3, 12): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/_ctypes.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/_ctypes.pyi index 108b7b7117..98562afb07 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/_ctypes.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/_ctypes.pyi @@ -492,3 +492,9 @@ def buffer_info(o: _CData | _CDataType | type[_CData | _CDataType], /) -> tuple[ def call_cdeclfunction(address: int, arguments: tuple[Any, ...], /) -> Any: ... def call_function(address: int, arguments: tuple[Any, ...], /) -> Any: ... + +# dllist() is available on Linux and other platforms like NetBSD +if sys.version_info >= (3, 14) and sys.platform != "win32" and sys.platform != "darwin": + # Added in Python 3.14.7 + def dllist() -> list[str]: + """dllist() return a list of loaded shared libraries""" diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/__init__.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/__init__.pyi index 5b2d7f7c54..98dfb02d04 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/__init__.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/_typeshed/__init__.pyi @@ -168,6 +168,18 @@ class SupportsTrunc(Protocol): # Mapping-like protocols +# The second and third overload could technically be combined, but splitting +# them works better with some type checkers. +class SupportsGet(Protocol[_KT_contra, _VT_co]): # type: ignore[misc] # Covariant type as parameter + @overload + def get(self, key: _KT_contra, /) -> _VT_co | None: ... + @overload + def get( # pyrefly: ignore[invalid-variance] + self, key: _KT_contra, default: _VT_co, / # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter + ) -> _VT_co: ... + @overload + def get(self, key: _KT_contra, default: _T, /) -> _VT_co | _T: ... + # stable class SupportsItems(Protocol[_KT_co, _VT_co]): def items(self) -> AbstractSet[tuple[_KT_co, _VT_co]]: ... @@ -193,6 +205,8 @@ class SupportsItemAccess(Protocol[_KT_contra, _VT]): def __setitem__(self, key: _KT_contra, value: _VT, /) -> None: ... def __delitem__(self, key: _KT_contra, /) -> None: ... +# Path and file handling + StrPath: TypeAlias = str | PathLike[str] # stable BytesPath: TypeAlias = bytes | PathLike[bytes] # stable GenericPath: TypeAlias = AnyStr | PathLike[AnyStr] diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/_winapi.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/_winapi.pyi index dc5d8ad02d..f90822f887 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/_winapi.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/_winapi.pyi @@ -450,6 +450,7 @@ if sys.platform == "win32": def NeedCurrentDirectoryForExePath(exe_name: str, /) -> bool: ... - if sys.version_info >= (3, 15): + if sys.version_info >= (3, 13): + # Added in Python 3.13.15, 3.14.7 def GetTickCount64() -> int: """Number of milliseconds that have elapsed since the system was started.""" diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/argparse.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/argparse.pyi index 3abc52ddbe..854186c9ef 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/argparse.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/argparse.pyi @@ -441,8 +441,8 @@ class HelpFormatter: _current_indent: int _level: int _action_max_length: int - _root_section: _Section - _current_section: _Section + _root_section: _Section # pyrefly: ignore [unknown-name] + _current_section: _Section # pyrefly: ignore [unknown-name] _whitespace_matcher: Pattern[str] _long_break_matcher: Pattern[str] diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/base_events.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/base_events.pyi index 62d24a994f..4e79aa1952 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/base_events.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/base_events.pyi @@ -185,7 +185,8 @@ class BaseEventLoop(AbstractEventLoop): """Create a Future object attached to the loop.""" # Tasks methods - if sys.version_info >= (3, 14): + # `eager_start` is supported as an arbitrary kwarg starting in 3.13.3. + if sys.version_info >= (3, 13): def create_task( self, coro: _CoroutineLike[_T], diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/events.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/events.pyi index 4d901d2741..973ac7c17b 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/events.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/events.pyi @@ -226,7 +226,8 @@ class AbstractEventLoop: @abstractmethod def create_future(self) -> Future[Any]: ... # Tasks methods - if sys.version_info >= (3, 14): + # `eager_start` is supported as an arbitrary kwarg starting in 3.13.3. + if sys.version_info >= (3, 13): @abstractmethod def create_task( self, diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/protocols.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/protocols.pyi index c0058d7183..9fcc1472cb 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/protocols.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/asyncio/protocols.pyi @@ -167,11 +167,9 @@ class DatagramProtocol(BaseProtocol): When the connection is closed, connection_lost() is called. """ - # addr can be a tuple[int, int] for some unusual protocols like socket.AF_NETLINK. - # Use tuple[str | Any, int] to not cause typechecking issues on most usual cases. - # This could be improved by using tuple[AnyOf[str, int], int] if the AnyOf feature is accepted. - # See https://github.com/python/typing/issues/566 - def datagram_received(self, data: bytes, addr: tuple[str | Any, int]) -> None: + # addr is a tuple[str, int] for IPv4 or tuple[str, int, int, int] for IPv6. + # It can also be a tuple[int, int] for unusual protocols like socket.AF_NETLINK. + def datagram_received(self, data: bytes, addr: tuple[Any, ...]) -> None: """Called when some datagram is received.""" def error_received(self, exc: Exception) -> None: diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/base64.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/base64.pyi index 2d8800faae..f03ce0f1a5 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/base64.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/base64.pyi @@ -241,7 +241,7 @@ else: Optional casefold is a flag specifying whether a lowercase alphabet is acceptable as input. For security purposes, the default is False. - RFC 3548 allows for optional mapping of the digit 0 (zero) to the + RFC 4648 allows for optional mapping of the digit 0 (zero) to the letter O (oh), and for optional mapping of the digit 1 (one) to either the letter I (eye) or letter L (el). The optional argument map01 when not None, specifies which letter the digit 1 should be diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/collections/__init__.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/collections/__init__.pyi index 9b6589d6db..449e591b06 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/collections/__init__.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/collections/__init__.pyi @@ -386,6 +386,9 @@ class Counter(dict[_T, int], Generic[_T]): or multiset. Elements are stored as dictionary keys and their counts are stored as dictionary values. + When constructed from a Mapping or Counter, the original object's + values will be used as the initial counts. + >>> c = Counter('abcdeabcdabcaba') # count elements from a string >>> c.most_common(3) # three most common elements @@ -680,17 +683,17 @@ class _OrderedDictValuesView(ValuesView[_VT_co]): # pyright doesn't have a specific error code for subclassing error! @final @type_check_only -class _odict_keys(dict_keys[_KT_co, _VT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[subclass-of-final-class] +class _odict_keys(dict_keys[_KT_co, _VT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[subclass-of-final-class] # pyrefly: ignore [invalid-inheritance] def __reversed__(self) -> Iterator[_KT_co]: ... @final @type_check_only -class _odict_items(dict_items[_KT_co, _VT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[subclass-of-final-class] +class _odict_items(dict_items[_KT_co, _VT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[subclass-of-final-class] # pyrefly: ignore [invalid-inheritance] def __reversed__(self) -> Iterator[tuple[_KT_co, _VT_co]]: ... @final @type_check_only -class _odict_values(dict_values[_KT_co, _VT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[subclass-of-final-class] +class _odict_values(dict_values[_KT_co, _VT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[subclass-of-final-class] # pyrefly: ignore [invalid-inheritance] def __reversed__(self) -> Iterator[_VT_co]: ... @disjoint_base diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/compression/zstd/_zstdfile.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/compression/zstd/_zstdfile.pyi index bb544afab0..8d27cd45da 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/compression/zstd/_zstdfile.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/compression/zstd/_zstdfile.pyi @@ -48,7 +48,7 @@ class ZstdFile(_streams.BaseStream): ) -> None: """Open a Zstandard compressed file in binary mode. - *file* can be either an file-like object, or a file name to open. + *file* can be either a file-like object, or a file name to open. *mode* can be 'r' for reading (default), 'w' for (over)writing, 'x' for creating exclusively, or 'a' for appending. These can diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/_base.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/_base.pyi index 7fc7ddeca5..edf10ca612 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/_base.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/_base.pyi @@ -35,7 +35,7 @@ class InvalidStateError(Error): class BrokenExecutor(RuntimeError): """ - Raised when a executor has become non-functional after a severe failure. + Raised when an executor has become non-functional after a severe failure. """ _T = TypeVar("_T") diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/process.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/process.pyi index 4625adfcf7..43dde182a0 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/process.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/process.pyi @@ -238,7 +238,8 @@ class _ExecutorManagerThread(Thread): def process_result_item(self, result_item: int | _ResultItem) -> None: ... def is_shutting_down(self) -> bool: ... - if sys.version_info >= (3, 15): + if sys.version_info >= (3, 14): + # bpe_message parameter added in 3.14.7 def terminate_broken(self, cause: str, bpe_message: str | None = None) -> None: ... else: def terminate_broken(self, cause: str) -> None: ... diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/thread.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/thread.pyi index 91756c39ef..4e353eda3d 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/thread.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/concurrent/futures/thread.pyi @@ -81,7 +81,8 @@ else: def __class_getitem__(cls, item: Any, /) -> GenericAlias: """Represent a PEP 585 generic type - E.g. for t = list[int], t.__origin__ is list and t.__args__ is (int,). + For example, for t = list[int], t.__origin__ is list and t.__args__ + is (int,). """ def _worker( diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/contextlib.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/contextlib.pyi index 0272813d77..4880ae23f8 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/contextlib.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/contextlib.pyi @@ -45,7 +45,7 @@ _CM_EF = TypeVar("_CM_EF", bound=AbstractContextManager[Any, Any] | _ExitFunc) # At runtime it inherits from ABC and is not a Protocol, but it is on the # allowlist for use as a Protocol. @runtime_checkable -class AbstractContextManager(ABC, Protocol[_T_co, _ExitT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-protocol] +class AbstractContextManager(ABC, Protocol[_T_co, _ExitT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-protocol] # pyrefly: ignore [invalid-inheritance] """An abstract base class for context managers.""" __slots__ = () @@ -62,7 +62,7 @@ class AbstractContextManager(ABC, Protocol[_T_co, _ExitT_co]): # type: ignore[m # At runtime it inherits from ABC and is not a Protocol, but it is on the # allowlist for use as a Protocol. @runtime_checkable -class AbstractAsyncContextManager(ABC, Protocol[_T_co, _ExitT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-protocol] +class AbstractAsyncContextManager(ABC, Protocol[_T_co, _ExitT_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-protocol] # pyrefly: ignore [invalid-inheritance] """An abstract base class for asynchronous context managers.""" __slots__ = () diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/copy.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/copy.pyi index 2f464f1e1c..cac6d201c1 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/copy.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/copy.pyi @@ -65,19 +65,27 @@ class _SupportsReplace(Protocol[_RT_co]): # None in CPython but non-None in Jython PyStringMap: Any -# Note: memo and _nil are internal kwargs. -def deepcopy(x: _T, memo: dict[int, Any] | None = None, _nil: Any = []) -> _T: - """Deep copy operation on arbitrary Python objects. - - See the module's __doc__ string for more info. - """ - def copy(x: _T) -> _T: """Shallow copy operation on arbitrary Python objects. See the module's __doc__ string for more info. """ +if sys.version_info >= (3, 15): + def deepcopy(x: _T, memo: dict[int, Any] | None = None) -> _T: + """Deep copy operation on arbitrary Python objects. + + See the module's __doc__ string for more info. + """ + +else: + # Note: memo and _nil are internal kwargs. + def deepcopy(x: _T, memo: dict[int, Any] | None = None, _nil: Any = []) -> _T: + """Deep copy operation on arbitrary Python objects. + + See the module's __doc__ string for more info. + """ + if sys.version_info >= (3, 13): __all__ += ["replace"] # The types accepted by `**changes` match those of `obj.__replace__`. diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/csv.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/csv.pyi index e2ef8fc468..4e59ceb28d 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/csv.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/csv.pyi @@ -244,6 +244,10 @@ class Sniffer: def sniff(self, sample: str, delimiters: str | None = None) -> type[Dialect]: """ Returns a dialect (or None) corresponding to the sample + + If several delimiters fit the sample equally well, the + delimiters listed in the preferred attribute are preferred, in + that order, no matter how many times each of them occurs. """ def has_header(self, sample: str) -> bool: ... diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/ctypes/util.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/ctypes/util.pyi index dc1a251365..3b815d8185 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/ctypes/util.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/ctypes/util.pyi @@ -8,6 +8,6 @@ if sys.platform == "win32": if sys.version_info >= (3, 14): def dllist() -> list[str]: - """Return a list of loaded shared libraries in the current process.""" + """dllist() return a list of loaded shared libraries""" def test() -> None: ... diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/dataclasses.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/dataclasses.pyi index 1db97b1893..c6bf7f69ec 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/dataclasses.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/dataclasses.pyi @@ -403,7 +403,7 @@ def fields(class_or_instance: DataclassInstance | type[DataclassInstance]) -> tu # HACK: `obj: Never` typing matches if object argument is using `Any` type. @overload -def is_dataclass(obj: Never) -> TypeIs[Top[DataclassInstance | type[DataclassInstance]]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] +def is_dataclass(obj: Never) -> TypeIs[Top[DataclassInstance | type[DataclassInstance]]]: # type: ignore[narrowed-type-not-subtype] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-type-guard-definition] # pyrefly: ignore [bad-function-definition] """Returns True if obj is a dataclass or an instance of a dataclass. """ diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/decimal.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/decimal.pyi index d43b2ac9de..af211d944e 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/decimal.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/decimal.pyi @@ -762,7 +762,7 @@ class Context: """Set all traps to False.""" def copy(self) -> Context: - """Return a duplicate of the context with all flags cleared.""" + """Return a duplicate of the context.""" def __copy__(self) -> Context: ... # see https://github.com/python/cpython/issues/94107 diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/doctest.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/doctest.pyi index e82f832bd6..75a9b78514 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/doctest.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/doctest.pyi @@ -423,6 +423,12 @@ class DocTestRunner: more information. """ + if sys.version_info >= (3, 15): + def report_skip(self, out: _Out, test: DocTest, example: Example) -> None: + """ + Report that the given example was skipped. + """ + def report_start(self, out: _Out, test: DocTest, example: Example) -> None: """ Report that the test runner is about to process the given diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/big5.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/big5.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/big5.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/big5.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/big5hkscs.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/big5hkscs.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/big5hkscs.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/big5hkscs.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp932.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp932.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp932.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp932.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp949.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp949.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp949.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp949.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp950.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp950.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp950.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/cp950.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jis_2004.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jis_2004.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jis_2004.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jis_2004.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jisx0213.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jisx0213.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jisx0213.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jisx0213.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jp.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jp.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jp.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_jp.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_kr.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_kr.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_kr.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/euc_kr.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gb18030.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gb18030.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gb18030.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gb18030.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gb2312.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gb2312.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gb2312.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gb2312.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gbk.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gbk.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gbk.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/gbk.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/hz.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/hz.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/hz.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/hz.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_1.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_1.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_1.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_1.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_2.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_2.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_2.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_2.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_2004.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_2004.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_2004.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_2004.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_3.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_3.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_3.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_3.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_ext.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_ext.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_ext.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_jp_ext.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_kr.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_kr.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_kr.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/iso2022_kr.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/johab.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/johab.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/johab.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/johab.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jis.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jis.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jis.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jis.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jis_2004.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jis_2004.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jis_2004.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jis_2004.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jisx0213.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jisx0213.pyi index d613026a5a..be96a1ba3a 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jisx0213.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/encodings/shift_jisx0213.pyi @@ -14,7 +14,7 @@ class IncrementalEncoder(mbc.MultibyteIncrementalEncoder, codecs.IncrementalEnco class IncrementalDecoder(mbc.MultibyteIncrementalDecoder, codecs.IncrementalDecoder): codec: ClassVar[mbc._MultibyteCodec] = ... -class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] +class StreamReader(Codec, mbc.MultibyteStreamReader, codecs.StreamReader): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] codec: ClassVar[mbc._MultibyteCodec] = ... class StreamWriter(Codec, mbc.MultibyteStreamWriter, codecs.StreamWriter): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/functools.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/functools.pyi index 4a50f409c6..e5881a89f3 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/functools.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/functools.pyi @@ -440,7 +440,7 @@ class cached_property(Generic[_T_co]): def __set_name__(self, owner: type[Any], name: str) -> None: ... # __set__ is not defined at runtime, but @cached_property is designed to be settable - def __set__(self, instance: object, value: _T_co) -> None: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] + def __set__(self, instance: object, value: _T_co) -> None: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # pyrefly: ignore [invalid-variance] def __class_getitem__(cls, item: Any, /) -> GenericAlias: """Represent a PEP 585 generic type diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/imaplib.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/imaplib.pyi index 0ae365b25e..c955a6d081 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/imaplib.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/imaplib.pyi @@ -292,7 +292,8 @@ class IMAP4: Note: 'duration' requires a socket connection (not IMAP4_stream). """ - if sys.version_info >= (3, 15): + if sys.version_info >= (3, 13): + # Default was fixed in Python 3.13.15, 3.14.7 def list(self, directory: str = "", pattern: str = "*") -> tuple[str, _AnyResponseData]: """List mailbox names in directory matching pattern. @@ -332,7 +333,8 @@ class IMAP4: Returns server 'BYE' response. """ - if sys.version_info >= (3, 15): + if sys.version_info >= (3, 13): + # Default was fixed in Python 3.13.15, 3.14.7 def lsub(self, directory: str = "", pattern: str = "*") -> _CommandResults: """List 'subscribed' mailbox names in directory matching pattern. @@ -419,7 +421,8 @@ class IMAP4: (typ, [data]) = .setacl(mailbox, who, what) """ - if sys.version_info >= (3, 15): + if sys.version_info >= (3, 13): + # Parameter "mailbox" was added in Python 3.13.15, 3.14.7 def setannotation(self, mailbox: str | bytes, *args: str) -> _CommandResults: """(typ, [data]) = .setannotation(mailbox[, entry, attribute]+) Set ANNOTATIONs. diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/importlib/abc.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/importlib/abc.pyi index 8751039e16..9d21cc9e79 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/importlib/abc.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/importlib/abc.pyi @@ -94,15 +94,28 @@ class InspectLoader(Loader): def exec_module(self, module: types.ModuleType) -> None: """Execute the module.""" - @staticmethod - def source_to_code( - data: ReadableBuffer | str | _ast.Module | _ast.Expression | _ast.Interactive, path: bytes | StrPath = "" - ) -> types.CodeType: - """Compile 'data' into a code object. - - The 'data' argument can be anything that compile() can handle. The'path' - argument should be where the data was retrieved (when applicable). - """ + if sys.version_info >= (3, 15): + @staticmethod + def source_to_code( + data: ReadableBuffer | str | _ast.Module | _ast.Expression | _ast.Interactive, + path: bytes | StrPath = "", + fullname: str | None = None, + ) -> types.CodeType: + """Compile 'data' into a code object. + + The 'data' argument can be anything that compile() can handle. The'path' + argument should be where the data was retrieved (when applicable). + """ + else: + @staticmethod + def source_to_code( + data: ReadableBuffer | str | _ast.Module | _ast.Expression | _ast.Interactive, path: bytes | StrPath = "" + ) -> types.CodeType: + """Compile 'data' into a code object. + + The 'data' argument can be anything that compile() can handle. The'path' + argument should be where the data was retrieved (when applicable). + """ class ExecutionLoader(InspectLoader): """Abstract base class for loaders that wish to support the execution of diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/inspect.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/inspect.pyi index 2827a72b3d..874f9733da 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/inspect.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/inspect.pyi @@ -462,18 +462,18 @@ def isroutine( def ismethoddescriptor(object: object) -> TypeIs[MethodDescriptorType]: """Return true if the object is a method descriptor. - But not if ismethod() or isclass() or isfunction() are true. + But not if ismethod(), isclass() or isfunction() is true. - This is new in Python 2.2, and, for example, is true of int.__add__. - An object passing this test has a __get__ attribute, but not a - __set__ attribute or a __delete__ attribute. Beyond that, the set - of attributes varies; __name__ is usually sensible, and __doc__ - often is. + An object passing this test (for example, int.__add__) has a __get__ + attribute, but not a __set__ attribute or a __delete__ attribute. + Beyond that, the set of attributes varies; __name__ is usually + sensible, and __doc__ often is. Methods implemented via descriptors that also pass one of the other - tests return false from the ismethoddescriptor() test, simply because - the other tests promise more -- you can, e.g., count on having the - __func__ attribute (etc) when an object passes ismethod(). + tests (ismethod(), isclass(), isfunction()) make this function return + false, simply because those other tests promise more -- you can, for + example, count on having the __func__ attribute when an object passes + ismethod(). """ def ismemberdescriptor(object: object) -> TypeIs[MemberDescriptorType]: @@ -496,8 +496,13 @@ def isgetsetdescriptor(object: object) -> TypeIs[GetSetDescriptorType]: def isdatadescriptor(object: object) -> TypeIs[_SupportsSet[Never, Never] | _SupportsDelete[Never]]: """Return true if the object is a data descriptor. + But not if ismethod(), isclass() or isfunction() is true. + Data descriptors have a __set__ or a __delete__ attribute. Examples are - properties (defined in Python) and getsets and members (defined in C). + properties, getsets, and members. For the latter two (defined only in C + extension modules) more specific tests are available as well: + isgetsetdescriptor() and ismemberdescriptor(), respectively. + Typically, data descriptors will also have __name__ and __doc__ attributes (properties, getsets, and members have both of these attributes), but this is not guaranteed. diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/mailbox.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/mailbox.pyi index 95166f5b9d..6218854b89 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/mailbox.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/mailbox.pyi @@ -186,6 +186,11 @@ class Mailbox(Generic[_MessageT_co]): def close(self) -> None: """Flush and close the mailbox.""" + if sys.version_info >= (3, 15): + def __enter__(self) -> Self: ... + def __exit__( + self, type: type[BaseException] | None, value: BaseException | None, traceback: TracebackType | None + ) -> None: ... # Undocumented, called by subclasses to parse added messages. def _dump_message(self, message: _MessageData, target: SupportsWrite[bytes], mangle_from_: bool = False) -> None: """Dump message contents to target file.""" @@ -581,7 +586,8 @@ class _ProxyFile: def __class_getitem__(cls, item: Any, /) -> GenericAlias: """Represent a PEP 585 generic type - E.g. for t = list[int], t.__origin__ is list and t.__args__ is (int,). + For example, for t = list[int], t.__origin__ is list and t.__args__ + is (int,). """ class _PartialFile(_ProxyFile): diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/os/__init__.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/os/__init__.pyi index e0bba440c9..9de0cd5ad6 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/os/__init__.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/os/__init__.pyi @@ -1025,7 +1025,7 @@ In the future, this property will contain the last metadata change time.""") # At runtime it inherits from ABC and is not a Protocol, but it will be # on the allowlist for use as a Protocol starting in 3.14. @runtime_checkable -class PathLike(ABC, Protocol[AnyStr_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-protocol] +class PathLike(ABC, Protocol[AnyStr_co]): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-protocol] # pyrefly: ignore [invalid-inheritance] """Abstract base class for implementing the file system path protocol.""" __slots__ = () diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/pydoc.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/pydoc.pyi index 1f8dd6d0b7..f508d08cbc 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/pydoc.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/pydoc.pyi @@ -189,8 +189,8 @@ class HTMLDoc(Doc): """Formatter class for HTML documentation.""" _repr_instance: HTMLRepr - repr = _repr_instance.repr - escape = _repr_instance.escape + repr = _repr_instance.repr # pyrefly: ignore [unknown-name] + escape = _repr_instance.escape # pyrefly: ignore [unknown-name] def page(self, title: str, contents: str) -> str: """Format an HTML page.""" @@ -353,7 +353,7 @@ class TextDoc(Doc): """Formatter class for text documentation.""" _repr_instance: TextRepr - repr = _repr_instance.repr + repr = _repr_instance.repr # pyrefly: ignore [unknown-name] def bold(self, text: str) -> str: """Format a string in bold by overstriking.""" diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/pyexpat/__init__.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/pyexpat/__init__.pyi index e3a6adfb49..131844d0c9 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/pyexpat/__init__.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/pyexpat/__init__.pyi @@ -1,6 +1,5 @@ """Python wrapper for Expat parser.""" -import sys from _typeshed import ReadableBuffer, SupportsRead from collections.abc import Callable from pyexpat import errors as errors, model as model @@ -102,41 +101,40 @@ class XMLParserType: library. """ - if sys.version_info >= (3, 13): - # Added in Python 3.13.4, 3.14.6 - def SetBillionLaughsAttackProtectionActivationThreshold(self, threshold: int, /) -> None: - """Sets the number of output bytes needed to activate protection against billion laughs attacks. - - The number of output bytes includes amplification from entity - expansion and reading DTD files. - - Parser objects usually have a protection activation threshold of - 8 MiB, but the actual default value depends on the underlying Expat - library. - - Activation thresholds below 4 MiB are known to break support for - DITA 1.3 payload and are hence not recommended. - """ - - def SetBillionLaughsAttackProtectionMaximumAmplification(self, max_factor: float, /) -> None: - """Sets the maximum tolerated amplification factor for protection against billion laughs attacks. - - The amplification factor is calculated as "(direct + indirect) / - direct" while parsing, where "direct" is the number of bytes read - from the primary document in parsing and "indirect" is the number of - bytes added by expanding entities and reading external DTD files, - combined. - - The 'max_factor' value must be a non-NaN floating point value - greater than or equal to 1.0. Amplification factors greater than - 30,000 can be observed in the middle of parsing even with benign - files in practice. In particular, the activation threshold should - be carefully chosen to avoid false positives. - - Parser objects usually have a maximum amplification factor of 100, - but the actual default value depends on the underlying Expat - library. - """ + # Added in Python 3.10.19, 3.11.14, 3.12.12, 3.13.4, 3.14.6 + def SetBillionLaughsAttackProtectionActivationThreshold(self, threshold: int, /) -> None: + """Sets the number of output bytes needed to activate protection against billion laughs attacks. + + The number of output bytes includes amplification from entity + expansion and reading DTD files. + + Parser objects usually have a protection activation threshold of + 8 MiB, but the actual default value depends on the underlying Expat + library. + + Activation thresholds below 4 MiB are known to break support for + DITA 1.3 payload and are hence not recommended. + """ + + def SetBillionLaughsAttackProtectionMaximumAmplification(self, max_factor: float, /) -> None: + """Sets the maximum tolerated amplification factor for protection against billion laughs attacks. + + The amplification factor is calculated as "(direct + indirect) / + direct" while parsing, where "direct" is the number of bytes read + from the primary document in parsing and "indirect" is the number of + bytes added by expanding entities and reading external DTD files, + combined. + + The 'max_factor' value must be a non-NaN floating point value + greater than or equal to 1.0. Amplification factors greater than + 30,000 can be observed in the middle of parsing even with benign + files in practice. In particular, the activation threshold should + be carefully chosen to avoid false positives. + + Parser objects usually have a maximum amplification factor of 100, + but the actual default value depends on the underlying Expat + library. + """ @property def intern(self) -> dict[str, str]: ... diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/subprocess.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/subprocess.pyi index afc25e69df..c4ab713906 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/subprocess.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/subprocess.pyi @@ -1310,9 +1310,9 @@ class Popen(Generic[AnyStr]): """ args: _CMD - stdin: IO[Any] | None - stdout: IO[Any] | None - stderr: IO[Any] | None + stdin: IO[AnyStr] | None + stdout: IO[AnyStr] | None + stderr: IO[AnyStr] | None pid: int returncode: int | MaybeNone universal_newlines: bool diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/tkinter/__init__.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/tkinter/__init__.pyi index 50089041f1..6c21a1799f 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/tkinter/__init__.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/tkinter/__init__.pyi @@ -633,6 +633,9 @@ class Misc: master: Misc | None tk: _tkinter.TkappType children: dict[str, Widget] + if sys.version_info >= (3, 15): + __iter__: ClassVar[None] # prevent using __getitem__ for iteration + def destroy(self) -> None: """Internal function. @@ -6383,6 +6386,9 @@ class Image(_Image): name: Incomplete tk: _tkinter.TkappType + if sys.version_info >= (3, 15): + __iter__: ClassVar[None] # prevent using __getitem__ for iteration + def __init__(self, imgtype, name=None, cnf={}, master: Misc | _tkinter.TkappType | None = None, **kw) -> None: ... def __del__(self) -> None: ... def __setitem__(self, key, value) -> None: ... diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/tkinter/font.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/tkinter/font.pyi index 2da1ac62ac..9eb7b22ea9 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/tkinter/font.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/tkinter/font.pyi @@ -2,6 +2,7 @@ import _tkinter import itertools +import sys import tkinter from typing import Any, ClassVar, Final, Literal, TypeAlias, TypedDict, overload, type_check_only from typing_extensions import Unpack @@ -64,6 +65,8 @@ class Font: name: str delete_font: bool + if sys.version_info >= (3, 15): + __iter__: ClassVar[None] # prevent using __getitem__ for iteration counter: ClassVar[itertools.count[int]] # undocumented def __init__( self, diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/turtle.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/turtle.pyi index bb8b0df0e1..e295dcd05d 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/turtle.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/turtle.pyi @@ -304,7 +304,7 @@ else: """rotate self counterclockwise by angle""" # Does not actually inherit from Canvas, but dynamically gets all methods of Canvas -class ScrolledCanvas(Canvas, Frame): # type: ignore[misc] +class ScrolledCanvas(Canvas, Frame): # type: ignore[misc] # pyrefly: ignore [inconsistent-inheritance] """Modeled after the scrolled canvas class from Grayons's Tkinter book. Used as the default canvas, which pops up automatically when @@ -1594,7 +1594,7 @@ class TPen: st = showturtle ht = hideturtle -class RawTurtle(TPen, TNavigator): # type: ignore[misc] # Conflicting methods in base classes +class RawTurtle(TPen, TNavigator): # type: ignore[misc] # Conflicting methods in base classes # pyrefly: ignore [inconsistent-inheritance] """Animation part of the RawTurtle. Puts RawTurtle upon a TurtleScreen and provides tools for its animation. diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/types.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/types.pyi index 2c258e31bc..91feb9cfd6 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/types.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/types.pyi @@ -301,14 +301,14 @@ class CodeType: """The same as replace().""" @final -class MappingProxyType(Mapping[_KT_co, _VT_co]): # type: ignore[type-var] # pyright: ignore[reportInvalidTypeArguments] # ty:ignore[invalid-generic-class] +class MappingProxyType(Mapping[_KT_co, _VT_co]): # type: ignore[type-var] # pyright: ignore[reportInvalidTypeArguments] # ty:ignore[invalid-generic-class] # pyrefly: ignore [invalid-variance] """Read-only proxy of a mapping.""" __hash__: ClassVar[None] # type: ignore[assignment] """Return hash(self).""" def __new__(cls, mapping: SupportsKeysAndGetItem[_KT_co, _VT_co]) -> Self: ... - def __getitem__(self, key: _KT_co, /) -> _VT_co: # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] + def __getitem__(self, key: _KT_co, /) -> _VT_co: # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # pyrefly: ignore [invalid-variance] """Return self[key].""" def __iter__(self) -> Iterator[_KT_co]: @@ -331,12 +331,12 @@ class MappingProxyType(Mapping[_KT_co, _VT_co]): # type: ignore[type-var] # py """D.items() -> a set-like object providing a view on D's items""" @overload - def get(self, key: _KT_co, /) -> _VT_co | None: # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter + def get(self, key: _KT_co, /) -> _VT_co | None: # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter # pyrefly: ignore [invalid-variance] """Return the value for key if key is in the mapping, else default.""" @overload - def get(self, key: _KT_co, default: _VT_co, /) -> _VT_co: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter + def get(self, key: _KT_co, default: _VT_co, /) -> _VT_co: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter # pyrefly: ignore [invalid-variance] @overload - def get(self, key: _KT_co, default: _T2, /) -> _VT_co | _T2: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter + def get(self, key: _KT_co, default: _T2, /) -> _VT_co | _T2: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter # pyrefly: ignore [invalid-variance] def __class_getitem__(cls, item: Any, /) -> GenericAlias: """mappingproxy objects are generic over two types, signifying (respectively) the types of their keys and values""" diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/typing.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/typing.pyi index 6c4746be17..43f30f2cfd 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/typing.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/typing.pyi @@ -1860,7 +1860,7 @@ class Mapping(Collection[_KT], Generic[_KT, _VT_co]): def get(self, key: object, /) -> _VT_co | None: """D.get(k[,d]) -> D[k] if k in D, else d. d defaults to None.""" @overload - def get(self, key: object, default: _VT_co, /) -> _VT_co: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter + def get(self, key: object, default: _VT_co, /) -> _VT_co: ... # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # Covariant type as parameter # pyrefly: ignore [invalid-variance] @overload def get(self, key: object, default: _T, /) -> _VT_co | _T: ... diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/typing_extensions.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/typing_extensions.pyi index 579c487db4..de85fd8386 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/typing_extensions.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/typing_extensions.pyi @@ -991,7 +991,7 @@ else: # At runtime it inherits from ABC and is not a Protocol, but it is on the # allowlist for use as a Protocol. @runtime_checkable - class Buffer(Protocol, abc.ABC): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-protocol] + class Buffer(Protocol, abc.ABC): # type: ignore[misc] # pyright: ignore[reportGeneralTypeIssues] # ty:ignore[invalid-protocol] # pyrefly: ignore [invalid-inheritance] """Base class for classes that implement the buffer protocol. The buffer protocol allows Python objects to expose a low-level diff --git a/crates/ty_vendored/vendor/typeshed/stdlib/webbrowser.pyi b/crates/ty_vendored/vendor/typeshed/stdlib/webbrowser.pyi index 4aff0e567c..72eeaecb1f 100644 --- a/crates/ty_vendored/vendor/typeshed/stdlib/webbrowser.pyi +++ b/crates/ty_vendored/vendor/typeshed/stdlib/webbrowser.pyi @@ -43,6 +43,8 @@ def open_new_tab(url: str) -> bool: If not possible, then the behavior becomes equivalent to open_new(). """ +def register_standard_browsers() -> None: ... + class BaseBrowser: """Parent class for all browsers. Do not use directly.""" From e534072484c7bc8d86694fbb28ec9bbd566249fc Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Sat, 15 Aug 2026 03:59:43 -0700 Subject: [PATCH 051/371] [ty] Preserve required stub fields in generated constructors (#27765) Annotation-only declarations in `.pyi` files act as bindings for attribute lookup, but they are not dataclass or `NamedTuple` field defaults. Previously, ty treated those synthetic bindings as defaults, making required constructor parameters optional and reporting false `dataclass-field-order` diagnostics when subclasses added required fields. Record whether an annotated assignment has a right-hand-side value in its semantic definition, and use that information when collecting generated fields from stubs. Only inline assignments, including `= ...`, count as stub-field defaults. Ordinary Python files and stub attribute lookup retain their existing behavior. Closes astral-sh/ty#4265. ## Test plan - Dataclass mdtests cover inherited required stub fields, constructor signatures and missing-argument diagnostics, explicit ellipsis defaults, and legitimate inherited field-order diagnostics. - Dataclass-transform mdtests cover inherited required stub-model fields, subclass constructor signatures, and missing arguments. - `NamedTuple` mdtests distinguish required annotation-only stub fields from explicit ellipsis defaults and verify constructor calls and missing-argument diagnostics. --- crates/ty_python_core/src/definition.rs | 9 ++- .../mdtest/dataclasses/dataclass_transform.md | 39 +++++++++++ .../mdtest/dataclasses/dataclasses.md | 69 +++++++++++++++++++ .../resources/mdtest/named_tuple.md | 27 ++++++++ .../src/types/class/static_literal.rs | 15 +++- 5 files changed, 155 insertions(+), 4 deletions(-) diff --git a/crates/ty_python_core/src/definition.rs b/crates/ty_python_core/src/definition.rs index 42ffb515b1..1c72e67bea 100644 --- a/crates/ty_python_core/src/definition.rs +++ b/crates/ty_python_core/src/definition.rs @@ -702,6 +702,7 @@ impl<'db> DefinitionNodeRef<'_, 'db> { node, }) => DefinitionKind::AnnotatedAssignment(AnnotatedAssignmentDefinitionKind { node: AstNodeRef::new(parsed, node), + has_value: node.value.is_some(), }), DefinitionNodeRef::AugmentedAssignment(augmented_assignment) => { DefinitionKind::AugmentedAssignment(AstNodeRef::new(parsed, augmented_assignment)) @@ -1142,7 +1143,7 @@ impl<'db> DefinitionKind<'db> { // Annotated assignment is always a declaration. It is also a binding if there is a RHS // or if we are in a stub file. Unfortunately, it is common for stubs to omit even an `...` value placeholder. DefinitionKind::AnnotatedAssignment(ann_assign) => { - if in_stub || ann_assign.value(module).is_some() { + if in_stub || ann_assign.has_value() { DefinitionCategory::DeclarationAndBinding } else { DefinitionCategory::Declaration @@ -1445,6 +1446,7 @@ impl<'db> AssignmentDefinitionKind<'db> { #[derive(Clone, Debug, get_size2::GetSize)] pub struct AnnotatedAssignmentDefinitionKind { node: AstNodeRef, + has_value: bool, } impl AnnotatedAssignmentDefinitionKind { @@ -1456,6 +1458,11 @@ impl AnnotatedAssignmentDefinitionKind { self.node(module).value.as_deref() } + /// Returns whether this annotated assignment has a right-hand-side value. + pub const fn has_value(&self) -> bool { + self.has_value + } + pub fn annotation<'ast>(&self, module: &'ast ParsedModuleRef) -> &'ast ast::Expr { &self.node(module).annotation } diff --git a/crates/ty_python_semantic/resources/mdtest/dataclasses/dataclass_transform.md b/crates/ty_python_semantic/resources/mdtest/dataclasses/dataclass_transform.md index 54671cc27b..c87abf4493 100644 --- a/crates/ty_python_semantic/resources/mdtest/dataclasses/dataclass_transform.md +++ b/crates/ty_python_semantic/resources/mdtest/dataclasses/dataclass_transform.md @@ -1411,6 +1411,45 @@ class Outer: Field ordering checks apply to classes created via `dataclass_transform`, just like normal `dataclass`es. +### Required fields inherited from stub models + +An annotation-only field in a stub remains required when its model is generated by a +`dataclass_transform` base class. + +`models.pyi`: + +```pyi +from typing_extensions import dataclass_transform + +@dataclass_transform() +class ModelBase: ... + +class RequiredModel(ModelBase): + required: int +``` + +Adding another required field does not cause an ordering violation, and both constructors reject +calls that omit their required parameters. + +```py +from models import RequiredModel + +class Child(RequiredModel): + added: str + +reveal_type(RequiredModel.__init__) # revealed: (self: RequiredModel, required: int) -> None +reveal_type(Child.__init__) # revealed: (self: Child, required: int, added: str) -> None + +RequiredModel(1) +Child(1, "value") + +# error: [missing-argument] "No argument provided for required parameter `required`" +RequiredModel() + +# error: [missing-argument] "No argument provided for required parameter `added`" +Child(1) +``` + ### For function-based transformers ```py diff --git a/crates/ty_python_semantic/resources/mdtest/dataclasses/dataclasses.md b/crates/ty_python_semantic/resources/mdtest/dataclasses/dataclasses.md index 76a67c974c..2c1f2441ae 100644 --- a/crates/ty_python_semantic/resources/mdtest/dataclasses/dataclasses.md +++ b/crates/ty_python_semantic/resources/mdtest/dataclasses/dataclasses.md @@ -1888,6 +1888,75 @@ Derived(1, "a") Derived(True) ``` +### Required fields inherited from stub dataclasses + +An annotation without an assigned value in a stub does not give a dataclass field a default. + +`base.pyi`: + +```pyi +from dataclasses import dataclass + +@dataclass +class Base: + required: int +``` + +The inherited field remains required in both constructors, and adding another required field does +not introduce a field-ordering violation. + +```py +from dataclasses import dataclass +from base import Base + +@dataclass +class Child(Base): + added: str + +reveal_type(Base.__init__) # revealed: (self: Base, required: int) -> None +reveal_type(Child.__init__) # revealed: (self: Child, required: int, added: str) -> None + +Base(1) +Child(1, "value") + +# error: [missing-argument] "No argument provided for required parameter `required`" +Base() + +# error: [missing-argument] "No argument provided for required parameter `added`" +Child(1) +``` + +### Defaulted fields inherited from stub dataclasses + +An ellipsis assigned to a stub field indicates an actual default. + +`base.pyi`: + +```pyi +from dataclasses import dataclass + +@dataclass +class EllipsisDefault: + required: int + optional: int = ... +``` + +The field produces an optional constructor parameter, so adding a required field in a subclass is +invalid. + +```py +from dataclasses import dataclass +from base import EllipsisDefault + +reveal_type(EllipsisDefault.__init__) # revealed: (self: EllipsisDefault, required: int, optional: int = ...) -> None + +EllipsisDefault(1) + +@dataclass +class InvalidEllipsisChild(EllipsisDefault): + added: str # error: [dataclass-field-order] +``` + ### Required fields after inherited defaults A required positional field cannot follow a positional field with a default inherited from a diff --git a/crates/ty_python_semantic/resources/mdtest/named_tuple.md b/crates/ty_python_semantic/resources/mdtest/named_tuple.md index a10aea4fbe..ccf335b9a6 100644 --- a/crates/ty_python_semantic/resources/mdtest/named_tuple.md +++ b/crates/ty_python_semantic/resources/mdtest/named_tuple.md @@ -132,6 +132,33 @@ reveal_type(alice5.id) # revealed: int reveal_type(alice5.name) # revealed: str ``` +### Fields declared in stubs + +An annotation-only field in a stub remains a required constructor argument. An explicit ellipsis +assignment represents a default and makes its field optional. + +`records.pyi`: + +```pyi +from typing import NamedTuple + +class Record(NamedTuple): + required: int + optional: str = ... +``` + +The generated constructor requires the first field but permits omitting the second: + +```py +from records import Record + +reveal_type(Record.__new__) # revealed: [Self](_cls: type[Self], required: int, optional: str = ...) -> Self + +Record(1) +Record(1, "value") +Record() # error: [missing-argument] +``` + ### Name mismatch diagnostics diff --git a/crates/ty_python_semantic/src/types/class/static_literal.rs b/crates/ty_python_semantic/src/types/class/static_literal.rs index 8dd37c408e..691fb6236f 100644 --- a/crates/ty_python_semantic/src/types/class/static_literal.rs +++ b/crates/ty_python_semantic/src/types/class/static_literal.rs @@ -2625,11 +2625,20 @@ impl<'db> StaticClassLiteral<'db> { } if let Some(attr_ty) = attr.place.ignore_possibly_undefined() { - let mut default_ty = if field_policy == CodeGeneratorKind::TypedDict { + // Annotation-only declarations in stubs also act as bindings for attribute + // lookup, but they do not supply field defaults. + let mut default_ty = if field_policy == CodeGeneratorKind::TypedDict + || (self.file(db).is_stub(db) + && !first_declaration.is_some_and(|definition| { + matches!( + definition.kind(db), + DefinitionKind::AnnotatedAssignment(annotation) + if annotation.has_value() + ) + })) { None } else { - let bindings = use_def.end_of_scope_symbol_bindings(symbol_id); - place_from_bindings(db, &env, bindings) + place_from_bindings(db, &env, use_def.end_of_scope_symbol_bindings(symbol_id)) .place .ignore_possibly_undefined() }; From 239f2a8ef38d30282592d61b5fb947d99d18a4dc Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Sat, 15 Aug 2026 12:59:55 +0200 Subject: [PATCH 052/371] Fix s390x stacker assembly in release builds (#27776) ## Summary Pass the s390x architecture flag to release builds so that stacker builds successfully ## Test Plan Testing: Focused repository hooks passed. --- .github/workflows/build-binaries.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/build-binaries.yml b/.github/workflows/build-binaries.yml index e25be8d8e6..7caae1789b 100644 --- a/.github/workflows/build-binaries.yml +++ b/.github/workflows/build-binaries.yml @@ -411,6 +411,7 @@ jobs: - target: s390x-unknown-linux-gnu arch: s390x manylinux: 2_17 + maturin_docker_options: -e CFLAGS_s390x_unknown_linux_gnu=-march=z10 - target: powerpc64le-unknown-linux-gnu arch: ppc64le manylinux: 2_17 From 8a772d319b8ef78d6b61daa4de981776680e2286 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sat, 15 Aug 2026 12:23:02 +0100 Subject: [PATCH 053/371] [ty] Reclassify fully static property-test types (#27780) ## Summary - Reclassify `Ty::TopDivergent`, `Ty::BottomDivergent`, and `Ty::UnittestMockLiteral` as fully static in the property-test generator, per @carljm's review comments in https://github.com/astral-sh/ruff/pull/27693#discussion_r3768752055 and https://github.com/astral-sh/ruff/pull/27693#discussion_r3768762550 - Exercise these types in fully static property tests, matching their existing `Type::is_fully_static` behavior. ## Test plan Codex ran the stable property tests with 100,000 QuickCheck cases. Two tests failed, `disjoint_from_is_irreflexive` and `subtype_of_implies_not_disjoint_from`, but both failures appear to be pre-existing: they reproduce on the unmodified base and were traced to an earlier change that preserved tuple types containing `Never`. See [my comment on the PR that introduced the failures](https://github.com/astral-sh/ruff/pull/27580#issuecomment-5292594407) and [the automatically opened tracking issue](https://github.com/astral-sh/ty/issues/4263). --- .../src/types/property_tests/type_generation.rs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/crates/ty_python_semantic/src/types/property_tests/type_generation.rs b/crates/ty_python_semantic/src/types/property_tests/type_generation.rs index 9bef4c30d7..fa14aa00f9 100644 --- a/crates/ty_python_semantic/src/types/property_tests/type_generation.rs +++ b/crates/ty_python_semantic/src/types/property_tests/type_generation.rs @@ -458,14 +458,13 @@ fn arbitrary_core_type(g: &mut Gen, fully_static: bool) -> Ty { Ty::Any, Ty::Unknown, Ty::Divergent, - Ty::TopDivergent, - Ty::BottomDivergent, Ty::SubclassOfAny, - Ty::UnittestMockLiteral, Ty::UnittestMockInstance, ], fully_static_types: [ Ty::Never, + Ty::TopDivergent, + Ty::BottomDivergent, Ty::None, int_lit, bool_lit, @@ -490,6 +489,7 @@ fn arbitrary_core_type(g: &mut Gen, fully_static: bool) -> Ty { Ty::KnownClassInstance(KnownClass::TypeAliasType), Ty::KnownClassInstance(KnownClass::NoDefaultType), Ty::TypingLiteral, + Ty::UnittestMockLiteral, Ty::BuiltinClassLiteral("str"), Ty::BuiltinClassLiteral("int"), Ty::BuiltinClassLiteral("bool"), From 6d4d692526580e0b4b1e38e52c4a6dd4786868d2 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sat, 15 Aug 2026 12:26:46 +0100 Subject: [PATCH 054/371] [ty] Update ecosystem-analyzer workflow pins (#27523) Updates the ecosystem-analyzer revisions used by the ty ecosystem-analyzer and ty ecosystem-report workflows to the latest upstream `main` commit, `585f99dd3eb90f992d6764684fb1b83506ceb938`. This keeps both workflows aligned with the latest ecosystem-analyzer improvements. --- .github/workflows/ty-ecosystem-analyzer.yaml | 2 +- .github/workflows/ty-ecosystem-report.yaml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ty-ecosystem-analyzer.yaml b/.github/workflows/ty-ecosystem-analyzer.yaml index 8935ebac58..dea4e2ac02 100644 --- a/.github/workflows/ty-ecosystem-analyzer.yaml +++ b/.github/workflows/ty-ecosystem-analyzer.yaml @@ -44,7 +44,7 @@ env: CARGO_PROFILE_PROFILING_DEBUG: line-tables-only # TODO: Update the mypy-primer revision in scripts/setup_primer_project.py # and regenerate its lockfile when updating ecosystem-analyzer. - ECOSYSTEM_ANALYZER_COMMIT: 27b644f296d70fccacb7d7c23c91c5d6ccd8713d + ECOSYSTEM_ANALYZER_COMMIT: 2b409ad30445f16b863919e0799b438c6b04c645 jobs: build-ty: diff --git a/.github/workflows/ty-ecosystem-report.yaml b/.github/workflows/ty-ecosystem-report.yaml index 9bcfc24f1f..b37d675328 100644 --- a/.github/workflows/ty-ecosystem-report.yaml +++ b/.github/workflows/ty-ecosystem-report.yaml @@ -20,7 +20,7 @@ env: RUST_BACKTRACE: 1 # Line-tables-only debug info: faster builds, backtraces still work. CARGO_PROFILE_PROFILING_DEBUG: line-tables-only - ECOSYSTEM_ANALYZER_COMMIT: 27b644f296d70fccacb7d7c23c91c5d6ccd8713d + ECOSYSTEM_ANALYZER_COMMIT: 2b409ad30445f16b863919e0799b438c6b04c645 jobs: ty-ecosystem-report: From 36cc91ffb72adde11f0c4dfb76330a07481f6153 Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Sat, 15 Aug 2026 07:31:55 -0400 Subject: [PATCH 055/371] Optimize development builds by default (#27562) ## Summary Make `opt-level = 1`, `debug = "line-tables-only"`, and `lto = "off"` the default development settings. This makes ordinary Cargo commands match our existing agent workflow, avoids local ThinLTO, and removes repeated profile overrides from `AGENTS.md`. Preserve existing unoptimized CI behavior by explicitly setting `CARGO_PROFILE_DEV_OPT_LEVEL=0` and `CARGO_PROFILE_DEV_LTO=false` in the affected workflows, including documentation generation, playground validation, typeshed synchronization, typing conformance, and daily fuzzing. Existing CI commands, cache configuration, and cache producers remain unchanged. Release and profiling settings are unaffected. This follows the discussion in https://github.com/astral-sh/ruff/pull/27526#issuecomment-5206579931. --- .github/workflows/ci.yaml | 2 ++ .github/workflows/daily_fuzz.yaml | 2 ++ .github/workflows/publish-docs.yml | 4 ++++ .github/workflows/publish-playground.yml | 2 ++ .github/workflows/publish-ty-playground.yml | 2 ++ .github/workflows/sync_typeshed.yaml | 2 ++ .github/workflows/typing_conformance.yaml | 2 ++ AGENTS.md | 24 ++++++++++----------- Cargo.toml | 6 ++++++ 9 files changed, 34 insertions(+), 12 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 2a3953afb1..3b2e823123 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -19,6 +19,8 @@ defaults: env: CARGO_INCREMENTAL: 0 CARGO_NET_RETRY: 10 + CARGO_PROFILE_DEV_LTO: "false" + CARGO_PROFILE_DEV_OPT_LEVEL: 0 CARGO_TERM_COLOR: always RUSTUP_MAX_RETRIES: 10 PACKAGE_NAME: ruff diff --git a/.github/workflows/daily_fuzz.yaml b/.github/workflows/daily_fuzz.yaml index 3411600ca2..230c8e1265 100644 --- a/.github/workflows/daily_fuzz.yaml +++ b/.github/workflows/daily_fuzz.yaml @@ -22,6 +22,8 @@ env: RUSTUP_MAX_RETRIES: 10 # Line-tables-only debug info: faster builds, backtraces still work. CARGO_PROFILE_DEV_DEBUG: line-tables-only + CARGO_PROFILE_DEV_LTO: "false" + CARGO_PROFILE_DEV_OPT_LEVEL: 0 PACKAGE_NAME: ruff FORCE_COLOR: 1 diff --git a/.github/workflows/publish-docs.yml b/.github/workflows/publish-docs.yml index e773a08a44..b95f7d0b6e 100644 --- a/.github/workflows/publish-docs.yml +++ b/.github/workflows/publish-docs.yml @@ -20,6 +20,10 @@ on: permissions: contents: read +env: + CARGO_PROFILE_DEV_LTO: "false" + CARGO_PROFILE_DEV_OPT_LEVEL: 0 + jobs: mkdocs: environment: diff --git a/.github/workflows/publish-playground.yml b/.github/workflows/publish-playground.yml index 4f407c0618..cd793d8257 100644 --- a/.github/workflows/publish-playground.yml +++ b/.github/workflows/publish-playground.yml @@ -15,6 +15,8 @@ on: env: CARGO_INCREMENTAL: 0 CARGO_NET_RETRY: 10 + CARGO_PROFILE_DEV_LTO: "false" + CARGO_PROFILE_DEV_OPT_LEVEL: 0 CARGO_TERM_COLOR: always RUSTUP_MAX_RETRIES: 10 diff --git a/.github/workflows/publish-ty-playground.yml b/.github/workflows/publish-ty-playground.yml index 0dee4e2491..03c84b620b 100644 --- a/.github/workflows/publish-ty-playground.yml +++ b/.github/workflows/publish-ty-playground.yml @@ -21,6 +21,8 @@ concurrency: env: CARGO_INCREMENTAL: 0 CARGO_NET_RETRY: 10 + CARGO_PROFILE_DEV_LTO: "false" + CARGO_PROFILE_DEV_OPT_LEVEL: 0 CARGO_TERM_COLOR: always RUSTUP_MAX_RETRIES: 10 diff --git a/.github/workflows/sync_typeshed.yaml b/.github/workflows/sync_typeshed.yaml index 6919d9f143..f71dff9971 100644 --- a/.github/workflows/sync_typeshed.yaml +++ b/.github/workflows/sync_typeshed.yaml @@ -44,6 +44,8 @@ env: CARGO_NET_RETRY: 10 CARGO_PROFILE_DEV_DEBUG: line-tables-only + CARGO_PROFILE_DEV_LTO: "false" + CARGO_PROFILE_DEV_OPT_LEVEL: 0 CARGO_TERM_COLOR: always NEXTEST_PROFILE: "ci" RUSTUP_MAX_RETRIES: 10 diff --git a/.github/workflows/typing_conformance.yaml b/.github/workflows/typing_conformance.yaml index bd276058ac..1c8b9bc594 100644 --- a/.github/workflows/typing_conformance.yaml +++ b/.github/workflows/typing_conformance.yaml @@ -36,6 +36,8 @@ env: RUST_BACKTRACE: 1 # Line-tables-only debug info: faster builds, backtraces still work. CARGO_PROFILE_DEV_DEBUG: line-tables-only + CARGO_PROFILE_DEV_LTO: "false" + CARGO_PROFILE_DEV_OPT_LEVEL: 0 CONFORMANCE_SUITE_COMMIT: bee91c2646261629c9835dc0adad16e0d554b4a5 PYTHON_VERSION: 3.12 diff --git a/AGENTS.md b/AGENTS.md index 73d93ad54c..86874bef46 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -21,7 +21,7 @@ instructions to PR authors or flag unrelated pre-existing issues. Run all tests (using `nextest` for faster execution and setting `INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1` to ensure all snapshots are updated): ```sh -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run +INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run ``` File-watcher tests do not work inside the sandbox. It is usually unnecessary to run them locally before filing a change unless you are certain that the change affects file-watching behavior. @@ -29,19 +29,19 @@ File-watcher tests do not work inside the sandbox. It is usually unnecessary to Run tests for a specific crate: ```sh -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run -p ty_python_semantic +INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run -p ty_python_semantic ``` Run a single mdtest file. The path to the mdtest file should be relative to the `crates/ty_python_semantic/resources/mdtest` folder. Include `--test mdtest` to avoid building unrelated test binaries: ```sh -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run -p ty_python_semantic --test mdtest -- mdtest:: +INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run -p ty_python_semantic --test mdtest -- mdtest:: ``` To run a specific mdtest within a file, use a substring of the Markdown header text as `MDTEST_TEST_FILTER`. Only use this if it's necessary to isolate a single test case: ```sh -MDTEST_TEST_FILTER="" CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run -p ty_python_semantic --test mdtest -- mdtest:: +MDTEST_TEST_FILTER="" INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1 cargo nextest run -p ty_python_semantic --test mdtest -- mdtest:: ``` ### Fallback without nextest @@ -50,16 +50,16 @@ If `cargo nextest` is not available, use `cargo test` with the same environment ```sh # Run all tests. -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo test +INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1 cargo test # Run tests for a specific crate. -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo test -p ty_python_semantic +INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1 cargo test -p ty_python_semantic # Run a single mdtest file. -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo test -p ty_python_semantic --test mdtest -- +INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1 cargo test -p ty_python_semantic --test mdtest -- # Run a specific mdtest within a file. -MDTEST_TEST_FILTER="" CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off INSTA_FORCE_PASS=1 INSTA_UPDATE=always CARGO_PROFILE_DEV_DEBUG="line-tables-only" MDTEST_UPDATE_SNAPSHOTS=1 cargo test -p ty_python_semantic --test mdtest -- +MDTEST_TEST_FILTER="" INSTA_FORCE_PASS=1 INSTA_UPDATE=always MDTEST_UPDATE_SNAPSHOTS=1 cargo test -p ty_python_semantic --test mdtest -- ``` ### Snapshot updates @@ -82,7 +82,7 @@ Never edit snapshot files or inline snapshot bodies manually. Regenerate them by ## Running Clippy ```sh -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off CARGO_PROFILE_DEV_DEBUG="line-tables-only" cargo clippy --workspace --all-targets --all-features -- -D warnings +cargo clippy --workspace --all-targets --all-features -- -D warnings ``` ## Running Debug Builds @@ -92,13 +92,13 @@ Use debug builds (not `--release`) when developing, as release builds lack debug Run Ruff: ```sh -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off CARGO_PROFILE_DEV_DEBUG="line-tables-only" cargo run --bin ruff -- check path/to/file.py +cargo run --bin ruff -- check path/to/file.py ``` Run ty: ```sh -CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off CARGO_PROFILE_DEV_DEBUG="line-tables-only" cargo run --bin ty -- check path/to/file.py +cargo run --bin ty -- check path/to/file.py ``` ## Working on ty @@ -176,7 +176,7 @@ Parts of `.github/workflows/release.yml` are generated by cargo-dist from `dist- - Prefer let chains (`if let` combined with `&&`) and let guards (`PAT if let ... =>`) over nested `if let` statements to reduce indentation and improve readability. At the end of a task, always check your work to see if you missed opportunities to use `let` chains or `let` guards. - If you _have_ to suppress a Clippy lint, prefer to use `#[expect()]` over `[allow()]`, where possible. But if a lint is complaining about unused/dead code, it's usually best to just delete the unused code. - Don't use comments to narrate code, but do use them to explain invariants and why something unusual was done a particular way. Make sure that a comment will make sense to somebody who's reading the code for the first time. Prefer plain language, avoid jargon, and don't be afraid to be more verbose if it's necessary to explain something well. Giving examples of the kind of Python code we're trying to model at this particular point in Ruff or ty can often be very helpful for future readers of the code. -- Run `CARGO_PROFILE_DEV_OPT_LEVEL=1 CARGO_PROFILE_DEV_LTO=off CARGO_PROFILE_DEV_DEBUG="line-tables-only" cargo dev generate-all` after changing configuration options, CLI arguments, lint rules, or environment variable definitions, as these changes require regeneration of schemas, docs, and CLI references. +- Run `cargo dev generate-all` after changing configuration options, CLI arguments, lint rules, or environment variable definitions, as these changes require regeneration of schemas, docs, and CLI references. - Don't prefix tests with `test_`. - Don't separate struct definitions from their `impl` blocks unless the `impl` is deliberately placed in a separate file, as for large structs. - Avoid running `uv run` for any scripts from the repository root unless you use `--no-project`, `--script` or similar. Using `uv run` from the Ruff repo root without these flags will build Ruff from source, which is very slow and usually unnecessary. diff --git a/Cargo.toml b/Cargo.toml index 930cee4ce2..9c854d1227 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -301,6 +301,12 @@ inherits = "release" opt-level = "z" codegen-units = 1 +[profile.dev] +opt-level = 1 +debug = "line-tables-only" +# Avoid the local ThinLTO that Cargo enables at nonzero optimization levels. +lto = "off" + [profile.dev.package.insta] opt-level = 3 From ae40859adde4a9b87b3e2bcee020e0c26131ab54 Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Sat, 15 Aug 2026 09:32:05 -0400 Subject: [PATCH 056/371] [ty] Model exception-suppressing context managers (#27219) ## Summary Previously, we assumed that a `with` body ran to completion whenever execution continued past it. A context manager can instead suppress an exception, leaving bindings from before the raising operation visible after the statement: ```python from contextlib import suppress def first(values: list[int]) -> int | None: result = None with suppress(StopIteration): result = next(iter(values)) return result ``` We now reuse exception checkpoints to model synchronous and asynchronous context managers, including exceptions raised while assigning an `as` target, entering a later manager, or evaluating a return expression. Bare control-flow transfers remain terminal, and checkpoints respect nested exception handlers, eager versus lazy evaluation, deleted bindings, and enclosing `finally` blocks. Following the typing specification, a concrete manager can suppress exceptions when its exit method returns `bool` or `Literal[True]`; `None`, `Literal[False]`, `Any`, and `bool | None` do not. For a union of manager types, we classify each alternative independently, so the union can suppress exceptions whenever any concrete manager can. Closes https://github.com/astral-sh/ty/issues/152. --- crates/ty_python_core/src/builder.rs | 218 ++++++-- .../src/builder/except_handlers.rs | 289 +++++++--- crates/ty_python_core/src/predicate.rs | 18 + .../resources/mdtest/terminal_statements.md | 393 +++++++++++++ .../resources/mdtest/with/async.md | 237 ++++++++ .../resources/mdtest/with/sync.md | 520 ++++++++++++++++++ crates/ty_python_semantic/src/reachability.rs | 58 +- .../src/types/context_manager.rs | 144 ++++- .../src/types/infer/builder.rs | 4 +- crates/ty_python_semantic/src/types/narrow.rs | 12 +- 10 files changed, 1765 insertions(+), 128 deletions(-) diff --git a/crates/ty_python_core/src/builder.rs b/crates/ty_python_core/src/builder.rs index c812523590..a446878577 100644 --- a/crates/ty_python_core/src/builder.rs +++ b/crates/ty_python_core/src/builder.rs @@ -1,7 +1,7 @@ use std::cell::{OnceCell, RefCell}; use std::sync::Arc; -use except_handlers::{ExceptionHandlers, TryNodeContextStackManager}; +use except_handlers::{ExceptionContextStackManager, ExceptionHandlers}; use itertools::Itertools; use ruff_python_ast::helpers::{Truthiness, any_over_expr, is_dotted_name}; use rustc_hash::{FxHashMap, FxHashSet}; @@ -244,8 +244,8 @@ pub(super) struct SemanticIndexBuilder<'db, 'ast> { /// The name of the first function parameter of the innermost function that we're currently visiting. current_first_parameter_name: Option<&'ast str>, - /// Per-scope contexts regarding nested `try`/`except` statements - try_node_context_stack_manager: TryNodeContextStackManager, + /// Per-scope exception contexts for nested `try` and `with` statements. + exception_context_stack_manager: ExceptionContextStackManager, /// Flags about the file's global scope has_future_annotations: bool, @@ -257,7 +257,9 @@ pub(super) struct SemanticIndexBuilder<'db, 'ast> { python_version: PythonVersion, source_text: OnceCell, semantic_checker: SemanticSyntaxChecker, - in_try: bool, + /// Whether the current statement is inside a `try` statement, including its `except`, `else`, + /// and `finally` suites. Used for semantic syntax checks independently of handler activity. + in_try_statement: bool, // Semantic Index fields scopes: IndexVec, @@ -316,7 +318,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { current_statements: Vec::new(), current_match_case: None, current_first_parameter_name: None, - try_node_context_stack_manager: TryNodeContextStackManager::default(), + exception_context_stack_manager: ExceptionContextStackManager::default(), has_future_annotations: false, in_type_checking_block: false, @@ -349,7 +351,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { python_version: file.python_version(db), source_text: OnceCell::new(), semantic_checker: SemanticSyntaxChecker::default(), - in_try: false, + in_try_statement: false, semantic_syntax_errors: RefCell::default(), narrowing_aliases: FxHashMap::default(), alias_predicates: FxHashMap::default(), @@ -509,7 +511,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { let scope = Scope::new(parent, node_with_kind, children_start..children_start); let is_class_scope = scope.kind().is_class(); - self.try_node_context_stack_manager.enter_nested_scope(); + self.exception_context_stack_manager.enter_nested_scope(); let file_scope_id = self.scopes.push(scope); self.place_tables.push(PlaceTableBuilder::default()); @@ -898,7 +900,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { /// scope, including those contributed by `global` and `nonlocal` keywords in the popped scope, /// but excluding nested `nonlocal`s that resolved to the popped scope. fn pop_scope(&mut self) -> NestedGlobalOrNonlocalDeclarations { - self.try_node_context_stack_manager.exit_scope(); + self.exception_context_stack_manager.exit_scope(); let ScopeInfo { file_scope_id: popped_scope_id, @@ -2172,6 +2174,8 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { } PredicateNode::SubjectElementPattern(_) | PredicateNode::IsNonTerminalCall(_) + | PredicateNode::ContextManagerSuppresses { .. } + | PredicateNode::FinallyNormalPathImpossible { .. } | PredicateNode::IsNonEmptyIterable(_) | PredicateNode::OrPatternAlternative(_) | PredicateNode::StarImportPlaceholder(_) => { @@ -2207,9 +2211,10 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { /// Records that the current state can enter any active `finally` suites before the current /// terminal control-flow transfer reaches its destination. fn record_terminal_finally_entry(&mut self) { - let mut try_node_stack_manager = std::mem::take(&mut self.try_node_context_stack_manager); - try_node_stack_manager.record_terminal_finally_entry(self); - self.try_node_context_stack_manager = try_node_stack_manager; + let mut exception_context_stack_manager = + std::mem::take(&mut self.exception_context_stack_manager); + exception_context_stack_manager.record_terminal_finally_entry(self); + self.exception_context_stack_manager = exception_context_stack_manager; } /// Returns whether an exception raised while evaluating `scope` can propagate directly to its @@ -2242,19 +2247,19 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { /// reveal_type(state) # Literal[1] /// ``` /// - /// Skips snapshot construction entirely when no enclosing `try` suite has active handlers. + /// Skips snapshot construction when no enclosing `try` or `with` context can handle exceptions. fn record_exception_checkpoint(&mut self) { - if !self.in_try - || !self - .try_node_context_stack_manager - .has_active_exception_handler(self) + if !self + .exception_context_stack_manager + .has_active_exception_handler(self) { return; } - let mut try_node_stack_manager = std::mem::take(&mut self.try_node_context_stack_manager); - try_node_stack_manager.record_exception_checkpoint(self); - self.try_node_context_stack_manager = try_node_stack_manager; + let mut exception_context_stack_manager = + std::mem::take(&mut self.exception_context_stack_manager); + exception_context_stack_manager.record_exception_checkpoint(self); + self.exception_context_stack_manager = exception_context_stack_manager; } fn record_exception_checkpoint_if(&mut self, can_raise: bool) { @@ -2273,9 +2278,8 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { }; is_use - && self.in_try && self - .try_node_context_stack_manager + .exception_context_stack_manager .has_active_exception_handler(self) && self .current_place_table() @@ -3647,7 +3651,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { if msg.is_some() || self - .try_node_context_stack_manager + .exception_context_stack_manager .has_active_exception_handler(self) { let truthy = if let Some(snapshots) = condition_flow_snapshot.into_branches() { @@ -4021,6 +4025,9 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { self.visit_expr(context_expr); self.record_exception_checkpoint(); + self.exception_context_stack_manager + .push_context_manager_context(); + if let Some(optional_vars) = optional_vars.as_deref() { let context_manager = self.add_standalone_expression(context_expr); self.add_unpackable_assignment( @@ -4033,8 +4040,62 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { ); } } + self.visit_body(body); - self.record_exception_checkpoint(); + + for item in items.iter().rev() { + let mut exceptional_entries = self + .exception_context_stack_manager + .finish_context_manager_context() + .into_iter(); + + if let Some(exceptional_entry) = exceptional_entries.next() { + let normal_exit = self.flow_snapshot(); + if normal_exit.is_always_unreachable() { + self.exception_context_stack_manager + .record_deferred_terminal_context_manager_exit(); + } + let context_expr = &item.context_expr; + let expression = self + .expressions_by_node + .get(&ExpressionNodeKey::from(context_expr)) + .copied() + .unwrap_or_else(|| self.add_standalone_expression(context_expr)); + let predicate = PredicateOrLiteral::Predicate(Predicate { + node: PredicateNode::ContextManagerSuppresses { + expression, + is_async: *is_async, + }, + is_positive: true, + }); + let predicate_id = self.add_predicate(predicate); + + self.flow_restore(exceptional_entry); + for exceptional_entry in exceptional_entries { + self.flow_merge(exceptional_entry); + } + + self.record_ambiguous_reachability(); + let reachability_constraint = self + .current_reachability_constraints_mut() + .add_atom(predicate_id); + let narrowing_constraint = self + .current_use_def_map_mut() + .narrowing_constraints + .add_atom(predicate_id); + self.current_use_def_map_mut() + .record_non_terminal_call_constraints( + reachability_constraint, + narrowing_constraint, + ); + + self.flow_merge(normal_exit); + } + + // A manager cannot suppress an exception raised by its own exit method, but + // an earlier manager or enclosing `try` statement can still receive it. + self.record_exception_checkpoint(); + } } ast::Stmt::For( @@ -4337,7 +4398,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { range: _, node_index: _, }) => { - let was_in_try = std::mem::replace(&mut self.in_try, true); + let was_in_try_statement = std::mem::replace(&mut self.in_try_statement, true); self.record_ambiguous_reachability(); let exception_handlers = if handlers.is_empty() { @@ -4350,8 +4411,8 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { } else { ExceptionHandlers::propagating() }; - self.try_node_context_stack_manager - .push_context(exception_handlers); + self.exception_context_stack_manager + .push_try_context(exception_handlers, !finalbody.is_empty()); // Visit the `try` block! self.visit_body(body); @@ -4362,7 +4423,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { // that may raise. Keep the context itself on the stack so that terminal statements // in `except` and `else` suites can still be recorded as entries to the associated // `finally` suite. - let try_block_snapshots = self.try_node_context_stack_manager.end_try_suite(); + let try_block_snapshots = self.exception_context_stack_manager.end_try_suite(); if !handlers.is_empty() { // Save the state immediately *after* visiting the `try` block @@ -4449,9 +4510,13 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { } let normal_pre_finally_state = self.flow_snapshot(); - let (terminal_finally_entry_snapshots, has_escaping_exception) = self - .try_node_context_stack_manager - .pop_context() + let ( + terminal_finally_entry_snapshots, + has_escaping_exception, + has_deferred_terminal_context_manager_exit, + ) = self + .exception_context_stack_manager + .pop_try_context() .into_finally_entry_state(); // TODO: there's lots of complexity here that isn't yet handled by our model. // In order to accurately model the semantics of `finally` suites, we in fact need to visit @@ -4481,6 +4546,53 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { } self.mark_unreachable(); } else { + let mut post_finally_terminal_predicate = None; + let mut terminal_snapshots = terminal_finally_entry_snapshots.into_iter(); + if has_deferred_terminal_context_manager_exit + && let Some(snapshot) = terminal_snapshots.next() + { + let continuation = self.current_use_def_map().reachability; + self.current_reachability_constraints_mut() + .mark_used(continuation); + let predicate_id = + self.add_predicate(PredicateOrLiteral::Predicate(Predicate { + node: PredicateNode::FinallyNormalPathImpossible { + scope: self.current_scope_id(), + continuation, + }, + is_positive: true, + })); + + self.flow_restore(snapshot); + for snapshot in terminal_snapshots { + self.flow_merge(snapshot); + } + + let reachability_constraint = self + .current_reachability_constraints_mut() + .add_atom(predicate_id); + let narrowing_constraint = self + .current_use_def_map_mut() + .narrowing_constraints + .add_atom(predicate_id); + self.current_use_def_map_mut() + .record_non_terminal_call_constraints( + reachability_constraint, + narrowing_constraint, + ); + + if finalbody.is_empty() { + let terminal_snapshot = self.flow_snapshot(); + self.flow_restore(normal_pre_finally_state); + self.exception_context_stack_manager + .propagate_deferred_terminal_context_manager_exit( + terminal_snapshot, + ); + } else { + self.flow_merge(normal_pre_finally_state); + post_finally_terminal_predicate = Some(predicate_id); + } + } // Mixed normal and terminal entry states are still handled by the normal path // only. See the corresponding TODO tests in `terminal_statements.md`. self.visit_body(finalbody); @@ -4491,8 +4603,44 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { { self.record_exception_checkpoint(); } + + if let Some(predicate_id) = post_finally_terminal_predicate + && self.current_use_def_map().reachability + != ScopedReachabilityConstraintId::ALWAYS_FALSE + { + let post_finally_state = self.flow_snapshot(); + let terminal_reachability = self + .current_reachability_constraints_mut() + .add_atom(predicate_id); + let terminal_narrowing = self + .current_use_def_map_mut() + .narrowing_constraints + .add_atom(predicate_id); + self.current_use_def_map_mut() + .record_non_terminal_call_constraints( + terminal_reachability, + terminal_narrowing, + ); + let terminal_snapshot = self.flow_snapshot(); + self.flow_restore(post_finally_state); + self.exception_context_stack_manager + .propagate_deferred_terminal_context_manager_exit(terminal_snapshot); + + let normal_reachability = self + .current_reachability_constraints_mut() + .add_not_constraint(terminal_reachability); + let normal_narrowing = self + .current_use_def_map_mut() + .narrowing_constraints + .add_negated_atom(predicate_id); + self.current_use_def_map_mut() + .record_non_terminal_call_constraints( + normal_reachability, + normal_narrowing, + ); + } } - self.in_try = was_in_try; + self.in_try_statement = was_in_try_statement; } ast::Stmt::Raise(_) => { @@ -4511,6 +4659,12 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { } ast::Stmt::Continue(_) | ast::Stmt::Break(_) => { + if self + .exception_context_stack_manager + .has_context_manager_exception_checkpoint() + { + self.record_ambiguous_reachability(); + } let snapshot = self.flow_snapshot(); if let Some(current_loop) = self.current_loop_mut() { if stmt.is_continue_stmt() { @@ -5382,7 +5536,7 @@ impl SemanticSyntaxContext for SemanticIndexBuilder<'_, '_> { | ScopeKind::TypeParams => {} } - if self.in_try { + if self.in_try_statement { return Some(LazyImportContext::TryExceptBlocks); } diff --git a/crates/ty_python_core/src/builder/except_handlers.rs b/crates/ty_python_core/src/builder/except_handlers.rs index 44a612bfe1..750d213b24 100644 --- a/crates/ty_python_core/src/builder/except_handlers.rs +++ b/crates/ty_python_core/src/builder/except_handlers.rs @@ -33,23 +33,27 @@ impl ExceptionHandlers { } } -/// An abstraction over the fact that each scope should have its own [`TryNodeContextStack`] +/// Maintains a separate [`ExceptionContextStack`] for each scope. #[derive(Debug, Default)] -pub(super) struct TryNodeContextStackManager(Vec); +pub(super) struct ExceptionContextStackManager { + stacks: Vec, + /// Number of `try` and `with` contexts still collecting exception checkpoints. + active_handler_count: usize, +} -impl TryNodeContextStackManager { - /// Push a new [`TryNodeContextStack`] onto the stack of stacks. +impl ExceptionContextStackManager { + /// Push a new [`ExceptionContextStack`] onto the stack of stacks. /// - /// Each [`TryNodeContextStack`] is only valid for a single scope + /// Each [`ExceptionContextStack`] is only valid for a single scope. pub(super) fn enter_nested_scope(&mut self) { - self.0.push(TryNodeContextStack::default()); + self.stacks.push(ExceptionContextStack::default()); } - /// Pop a new [`TryNodeContextStack`] off the stack of stacks. + /// Pop an [`ExceptionContextStack`] off the stack of stacks. /// - /// Each [`TryNodeContextStack`] is only valid for a single scope + /// Each [`ExceptionContextStack`] is only valid for a single scope. pub(super) fn exit_scope(&mut self) { - let popped_context = self.0.pop(); + let popped_context = self.stacks.pop(); debug_assert!( popped_context.is_some(), "exit_scope() should never be called on an empty stack \ @@ -57,37 +61,64 @@ impl TryNodeContextStackManager { ); } - /// Push a [`TryNodeContext`] onto the [`TryNodeContextStack`] at the top of our stack of - /// stacks. + /// Registers a `try` statement on the current scope's exception-context stack. /// /// Only suites with handlers collect exception checkpoints; a bare handler prevents those /// exceptions from propagating to enclosing suites. - pub(super) fn push_context(&mut self, exception_handlers: ExceptionHandlers) { - self.current_try_context_stack() - .push_context(exception_handlers); + pub(super) fn push_try_context( + &mut self, + exception_handlers: ExceptionHandlers, + has_finally: bool, + ) { + self.active_handler_count += usize::from(exception_handlers.is_active()); + self.current_exception_context_stack() + .push_try_context(exception_handlers, has_finally); + } + + /// Registers a context manager after it enters but before its target is assigned. + pub(super) fn push_context_manager_context(&mut self) { + self.active_handler_count += 1; + self.current_exception_context_stack() + .push_context_manager_context(); + } + + /// Removes the innermost context manager and returns the exceptions it could suppress. + /// + /// Removing the context before its exit method runs prevents it from suppressing exceptions + /// raised by its own exit method. + pub(super) fn finish_context_manager_context(&mut self) -> Vec { + let snapshots = self.take_exception_snapshots(); + let context = self.current_exception_context_stack().pop_context(); + debug_assert!(matches!(context.kind, ExceptionContextKind::With)); + snapshots } - /// Pop a [`TryNodeContext`] off the [`TryNodeContextStack`] at the top of our stack of stacks. - pub(super) fn pop_context(&mut self) -> TryNodeContext { - self.current_try_context_stack().pop_context() + /// Removes the current `try` context after its handlers have been deactivated. + pub(super) fn pop_try_context(&mut self) -> ExceptionContext { + let context = self.current_exception_context_stack().pop_context(); + debug_assert!(matches!(context.kind, ExceptionContextKind::Try { .. })); + debug_assert!(!context.exception_handlers.is_active()); + context } - /// Retrieve the [`TryNodeContext`] that is currently at the top of the stack, and take all + /// Retrieve the [`ExceptionContext`] at the top of the stack, and take all /// snapshots recorded while visiting the `try` suite. /// /// Taking the snapshots deactivates the suite's handlers before their bodies are visited. pub(super) fn end_try_suite(&mut self) -> Vec { - self.current_try_context_stack().end_try_suite() + self.take_exception_snapshots() } - /// Record a checkpoint for every active `try` suite that could handle an exception raised at - /// the current point in control flow. + /// Records a checkpoint for every active `try` or `with` context that could handle an + /// exception raised at the current point in control flow. /// /// Crosses eager scopes, but stops at lazy scopes, unreachable flow, and bare handlers. pub(super) fn record_exception_checkpoint(&mut self, builder: &mut SemanticIndexBuilder) { - debug_assert_eq!(self.0.len(), builder.scope_stack.len()); + debug_assert_eq!(self.stacks.len(), builder.scope_stack.len()); - for (scope_stack_index, try_context_stack) in self.0.iter_mut().enumerate().rev() { + let mut has_intervening_finally = false; + for (scope_stack_index, exception_context_stack) in self.stacks.iter_mut().enumerate().rev() + { let scope_id = builder.scope_stack[scope_stack_index].file_scope_id; let use_def_map = &builder.use_def_maps[scope_id]; @@ -97,7 +128,9 @@ impl TryNodeContextStackManager { break; } - if !try_context_stack.record_exception_checkpoint(use_def_map) { + if !exception_context_stack + .record_exception_checkpoint(use_def_map, &mut has_intervening_finally) + { break; } @@ -107,14 +140,18 @@ impl TryNodeContextStackManager { } } - /// Returns whether an active `try` suite can receive an exception from the current scope. + /// Returns whether an active `try` or `with` context can receive an exception from this scope. /// /// A context can remain on the stack for its `finally` suite after its handlers become inactive. pub(super) fn has_active_exception_handler(&self, builder: &SemanticIndexBuilder) -> bool { - debug_assert_eq!(self.0.len(), builder.scope_stack.len()); + if self.active_handler_count == 0 { + return false; + } + + debug_assert_eq!(self.stacks.len(), builder.scope_stack.len()); - for (scope_stack_index, try_context_stack) in self.0.iter().enumerate().rev() { - if try_context_stack.has_active_exception_handler() { + for (scope_stack_index, exception_context_stack) in self.stacks.iter().enumerate().rev() { + if exception_context_stack.has_active_exception_handler() { return true; } @@ -127,84 +164,139 @@ impl TryNodeContextStackManager { false } - /// Retrieve the stack that is at the top of our stack of stacks. - /// Push the snapshot onto the innermost `try` block's terminal-entry snapshots for its - /// `finally` suite. + /// Returns whether an enclosing context manager has already seen an exception checkpoint. + pub(super) fn has_context_manager_exception_checkpoint(&self) -> bool { + self.stacks.last().is_some_and(|stack| { + stack.0.iter().any(|context| { + matches!(context.kind, ExceptionContextKind::With) + && context.last_checkpoint_key.is_some() + }) + }) + } + + /// Records that a context manager makes an apparently terminal control-flow path possibly + /// non-terminal because it may silence an earlier exception. Whether it actually suppresses + /// exceptions is determined during type inference. + pub(super) fn record_deferred_terminal_context_manager_exit(&mut self) { + if let Some(context) = self.current_exception_context_stack().innermost_try() { + context.has_deferred_terminal_context_manager_exit = true; + } + } + + /// Forwards a deferred terminal state to the nearest enclosing `try`. + pub(super) fn propagate_deferred_terminal_context_manager_exit( + &mut self, + terminal_snapshot: FlowSnapshot, + ) { + if let Some(context) = self.current_exception_context_stack().innermost_try() { + context.has_deferred_terminal_context_manager_exit = true; + context + .terminal_finally_entry_snapshots + .push(terminal_snapshot); + } + } + + /// Records a terminal entry for the nearest enclosing `try`, skipping `with` contexts. pub(super) fn record_terminal_finally_entry(&mut self, builder: &SemanticIndexBuilder) { - self.current_try_context_stack() + self.current_exception_context_stack() .record_terminal_finally_entry(builder); } - /// Retrieve the [`TryNodeContextStack`] that is relevant for the current scope. - fn current_try_context_stack(&mut self) -> &mut TryNodeContextStack { - self.0 + /// Takes the current context's snapshots and updates the number of active handlers. + fn take_exception_snapshots(&mut self) -> Vec { + if let Some(snapshots) = self + .current_exception_context_stack() + .take_exception_snapshots() + { + self.active_handler_count -= 1; + snapshots + } else { + Vec::new() + } + } + + /// Retrieve the [`ExceptionContextStack`] that is relevant for the current scope. + fn current_exception_context_stack(&mut self) -> &mut ExceptionContextStack { + self.stacks .last_mut() - .expect("There should always be at least one `TryBlockContexts` on the stack") + .expect("There should always be at least one `ExceptionContextStack` on the stack") } } -/// The contexts of nested `try`/`except` blocks for a single scope +/// The contexts of nested `try` and `with` statements for a single scope. #[derive(Debug, Default)] -struct TryNodeContextStack(Vec); +struct ExceptionContextStack(Vec); -impl TryNodeContextStack { - /// Returns whether a `try` suite in this scope is still collecting exception checkpoints. +impl ExceptionContextStack { + /// Returns whether a `try` or `with` context is still collecting exception checkpoints. fn has_active_exception_handler(&self) -> bool { self.0 .iter() .any(|context| context.exception_handlers.is_active()) } - /// Push a new [`TryNodeContext`] for recording exception checkpoints and terminal entries - /// while visiting a [`ruff_python_ast::StmtTry`] node. - fn push_context(&mut self, exception_handlers: ExceptionHandlers) { - self.0.push(TryNodeContext { + /// Registers a `try` statement and whether exceptions must first pass through cleanup. + fn push_try_context(&mut self, exception_handlers: ExceptionHandlers, has_finally: bool) { + self.0.push(ExceptionContext::new( + ExceptionContextKind::Try { has_finally }, exception_handlers, - ..TryNodeContext::default() - }); + )); + } + + /// Registers a context manager that may receive exceptions from its body. + fn push_context_manager_context(&mut self) { + self.0.push(ExceptionContext::new( + ExceptionContextKind::With, + ExceptionHandlers::propagating(), + )); } - /// Pop a [`TryNodeContext`] off the stack. - fn pop_context(&mut self) -> TryNodeContext { + /// Pop an [`ExceptionContext`] off the stack. + fn pop_context(&mut self) -> ExceptionContext { self.0 .pop() - .expect("Cannot pop a `try` block off an empty `TryBlockContexts` stack") + .expect("Cannot pop an exception context off an empty `ExceptionContextStack`") } - /// Take all snapshots recorded while visiting the `try` suite and deactivate its handlers. - fn end_try_suite(&mut self) -> Vec { + /// Takes the innermost context's snapshots if it has active handlers, deactivating them. + fn take_exception_snapshots(&mut self) -> Option> { let context = self .0 .last_mut() - .expect("Cannot take snapshots from an empty `TryBlockContexts` stack"); + .expect("Cannot take snapshots from an empty `ExceptionContextStack`"); match std::mem::take(&mut context.exception_handlers) { - ExceptionHandlers::None => Vec::new(), + ExceptionHandlers::None => None, ExceptionHandlers::Propagating(snapshots) | ExceptionHandlers::CatchAll(snapshots) => { - snapshots + Some(snapshots) } } } - /// Records the checkpoint for all enclosing active `try` suites in this scope. Returns whether - /// the checkpoint should continue propagating to an enclosing scope. + /// Records a checkpoint for every active `try` or `with` context in this scope. + /// Returns whether the checkpoint should continue propagating to an enclosing scope. /// - /// A bare handler consumes the exception, preventing any outer handler from seeing it. The - /// snapshot is constructed only if a handler has not already observed the current flow state. - fn record_exception_checkpoint(&mut self, use_def_map: &UseDefMapBuilder<'_>) -> bool { + /// A bare handler consumes the exception, preventing any outer handler from seeing it. A + /// `finally` suite prevents enclosing context managers from receiving a checkpoint until its + /// cleanup has run, while preserving existing outer-`try` handler behavior. The snapshot is + /// constructed only if a handler has not already observed the current flow state. + fn record_exception_checkpoint( + &mut self, + use_def_map: &UseDefMapBuilder<'_>, + has_intervening_finally: &mut bool, + ) -> bool { let checkpoint_key = use_def_map.exception_checkpoint_key(); - let mut snapshot = None; for context in self.0.iter_mut().rev() { + if *has_intervening_finally && matches!(context.kind, ExceptionContextKind::With) { + continue; + } + match &mut context.exception_handlers { ExceptionHandlers::None => context.has_escaping_exception = true, ExceptionHandlers::Propagating(snapshots) | ExceptionHandlers::CatchAll(snapshots) => { if context.last_checkpoint_key != Some(checkpoint_key) { - snapshots.push( - snapshot - .get_or_insert_with(|| use_def_map.snapshot()) - .clone(), - ); + snapshots.push(use_def_map.snapshot()); context.last_checkpoint_key = Some(checkpoint_key); } if context.exception_handlers.is_catch_all() { @@ -213,45 +305,76 @@ impl TryNodeContextStack { context.has_escaping_exception = true; } } + + *has_intervening_finally |= matches!( + context.kind, + ExceptionContextKind::Try { has_finally: true } + ); } true } - /// Push the snapshot onto the innermost `try` block's terminal-entry snapshots for its - /// `finally` suite. + /// Records a terminal entry for the nearest `try` context, skipping intervening `with` contexts. fn record_terminal_finally_entry(&mut self, builder: &SemanticIndexBuilder) { - if let Some(context) = self.0.last_mut() { - context.record_terminal_finally_entry(builder.flow_snapshot()); + if let Some(context) = self.innermost_try() { + context + .terminal_finally_entry_snapshots + .push(builder.flow_snapshot()); } } + + /// Finds the nearest enclosing `try`, skipping context managers. + fn innermost_try(&mut self) -> Option<&mut ExceptionContext> { + self.0 + .iter_mut() + .rev() + .find(|context| matches!(context.kind, ExceptionContextKind::Try { .. })) + } } -/// Context for tracking exception and `finally` entry states for a single -/// [`ruff_python_ast::StmtTry`] node. +/// Distinguishes `try` exception contexts from `with` exception contexts. +#[derive(Debug, Copy, Clone, PartialEq, Eq)] +enum ExceptionContextKind { + Try { has_finally: bool }, + With, +} + +/// Exception-entry states for one `try` or `with` statement. /// -/// It will likely be necessary to add more fields to this struct in the future -/// when we add more advanced handling of `finally` branches. -#[derive(Debug, Default)] -pub(super) struct TryNodeContext { +/// Only `try` contexts also collect terminal entries for a `finally` suite. +#[derive(Debug)] +pub(super) struct ExceptionContext { exception_handlers: ExceptionHandlers, + kind: ExceptionContextKind, last_checkpoint_key: Option<(ScopedDefinitionId, ControlFlowRevision)>, /// Whether an exception escaped this suite and must also propagate after its cleanup. has_escaping_exception: bool, + /// Whether apparently terminal control flow in a nested context-manager body, such as a + /// `return` or `raise`, may become non-terminal if type inference determines that the context + /// manager suppresses exceptions. This flag belongs to the enclosing `try` context because it + /// affects control flow into its `finally` suite. + has_deferred_terminal_context_manager_exit: bool, terminal_finally_entry_snapshots: Vec, } -impl TryNodeContext { - pub(super) fn into_finally_entry_state(self) -> (Vec, bool) { +impl ExceptionContext { + fn new(kind: ExceptionContextKind, exception_handlers: ExceptionHandlers) -> Self { + Self { + exception_handlers, + kind, + last_checkpoint_key: None, + has_escaping_exception: false, + has_deferred_terminal_context_manager_exit: false, + terminal_finally_entry_snapshots: Vec::new(), + } + } + + pub(super) fn into_finally_entry_state(self) -> (Vec, bool, bool) { ( self.terminal_finally_entry_snapshots, self.has_escaping_exception, + self.has_deferred_terminal_context_manager_exit, ) } - - /// Take a record of what the internal state looked like before a terminal control-flow - /// transfer that will pass through the `finally` suite. - fn record_terminal_finally_entry(&mut self, snapshot: FlowSnapshot) { - self.terminal_finally_entry_snapshots.push(snapshot); - } } diff --git a/crates/ty_python_core/src/predicate.rs b/crates/ty_python_core/src/predicate.rs index b9af38d627..68ffc52942 100644 --- a/crates/ty_python_core/src/predicate.rs +++ b/crates/ty_python_core/src/predicate.rs @@ -18,6 +18,7 @@ use crate::ast_ids::ExpressionNodeKey; use crate::db::Db; use crate::expression::Expression; use crate::global_scope; +use crate::reachability_constraints::ScopedReachabilityConstraintId; use crate::scope::{FileScopeId, ScopeId}; use crate::symbol::ScopedSymbolId; @@ -114,6 +115,23 @@ pub struct CallableAndCallExpr<'db> { #[derive(Clone, Copy, Debug, Hash, PartialEq, Eq, get_size2::GetSize, salsa::SalsaValue)] pub enum PredicateNode<'db> { Expression(Expression<'db>), + /// Whether a context manager's exit return type allows an exception to be suppressed. + /// + /// Resolved during type inference because the context manager's type is unavailable during + /// semantic indexing. + ContextManagerSuppresses { + expression: Expression<'db>, + is_async: bool, + }, + /// Whether semantic evaluation rules out every normal entry into a `finally` suite. + /// + /// The continuation is captured before constructing this predicate, so its constraint cannot + /// depend on the predicate itself. Deferring evaluation preserves terminal cleanup paths when + /// a context manager's suppression behavior is unavailable during semantic indexing. + FinallyNormalPathImpossible { + scope: ScopeId<'db>, + continuation: ScopedReachabilityConstraintId, + }, /// These predicates are recorded for statements with call expressions. As part of /// reachability constraints, they are used to determine whether control flow can /// continue past this statement or not. diff --git a/crates/ty_python_semantic/resources/mdtest/terminal_statements.md b/crates/ty_python_semantic/resources/mdtest/terminal_statements.md index e960d2f79d..6098b8d3e2 100644 --- a/crates/ty_python_semantic/resources/mdtest/terminal_statements.md +++ b/crates/ty_python_semantic/resources/mdtest/terminal_statements.md @@ -578,6 +578,399 @@ def finally_assignment_runs_before_break(): reveal_type(x) # revealed: Literal[1] ``` +## Returning from a context manager inside `try` + +A context manager cannot prevent a `return` from reaching the enclosing `finally` block. The block +still sees assignments made before the return. + +```py +from contextlib import suppress + +def returns_through_finally() -> None: + value = "before" + try: + with suppress(ValueError): + value = "returned" + return + finally: + reveal_type(value) # revealed: Literal["returned"] +``` + +## Continuing after a suppressing context manager inside `try` + +When a context manager suppresses an exception, a later assignment determines the value observed by +the `finally` block: + +```py +from contextlib import suppress + +value = "before" +try: + with suppress(ValueError): + raise ValueError + value = "continuing" +finally: + reveal_type(value) # revealed: Literal["continuing"] +``` + +## Continuing after a suppressing context manager and `finally` + +After an exception is suppressed, assignments in the `finally` block remain visible on the +continuing path: + +```py +from contextlib import suppress + +def continues_after_finally() -> str: + try: + with suppress(ValueError): + raise ValueError + finally: + value = "cleanup" + reveal_type(value) # revealed: Literal["cleanup"] + return value +``` + +## Raising from a context manager inside `try` + +A `finally` block remains reachable when a context manager propagates an exception: + +```py +from contextlib import nullcontext + +try: + with nullcontext(): + raise ValueError +finally: + # The diagnostic confirms that `finally` is reachable. + missing_name # error: [unresolved-reference] +``` + +## Unreachable bindings after a context manager inside `try` + +Assignments and imports after the propagating context manager cannot make the `finally` block +unreachable: + +```py +from contextlib import nullcontext + +try: + with nullcontext(): + raise ValueError + unreachable = 1 + import sys +finally: + # The diagnostic confirms that `finally` is reachable. + missing_after_unreachable_bindings # error: [unresolved-reference] +``` + +## Code after a terminal context manager and `finally` + +A non-suppressing manager does not allow a raised exception to continue past `finally` or implicitly +return from an annotated function: + +```py +from contextlib import nullcontext + +def does_not_continue() -> int: + try: + with nullcontext(): + raise ValueError + finally: + pass + # The absence of a diagnostic confirms that this code is unreachable. + missing_after_finally +``` + +## Narrowing after a terminal context manager and `finally` + +A branch that raises through a non-suppressing manager remains terminal after its cleanup: + +```py +from contextlib import nullcontext + +def narrows_after_finally(value: str | None) -> None: + if value is None: + try: + with nullcontext(): + raise ValueError + finally: + pass + reveal_type(value) # revealed: str +``` + +## Loop control after a terminal context manager and `finally` + +A `break` through a non-suppressing manager and its enclosing cleanup cannot reach a later +assignment in the loop: + +```py +from contextlib import nullcontext + +for _ in [1]: + try: + with nullcontext(): + break + finally: + pass + after_break = 1 + +after_break # error: [unresolved-reference] +``` + +The same applies to `continue`: + +```py +for _ in [1]: + try: + with nullcontext(): + continue + finally: + pass + after_continue = 1 + +after_continue # error: [unresolved-reference] +``` + +## Nested `finally` suites after a terminal context manager + +The outer cleanup observes assignments made in the inner cleanup, but execution does not continue +after either suite: + +```py +from contextlib import nullcontext + +def nested_cleanup() -> None: + try: + try: + with nullcontext(): + raise ValueError + finally: + value = "cleanup" + finally: + reveal_type(value) # revealed: Literal["cleanup"] + # The absence of a diagnostic confirms that this code is unreachable. + missing_after_nested_finally +``` + +## Terminal `except` branches after a context manager + +An `except` branch that assigns a value before returning still contributes that value to the +`finally` block: + +```py +from contextlib import nullcontext + +def unknown_exception() -> Exception: + return ValueError() + +def handler_returns() -> None: + value = "before" + try: + with nullcontext(): + raise unknown_exception() + except ValueError: + value = "returned" + return + finally: + reveal_type(value) # revealed: Literal["before", "returned"] +``` + +## Named `except` branches after a context manager + +Binding an exception does not make a terminal `except` branch a continuing entry into `finally`: + +```py +from contextlib import nullcontext + +def unknown_exception() -> Exception: + return ValueError() + +def named_handler() -> None: + value = "before" + try: + with nullcontext(): + raise unknown_exception() + except ValueError as error: + value = error + return + finally: + reveal_type(value) # revealed: Literal["before"] | ValueError +``` + +## Multiple terminal `except` branches after a context manager + +Every terminal `except` branch contributes its assignment to the `finally` block: + +```py +from contextlib import nullcontext + +def unknown_exception() -> Exception: + return ValueError() + +def multiple_handlers() -> None: + value = "before" + try: + with nullcontext(): + raise unknown_exception() + except ValueError: + value = "value-error" + return + except TypeError: + value = "type-error" + raise RuntimeError + finally: + reveal_type(value) # revealed: Literal["before", "value-error", "type-error"] +``` + +## `except` branches without terminal statements after a context manager + +An `except` branch with no terminal statements determines the value observed by `finally`: + +```py +from contextlib import nullcontext + +value = "before" +try: + with nullcontext(): + raise ValueError +except ValueError: + value = "continuing" +finally: + reveal_type(value) # revealed: Literal["continuing"] +``` + +## Unreachable assignments after a context manager inside `except` + +A context manager propagates an exception from an `except` branch even when an unreachable +assignment follows: + +```py +from contextlib import nullcontext + +try: + raise ValueError +except ValueError: + with nullcontext(): + raise RuntimeError + unreachable = 1 +finally: + # The diagnostic confirms that `finally` is reachable. + missing_after_unreachable_handler_assignment # error: [unresolved-reference] +``` + +## Raising from a context manager inside a named `except` branch + +Clearing a named exception does not hide the terminal path from `finally`: + +```py +from contextlib import nullcontext + +try: + raise ValueError +except ValueError as error: + with nullcontext(): + raise RuntimeError +finally: + # The diagnostic confirms that `finally` is reachable. + missing_name # error: [unresolved-reference] +``` + +## Terminal nested `except` branches without their own `finally` + +An unreachable assignment and a binding in the terminal inner `except` branch do not prevent the +path from reaching the outer `finally` block: + +```py +from contextlib import nullcontext + +def nested_unreachable_assignment() -> None: + try: + try: + with nullcontext(): + raise ValueError + unreachable = 1 + except ValueError: + local = 1 + return + finally: + # The diagnostic confirms that `finally` is reachable. + missing_after_nested_unreachable_assignment # error: [unresolved-reference] +``` + +A `break` through an inner `except` branch also reaches the outer `finally` block: + +```py +for _ in [1]: + try: + try: + with nullcontext(): + raise ValueError + except ValueError: + break + finally: + # The diagnostic confirms that `finally` is reachable. + missing_name # error: [unresolved-reference] +``` + +## Unreachable assignments after a context manager inside `else` + +A context manager propagates an exception from `else` even when an unreachable assignment follows: + +```py +from contextlib import nullcontext + +try: + pass +except ValueError: + pass +else: + with nullcontext(): + raise RuntimeError + unreachable = 1 +finally: + # The diagnostic confirms that `finally` is reachable. + missing_after_unreachable_else_assignment # error: [unresolved-reference] +``` + +## Possibly unbound names in `finally` after a context manager + +When an assignment raises before binding a name, a `finally` block can observe that the name is +undefined: + +```py +from contextlib import nullcontext + +def may_raise() -> str: + raise RuntimeError + +def without_context_manager() -> str | None: + try: + value = may_raise() + return may_raise() + except ValueError: + return None + finally: + # error: [possibly-unresolved-reference] + reveal_type(value) # revealed: str +``` + +A non-suppressing context manager does not prevent the `finally` block from observing that the name +may remain undefined. + +```py +def with_context_manager() -> str | None: + try: + value = may_raise() + with nullcontext(): + return may_raise() + except ValueError: + return None + finally: + # error: [possibly-unresolved-reference] + reveal_type(value) # revealed: str +``` + ## Calls to functions returning `Never` / `NoReturn` These calls should be treated as terminal statements. diff --git a/crates/ty_python_semantic/resources/mdtest/with/async.md b/crates/ty_python_semantic/resources/mdtest/with/async.md index 7a745c9db7..c5311a7053 100644 --- a/crates/ty_python_semantic/resources/mdtest/with/async.md +++ b/crates/ty_python_semantic/resources/mdtest/with/async.md @@ -18,6 +18,243 @@ async def test(): reveal_type(f) # revealed: Target ``` +## Exception-suppressing async context managers and union aliases + +An asynchronous context manager can suppress exceptions if its `__aexit__` method returns `bool`: + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Literal + +class Suppresses: + async def __aenter__(self) -> None: ... + async def __aexit__(self, exc_type, exc_value, traceback) -> bool: + return True + +async def may_raise() -> str: + raise ValueError + +async def preserved_binding() -> None: + result = None + async with Suppresses(): + result = await may_raise() + reveal_type(result) # revealed: None | str +``` + +If an exception interrupts an assignment to a new name, that name may remain undefined: + +```py +async def missing_binding() -> None: + async with Suppresses(): + value = await may_raise() + # error: [possibly-unresolved-reference] + reveal_type(value) # revealed: str +``` + +An `__aexit__` return type of `None` does not suppress exceptions: + +```py +class Propagates: + async def __aenter__(self) -> None: ... + async def __aexit__(self, exc_type, exc_value, traceback) -> None: ... + +async def propagating_exit() -> None: + result = None + async with Propagates(): + result = await may_raise() + reveal_type(result) # revealed: str +``` + +[The typing specification](https://typing.python.org/en/latest/spec/exceptions.html#context-managers) +treats an awaited `Literal[True] | None` return type as non-suppressing, even though a truthy return +value would suppress an exception at runtime: + +```py +class OptionalTrueExit: + async def __aenter__(self) -> None: ... + async def __aexit__(self, exc_type, exc_value, traceback) -> Literal[True] | None: + return True + +async def optional_true_exit() -> None: + result = None + async with OptionalTrueExit(): + result = await may_raise() + reveal_type(result) # revealed: str +``` + +A PEP 695 alias does not prevent a suppressing union member from preserving an earlier binding: + +```py +type Managers = Suppresses | Propagates + +async def preserved_union_binding(manager: Managers) -> None: + result = None + async with manager: + result = await may_raise() + reveal_type(result) # revealed: None | str +``` + +A suppressed exception can also leave a new binding undefined: + +```py +async def missing_union_binding(manager: Managers) -> None: + async with manager: + result = await may_raise() + # error: [possibly-unresolved-reference] + reveal_type(result) # revealed: str +``` + +## Earlier async context managers can suppress later entry failures + +If an earlier async context manager suppresses an exception while a later manager enters, the later +manager's target may never be assigned: + +```py +class Suppresses: + async def __aenter__(self) -> None: ... + async def __aexit__(self, exc_type, exc_value, traceback) -> bool: + return True + +class EnterFails: + async def __aenter__(self) -> str: + raise ValueError + + async def __aexit__(self, exc_type, exc_value, traceback) -> None: ... + +async def later_entry_fails() -> None: + async with Suppresses(), EnterFails() as target: + pass + # error: [possibly-unresolved-reference] + reveal_type(target) # revealed: str +``` + +## Returning from an exception-suppressing async context manager + +A context manager cannot suppress a return statement: + +```py +class Suppresses: + async def __aenter__(self) -> None: ... + async def __aexit__(self, exc_type, exc_value, traceback) -> bool: + return True + +async def bare_return() -> str: + async with Suppresses(): + return "finished" +``` + +An exception raised while evaluating an awaited return expression can be suppressed instead: + +```py +async def may_raise() -> str: + raise ValueError + +async def interrupted_return() -> str: # error: [invalid-return-type] + async with Suppresses(): + return await may_raise() +``` + +## Overloaded async context manager exit methods + +An overloaded async exit method can distinguish normal exits from exceptions: + +```py +from typing import Awaitable, Literal, overload +from typing_extensions import Never + +async def may_raise() -> str: + raise ValueError +``` + +An overload returning `True` only on a normal exit cannot suppress an exception: + +```py +class NormalExitOnly: + async def __aenter__(self) -> None: ... + @overload + async def __aexit__(self, exc_type: None, exc_value, traceback) -> Literal[True]: ... + @overload + async def __aexit__(self, exc_type: type[BaseException], exc_value, traceback) -> Literal[False]: ... + async def __aexit__(self, exc_type, exc_value, traceback) -> bool: + return exc_type is None + +async def normal_exit_only() -> None: + result = None + async with NormalExitOnly(): + result = await may_raise() + reveal_type(result) # revealed: str +``` + +Of the following three overloads, the second applies when an exception is raised, and the third +applies when the suite exits without an exception. The first overload never applies because its +exception argument is `Never`: + +```py +class NeverExit: + async def __aenter__(self) -> None: ... + @overload + async def __aexit__(self, exc_type: Never, exc_value, traceback) -> Literal[True]: ... + @overload + async def __aexit__(self, exc_type: type[BaseException], exc_value, traceback) -> Literal[False]: ... + @overload + async def __aexit__(self, exc_type: None, exc_value, traceback) -> Literal[False]: ... + async def __aexit__(self, exc_type, exc_value, traceback) -> bool: + return False + +async def impossible_exit() -> None: + result = None + async with NeverExit(): + result = await may_raise() + reveal_type(result) # revealed: str +``` + +An exceptional overload can suppress its exception even if another exceptional overload cannot: + +```py +class SuppressesValueError: + async def __aenter__(self) -> None: ... + @overload + async def __aexit__(self, exc_type: type[ValueError], exc_value: ValueError, traceback: object) -> Literal[True]: ... + @overload + async def __aexit__(self, exc_type: type[TypeError], exc_value: TypeError, traceback: object) -> None: ... + @overload + async def __aexit__(self, exc_type: None, exc_value: None, traceback: None) -> None: ... + async def __aexit__(self, exc_type, exc_value, traceback) -> Literal[True] | None: + return True if exc_type is ValueError else None + +async def mixed_exceptional_exits() -> None: + result = None + async with SuppressesValueError(): + result = await may_raise() + reveal_type(result) # revealed: None | str +``` + +An exceptional overload that returns a non-awaitable does not prevent a later awaitable overload +from suppressing a different exception: + +```py +class SuppressesAfterNonAwaitable: + async def __aenter__(self) -> None: ... + @overload + def __aexit__(self, exc_type: type[TypeError], exc_value: TypeError, traceback: object) -> bool: ... + @overload + def __aexit__(self, exc_type: type[ValueError], exc_value: ValueError, traceback: object) -> Awaitable[Literal[True]]: ... + @overload + def __aexit__(self, exc_type: None, exc_value: None, traceback: None) -> Awaitable[None]: ... + def __aexit__(self, exc_type, exc_value, traceback) -> bool | Awaitable[Literal[True]] | Awaitable[None]: + raise NotImplementedError + +async def suppresses_after_non_awaitable() -> None: + result = None + async with SuppressesAfterNonAwaitable(): + result = await may_raise() + reveal_type(result) # revealed: None | str +``` + ## Multiple targets ```py diff --git a/crates/ty_python_semantic/resources/mdtest/with/sync.md b/crates/ty_python_semantic/resources/mdtest/with/sync.md index 0669d941dd..7e0f781e52 100644 --- a/crates/ty_python_semantic/resources/mdtest/with/sync.md +++ b/crates/ty_python_semantic/resources/mdtest/with/sync.md @@ -18,6 +18,526 @@ with Manager() as f: reveal_type(f) # revealed: Target ``` +## Exception-suppressing context managers + +When a context manager suppresses an exception during an assignment, the previous binding remains +visible after the `with` statement: + +```py +from contextlib import suppress + +def may_raise() -> str: + raise ValueError + +result = None +with suppress(ValueError): + result = may_raise() + +reveal_type(result) # revealed: None | str +``` + +A new name may remain undefined when an exception interrupts its assignment: + +```py +with suppress(ValueError): + value = may_raise() + +# error: [possibly-unresolved-reference] +reveal_type(value) # revealed: str +``` + +A deleted binding is not restored if a later exception is suppressed: + +```py +deleted = 1 +with suppress(ValueError): + del deleted + may_raise() + +deleted # error: [unresolved-reference] +``` + +An assignment that cannot raise is not affected by exception suppression: + +```py +with suppress(ValueError): + safe_value = 1 + +reveal_type(safe_value) # revealed: Literal[1] +``` + +## Assigning a context manager target can raise + +Unpacking the result of `__enter__` can raise after the context manager has entered. Suppressing +that exception preserves an earlier binding, while a new target may remain undefined: + +```py +class EmptyIterableManager: + def __enter__(self) -> list[int]: + return [] + + def __exit__(self, exc_type, exc_value, traceback) -> bool: + return True + +value = "before" +with EmptyIterableManager() as (value, missing): + pass + +reveal_type(value) # revealed: Literal["before"] | int +# error: [possibly-unresolved-reference] +reveal_type(missing) # revealed: int +``` + +## Earlier context managers can suppress later entry failures + +If an earlier context manager suppresses an exception while a later manager enters, the later +manager's target may never be assigned: + +```py +from contextlib import suppress + +class EnterFails: + def __enter__(self) -> str: + raise ValueError + + def __exit__(self, exc_type, exc_value, traceback) -> None: ... + +with suppress(ValueError), EnterFails() as target: + pass + +# error: [possibly-unresolved-reference] +reveal_type(target) # revealed: str +``` + +## Loop exits inside multiple context managers + +A context manager cannot suppress a `break`, but it can suppress an exception while the next manager +enters. An assignment after the managers is therefore only possibly reached: + +```py +from contextlib import nullcontext, suppress + +for _ in [1]: + with suppress(ValueError), nullcontext(): + break + after_break = 1 + +after_break # error: [possibly-unresolved-reference] +``` + +It cannot suppress a `continue` either: + +```py +for _ in [1]: + with suppress(ValueError), nullcontext(): + continue + after_continue = 1 + +after_continue # error: [possibly-unresolved-reference] +``` + +An exception inside one manager can likewise be suppressed before a `break`: + +```py +for _ in [1]: + with suppress(ValueError): + int("invalid") + break + after_exception = 1 + +after_exception # error: [possibly-unresolved-reference] +``` + +## Loop exits inside nested context managers + +Nested context managers cannot suppress a `break`, but the outer manager can suppress an exception +while the inner manager enters: + +```py +from contextlib import nullcontext, suppress + +for _ in [1]: + with suppress(ValueError): + with nullcontext(): + break + after_break = 1 + +after_break # error: [possibly-unresolved-reference] +``` + +They cannot suppress a `continue` either: + +```py +for _ in [1]: + with suppress(ValueError): + with nullcontext(): + continue + after_continue = 1 + +after_continue # error: [possibly-unresolved-reference] +``` + +## Returning from an exception-suppressing context manager + +A context manager cannot suppress a return statement: + +```py +from contextlib import suppress + +def bare_return() -> int: + with suppress(ValueError): + return 1 +``` + +It can suppress an exception raised while evaluating the return expression, allowing the function to +continue without returning a value: + +```py +def may_raise() -> int: + raise ValueError + +# error: [invalid-return-type] "Function can implicitly return `None`, which is not assignable to return type `int`" +def interrupted_return() -> int: + with suppress(ValueError): + return may_raise() +``` + +## Exception handlers inside a suppressing context manager + +A bare `except:` catches an exception before it can reach the surrounding context manager: + +```py +from contextlib import suppress + +def caught_before_suppression() -> int: + with suppress(ValueError): + try: + raise ValueError + except: + return 1 +``` + +## A terminal `finally` prevents exception suppression + +A `return` in a `finally` block replaces the exception before it can reach an enclosing context +manager: + +```py +from contextlib import suppress + +def always_returns() -> int: + with suppress(ValueError): + try: + raise ValueError + finally: + return 1 +``` + +## Cleanup runs before an enclosing context manager suppresses an exception + +Assignments in a `finally` block are visible after an enclosing context manager suppresses the +exception: + +```py +from contextlib import suppress + +def cleanup_before_suppression() -> None: + result = None + with suppress(ValueError): + try: + raise ValueError + finally: + result = "cleaned" + reveal_type(result) # revealed: Literal["cleaned"] +``` + +## Eager expressions inside a suppressing context manager + +A list comprehension evaluates its body eagerly, so a context manager can suppress an exception +raised inside it: + +```py +from contextlib import suppress + +def may_raise() -> int: + raise ValueError + +# error: [invalid-return-type] "Function can implicitly return `None`, which is not assignable to return type `int`" +def eager_comprehension() -> int: + with suppress(ValueError): + [may_raise() for _ in [0]] + return 1 +``` + +Generator expressions are also assumed to run eagerly, so their exceptions can be suppressed: + +```py +# error: [invalid-return-type] "Function can implicitly return `None`, which is not assignable to return type `int`" +def eager_generator() -> int: + with suppress(ValueError): + (may_raise() for _ in [0]) + return 1 +``` + +## Context manager exit return types + +The typing specification treats an `__exit__` return type of `bool` as potentially suppressing: + +```py +from typing import Any, Literal + +class Manager: + def __enter__(self) -> None: ... + +class ReturnsBool(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> bool: + return True + +def may_raise() -> str: + raise ValueError + +bool_result = None +with ReturnsBool(): + bool_result = may_raise() +reveal_type(bool_result) # revealed: None | str +``` + +An `__exit__` return type of `Literal[True]` can also suppress exceptions: + +```py +class ReturnsTrue(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> Literal[True]: + return True + +true_result = None +with ReturnsTrue(): + true_result = may_raise() +reveal_type(true_result) # revealed: None | str +``` + +An `__exit__` return type of `Literal[False]` cannot suppress exceptions: + +```py +class ReturnsFalse(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> Literal[False]: + return False + +false_result = None +with ReturnsFalse(): + false_result = may_raise() +reveal_type(false_result) # revealed: str +``` + +An `__exit__` return type of `None` cannot suppress exceptions: + +```py +class ReturnsNone(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> None: ... + +none_result = None +with ReturnsNone(): + none_result = may_raise() +reveal_type(none_result) # revealed: str +``` + +[The typing specification](https://typing.python.org/en/latest/spec/exceptions.html#context-managers) +classifies `bool | None` as non-suppressing for compatibility with common non-suppressing context +managers, even though a truthy return value can suppress an exception at runtime: + +```py +class ReturnsOptionalBool(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> bool | None: + return None + +optional_result = None +with ReturnsOptionalBool(): + optional_result = may_raise() +reveal_type(optional_result) # revealed: str +``` + +This convention also treats `Literal[True] | None` as non-suppressing: + +```py +class ReturnsOptionalTrue(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> Literal[True] | None: + return True + +optional_true_result = None +with ReturnsOptionalTrue(): + optional_true_result = may_raise() +reveal_type(optional_true_result) # revealed: str +``` + +An `__exit__` return type of `Literal[False] | None` cannot suppress exceptions either: + +```py +class ReturnsOptionalFalse(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> Literal[False] | None: + return False + +optional_false_result = None +with ReturnsOptionalFalse(): + optional_false_result = may_raise() +reveal_type(optional_false_result) # revealed: str +``` + +An `__exit__` return type of `Any` does not indicate exception suppression either: + +```py +class ReturnsAny(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> Any: + return False + +any_result = None +with ReturnsAny(): + any_result = may_raise() +reveal_type(any_result) # revealed: str +``` + +## Context managers with union and aliased union types + +A context manager with a union type may suppress an exception if any member can suppress it, even +when another member cannot: + +```toml +[environment] +python-version = "3.12" +``` + +```py +class Manager: + def __enter__(self) -> None: ... + +class Suppresses(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> bool: + return True + +class Propagates(Manager): + def __exit__(self, exc_type, exc_value, traceback) -> bool | None: + return None + +def may_raise() -> str: + raise ValueError + +def possibly_suppressing(manager: Suppresses | Propagates) -> None: + result = None + with manager: + result = may_raise() + reveal_type(result) # revealed: None | str +``` + +A PEP 695 alias does not prevent a suppressing union member from preserving an earlier binding: + +```py +type Managers = Suppresses | Propagates + +def preserved_binding(manager: Managers) -> None: + result = None + with manager: + result = may_raise() + reveal_type(result) # revealed: None | str +``` + +A suppressed exception can also leave a new binding undefined: + +```py +def missing_binding(manager: Managers) -> None: + with manager: + result = may_raise() + # error: [possibly-unresolved-reference] + reveal_type(result) # revealed: str +``` + +## Non-suppressing context managers preserve narrowing + +A non-suppressing manager does not change narrowing after an exception propagates: + +```py +class Manager: + def __enter__(self) -> None: ... + def __exit__(self, exc_type, exc_value, traceback) -> None: ... + +def propagating_exception(value: int | str) -> None: + if isinstance(value, int): + with Manager(): + raise ValueError + reveal_type(value) # revealed: str +``` + +## Overloaded context manager exit methods + +Whether an overloaded exit method can suppress an exception depends on the overload used when an +exception occurs, not the overload used when its suite exits without an exception. In the latter +case, Python calls `__exit__(None, None, None)`: + +```py +from typing import Literal, overload +from typing_extensions import Never + +class Manager: + def __enter__(self) -> None: ... + +def may_raise() -> str: + raise ValueError +``` + +A manager that returns `True` only during normal exit cannot suppress exceptions: + +```py +class NormalOnly(Manager): + @overload + def __exit__(self, exc_type: None, exc_value: None, traceback: None) -> Literal[True]: ... + @overload + def __exit__(self, exc_type: type[BaseException], exc_value: BaseException, traceback: object) -> Literal[False]: ... + def __exit__(self, exc_type, exc_value, traceback) -> bool: + return exc_type is None + +normal_value = None +with NormalOnly(): + normal_value = may_raise() +reveal_type(normal_value) # revealed: str +``` + +An exceptional overload cannot suppress an exception if either exception argument is uninhabited: + +```py +class ImpossibleExceptionalExit(Manager): + @overload + def __exit__(self, exc_type: Never, exc_value: BaseException, traceback: object) -> Literal[True]: ... + @overload + def __exit__(self, exc_type: type[BaseException], exc_value: Never, traceback: object) -> Literal[True]: ... + @overload + def __exit__( + self, exc_type: type[BaseException] | None, exc_value: BaseException | None, traceback: object | None + ) -> Literal[False]: ... + def __exit__(self, exc_type, exc_value, traceback) -> bool: + return False + +impossible_exception_value = None +with ImpossibleExceptionalExit(): + impossible_exception_value = may_raise() +reveal_type(impossible_exception_value) # revealed: str +``` + +An exceptional overload can suppress its exception even if another exceptional overload cannot: + +```py +class SuppressesValueError(Manager): + @overload + def __exit__(self, exc_type: type[ValueError], exc_value: ValueError, traceback: object) -> Literal[True]: ... + @overload + def __exit__(self, exc_type: type[TypeError], exc_value: TypeError, traceback: object) -> None: ... + @overload + def __exit__(self, exc_type: None, exc_value: None, traceback: None) -> None: ... + def __exit__(self, exc_type, exc_value, traceback) -> Literal[True] | None: + return True if exc_type is ValueError else None + +mixed_exceptional_value = None +with SuppressesValueError(): + mixed_exceptional_value = may_raise() +reveal_type(mixed_exceptional_value) # revealed: None | str +``` + ## Union context manager ```py diff --git a/crates/ty_python_semantic/src/reachability.rs b/crates/ty_python_semantic/src/reachability.rs index d6bfff29f5..f19ba7dcc6 100644 --- a/crates/ty_python_semantic/src/reachability.rs +++ b/crates/ty_python_semantic/src/reachability.rs @@ -215,8 +215,8 @@ use ruff_text_size::TextRange; use rustc_hash::{FxHashMap, FxHashSet}; use smallvec::SmallVec; use ty_python_core::{ - BindingWithConstraints, DeclarationWithConstraint, DeclarationsIterator, FileScopeId, - ScopedDefinitionId, SemanticIndex, Truthiness, UseDefMap, + BindingWithConstraints, DeclarationWithConstraint, DeclarationsIterator, EvaluationMode, + FileScopeId, ScopedDefinitionId, SemanticIndex, Truthiness, UseDefMap, definition::DefinitionState, expression::Expression, narrowing_constraints::{NarrowingConstraints, ScopedNarrowingConstraint}, @@ -542,11 +542,13 @@ const REACHABILITY_EVALUATION_CHUNK_SIZE: usize = 256; fn predicate_scope<'db>(db: &'db dyn Db, predicate: &Predicate<'db>) -> ScopeId<'db> { match predicate.node { - PredicateNode::Expression(expression) => expression.scope(db), + PredicateNode::Expression(expression) + | PredicateNode::ContextManagerSuppresses { expression, .. } => expression.scope(db), PredicateNode::IsNonTerminalCall(CallableAndCallExpr { callable, .. }) => { callable.scope(db) } PredicateNode::Pattern(pattern) => pattern.scope(db), + PredicateNode::FinallyNormalPathImpossible { scope, .. } => scope, PredicateNode::OrPatternAlternative(scope) => scope, PredicateNode::SubjectElementPattern(subject_element) => subject_element.pattern.scope(db), PredicateNode::IsNonEmptyIterable(expression) => expression.scope(db), @@ -703,6 +705,30 @@ fn evaluate_reachability_constraint<'db>( ) } +/// Evaluates the normal continuation captured by a deferred `finally` predicate. +/// +/// Unlike other reachability predicates, a deferred `finally` predicate recursively evaluates +/// another reachability constraint, which may contain earlier deferred `finally` predicates. +/// Caching these continuations prevents a sequence of `finally` suites from repeatedly evaluating +/// all preceding continuations, which would otherwise take exponential time. +/// +/// Other expensive predicates already use tracked queries, while ordinary reachability +/// constraints are cached within each inference region and at sparse checkpoints. Tracking +/// [`evaluate_reachability_constraint`] itself would instead retain a Salsa query key and memo for +/// every constraint. +#[salsa::tracked( + returns(copy), + cycle_initial = |_, _, _, _| Truthiness::Ambiguous, + heap_size = get_size2::GetSize::get_heap_size +)] +fn evaluate_finally_continuation<'db>( + db: &'db dyn Db, + scope: ScopeId<'db>, + continuation: ScopedReachabilityConstraintId, +) -> Truthiness { + evaluate_reachability_constraint(db, scope, continuation) +} + fn terminal_reachability(id: ScopedReachabilityConstraintId) -> Option { match id { ScopedReachabilityConstraintId::ALWAYS_TRUE => Some(Truthiness::AlwaysTrue), @@ -1134,7 +1160,12 @@ impl<'a, 'db> NarrowingProjector<'a, 'db> { let node = self.constraints.get_interior_node(id); let predicate = self.predicates[node.atom]; - if matches!(predicate.node, PredicateNode::IsNonTerminalCall(_)) { + if matches!( + predicate.node, + PredicateNode::IsNonTerminalCall(_) + | PredicateNode::ContextManagerSuppresses { .. } + | PredicateNode::FinallyNormalPathImpossible { .. } + ) { actions.push(Action::AnalyzeNonTerminal(id)); actions.push(Action::Visit(node.if_uncertain)); } else { @@ -1151,7 +1182,9 @@ impl<'a, 'db> NarrowingProjector<'a, 'db> { Truthiness::AlwaysTrue => node.if_true, Truthiness::AlwaysFalse => node.if_false, Truthiness::Ambiguous => { - unreachable!("`IsNonTerminalCall` predicates should never be Ambiguous") + unreachable!( + "statically decidable predicates should never be Ambiguous" + ) } }; @@ -1523,6 +1556,21 @@ fn analyze_single(db: &dyn Db, env: &ProgramEnvironment<'_>, predicate: &Predica .bool(db, env) .negate_if(!predicate.is_positive) } + PredicateNode::ContextManagerSuppresses { + expression, + is_async, + } => Truthiness::from( + infer_same_file_expression_type(db, expression, TypeContext::default()) + .can_suppress_exceptions(db, env, EvaluationMode::from_is_async(is_async)), + ) + .negate_if(!predicate.is_positive), + PredicateNode::FinallyNormalPathImpossible { + scope, + continuation, + } => Truthiness::from( + evaluate_finally_continuation(db, scope, continuation).is_always_false(), + ) + .negate_if(!predicate.is_positive), PredicateNode::IsNonTerminalCall(CallableAndCallExpr { callable, call_expr, diff --git a/crates/ty_python_semantic/src/types/context_manager.rs b/crates/ty_python_semantic/src/types/context_manager.rs index c4b9071837..dd24079502 100644 --- a/crates/ty_python_semantic/src/types/context_manager.rs +++ b/crates/ty_python_semantic/src/types/context_manager.rs @@ -1,16 +1,154 @@ use crate::Db; use crate::ProgramEnvironment; use crate::{ - FxOrderSet, + FxOrderSet, Program, types::{ - Bindings, CallArguments, CallDunderError, Type, TypeContext, call::CallErrorKind, - context::InferContext, diagnostic::INVALID_CONTEXT_MANAGER, + Bindings, CallArguments, CallDunderError, KnownClass, MemberLookupPolicy, Type, + TypeContext, call::CallErrorKind, context::InferContext, + diagnostic::INVALID_CONTEXT_MANAGER, }, }; use ruff_python_ast as ast; use ty_python_core::EvaluationMode; impl<'db> Type<'db> { + /// Returns whether this context manager can suppress an exception raised inside its suite. + /// + /// Following the [typing specification], only exit methods returning exactly `bool` or + /// `Literal[True]` are considered suppressing; `bool | None` and `Any` are not. This + /// intentionally differs from runtime truthiness: non-suppressing context managers are + /// commonly annotated as returning `bool | None`, so treating every potentially truthy return + /// type as suppressing would incorrectly preserve exception paths for ordinary managers. + /// Asynchronous exit results are awaited before applying this rule. + /// + /// [typing specification]: https://typing.python.org/en/latest/spec/exceptions.html#context-managers + /// + /// Suppression is cached by manager type because the same predicate can be evaluated repeatedly + /// for different bindings and context managers. Each alternative in a union is classified + /// separately: if any possible manager can suppress exceptions, the union can suppress + /// exceptions too. Exceptional-exit overloads are also classified independently. Merging the + /// return types of different manager alternatives or overloads could incorrectly classify a + /// suppressing exit alongside a non-suppressing exit as returning `bool | None`. + /// + /// Python passes `(None, None, None)` to an exit method when a suite completes normally and + /// passes the exception type, value, and traceback when it raises. Consequently, overloads + /// whose first two arguments cannot accept an exception type and instance cannot describe an + /// exceptional exit and must not affect the suppression result: + /// + /// ```python + /// @overload + /// def __exit__(self, typ: None, value: None, tb: None) -> None: ... + /// + /// @overload + /// def __exit__( + /// self, + /// typ: type[BaseException], + /// value: BaseException, + /// tb: TracebackType | None, + /// ) -> Literal[True]: ... + /// ``` + /// + /// This manager can suppress exceptions despite its normal-exit overload returning `None`. + /// Suppression preserves any state from before an operation that raises: + /// + /// ```python + /// from contextlib import suppress + /// + /// value = None + /// with suppress(ValueError): + /// value = int("invalid") + /// reveal_type(value) # int | None + /// ``` + pub(crate) fn can_suppress_exceptions( + self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + mode: EvaluationMode, + ) -> bool { + #[salsa::tracked( + returns(copy), + cycle_initial = |_, _, _, _, _| false, + heap_size = ruff_memory_usage::heap_size + )] + fn can_suppress_exceptions_impl<'db>( + db: &'db dyn Db, + program: Program<'db>, + manager: Type<'db>, + is_async: bool, + ) -> bool { + if let Some(union) = manager.as_union_like(db) { + return union + .elements(db) + .iter() + .any(|&element| can_suppress_exceptions_impl(db, program, element, is_async)); + } + + let env = ProgramEnvironment::from_program(program); + let method = if is_async { "__aexit__" } else { "__exit__" }; + let Some(callables) = manager + .member_lookup_with_policy( + db, + &env, + method, + MemberLookupPolicy::NO_INSTANCE_FALLBACK, + ) + .place + .ignore_possibly_undefined() + .and_then(|exit| exit.try_upcast_to_callable(db, &env)) + else { + return false; + }; + + let exception_type = KnownClass::BaseException.to_subclass_of(db, &env); + let exception_instance = KnownClass::BaseException.to_instance(db, &env); + for signature in callables + .iter() + .flat_map(|callable| callable.signatures(db)) + { + if signature + .parameters() + .get_positional(0) + .is_some_and(|parameter| { + parameter + .annotated_type() + .is_disjoint_from(db, &env, exception_type) + }) + || signature + .parameters() + .get_positional(1) + .is_some_and(|parameter| { + parameter.annotated_type().is_disjoint_from( + db, + &env, + exception_instance, + ) + }) + { + continue; + } + + let return_type = if is_async { + let Ok(awaited) = signature.return_ty.try_await(db, &env) else { + continue; + }; + awaited + } else { + signature.return_ty + }; + + if return_type.is_equivalent_to(db, &env, KnownClass::Bool.to_instance(db, &env)) + || return_type.is_equivalent_to(db, &env, Type::bool_literal(true)) + { + return true; + } + } + + false + } + + can_suppress_exceptions_impl(db, env.program(db), self, mode.is_async()) + } + /// Returns the type bound from a context manager with type `self`. /// /// This method should only be used outside of type checking because it omits any errors. diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index 4957cb307a..f5b5dfcbbf 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -2183,8 +2183,8 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { } else { // Call into the context expression inference to validate that it evaluates // to a valid context manager. - let context_expression_ty = - self.infer_expression(&item.context_expr, TypeContext::default()); + let context_expression_ty = self + .infer_maybe_standalone_expression(&item.context_expr, TypeContext::default()); self.infer_context_expression(&item.context_expr, context_expression_ty, *is_async); self.infer_optional_expression(target, TypeContext::default()); } diff --git a/crates/ty_python_semantic/src/types/narrow.rs b/crates/ty_python_semantic/src/types/narrow.rs index 15f226acbf..8ed9986248 100644 --- a/crates/ty_python_semantic/src/types/narrow.rs +++ b/crates/ty_python_semantic/src/types/narrow.rs @@ -101,7 +101,9 @@ pub(crate) fn infer_narrowing_constraints<'db>( .and_then(|constraints| constraints.get(&place).cloned()); (positive, None) } - PredicateNode::IsNonTerminalCall(_) + PredicateNode::ContextManagerSuppresses { .. } + | PredicateNode::FinallyNormalPathImpossible { .. } + | PredicateNode::IsNonTerminalCall(_) | PredicateNode::IsNonEmptyIterable(_) | PredicateNode::OrPatternAlternative(_) | PredicateNode::StarImportPlaceholder(_) => (None, None), @@ -1502,7 +1504,9 @@ impl<'db, 'ast> NarrowingConstraintsBuilder<'db, 'ast> { PredicateNode::SubjectElementPattern(subject_element) => { self.evaluate_subject_element_pattern(subject_element) } - PredicateNode::IsNonTerminalCall(_) => return None, + PredicateNode::ContextManagerSuppresses { .. } + | PredicateNode::FinallyNormalPathImpossible { .. } + | PredicateNode::IsNonTerminalCall(_) => return None, PredicateNode::IsNonEmptyIterable(_) => return None, PredicateNode::OrPatternAlternative(_) => return None, PredicateNode::StarImportPlaceholder(_) => return None, @@ -3194,8 +3198,10 @@ impl<'db> NarrowingConstraintsBuilder<'db, '_> { fn scope(&self) -> ScopeId<'db> { let db = self.db; match self.predicate { - PredicateNode::Expression(expression) => expression.scope(db), + PredicateNode::Expression(expression) + | PredicateNode::ContextManagerSuppresses { expression, .. } => expression.scope(db), PredicateNode::Pattern(pattern) => pattern.scope(db), + PredicateNode::FinallyNormalPathImpossible { scope, .. } => scope, PredicateNode::OrPatternAlternative(scope) => scope, PredicateNode::SubjectElementPattern(subject_element) => { subject_element.pattern.scope(db) From 52ec0cf408d3cabbfc180434db750187c55d82d7 Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Sat, 15 Aug 2026 17:43:07 +0200 Subject: [PATCH 057/371] [ty] Reduce notebook version logging to debug (#27783) --- crates/ty_server/src/session/index.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/crates/ty_server/src/session/index.rs b/crates/ty_server/src/session/index.rs index a46b23c012..cafb21f00c 100644 --- a/crates/ty_server/src/session/index.rs +++ b/crates/ty_server/src/session/index.rs @@ -109,8 +109,8 @@ impl Index { ) }); - tracing::info!( - "version: {}, new_version: {}", + tracing::debug!( + "Updating notebook document from version {} to version {}", notebook.version(), new_version ); From 5428c216bf56d28191e82d495de62d2f8025ba49 Mon Sep 17 00:00:00 2001 From: Auguste Lalande Date: Sat, 15 Aug 2026 12:26:53 -0400 Subject: [PATCH 058/371] [ty] Expand nested union aliases when finding a TypedDict or callable (#27740) ## Summary A union-valued type alias nested inside another union stayed unexpanded, hiding a TypedDict or callable from inference. Fixed by expanding aliases before shape checks. ## Test Plan Added mdtest. --------- Co-authored-by: Claude Opus 5 Co-authored-by: Carl Meyer --- .../resources/mdtest/attributes.md | 16 ++ .../resources/mdtest/bidirectional.md | 47 ++++ .../mdtest/generics/pep695/functions.md | 32 +++ .../resources/mdtest/typed_dict.md | 233 ++++++++++++++++++ crates/ty_python_semantic/src/types.rs | 50 ++-- .../ty_python_semantic/src/types/call/bind.rs | 8 +- .../src/types/call/bind/constructor.rs | 2 +- .../src/types/diagnostic.rs | 16 +- .../src/types/infer/builder.rs | 30 ++- .../src/types/infer/builder/dict.rs | 4 +- .../src/types/infer/builder/function.rs | 2 +- .../src/types/infer/builder/type_form.rs | 2 +- .../src/types/typed_dict.rs | 2 +- 13 files changed, 400 insertions(+), 44 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/attributes.md b/crates/ty_python_semantic/resources/mdtest/attributes.md index fc45a569fe..4f9fb45be7 100644 --- a/crates/ty_python_semantic/resources/mdtest/attributes.md +++ b/crates/ty_python_semantic/resources/mdtest/attributes.md @@ -2077,6 +2077,22 @@ class UsesMaybeGeneratedDescriptorWithDynamicBase(DynamicGeneratedBase, metaclas reveal_type(UsesMaybeGeneratedDescriptorWithDynamicBase().generated_descriptor) # revealed: Literal["descriptor"] | Any ``` +A union alias must not hide a non-descriptor member: the same dynamic fallback remains possible +after expanding it: + +```py +from typing_extensions import TypeAliasType + +GeneratedDescriptorOrInt = TypeAliasType("GeneratedDescriptorOrInt", GeneratedDescriptor | int) + +class AliasedDescriptorMeta(MaybeDescriptorMeta): + generated_descriptor: GeneratedDescriptorOrInt | GeneratedDescriptor + +class UsesAliasedDescriptorWithDynamicBase(DynamicGeneratedBase, metaclass=AliasedDescriptorMeta): ... + +reveal_type(UsesAliasedDescriptorWithDynamicBase().generated_descriptor) # revealed: Literal["descriptor"] | Any +``` + Dynamic bases are ignored when descriptor detection requires a concrete `__get__` method: ```py diff --git a/crates/ty_python_semantic/resources/mdtest/bidirectional.md b/crates/ty_python_semantic/resources/mdtest/bidirectional.md index 43e30e06d5..541568ba05 100644 --- a/crates/ty_python_semantic/resources/mdtest/bidirectional.md +++ b/crates/ty_python_semantic/resources/mdtest/bidirectional.md @@ -1827,6 +1827,53 @@ _: list[int | str] = f12() # error: [invalid-assignment] reveal_type(f12) # revealed: () -> list[int] ``` +## Lambda contextual inference through union type aliases + +A lambda parameter is inferred from a callable behind a union-valued type alias, including when that +alias is itself an element of another union: + +```py +from typing import Callable +from typing_extensions import TypeAliasType + +type IntCallback = Callable[[int], None] +type IntCallbackOrInt = Callable[[int], None] | int +IntCallbackOrIntAliasType = TypeAliasType("IntCallbackOrIntAliasType", Callable[[int], None] | int) + +def consume(value: int) -> None: + pass + +x1: Callable[[int], None] | str = lambda value: consume(reveal_type(value)) # revealed: int +x2: IntCallbackOrInt | str = lambda value: consume(reveal_type(value)) # revealed: int +x3: IntCallbackOrIntAliasType | str = lambda value: consume(reveal_type(value)) # revealed: int + +# TODO: An alias that does not resolve to a union is not expanded here, so the parameter is not +# inferred from the type context. +x4: IntCallback = lambda value: consume(reveal_type(value)) # revealed: Unknown +``` + +## Lambda contextual inference through `TypeAliasType` on Python 3.11 + +```toml +[environment] +python-version = "3.11" +``` + +On Python 3.11, `typing_extensions.TypeAliasType` provides the same alias semantics without the +`type` statement: + +```py +from typing import Callable +from typing_extensions import TypeAliasType + +IntCallbackOrInt = TypeAliasType("IntCallbackOrInt", Callable[[int], None] | int) + +def consume(value: int) -> None: + pass + +y1: IntCallbackOrInt | str = lambda value: consume(reveal_type(value)) # revealed: int +``` + ## Unified call inference Generic call arguments are inferred under fixpoint iteration, allowing constraints from call diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md index a9d39cc2f5..f6d917c67f 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md @@ -1386,6 +1386,38 @@ def _(x: list[int], y: dict[int, int]): reveal_type(h(y)) # revealed: int | None ``` +A bounded type variable should still be enforced when it appears in multiple union members and the +argument is itself a union. This currently exposes : + +```py +class Box[T]: ... + +def unbox[T: bytes](value: Box[T] | T) -> T: + raise NotImplementedError + +def invalid_union(value: int | str) -> None: + # TODO: This should report [invalid-argument-type]: neither `int` nor `str` satisfies `T: bytes`. + reveal_type(unbox(value)) # revealed: Unknown +``` + +The same missing constraint lets an incompatible generic overload win over a matching overload: + +```py +from typing import assert_type, overload + +@overload +def select[T: bytes](value: Box[T] | T) -> T: ... +@overload +def select(value: int | str) -> bool: ... +def select(value: object) -> object: + raise NotImplementedError + +def selects_invalid_overload(value: int | str) -> None: + # TODO: This should select the second overload and infer `bool`. + # error: [type-assertion-failure] "Type `Unknown` does not match asserted type `bool`" + assert_type(select(value), bool) +``` + ## Bounded typevar call context through a union Regression test for an `invalid-assignment` false positive: `list(items)` should be assignable to diff --git a/crates/ty_python_semantic/resources/mdtest/typed_dict.md b/crates/ty_python_semantic/resources/mdtest/typed_dict.md index 16e514dfcf..582341ff04 100644 --- a/crates/ty_python_semantic/resources/mdtest/typed_dict.md +++ b/crates/ty_python_semantic/resources/mdtest/typed_dict.md @@ -1080,6 +1080,239 @@ def takes_td_or_iterable(value: TD | Iterable[int]) -> None: takes_td_or_iterable({42: 42}) ``` +## Union of `TypedDict` behind a type alias + +```toml +[environment] +python-version = "3.12" +``` + +A `TypedDict` is still found when the annotation reaches it through a type alias, including when the +alias resolves to a union and is itself one element of a larger union: + +```py +from typing import TypedDict +from typing_extensions import TypeAliasType + +class Person(TypedDict): + name: str + age: int | None + +type PersonAlias = Person +type PersonOrId = Person | int +PersonOrIdAliasType = TypeAliasType("PersonOrIdAliasType", Person | int) + +aliased: PersonAlias = {"name": "Alice", "age": 30} +reveal_type(aliased) # revealed: Person + +aliased_in_union: PersonAlias | str = {"name": "Alice", "age": 30} +reveal_type(aliased_in_union) # revealed: Person + +union_alias: PersonOrId = {"name": "Alice", "age": 30} +reveal_type(union_alias) # revealed: Person + +union_alias_in_union: PersonOrId | str = {"name": "Alice", "age": 30} +reveal_type(union_alias_in_union) # revealed: Person + +alias_type_in_union: PersonOrIdAliasType | str = {"name": "Alice", "age": 30} +reveal_type(alias_type_in_union) # revealed: Person +``` + +A dictionary constructed with keyword arguments uses the same aliased `TypedDict` context: + +```py +constructed: PersonOrId | str = dict(name="Alice", age=30) +reveal_type(constructed) # revealed: Person +``` + +Keys are still validated against the aliased `TypedDict`: + +```py +# error: [invalid-key] "Unknown key "nickname" for TypedDict `Person`" +unknown_key: PersonOrId | str = {"name": "Alice", "age": 30, "nickname": "Ali"} +``` + +Expanding can leave a single `TypedDict` rather than a union, when every arm aliases the same one. +Such an annotation is still validated field by field: + +```py +type FirstPerson = Person +type SecondPerson = Person + +collapsed: FirstPerson | SecondPerson = {"name": "Alice", "age": 30} +reveal_type(collapsed) # revealed: Person + +collapsed_constructor: FirstPerson | SecondPerson = dict(name="Alice", age=30) +reveal_type(collapsed_constructor) # revealed: Person + +# error: [invalid-key] "Unknown key "nickname" for TypedDict `Person`" +collapsed_unknown_key: FirstPerson | SecondPerson = {"name": "Alice", "age": 30, "nickname": "Ali"} + +collapsed_constructor_unknown_key: FirstPerson | SecondPerson = dict( + name="Alice", + age=30, + # error: [invalid-key] "Unknown key "nickname" for TypedDict `Person`" + nickname="Ali", +) +``` + +The same holds where the annotation is a parameter default or a nested field: + +```py +class Team(TypedDict): + lead: FirstPerson | SecondPerson + +# error: [invalid-key] "Unknown key "nickname" for TypedDict `Person`" +def hire(person: FirstPerson | SecondPerson = {"name": "Alice", "age": 30, "nickname": "Ali"}): ... + +# error: [invalid-key] "Unknown key "nickname" for TypedDict `Person`" +team: Team = {"lead": {"name": "Alice", "age": 30, "nickname": "Ali"}} +``` + +Constructor inference currently ignores compatible non-`TypedDict` union members. This also occurs +without aliases; expansion only exposes the existing limitation: + +```py +# TODO: The `dict[str, str]` fallback should accept this constructor without errors. +# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor" +# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor" +# error: [invalid-key] "Unknown key "other" for TypedDict `Person`" +accepted_by_fallback: PersonOrId | dict[str, str] = dict(other="x") + +# TODO: This should reveal `dict[str, str]`, not `Person`. +reveal_type(accepted_by_fallback) # revealed: Person +``` + +The same limitation applies to arguments, return values, and nested `TypedDict` fields: + +```py +class Roster(TypedDict): + lead: PersonOrId | dict[str, str] + +def takes_fallback(value: PersonOrId | dict[str, str]) -> None: ... + +# TODO: The `dict[str, str]` fallback should accept this argument without errors. +# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor" +# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor" +# error: [invalid-key] "Unknown key "other" for TypedDict `Person`" +takes_fallback(dict(other="x")) + +def returns_fallback() -> PersonOrId | dict[str, str]: + # TODO: The `dict[str, str]` fallback should accept this return without errors. + # error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor" + # error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor" + # error: [invalid-key] "Unknown key "other" for TypedDict `Person`" + return dict(other="x") + +# TODO: The `dict[str, str]` fallback should accept this nested value without errors. +# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor" +# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor" +# error: [invalid-key] "Unknown key "other" for TypedDict `Person`" +nested_fallback: Roster = {"lead": dict(other="x")} +``` + +Broader fallback types are also ignored: + +```py +from typing import Any, Mapping + +# TODO: The `Any` fallback should accept this constructor without errors. +# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor" +# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor" +# error: [invalid-key] "Unknown key "other" for TypedDict `Person`" +any_fallback: PersonOrId | Any = dict(other="x") + +# TODO: The `Mapping[str, str]` fallback should accept this constructor without errors. +# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor" +# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor" +# error: [invalid-key] "Unknown key "other" for TypedDict `Person`" +mapping_fallback: PersonOrId | Mapping[str, str] = dict(other="x") +``` + +A constructor with an invalid key is correctly validated when no union member provides a compatible +dictionary fallback, whether the alias appears directly or in a larger union: + +```py +# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor" +# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor" +# error: [invalid-key] "Unknown key "other" for TypedDict `Person`" +no_fallback: PersonOrId = dict(other="x") + +# error: [missing-typed-dict-key] "Missing required key 'name' in TypedDict `Person` constructor" +# error: [missing-typed-dict-key] "Missing required key 'age' in TypedDict `Person` constructor" +# error: [invalid-key] "Unknown key "other" for TypedDict `Person`" +invalid_constructor: PersonOrId | str = dict(other="x") +``` + +## Overload selection with an aliased `TypedDict` + +```toml +[environment] +python-version = "3.12" +``` + +A dictionary literal selects the matching overload when its `TypedDict` type is nested inside a +union-valued alias: + +```py +from typing import TypedDict, assert_type, overload + +class Payload(TypedDict): + required: int + +type PayloadOrInt = Payload | int + +@overload +def select(value: PayloadOrInt | str) -> str: ... +@overload +def select(value: float) -> bytes: ... +def select(value: object) -> object: + return str(value) + +assert_type(select({"required": 1}), str) +``` + +## `TypedDict` behind a `TypeAliasType` alias on Python 3.11 + +```toml +[environment] +python-version = "3.11" +``` + +Expansion is not tied to the `type` statement. `TypeAliasType` is expanded the same way on versions +that predate it, and the `TypedDict` behind one is still found and validated: + +```py +from typing import TypedDict +from typing_extensions import TypeAliasType + +class Person(TypedDict): + name: str + age: int | None + +PersonOrId = TypeAliasType("PersonOrId", Person | int) + +union_alias_in_union: PersonOrId | str = {"name": "Alice", "age": 30} +reveal_type(union_alias_in_union) # revealed: Person +``` + +Dictionary constructors use the same aliased `TypedDict` context: + +```py +constructed: PersonOrId | str = dict(name="Alice", age=30) +reveal_type(constructed) # revealed: Person +``` + +Both dictionary literals and constructors reject unknown keys: + +```py +# error: [invalid-key] "Unknown key "nickname" for TypedDict `Person`" +unknown_key: PersonOrId | str = {"name": "Alice", "age": 30, "nickname": "Ali"} + +# error: [invalid-key] "Unknown key "nickname" for TypedDict `Person`" +invalid_constructor: PersonOrId | str = dict(name="Alice", age=30, nickname="Ali") +``` + ## Type ignore compatibility issues Users should be able to ignore TypedDict validation errors with `# type: ignore` diff --git a/crates/ty_python_semantic/src/types.rs b/crates/ty_python_semantic/src/types.rs index 6fb3645aae..4827dad134 100644 --- a/crates/ty_python_semantic/src/types.rs +++ b/crates/ty_python_semantic/src/types.rs @@ -2597,13 +2597,31 @@ impl<'db> Type<'db> { /// If the type is a union (or a type alias that resolves to a union), filters union elements /// based on the provided predicate. /// + /// Aliases among the elements are expanded first. An element may itself be an alias for a + /// union, which is otherwise left unexpanded so diagnostics can name it, but filtering is a + /// set operation and has to see the members rather than the name. + /// /// Otherwise, returns the type unchanged. - fn filter_union(self, db: &'db dyn Db, f: impl FnMut(&Type<'db>) -> bool) -> Type<'db> { - if let Type::Union(union) = self.resolve_type_alias(db) { - union.filter(db, f) + fn filter_union( + self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + mut f: impl FnMut(&Type<'db>) -> bool, + ) -> Type<'db> { + let Type::Union(union) = self.resolve_type_alias(db) else { + return self; + }; + let union = if union.has_aliases(db) { + match union.expand_aliases(db, env) { + Type::Union(expanded) => expanded, + // Expanding collapsed the union to a single type, leaving nothing to filter + // between, so apply the predicate to it directly. + expanded => return if f(&expanded) { expanded } else { Type::Never }, + } } else { - self - } + union + }; + union.filter(db, f) } /// If the type is a union, removes union elements that are disjoint from `target`. @@ -2617,7 +2635,7 @@ impl<'db> Type<'db> { inferable: TypeVarSet<'db>, ) -> Type<'db> { let constraints = ConstraintSetBuilder::new(); - self.filter_union(db, |elem| { + self.filter_union(db, env, |elem| { !elem .when_disjoint_from(db, env, target, &constraints, inferable) .is_always_satisfied(db, env) @@ -3605,20 +3623,14 @@ impl<'db> Type<'db> { else { return dynamic_instance_fallback; }; - let all_arms_are_possible_data_descriptors = declaration - .ty - .resolve_type_alias(db) - .as_union() - .is_none_or(|union| { - union - .elements(db) - .iter() - .all(|ty| ty.may_be_data_descriptor(db, env)) - }); + let mut all_arms_are_possible_data_descriptors = true; + let descriptor_ty = declaration.ty.filter_union(db, env, |ty| { + let is_possible_data_descriptor = ty.may_be_data_descriptor(db, env); + all_arms_are_possible_data_descriptors &= is_possible_data_descriptor; + is_possible_data_descriptor + }); Place::Defined(DefinedPlace { - ty: declaration - .ty - .filter_union(db, |ty| ty.may_be_data_descriptor(db, env)), + ty: descriptor_ty, definedness: if all_arms_are_possible_data_descriptors { declaration.definedness } else { diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index 490858c515..0e24da57da 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -5819,7 +5819,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { let preferred_type_mappings = return_with_tcx .and_then(|(return_ty, tcx)| { if !tcx - .filter_union(db, |ty| ty.may_prefer_declared_type(db, self.env)) + .filter_union(db, self.env, |ty| ty.may_prefer_declared_type(db, self.env)) .may_prefer_declared_type(db, self.env) { return None; @@ -5879,7 +5879,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { binding.bound_typevar, binding.solution, ) - .filter_union(db, |ty| { + .filter_union(db, self.env, |ty| { if ty.has_unspecialized_type_var(db, self.env) { partially_specialized_declared_type.insert(identity); return false; @@ -5894,8 +5894,8 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { // deeply contains non-inferable typevars. Such types (e.g., // `T@h | list[T@h]` from an outer generic scope) don't provide // useful concrete information and would cause over-expansion. - let concrete_content = - inferred_ty.filter_union(db, |ty| !ty.has_typevar(db, self.env)); + let concrete_content = inferred_ty + .filter_union(db, self.env, |ty| !ty.has_typevar(db, self.env)); if concrete_content.is_never() && inferred_ty.has_typevar(db, self.env) { continue; } diff --git a/crates/ty_python_semantic/src/types/call/bind/constructor.rs b/crates/ty_python_semantic/src/types/call/bind/constructor.rs index 28a61f6851..cf996fe726 100644 --- a/crates/ty_python_semantic/src/types/call/bind/constructor.rs +++ b/crates/ty_python_semantic/src/types/call/bind/constructor.rs @@ -412,7 +412,7 @@ impl<'db> ConstructorBinding<'db> { .copied() .map(|mapped_ty| { let without_unknown = - mapped_ty.filter_union(db, |element| !element.is_unknown()); + mapped_ty.filter_union(db, env, |element| !element.is_unknown()); let mapped_ty = if without_unknown.is_never() { mapped_ty } else { diff --git a/crates/ty_python_semantic/src/types/diagnostic.rs b/crates/ty_python_semantic/src/types/diagnostic.rs index 6ec1d917aa..635a02743b 100644 --- a/crates/ty_python_semantic/src/types/diagnostic.rs +++ b/crates/ty_python_semantic/src/types/diagnostic.rs @@ -1473,13 +1473,14 @@ pub(super) fn report_slice_step_size_zero(context: &InferContext, node: AnyNodeR // We avoid emitting invalid assignment diagnostic for literal assignments to a `TypedDict`, as // they can only occur if we already failed to validate the dict (and emitted some diagnostic). -pub(crate) fn is_invalid_typed_dict_literal( - db: &dyn Db, - target_ty: Type, +pub(crate) fn is_invalid_typed_dict_literal<'db>( + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + target_ty: Type<'db>, source: AnyNodeRef<'_>, ) -> bool { target_ty - .filter_union(db, Type::is_typed_dict) + .filter_union(db, env, Type::is_typed_dict) .as_typed_dict() .is_some() && matches!(source, AnyNodeRef::ExprDict(_)) @@ -1650,7 +1651,12 @@ pub(super) fn report_invalid_assignment<'db>( }; if let Some(value_node) = value_node - && is_invalid_typed_dict_literal(db, target_ty, value_node.into()) + && is_invalid_typed_dict_literal( + db, + context.program_environment(), + target_ty, + value_node.into(), + ) { return; } diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index f5b5dfcbbf..8917ffa79e 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -6412,7 +6412,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { && let Some(tcx) = tcx.annotation && let literal_tcx @ (Type::Union(_) | Type::LiteralValue(_)) = tcx .resolve_type_alias(db) - .filter_union(db, |ty| ty.as_literal_value().is_some()) + .filter_union(db, env, |ty| ty.as_literal_value().is_some()) && ty.is_assignable_to(db, env, literal_tcx) { ty = Type::LiteralValue(literal.to_unpromotable()); @@ -6541,7 +6541,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { for binding in solution { let inferred_ty = binding .solution - .filter_union(db, |ty| !ty.has_unspecialized_type_var(db, env)); + .filter_union(db, env, |ty| !ty.has_unspecialized_type_var(db, env)); if inferred_ty.has_unspecialized_type_var(db, env) { continue; } @@ -7069,12 +7069,17 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { let mut item_types = FxHashMap::default(); // Validate `TypedDict` dictionary literal assignments. - if let Some(annotation) = tcx - .annotation - .map(|annotation| annotation.resolve_type_alias(db)) + if let Some(annotation) = + tcx.annotation + .map(|annotation| match annotation.resolve_type_alias(db) { + Type::Union(union) if union.has_aliases(db) => union.expand_aliases(db, env), + annotation => annotation, + }) { if let Some(typed_dict) = annotation.as_typed_dict() { - // If there is a single typed dict annotation, infer against it directly. + // If there is a single typed dict annotation, infer against it directly. Expanding + // first means a union whose arms all alias the same `TypedDict` reaches this + // branch rather than neither. if let Some(ty) = self.infer_typed_dict_expression(dict, typed_dict, &mut item_types) { @@ -7321,11 +7326,11 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { .zip(specialization.types(self.db())) { let inferred_ty = inferred_ty - .filter_union(db, |ty| { + .filter_union(db, env, |ty| { !ty.as_typevar() .is_some_and(|tv| tv.is_inferable(self.db(), inferable)) }) - .filter_union(db, |ty| !ty.has_unspecialized_type_var(db, env)); + .filter_union(db, env, |ty| !ty.has_unspecialized_type_var(db, env)); if inferred_ty.has_unspecialized_type_var(db, env) { continue; } @@ -7375,8 +7380,9 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // Avoid inferring a preferred type based on partially specialized // type context from an outer generic call. If the type context is // a union, we try to keep any concrete elements. - let inferred_ty = inferred_ty - .filter_union(db, |ty| !ty.has_unspecialized_type_var(db, env)); + let inferred_ty = inferred_ty.filter_union(db, env, |ty| { + !ty.has_unspecialized_type_var(db, env) + }); if inferred_ty.has_unspecialized_type_var(db, env) { continue; } @@ -8355,7 +8361,9 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // TODO: We could perform multi-inference here if there are multiple `Callable` annotations // in the union/intersection. let callable_tcx = if let Some(tcx) = tcx.annotation - && let Some(callable) = tcx.filter_union(db, Type::is_callable_type).as_callable() + && let Some(callable) = tcx + .filter_union(db, env, Type::is_callable_type) + .as_callable() { match callable.signatures(self.db()).overloads.as_slice() { [signature] => Some(signature), diff --git a/crates/ty_python_semantic/src/types/infer/builder/dict.rs b/crates/ty_python_semantic/src/types/infer/builder/dict.rs index 9a6c68ac1b..bd6d41af8b 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/dict.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/dict.rs @@ -26,7 +26,9 @@ impl<'db> TypeInferenceBuilder<'db, '_> { // then validate and return the TypedDict type. This also covers `dict(**src)` when `src` // is `TypedDict`-shaped. if let Some(tcx) = call_expression_tcx.annotation - && let Some(typed_dict) = tcx.filter_union(db, Type::is_typed_dict).as_typed_dict() + && let Some(typed_dict) = tcx + .filter_union(db, self.program_environment(), Type::is_typed_dict) + .as_typed_dict() { // Only speculate the `**kwargs` applicability check. Assignability handles inputs that // are already valid for the target, including gradual and bottom types. The additional diff --git a/crates/ty_python_semantic/src/types/infer/builder/function.rs b/crates/ty_python_semantic/src/types/infer/builder/function.rs index ae2c79e0fc..98b0d99608 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/function.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/function.rs @@ -1058,7 +1058,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // Avoid duplicate diagnostics: invalid TypedDict literals already emit specific errors. let suppress_invalid_default = - is_invalid_typed_dict_literal(db, declared_ty, default_expr.into()); + is_invalid_typed_dict_literal(db, env, declared_ty, default_expr.into()); if !default_ty.is_assignable_to(db, env, declared_ty) && !suppress_invalid_default && !((self.in_stub() diff --git a/crates/ty_python_semantic/src/types/infer/builder/type_form.rs b/crates/ty_python_semantic/src/types/infer/builder/type_form.rs index 88dfa725aa..355b0d85c1 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/type_form.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/type_form.rs @@ -28,7 +28,7 @@ impl<'db> TypeInferenceBuilder<'db, '_> { .iter() .any(|element| matches!(element.resolve_type_alias(db), Type::TypeForm(_))) => { - Some(target.filter_union(db, |element| { + Some(target.filter_union(db, env, |element| { !matches!(element.resolve_type_alias(db), Type::TypeForm(_)) })) } diff --git a/crates/ty_python_semantic/src/types/typed_dict.rs b/crates/ty_python_semantic/src/types/typed_dict.rs index 1d3225c596..78cc9a5b23 100644 --- a/crates/ty_python_semantic/src/types/typed_dict.rs +++ b/crates/ty_python_semantic/src/types/typed_dict.rs @@ -1533,7 +1533,7 @@ impl<'db> TypedDictKeyAssignment<'_, 'db, '_> { return true; } - if diagnostic::is_invalid_typed_dict_literal(db, item.declared_ty, self.value_node) { + if diagnostic::is_invalid_typed_dict_literal(db, env, item.declared_ty, self.value_node) { return false; } From a1a17be7ceb44e06223076f78db34747b9161a3f Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Sat, 15 Aug 2026 09:42:41 -0700 Subject: [PATCH 059/371] [ty] Preserve variadic generics in functools.partial (#27774) `functools.partial` incorrectly inferred an untouched `TypeVarTuple` as an empty tuple, so later calls rejected valid positional arguments. Preserve unresolved variadic type parameters while constructing a partial, while retaining correct empty-pack inference for completed calls. Fixes astral-sh/ty#4271. ## Test plan Added mdtests covering partials with no bound arguments, fixed and generic bound leading parameters, empty and nonempty subsequent calls, and the original Python 3.14 `asyncio.run_in_executor` regression. --- .../mdtest/call/functools_partial.md | 84 +++++++++++++++++++ .../ty_python_semantic/src/types/call/bind.rs | 34 ++++++-- 2 files changed, 112 insertions(+), 6 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/call/functools_partial.md b/crates/ty_python_semantic/resources/mdtest/call/functools_partial.md index 83c80a92cf..7886deaae0 100644 --- a/crates/ty_python_semantic/resources/mdtest/call/functools_partial.md +++ b/crates/ty_python_semantic/resources/mdtest/call/functools_partial.md @@ -393,6 +393,90 @@ reveal_type(p(2)) # revealed: tuple[int, int] reveal_type(p(2)[1]) # revealed: int ``` +### Variadic generic functions with no bound arguments + +A partial with no bound arguments preserves its variadic type parameter until the resulting callable +is invoked. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from functools import partial + +def collect[*Ts](*values: *Ts) -> tuple[*Ts]: + return values + +bound = partial(collect) +reveal_type(bound) # revealed: partial[[*Ts](*values: Ts) -> tuple[*Ts]] +reveal_type(bound()) # revealed: tuple[()] +reveal_type(bound("x", 1)) # revealed: tuple[Literal["x"], Literal[1]] +``` + +### Variadic generic functions with a bound leading parameter + +Binding a fixed leading parameter leaves the variadic type parameter available for later arguments. +A completed call with no variadic arguments still infers an empty tuple. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from functools import partial + +def collect[*Ts](prefix: int, *values: *Ts) -> tuple[*Ts]: + return values + +bound = partial(collect, 1) +reveal_type(bound) # revealed: partial[[*Ts](*values: Ts) -> tuple[*Ts]] +reveal_type(bound()) # revealed: tuple[()] +reveal_type(bound("x", 2)) # revealed: tuple[Literal["x"], Literal[2]] +reveal_type(collect(1)) # revealed: tuple[()] +``` + +### Variadic generic functions with a bound generic leading parameter + +A bound ordinary type parameter is specialized while an untouched variadic type parameter remains +generic. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from functools import partial + +def collect[T, *Ts](prefix: T, *values: *Ts) -> tuple[T, *Ts]: + return (prefix, *values) + +bound = partial(collect, 1) +reveal_type(bound) # revealed: partial[[*Ts](*values: Ts) -> tuple[Literal[1], *Ts]] +reveal_type(bound()) # revealed: tuple[Literal[1]] +reveal_type(bound("x", True)) # revealed: tuple[Literal[1], Literal["x"], Literal[True]] +``` + +### Partially bound asyncio executor callback + +Binding the executor must not consume the callback's variadic arguments before it is called. + +```toml +[environment] +python-version = "3.14" +``` + +```py +import asyncio +from functools import partial + +callback = partial(asyncio.get_running_loop().run_in_executor, None) +asyncio.run(callback(print, "")) +``` + ### Generic functions preserve defaults for no-longer-inferable type params ```py diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index 0e24da57da..59edb78e66 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -1098,7 +1098,7 @@ impl<'db> Bindings<'db> { .bindings(db, env) .match_parameters(db, env, &bound_call_arguments); for binding in partial_bindings.iter_flat_mut() { - binding.clear_missing_argument_errors_for_partial_application(); + binding.prepare_for_partial_application(); } for constructor in partial_bindings.iter_constructor_items_mut() { if let Some(downstream) = constructor.downstream_constructor_mut() { @@ -3443,13 +3443,13 @@ impl<'db> CallableBinding<'db> { } } - /// Ignore missing-argument errors when constructing `functools.partial(...)`. + /// Prepare these overloads for constructing `functools.partial(...)`. /// /// Partial application intentionally leaves some parameters unbound, so we still want to - /// type-check all explicitly bound arguments against each overload. - fn clear_missing_argument_errors_for_partial_application(&mut self) { + /// type-check all explicitly bound arguments without treating unbound parameters as absent. + fn prepare_for_partial_application(&mut self) { for overload in &mut self.overloads { - overload.clear_missing_argument_errors_for_partial_application(); + overload.prepare_for_partial_application(); } } @@ -5422,6 +5422,7 @@ struct ArgumentTypeChecker<'a, 'db> { call_expression_tcx: TypeContext<'db>, return_ty: Type<'db>, errors: &'a mut Vec>, + is_partial_application: bool, inferable_typevars: TypeVarSet<'db>, inference: Option>, @@ -5531,6 +5532,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { call_expression_tcx: TypeContext<'db>, return_ty: Type<'db>, errors: &'a mut Vec>, + is_partial_application: bool, ) -> Self { Self { db, @@ -5545,6 +5547,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { call_expression_tcx, return_ty, errors, + is_partial_application, inferable_typevars: TypeVarSet::None, inference: None, constraint_set_errors: vec![false; arguments.len()], @@ -6066,6 +6069,18 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { return Ok(()); } + // An untouched variadic parameter remains available to future calls of a partial. In + // contrast, an ordinary completed call with no variadic arguments infers an empty pack. + if self.is_partial_application + && !self.argument_matches.iter().any(|argument| { + argument + .iter() + .any(|matched| matched.index == parameter_index) + }) + { + return Ok(()); + } + let (formal, typevartuple) = match parameter.annotated_type() { Type::TypeVar(typevar) if typevar.is_typevartuple(db) => ( Type::tuple(TupleType::unpacked_typevartuple(db, self.env, typevar)), @@ -7192,6 +7207,9 @@ pub(crate) struct Binding<'db> { /// The type-variable inference result for this binding, if the callable is generic. inference: Option>, + /// Whether these arguments construct a partial instead of completing a call. + is_partial_application: bool, + /// Information about which parameter(s) each argument was matched with, in argument source /// order. argument_matches: Box<[MatchedArgument<'db>]>, @@ -7276,6 +7294,7 @@ impl<'db> Binding<'db> { constructor_context: None, inferable_typevars: TypeVarSet::None, inference: None, + is_partial_application: false, argument_matches: Box::from([]), variadic_argument_matched_to_variadic_parameter: false, parameter_tys: Box::from([]), @@ -7821,6 +7840,7 @@ impl<'db> Binding<'db> { call_expression_tcx, self.return_ty, &mut self.errors, + self.is_partial_application, ); // If this overload is generic, first see if we can infer a specialization of the function @@ -7930,13 +7950,15 @@ impl<'db> Binding<'db> { } /// `functools.partial(...)` is allowed to leave required parameters unbound. - fn clear_missing_argument_errors_for_partial_application(&mut self) { + fn prepare_for_partial_application(&mut self) { + self.is_partial_application = true; self.errors .retain(|error| !matches!(error, BindingError::MissingArguments { .. })); } /// Downstream constructor validation is deferred until after partial signatures are merged. fn clear_deferred_constructor_errors_for_partial_application(&mut self) { + self.is_partial_application = true; self.errors.retain(|error| { !matches!( error, From 370f735ce20e5c79777e773249753e006cb2e04e Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Sat, 15 Aug 2026 18:06:39 +0100 Subject: [PATCH 060/371] [ty] Improve ecosystem report summaries (#27785) --- .../minimizing-ty-ecosystem-changes/SKILL.md | 44 ++++++++++++------- .../references/advanced-minimization.md | 10 ++--- .../summarise-ecosystem-results/SKILL.md | 32 +++++++++----- .../references/evidence-acquisition.md | 38 +++++++++++++--- .../references/subagent-handoff.md | 32 +++++++++----- .github/workflows/ty-ecosystem-analyzer.yaml | 9 ++-- 6 files changed, 112 insertions(+), 53 deletions(-) diff --git a/.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md b/.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md index f6baa25367..ea77b4fe73 100644 --- a/.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md +++ b/.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md @@ -17,9 +17,9 @@ Start each investigation from fresh artifacts. Do not trust retained memories, p ## Collect Exact-Run Metadata -If a primary agent supplied an existing run-metadata manifest, verify that its run ID and attempt match the frozen report and that it contains each assigned project. Reuse the manifest without modifying it. +If the primary agent supplied an immutable `TY_ECOSYSTEM_RUN_METADATA` manifest, verify that its run ID and attempt match the frozen report and that it contains each assigned project. All subagents reuse the same read-only manifest; never modify it or generate another shared manifest. -Otherwise, run the bundled helper with the Actions run ID or URL, matching attempt, and every affected mypy-primer project name: +Otherwise, run the bundled helper once with the Actions run ID or URL, matching attempt, and every affected mypy-primer project name: ```bash scripts/collect_ty_ecosystem_run_metadata.py \ @@ -34,9 +34,9 @@ The current workflow splits compilation into `Build ty (base)` and `Build ty (pr ## Prepare ty -If a primary agent supplied freshly copied base and PR profiling binaries plus the PR ecosystem config, preserve their absolute paths as `TY_ECOSYSTEM_BASE_BINARY` and `TY_ECOSYSTEM_PR_BINARY`, verify they exist, and reuse them. Do not rebuild those binaries, switch shared Ruff refs, or overwrite the shared artifacts. An agent may build an exact-revision debug binary on demand to identify an ambiguous internal type, using an isolated worktree if necessary; the profiling binaries remain the behavioral oracle. +If a primary agent supplied freshly copied base and PR profiling binaries plus the PR ecosystem config, preserve their absolute paths as `TY_ECOSYSTEM_BASE_BINARY` and `TY_ECOSYSTEM_PR_BINARY`, verify they exist, and reuse them. Do not rebuild those binaries, switch shared Ruff refs, or overwrite the shared artifacts. If an exact-revision debug binary is needed to identify an ambiguous internal type, request it from the primary agent; the profiling binaries remain the behavioral oracle. -Otherwise, require a clean working tree, copy `.github/ty-ecosystem.toml` from the PR revision, and build ty on the manifest's merge base and PR revision: +Otherwise, require a clean working tree, remember its original ref, and build both exact revisions before assigning any subagent work. Reuse the checkout's existing Cargo target directory, copy the profiling binaries and PR ecosystem config to `target/ty-ecosystem-bins`, and restore the original ref when finished: Fetch the PR revision explicitly because pull-request runs usually use a synthetic GitHub merge commit that a normal clone does not contain: @@ -44,23 +44,30 @@ Fetch the PR revision explicitly because pull-request runs usually use a synthet set -euo pipefail test -z "$(git status --short)" || { git status --short; exit 1; } -git fetch origin +original_ref="$(git symbolic-ref --quiet --short HEAD || git rev-parse HEAD)" +git fetch https://github.com/astral-sh/ruff.git mkdir -p target/ty-ecosystem-bins +trap 'git checkout "$original_ref"' EXIT + +artifact_dir="$PWD/target/ty-ecosystem-bins" +build_target_dir="${CARGO_TARGET_DIR:-target}" export CARGO_PROFILE_PROFILING_DEBUG=line-tables-only -git checkout +git checkout --detach cargo build --package ty --profile profiling -cp target/profiling/ty target/ty-ecosystem-bins/ty-base +cp "$build_target_dir/profiling/ty" "$artifact_dir/ty-base" -git checkout -cp .github/ty-ecosystem.toml target/ty-ecosystem-bins/ty-ecosystem.toml +git checkout --detach +cp .github/ty-ecosystem.toml "$artifact_dir/ty-ecosystem.toml" cargo build --package ty --profile profiling -cp target/profiling/ty target/ty-ecosystem-bins/ty-pr +cp "$build_target_dir/profiling/ty" "$artifact_dir/ty-pr" ``` +After restoring the original ref, inspect vendored definitions and Rust implementations with `git -C show :`, selecting the merge-base or PR revision from the immutable manifest. Never assume working-tree files match either analyzed binary or switch the shared checkout's ref. + ## Reproduce -Create a unique temporary directory for each project and use its absolute path. Read its Python version and the pinned mypy-primer revision from the manifest. Obtain the project revision from the `/blob//` component of the original diagnostic's source permalink, and check that links for the same project agree. If no diagnostic permalink exists, inspect the matching diagnostics shard or Actions logs; if the exact revision cannot be recovered, explicitly report that limitation. Then bypass the adjacent script lockfile: +Create a unique temporary directory for each project and use its absolute path. Read its Python version and the pinned mypy-primer revision from the shared manifest. Obtain the project revision from the `/blob//` component of the original diagnostic's source permalink, and check that links for the same project agree. If no diagnostic permalink exists, inspect the matching diagnostics shard or Actions logs; if the exact revision cannot be recovered, explicitly report that limitation. Then bypass the adjacent script lockfile: ```bash uv run \ @@ -73,7 +80,7 @@ uv run \ --exclude-newer ``` -Use the ecosystem config as user-level configuration, matching CI without replacing project-level config discovery, and re-export `XDG_CONFIG_HOME` in each new shell. If a primary agent supplied `TY_ECOSYSTEM_CONFIG_HOME`, reuse its installed config without modifying it; otherwise, install the copied config locally. Read the project's `strict` or `non-strict` label from the frozen detailed report, or its `strict_settings` value from the matching diagnostics shard. Preserve that mode when running either binary: +Use the ecosystem config as user-level configuration, matching CI without replacing project-level config discovery, and re-export `XDG_CONFIG_HOME` and `RUST_BACKTRACE=1` in each new shell. If a primary agent supplied `TY_ECOSYSTEM_CONFIG_HOME`, reuse its installed config without modifying it; otherwise, install the copied config locally. Read the project's `strict` or `non-strict` label from the frozen detailed report, or its `strict_settings` value from the matching diagnostics shard. Preserve that mode when running either binary: ```bash if [[ -n "${TY_ECOSYSTEM_CONFIG_HOME:-}" ]]; then @@ -85,6 +92,7 @@ else cp "$PWD/target/ty-ecosystem-bins/ty-ecosystem.toml" "$XDG_CONFIG_HOME/ty/ty.toml" fi unset TY_CONFIG_FILE +export RUST_BACKTRACE=1 project_dir="" ty_base="${TY_ECOSYSTEM_BASE_BINARY:-$PWD/target/ty-ecosystem-bins/ty-base}" @@ -116,16 +124,18 @@ pr_exit_status=0 run_ecosystem_ty || pr_exit_status=$? ``` -Confirm the detailed report's difference exactly, including duplicate diagnostics and both exit statuses. Ordinary diagnostics can produce exit status 1; do not mistake that for a failed reproduction. +Confirm the detailed report's difference exactly, including duplicate diagnostics and both exit statuses. When reproducing an intermittent severe failure, repeat each side using its reported run count. Ordinary diagnostics can produce exit status 1; do not mistake that for a failed reproduction. For panics, identify the stable fingerprint by comparing the Rust panic site or decisive causal frame and panic payload; ignore checked Python-file paths and incidental backtrace differences. ## Minimize -Reduce the reproduced project toward a self-contained single-file reproducer with minimal code and dependencies. A reduction is trivial only when the difference already occurs in one self-contained file and can be preserved solely by deleting obviously unrelated code. Multiple files, imports or dependencies, inlining, replacing language constructs, ambiguous types such as `@Todo`, or an uncertain cause make a reduction nontrivial. Before attempting any nontrivial reduction, read and follow [references/advanced-minimization.md](references/advanced-minimization.md). If in doubt, treat the reduction as nontrivial. +The target is a fully minimized, provenance-preserving reproducer: preferably one self-contained file, with no avoidable third-party or standard-library imports and no unnecessary definitions, annotations, branches, or advanced language features. Retain a third-party import only if identified ty behavior depends on that library's identity or third-party search-path classification. + +Before minimizing any ecosystem change, read and follow [references/advanced-minimization.md](references/advanced-minimization.md). Exhaust its complete reduction loop, including third-party dependency and standard-library inlining, and retain an import only after verifying that neither removing it nor inlining its definitions preserves the underlying behavior. -Matching diagnostics or displayed types do not establish a shared cause. When the output is ambiguous, identify the original and minimized triggers using exact-revision debug output, a targeted `reveal_type`, or the producing Rust call site. +Matching diagnostics or displayed types do not establish a shared cause. When the output is ambiguous, identify and compare the original and minimized triggers using exact-revision debug output, a targeted `reveal_type`, or the producing Rust call site from the matching analyzed revision. -Record the original source permalink, accepted reductions, both binaries' results, and any causal fingerprint. If source provenance or a matching cause cannot be established, return the original project excerpt explicitly marked as unminimized. +A minimization is complete only when a verified reduction chain connects the reproducer to the original ecosystem entry and an exhaustive pass finds no further reduction. If a genuine external blocker prevents completion, report the blocker and identify the minimization as incomplete; an original source excerpt is not a successfully minimized result. ## Return -Provide the original permalinked report entry, exact base and PR behavior, minimal code, full diagnostic messages and error codes, and the manifest/commands needed to reproduce it. When called from the summary workflow, return import-audit and reduction notes separately from report-ready Markdown. +Provide the original permalinked report entry, exact base and PR behavior, minimal code, full diagnostic messages and error codes or the panic fingerprint, and the manifest/commands needed to reproduce it. When called from the summary workflow, return import-audit and reduction notes separately from report-ready Markdown. diff --git a/.agents/skills/minimizing-ty-ecosystem-changes/references/advanced-minimization.md b/.agents/skills/minimizing-ty-ecosystem-changes/references/advanced-minimization.md index 056b8f0c45..b374f35171 100644 --- a/.agents/skills/minimizing-ty-ecosystem-changes/references/advanced-minimization.md +++ b/.agents/skills/minimizing-ty-ecosystem-changes/references/advanced-minimization.md @@ -4,7 +4,7 @@ Use this reference after the reported difference reproduces against the copied b ## Target -Prefer a single-file reproducer with no third-party imports, few definitions, and the least complex typing or language features that still demonstrate the difference. Keep special modules such as `typing`, `abc`, `enum`, `types`, and `typing_extensions` only when removing them changes the behavior. +Prefer a single-file reproducer with no avoidable third-party imports, few definitions, and the least complex typing or language features that still demonstrate the difference. Keep special modules such as `typing`, `abc`, `enum`, `types`, and `typing_extensions` only when neither removing them nor inlining their definitions preserves the behavior; retain a third-party import only after identifying ty behavior that depends on that library's identity or third-party search-path classification. ## Reduction Loop @@ -13,16 +13,16 @@ Work systematically from the reproduced project. NEVER skip ahead to an explanat 1. Delete unrelated files. 2. Remove imports, definitions, decorators, annotations, statements, and branches. 3. Inline first-party definitions into the reproducer. -4. For each required third-party dependency, copy the entire installed dependency into the source tree as first-party code, including every package directory and module it provides. Do this before attempting to minimize any part of the dependency. Adjust imports, verify that the difference still reproduces with the complete copy, and only then begin deleting files or definitions from it. Never start by copying only apparently relevant files or definitions. If cloning a dependency is unavoidable, use the exact installed revision or version and copy the complete dependency into the source tree before reducing it. -5. Inline the relevant standard-library definitions from `crates/ty_vendored`, which is ty's source of truth for stdlib types. +4. For each required third-party dependency, copy the entire installed dependency into the source tree as first-party code, including every package directory and module it provides. Do this before attempting to minimize any part of the dependency. Adjust imports, verify that the difference still reproduces with the complete copy, and only then begin deleting files or definitions from it. If the complete copy changes the behavior because ty special-cases that library or distinguishes first-party from third-party search paths, identify the relevant ty implementation at the matching analyzed Ruff revision before retaining the original import. Never start by copying only apparently relevant files or definitions. If cloning a dependency is unavoidable, use the exact installed revision or version and copy the complete dependency into the source tree before reducing it. +5. Inline the relevant standard-library definitions from the analyzed revision of `crates/ty_vendored`, using `git -C show :crates/ty_vendored/`; compare the merge-base and PR definitions when they differ. 6. Replace complex constructs with simpler equivalents, such as removing a walrus expression or replacing a protocol when the difference survives. Repeat the full loop until an exhaustive pass through every stage finds no further reduction that preserves the difference. Do not stop merely because the likely cause is understood or the reproducer is already small. ## Final Audit -Attempt to remove every remaining import and inline every remaining third-party definition. Record why any surviving import is essential. Keep these notes as working evidence; the caller decides whether they belong in its final artifact. +Attempt to remove every remaining import and inline its definitions, including remaining third-party and standard-library definitions. Retain an import only after verifying that neither removal nor inlining preserves the underlying behavior. For a third-party import, additionally verify that its module identity or third-party search-path classification is essential and identify the relevant ty implementation. Convenience, familiar APIs, matching class names, or preserving the diagnostic's module spelling do not justify keeping an import. Record why any surviving import is essential and, for a third-party import, where ty implements the relevant behavior. Keep these notes as working evidence; the caller decides whether they belong in its final artifact. -Verify that the recorded reduction chain connects the final reproducer to the original ecosystem entry and, when diagnostic output is ambiguous, preserves the original causal fingerprint. If either check fails, return the original project excerpt as unminimized instead of substituting an unrelated example. +Verify that the recorded reduction chain connects the final reproducer to the original ecosystem entry and, when diagnostic output is ambiguous, preserves the original causal fingerprint. If a required check fails, continue investigating; if a genuine external blocker prevents completion, report the blocker and mark the minimization as incomplete instead of presenting an unrelated example or original excerpt as a minimized result. Delete transient project and dependency copies after the investigation. diff --git a/.agents/skills/summarise-ecosystem-results/SKILL.md b/.agents/skills/summarise-ecosystem-results/SKILL.md index 875f983d69..1136e97397 100644 --- a/.agents/skills/summarise-ecosystem-results/SKILL.md +++ b/.agents/skills/summarise-ecosystem-results/SKILL.md @@ -7,9 +7,11 @@ description: Use when a user says "summarise ecosystem results", "summarize this ## Priorities -1. Reproduce every retained behavior with the exact environment used by the Actions run. -2. Lead the report with new or meaningfully changed project failures, including intermittent severe failures, then cover stable diagnostic changes and clear minimized examples. -3. Keep execution, audit, and traceability bookkeeping out of the report. +1. Reproduce every retained source-attributable behavior with the exact environment used by the Actions run. +2. For every distinct source-attributable behavior change, produce the smallest provenance-preserving reproducer obtainable through the complete advanced-minimization workflow. +3. Eliminate every third-party import unless identified ty behavior depends on that library's identity or third-party search-path classification, and eliminate every unnecessary standard-library import. Retain an import only after verifying that neither removing it nor inlining its definitions preserves the underlying behavior. +4. Lead the report with new or meaningfully changed project failures, including intermittent severe failures, then cover stable diagnostic changes and fully minimized examples. +5. Keep execution, audit, and traceability bookkeeping out of the report. ## Deliverable @@ -23,16 +25,26 @@ If summarising an ecosystem report is the only thing you're asked to do in a Cod - Focus on new or meaningfully changed behavior relative to the merge base. Evaluate individual diagnostics and failure outcomes, not a project's overall flaky or persistent status. - Omit flaky diagnostic changes, unchanged failures, and frequency fluctuations that leave the observed outcomes unchanged. -- Report new or changed panics, crashes, overflows, and timeouts, including merge-base and PR run frequencies when intermittent behavior is involved. +- Report new, fixed, or meaningfully changed panics, crashes, overflows, and timeouts, including merge-base and PR run frequencies when intermittent behavior is involved. ## Workflow -1. **Freeze the evidence.** Preserve any report URL or ecosystem-results comment explicitly supplied by the user before identifying the PR. For PR-only input, find its ecosystem-results comment and linked detailed report. Capture the matching Actions run and attempt as described in [references/evidence-acquisition.md](references/evidence-acquisition.md); never replace a supplied report with the PR's current report. Ignore later comment edits, PR updates, and workflow runs. Use the frozen detailed report as the authoritative change list and the comment for orientation when available. -2. **Identify changed outcomes.** Check the detailed report for new, fixed, or changed project failures, panics, overflows, timeouts, abnormal exits, and diagnostic changes, applying the reporting policy to each entry and outcome. -3. **Reproduce from scratch.** Ignore retained memories and previous local artifacts. Load the `minimizing-ty-ecosystem-changes` skill, use its metadata helper and exact-run workflow, and reproduce each report entry before explaining or minimizing it. Reproduce intermittent severe failure changes with the reported run counts. -4. **Minimize with provenance.** Include a standalone reproducer only when a verified reduction chain connects it to a cited ecosystem entry and preserves the same underlying trigger. If either cannot be verified, retain the original source excerpt and identify it as unminimized. +1. **Freeze the evidence.** Preserve any report URL or ecosystem-results comment explicitly supplied by the user before identifying the PR. For PR-only input, find its ecosystem-results comment and linked detailed report. Capture the matching Actions run and attempt as described in [references/evidence-acquisition.md](references/evidence-acquisition.md); never replace a supplied report with the PR's current report. Recover exact-run metadata promptly, then prepare both exact-revision profiling binaries and the shared configuration before assigning subagent work. Ignore later comment edits, PR updates, and workflow runs. Prefer the selected attempt's validated `full-report/diff.json` as the authoritative structured change inventory, retain its matching frozen HTML report, and use the comment for orientation when available. Fall back to the frozen HTML report if the JSON report is unavailable. +2. **Identify changed outcomes.** Inspect the structured diff for added, removed, and modified projects; stable diagnostic additions, removals, and rewrites; project failures; and intermittent exit-status changes. Preserve diagnostic levels, duplicate occurrences, source permalinks, project strictness, panic evidence, and observed run frequencies. Exclude flaky diagnostics and frequency-only noise without excluding stable diagnostics or changed severe failures from flaky projects. Use the matching HTML report for visual context, or as the primary evidence when structured JSON cannot be obtained safely. +3. **Reproduce from scratch.** Ignore retained memories and previous local artifacts. Load the `minimizing-ty-ecosystem-changes` skill, collect exact-run metadata once, and reproduce every retained, source-attributable diagnostic or panic before explaining or minimizing it. Reproduce intermittent severe failure changes with the reported merge-base and PR run counts. Verify retained outcomes without recoverable source against their captured statuses, stderr, panic evidence, and run frequencies. +4. **Minimize to completion with provenance.** For each distinct source-attributable behavior change, follow the complete advanced-minimization workflow until an exhaustive pass finds no further reduction. Derive the reproducer from a cited ecosystem entry through a verified reduction chain; never replace that entry with an independently invented example demonstrating superficially similar behavior. Before accepting a reproducer, attempt to remove every import, inline every third-party definition, and inline relevant standard-library definitions. Retain a third-party import only when identified ty behavior depends on that library's identity or third-party search-path classification and neither removing the import nor inlining its definitions preserves the underlying behavior. If a genuine external blocker prevents completion, report that blocker to the user and identify the task as incomplete. Do not silently substitute an unminimized excerpt or present a partially minimized report as finished. 5. **Group by cause.** Group entries only when the same base-to-PR behavior, underlying trigger, explanation, and reproducer account for every entry. Identical diagnostic text or displayed `@Todo` types do not establish equivalence. 6. **Find existing ty issues.** When a diagnostic change exposes a pre-existing shortcoming in ty, search the `astral-sh/ty` issue tracker for the precise underlying behavior. Link matching issues directly from the relevant report section; do not mistake incorrect or incomplete third-party annotations for ty shortcomings. -7. **Write and verify.** Fill the report template, record each affected project's strict or non-strict analysis mode, and include both strict-analysis flags in the comparison method when applicable. Check every change number, link, diagnostic, reproducer's source provenance, and causal fingerprint when required, then run `uv run --only-group dev --locked prek run --files PR__ECOSYSTEM_SUMMARY.md`. Present the Markdown file as the finished product. +7. **Write and verify.** Fill the report template and verify that every source-attributable behavior change has a fully minimized, provenance-preserving reproducer. Check every change number, link, diagnostic, retained import, reproducer's source provenance, and causal fingerprint when required. Verify that every retained third-party import is essential to identified ty behavior that depends on that library's identity or third-party search-path classification, that no avoidable standard-library import remains, and that no source-attributable section contains an unminimized excerpt. Then run `uv run --only-group dev --locked prek run --files PR__ECOSYSTEM_SUMMARY.md`. Present the Markdown file as the finished product only after these checks pass. -When parallelizing reproduction or minimization, read [references/subagent-handoff.md](references/subagent-handoff.md). Otherwise, keep batches small and work through them sequentially. +## Parallel execution + +This skill explicitly requests subagents when the report contains multiple affected projects or independently investigable entries. + +Once the exact-run metadata, both profiling binaries, and shared configuration are ready, spawn as many subagents as the available concurrency budget and independent work allow, reserving one slot for the primary agent. Keep available slots occupied by assigning further work as subagents finish. + +Assign disjoint projects or explicit report entries. Apparent similarity may guide scheduling, but does not establish causal equivalence. Follow all existing requirements for exhaustive reproduction, verified reduction chains, exhaustive minimization, and grouping by verified cause. + +The primary agent owns the frozen evidence, shared profiling binaries, configuration, coordination, and final report. Follow [references/subagent-handoff.md](references/subagent-handoff.md) for handoff and shared-artifact requirements. + +If multiple independent assignments exist but no subagents are spawned, record the specific reason. diff --git a/.agents/skills/summarise-ecosystem-results/references/evidence-acquisition.md b/.agents/skills/summarise-ecosystem-results/references/evidence-acquisition.md index 4c05457817..f68ae78d53 100644 --- a/.agents/skills/summarise-ecosystem-results/references/evidence-acquisition.md +++ b/.agents/skills/summarise-ecosystem-results/references/evidence-acquisition.md @@ -7,6 +7,8 @@ If no comment matching an explicitly supplied report remains, continue with the Create a unique snapshot directory, save the matching comment when available and the selected attempt's effective job graph, and inspect the run's available artifacts: ```bash +set -euo pipefail + snapshot_dir="$(mktemp -d "${TMPDIR:-/tmp}/ty-ecosystem-report.XXXXXX")" ecosystem_comment_id="" if [[ -n "$ecosystem_comment_id" ]]; then @@ -14,18 +16,40 @@ if [[ -n "$ecosystem_comment_id" ]]; then fi gh run view --repo astral-sh/ruff --attempt \ --json attempt,headSha,jobs,startedAt,updatedAt,url > "$snapshot_dir/run.json" -gh api "repos/astral-sh/ruff/actions/runs//artifacts" > "$snapshot_dir/artifacts.json" +gh api --paginate --slurp \ + "repos/astral-sh/ruff/actions/runs//artifacts?per_page=100" | + jq '{artifacts: [.[].artifacts[]]}' > "$snapshot_dir/artifacts.json" +printf 'TY_ECOSYSTEM_SNAPSHOT_DIR=%s\n' "$snapshot_dir" ``` -`gh run download` cannot select an attempt, and a newer rerun can replace an older attempt's artifacts without changing Ruff's revisions. Before downloading, verify that `full-report` was created during the selected attempt's report-generation job and that each diagnostics shard was created during its matching successful shard job. Use the effective job graph, not the attempt start time: partial reruns legitimately inherit successful jobs and artifacts from earlier attempts. +`gh run download` cannot select an attempt, and a newer rerun can replace an older attempt's artifacts without changing Ruff's revisions. Before downloading, verify that `full-report` was created during the selected attempt's report-generation job and that each diagnostics shard was created during its matching successful shard job. Use the effective job graph, not the attempt start time: partial reruns legitimately inherit successful jobs and artifacts from earlier attempts. Preserve each validated artifact's immutable ID; never re-resolve its mutable name during download. -Download only artifacts that pass these checks; use the shard glob only when every matching artifact belongs to the selected job graph: +Download the validated report and each available, validated shard directly by artifact ID: ```bash -gh run download --repo astral-sh/ruff --name full-report --dir "$snapshot_dir/full-report" -gh run download --repo astral-sh/ruff --pattern 'diagnostics-shard-*' --dir "$snapshot_dir/shards" +download_validated_artifact() { + local artifact_id="$1" + local destination="$2" + + mkdir -p "$destination" + gh api "repos/astral-sh/ruff/actions/artifacts/$artifact_id/zip" \ + > "$snapshot_dir/artifact-$artifact_id.zip" + unzip -q "$snapshot_dir/artifact-$artifact_id.zip" -d "$destination" +} + +download_validated_artifact "$snapshot_dir/full-report" +download_validated_artifact \ + "$snapshot_dir/shards/diagnostics-shard-" ``` -Record the selected Actions attempt and pass it to `scripts/collect_ty_ecosystem_run_metadata.py` with `--attempt `. Verify that the frozen report's Ruff base and PR revisions agree with the resulting manifest, then use the saved report, shards, run, attempt, and matching comment when available throughout the investigation. +When the selected deployed HTML report was available, compare it byte-for-byte with the downloaded artifact's `diff.html` before trusting the adjacent JSON. Record the selected Actions attempt and pass it to `scripts/collect_ty_ecosystem_run_metadata.py` with `--attempt ` once for all projects requiring reproduction. Verify that the frozen HTML report's Ruff base and PR revisions agree with the resulting immutable manifest, then use the saved report, shards, run, attempt, and matching comment when available throughout the investigation. + +## Prefer the Exact Attempt's Structured Diff + +When the validated `full-report` artifact contains `diff.json`, use `$snapshot_dir/full-report/diff.json` as the authoritative structured change inventory and the adjacent `diff.html` as its human-readable counterpart. The JSON contains no Ruff revisions, Actions run ID, or attempt number: its provenance comes from the verified artifact and its matching HTML report, not from its contents or a coincidentally matching PR revision. + +The deployment also exposes a sibling `diff.json` next to the detailed HTML report. Use a deployed JSON file only when it was frozen from the same immutable deployment as the selected HTML report and that deployment can be tied to the selected Actions run and attempt. A current PR deployment, a later artifact, or matching Ruff commits alone cannot establish this provenance. + +Inspect the structured report and the schema or diff-generation code at the exact ecosystem-analyzer revision; do not assume JSON field names or classifications remain stable across revisions. Record each affected project's strict or non-strict analysis mode and include both strict-analysis flags in the comparison method when applicable. -If the selected report's artifacts were replaced or are unavailable, use its frozen deployed report and explicitly describe any unavailable shards or resulting verification limitations. Never silently substitute artifacts produced by an unrelated reporting attempt. +If the selected attempt's JSON report or artifacts are unavailable or replaced, or its artifact HTML differs from the frozen deployed report, use the frozen HTML report, disclose unavailable shards and resulting verification limitations, and never substitute another attempt's artifacts. diff --git a/.agents/skills/summarise-ecosystem-results/references/subagent-handoff.md b/.agents/skills/summarise-ecosystem-results/references/subagent-handoff.md index 78f2bf6565..aa945c91f2 100644 --- a/.agents/skills/summarise-ecosystem-results/references/subagent-handoff.md +++ b/.agents/skills/summarise-ecosystem-results/references/subagent-handoff.md @@ -1,29 +1,39 @@ # Subagent Handoff -Use this reference only when parallelizing reproduction and minimization. +Use this reference whenever the summary skill delegates reproduction or minimization. ## Primary-Agent Responsibilities -Prepare the frozen evidence snapshot, copied base binary, PR binary, PR ecosystem config, and one run-metadata manifest covering every affected project. Record the binaries' absolute paths as `TY_ECOSYSTEM_BASE_BINARY` and `TY_ECOSYSTEM_PR_BINARY`. Choose an absolute `TY_ECOSYSTEM_CONFIG_HOME` and install the copied config once at `$TY_ECOSYSTEM_CONFIG_HOME/ty/ty.toml`. Treat the snapshot, binaries, manifest, copied config, and installed configuration as read-only shared inputs. Batch related entries without creating more assignments than can run concurrently. +Freeze the exact report, Actions run, and attempt. Run `scripts/collect_ty_ecosystem_run_metadata.py` once for all projects with retained source-attributable diagnostics or new, fixed, or meaningfully changed reproducible failure outcomes, then build and copy both exact-revision profiling binaries before assigning any subagent work. + +Publish the immutable `TY_ECOSYSTEM_RUN_METADATA`, `TY_ECOSYSTEM_BASE_BINARY`, and `TY_ECOSYSTEM_PR_BINARY` absolute paths. Install the copied PR ecosystem config once at `$TY_ECOSYSTEM_CONFIG_HOME/ty/ty.toml` and publish the absolute `TY_ECOSYSTEM_CONFIG_HOME` path. Treat the snapshot, optional structured JSON, binaries, metadata, copied config, and installed configuration as read-only shared inputs. + +If a subagent requests an exact-revision debug binary, only the primary agent may build it. Pause every active worker and wait for acknowledgment, verify the shared checkout is clean, remember its original ref, then build and copy the requested binary. Always restore the original ref before resuming workers, even if the build fails; publish the binary's immutable path only after a successful build and restoration. The profiling binaries remain the behavioral oracle. + +Subagents must not regenerate shared manifests, rebuild shared profiling binaries, switch shared Ruff refs, rewrite shared configuration, or overwrite another agent's working files. ## Assignment Checklist Give each subagent: - The PR and detailed report links, plus the ecosystem comment link when available. -- The paths to the frozen detailed report and available diagnostics shards, plus the frozen comment path when available and the selected Actions run and attempt; use these captured inputs instead of refetching live evidence. -- The exact report entries assigned to it. -- The copied-config and metadata-manifest paths, plus the shared `TY_ECOSYSTEM_BASE_BINARY`, `TY_ECOSYSTEM_PR_BINARY`, and `TY_ECOSYSTEM_CONFIG_HOME` values. -- The instruction to follow the `minimizing-ty-ecosystem-changes` skill using a unique temporary directory. -- The instruction to preserve a verified reduction chain and underlying trigger, or return the original source explicitly marked as unminimized. -- Permission to build an exact-revision debug binary on demand for causal inspection, using an isolated worktree if necessary and retaining the profiling binaries as the behavioral oracle. -- The instruction not to rebuild profiling binaries, regenerate the supplied manifest, rewrite the installed configuration, switch shared Ruff refs, overwrite shared artifacts, trust previous local reproductions, or substitute current dependency metadata. +- The paths to the frozen HTML report, optional matching structured JSON, and available diagnostics shards, plus the frozen comment path when available and the selected Actions run and attempt; use these captured inputs instead of refetching live evidence. +- The exact assigned entries from the structured JSON when available, or from the frozen HTML report otherwise; distinguish source-attributable changes from outcomes without recoverable source, and provide the reported merge-base and PR run counts for intermittent severe failures. +- The immutable `TY_ECOSYSTEM_RUN_METADATA`, `TY_ECOSYSTEM_BASE_BINARY`, `TY_ECOSYSTEM_PR_BINARY`, and `TY_ECOSYSTEM_CONFIG_HOME` absolute paths. +- For assignments requiring reproduction, the instruction to use the `minimizing-ty-ecosystem-changes` skill with the shared manifest, copied profiling binaries, installed configuration, and a unique temporary directory; never generate another manifest. +- For source-attributable assignments, the instruction to produce a fully minimized, provenance-preserving reproducer by exhausting the complete advanced-minimization workflow, including third-party dependency inlining, standard-library inlining, and an audit of every remaining import. +- The instruction to inspect vendored definitions and Rust implementations with `git -C show :`, using the analyzed revisions from the immutable manifest rather than the restored working tree. +- The instruction that an independently invented analogue does not satisfy a source-attributable assignment, and that a partially minimized example or an original source excerpt never satisfies any minimization assignment. If a genuine external blocker prevents minimization, return the blocker and mark the assignment as incomplete. +- For outcomes without recoverable source evidence, the instruction to verify and report the captured outcomes, stderr, panic evidence, and run frequencies without requiring a source reproducer or minimized code. +- The instruction to request any exact-revision debug binary from the primary agent instead of building one or switching shared Ruff refs. +- The instruction to stop all checkout-dependent work and subprocesses when the primary agent requests a pause, acknowledge only after they have stopped, and remain paused until explicitly resumed, even if the debug build fails. +- The instruction not to rebuild profiling binaries, regenerate published metadata, rewrite the installed configuration, switch shared Ruff refs, overwrite shared artifacts, trust previous local reproductions, or substitute current dependency metadata. ## Required Return Request: -- Report-ready GitHub-flavored Markdown describing the exact base-versus-PR behavior and minimized code. -- Separate working notes covering the original source permalink, reproduction, accepted reductions, both binaries' results, any necessary causal fingerprint, and the import audit. +- For source-attributable assignments, report-ready GitHub-flavored Markdown describing the exact base-versus-PR behavior and minimized code, plus separate working notes covering the original source permalink, reproduction, accepted reductions, both binaries' results, per-side run counts for intermittent severe failures, any necessary causal fingerprint, and the import audit. +- For outcomes without recoverable source evidence, report-ready GitHub-flavored Markdown describing the verified project outcomes, relevant stderr, panic evidence, and run frequencies. If a later entry has exactly the same behavior change and cause as an already minimized entry, the subagent may classify it as a duplicate instead of repeating the full minimization, but it must explain the match. diff --git a/.github/workflows/ty-ecosystem-analyzer.yaml b/.github/workflows/ty-ecosystem-analyzer.yaml index dea4e2ac02..13a9e53125 100644 --- a/.github/workflows/ty-ecosystem-analyzer.yaml +++ b/.github/workflows/ty-ecosystem-analyzer.yaml @@ -220,13 +220,16 @@ jobs: mkdir dist + # Upload a structured JSON report alongside the HTML report to make + # ecosystem changes easier for agents to analyze. ecosystem-analyzer \ generate-diff \ diagnostics-base.json \ diagnostics-PR.json \ --old-name "${BASE_REF_NAME} (merge base)" \ --new-name "$REF_NAME" \ - --output-html dist/diff.html + --output-html dist/diff.html \ + --output-json dist/diff.json set +e ecosystem-analyzer \ @@ -254,8 +257,8 @@ jobs: echo "diff_statistics_exit_code=$DIFF_STATISTICS_EXIT_CODE" >> "$GITHUB_OUTPUT" - # NOTE: astral-bot deploys both HTML files in this artifact and uses the - # deployed URLs for the report links in its PR comment. + # NOTE: astral-bot deploys every file in this artifact and uses the + # deployed HTML URLs for the report links in its PR comment. # Make sure to update the bot if you rename the artifact. - name: "Upload full report" uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 From b96364cbf66e27df3c87beed5923f561f033b52c Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Sat, 15 Aug 2026 17:39:17 -0400 Subject: [PATCH 061/371] [ty] Extract implicit-attribute inference into a dedicated module (#27790) ## Summary Move implicit-attribute inference out of the large static-class implementation and into `types/class/implicit_attributes.rs`. --- crates/ty_python_semantic/src/types/class.rs | 12 +- .../src/types/class/implicit_attributes.rs | 500 ++++++++++++++++ .../src/types/class/static_literal.rs | 532 +----------------- 3 files changed, 523 insertions(+), 521 deletions(-) create mode 100644 crates/ty_python_semantic/src/types/class/implicit_attributes.rs diff --git a/crates/ty_python_semantic/src/types/class.rs b/crates/ty_python_semantic/src/types/class.rs index 08c587b92a..7221d2a199 100644 --- a/crates/ty_python_semantic/src/types/class.rs +++ b/crates/ty_python_semantic/src/types/class.rs @@ -5,12 +5,12 @@ pub(crate) use self::dynamic_literal::{ DynamicClassAnchor, DynamicClassLiteral, DynamicMetaclassConflict, dynamic_class_bases_argument, }; pub(super) use self::enum_literal::{DynamicEnumAnchor, DynamicEnumLiteral, EnumSpec}; +use self::implicit_attributes::{AugmentedBindings, ImplicitAttribute}; pub use self::known::KnownClass; use self::named_tuple::synthesize_namedtuple_class_member; pub(super) use self::named_tuple::{ DynamicNamedTupleAnchor, DynamicNamedTupleLiteral, NamedTupleField, NamedTupleSpec, }; -use self::static_literal::{AugmentedBindings, ImplicitAttribute}; pub(crate) use self::static_literal::{ ExpandedClassBaseEntry, FrozenDataclassDispatch, StaticClassLiteral, expanded_class_base_entries, @@ -68,6 +68,7 @@ use ty_python_core::{ProgramFile, place_table, use_def_map}; mod dynamic_literal; mod enum_literal; +mod implicit_attributes; mod known; mod named_tuple; mod static_literal; @@ -2227,14 +2228,7 @@ impl<'db> ClassType<'db> { ) -> ImplicitAttribute<'db> { let augmented_bindings = self .static_class_literal(db) - .map(|(class, _)| { - StaticClassLiteral::implicit_attribute_bindings( - db, - class.body_scope(db), - name, - target_method_decorator, - ) - }) + .map(|(class, _)| class.implicit_attribute_bindings(db, name, target_method_decorator)) .filter(|implicit| member.is_undefined() == implicit.member.is_undefined()) .and_then(|implicit| implicit.augmented_bindings); diff --git a/crates/ty_python_semantic/src/types/class/implicit_attributes.rs b/crates/ty_python_semantic/src/types/class/implicit_attributes.rs new file mode 100644 index 0000000000..29ad70f738 --- /dev/null +++ b/crates/ty_python_semantic/src/types/class/implicit_attributes.rs @@ -0,0 +1,500 @@ +//! Implicit instance and class attributes inferred from method assignments. + +use super::{MethodDecorator, static_literal::StaticClassLiteral}; +use crate::{ + Db, ProgramEnvironment, TypeQualifiers, attribute_assignments, attribute_declarations, + place::{Place, Provenance}, + reachability::binding_reachability, + types::{ + KnownClass, Truthiness, Type, TypeContext, UnionBuilder, definition_expression_type, + function::{is_implicit_classmethod, is_implicit_staticmethod}, + infer::infer_unpack_types, + infer_expression_type, inferred_declaration, + member::Member, + }, +}; +use ruff_db::parsed::parsed_module; +use ruff_python_ast::name::Name; +use ty_python_core::{ + attribute_scopes, + definition::{Definition, DefinitionKind, DefinitionState, TargetKind}, + place_table, + scope::{Scope, ScopeId}, + semantic_index, use_def_map, +}; + +#[salsa::tracked] +impl<'db> StaticClassLiteral<'db> { + /// Tries to find declarations/bindings of an attribute named `name` that are only + /// "implicitly" defined (`self.x = …`, `cls.x = …`) in a method of this class. + /// The `target_method_decorator` parameter is used to skip methods that do not have the + /// expected decorator. + pub(super) fn implicit_attribute( + self, + db: &'db dyn Db, + name: &str, + target_method_decorator: MethodDecorator, + ) -> Member<'db> { + self.implicit_attribute_bindings(db, name, target_method_decorator) + .member + } + + /// Separate assignments that establish an attribute from assignments that must first read it. + /// + /// ```python + /// class Counter: + /// def increment(self): + /// self.value += 1 + /// ``` + /// + /// Here, `value` remains undefined until MRO lookup finds an independent class or instance + /// attribute. The same rule applies to `cls.value` in a classmethod. + pub(super) fn implicit_attribute_bindings( + self, + db: &'db dyn Db, + name: &str, + target_method_decorator: MethodDecorator, + ) -> ImplicitAttribute<'db> { + let class_body_scope = self.body_scope(db); + // Collect names in a tracked query so unrelated edits can preserve dependent member + // lookups, and avoid retaining query entries for names that no method can define. + let names = implicit_attribute_names(db, class_body_scope); + let Ok(name_index) = names.binary_search_by(|candidate| candidate.as_str().cmp(name)) + else { + return ImplicitAttribute { + member: Member::unbound(), + augmented_bindings: None, + }; + }; + + Self::implicit_attribute_inner( + db, + ImplicitAttributeName::new( + db, + class_body_scope, + &names[name_index], + target_method_decorator, + ), + ) + } + + #[salsa::tracked( + returns(copy), + cycle_fn=implicit_attribute_cycle_recover, + cycle_initial=|_, id, _| ImplicitAttribute { + member: Member { + inner: Place::bound(Type::divergent(id)).into(), + }, + augmented_bindings: None, + }, + heap_size=ruff_memory_usage::heap_size, + )] + fn implicit_attribute_inner( + db: &'db dyn Db, + attribute: ImplicitAttributeName<'db>, + ) -> ImplicitAttribute<'db> { + Self::implicit_attribute_impl(db, attribute) + } + + fn implicit_attribute_impl( + db: &'db dyn Db, + attribute: ImplicitAttributeName<'db>, + ) -> ImplicitAttribute<'db> { + let class_body_scope = attribute.class_body_scope(db); + let name = attribute.name(db).as_str(); + let target_method_decorator = attribute.target_method_decorator(db); + let program_file = class_body_scope.program_file(db); + let python_file = program_file.python_file(db); + let env = &ProgramEnvironment::from_file(program_file); + + // If we do not see any declarations of an attribute, neither in the class body nor in + // any method, we build a union of the raw types inferred from all bindings of that + // attribute, then apply public-type promotion to the final union. + let mut union_of_inferred_types = UnionBuilder::new(db, env); + let mut qualifiers = TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE; + + let mut is_attribute_bound = false; + let mut augmented_bindings = Vec::new(); + let mut provenance = Provenance::Unknown; + + let module = parsed_module(db, python_file).load(db); + let index = semantic_index(db, program_file); + let class_map = use_def_map(db, class_body_scope); + let class_table = place_table(db, class_body_scope); + let is_valid_scope = |method_scope: &Scope| { + let Some(method_def) = method_scope.node().as_function() else { + return true; + }; + + // Check the decorators directly on the AST node to determine if this method + // is a classmethod or staticmethod. This is more reliable than checking the + // final evaluated type, which may be wrapped by other decorators like @cache. + let function_node = method_def.node(&module); + let definition = index.expect_single_definition(method_def); + + let mut is_classmethod = false; + let mut is_staticmethod = false; + + for decorator in &function_node.decorator_list { + let decorator_ty = + definition_expression_type(db, definition, &decorator.expression); + if let Type::ClassLiteral(class) = decorator_ty { + match class.known(db) { + Some(KnownClass::Classmethod) => is_classmethod = true, + Some(KnownClass::Staticmethod) => is_staticmethod = true, + _ => {} + } + } + } + + // Also check for implicit classmethods/staticmethods based on method name + let method_name = function_node.name.as_str(); + if is_implicit_classmethod(method_name) { + is_classmethod = true; + } + if is_implicit_staticmethod(method_name) { + is_staticmethod = true; + } + + match target_method_decorator { + MethodDecorator::None => !is_classmethod && !is_staticmethod, + MethodDecorator::ClassMethod => is_classmethod, + MethodDecorator::StaticMethod => is_staticmethod, + } + }; + + // First check declarations + for (attribute_declarations, method_scope_id) in + attribute_declarations(db, class_body_scope, name) + { + let method_scope = index.scope(method_scope_id); + if !is_valid_scope(method_scope) { + continue; + } + + for attribute_declaration in attribute_declarations { + let DefinitionState::Defined(declaration) = attribute_declaration.declaration + else { + continue; + }; + + let DefinitionKind::AnnotatedAssignment(assignment) = declaration.kind(db) else { + continue; + }; + + // We found an annotated assignment of one of the following forms (using 'self' in these + // examples, but we support arbitrary names for the first parameters of methods): + // + // self.name: + // self.name: = … + + let Some(annotation) = inferred_declaration(db, declaration).declared() else { + continue; + }; + let annotation = Place::declared(annotation.inner) + .with_definition(declaration) + .with_qualifiers( + annotation.qualifiers | TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE, + ); + + if let Some(all_qualifiers) = annotation.is_bare_final() { + if let Some(value) = assignment.value(&module) { + // If we see an annotated assignment with a bare `Final` as in + // `self.SOME_CONSTANT: Final = 1`, infer the type from the value + // on the right-hand side. + + let inferred_ty = infer_expression_type( + db, + index.expression(value), + TypeContext::default(), + ); + return ImplicitAttribute { + member: Member { + inner: Place::bound(inferred_ty) + .with_definition(declaration) + .with_qualifiers(all_qualifiers), + }, + augmented_bindings: None, + }; + } + + // If there is no right-hand side, just record that we saw a `Final` qualifier + qualifiers |= all_qualifiers; + continue; + } + + return ImplicitAttribute { + member: Member { inner: annotation }, + augmented_bindings: None, + }; + } + } + + for (attribute_assignments, attribute_binding_scope_id) in + attribute_assignments(db, class_body_scope, name) + { + let binding_scope = index.scope(attribute_binding_scope_id); + if !is_valid_scope(binding_scope) { + continue; + } + + let scope_for_reachability_analysis = { + if binding_scope.node().as_function().is_some() { + binding_scope + } else if binding_scope.is_eager() { + let mut eager_scope_parent = binding_scope; + while eager_scope_parent.is_eager() + && let Some(parent) = eager_scope_parent.parent() + { + eager_scope_parent = index.scope(parent); + } + eager_scope_parent + } else { + binding_scope + } + }; + + // The attribute assignment inherits the reachability of the method which contains it + let is_method_reachable = + if let Some(method_def) = scope_for_reachability_analysis.node().as_function() { + let method = index.expect_single_definition(method_def); + let method_place = class_table + .symbol_id(&method_def.node(&module).name) + .unwrap(); + class_map + .reachable_symbol_bindings(method_place) + .find_map(|bind| { + (bind.binding.is_defined_and(|def| def == method)) + .then(|| binding_reachability(db, class_map, &bind)) + }) + .unwrap_or(Truthiness::AlwaysFalse) + } else { + Truthiness::AlwaysFalse + }; + if is_method_reachable.is_always_false() { + continue; + } + + for attribute_assignment in attribute_assignments { + if let DefinitionState::Undefined = attribute_assignment.binding { + continue; + } + + let DefinitionState::Defined(binding) = attribute_assignment.binding else { + continue; + }; + + if matches!(binding.kind(db), DefinitionKind::AugmentedAssignment(_)) { + augmented_bindings.push(binding); + continue; + } + + if !is_method_reachable.is_always_false() { + is_attribute_bound = true; + } + + let inferred_ty = implicit_attribute_binding_type(db, binding); + + if let Some(inferred_ty) = inferred_ty { + provenance = provenance.or(Provenance::SingleDefinition(binding)); + union_of_inferred_types = union_of_inferred_types.add(inferred_ty); + } + } + } + + let member = if is_attribute_bound { + Member { + inner: Place::bound( + union_of_inferred_types + .build() + .promote(db, env) + .promote_singletons(db, env), + ) + .with_provenance(provenance) + .with_qualifiers(qualifiers), + } + } else { + Member::unbound() + }; + + ImplicitAttribute { + member, + augmented_bindings: (!augmented_bindings.is_empty()) + .then(|| AugmentedBindings::new(db, augmented_bindings.into_boxed_slice())), + } + } +} + +/// Attributes assigned by instance methods or classmethods on a single class. +/// +/// Ordinary assignments such as `self.value = 1` or `cls.value = 1` establish an attribute +/// directly. Augmented assignments first require an existing instance or class attribute to supply +/// the value they read. +#[derive(Debug, Clone, Copy, PartialEq, Eq, get_size2::GetSize, salsa::SalsaValue)] +pub(super) struct ImplicitAttribute<'db> { + /// The attribute established by assignments that do not depend on an existing value. + pub(super) member: Member<'db>, + /// Augmented assignments that require an existing instance or class attribute. + pub(super) augmented_bindings: Option>, +} + +/// Augmented assignments deferred until MRO lookup finds the attribute they read. +#[salsa::interned(debug, heap_size=ruff_memory_usage::heap_size)] +pub(super) struct AugmentedBindings<'db> { + #[returns(deref)] + pub(super) definitions: Box<[Definition<'db>]>, +} + +// The Salsa heap is tracked separately. +impl get_size2::GetSize for AugmentedBindings<'_> {} + +#[salsa::interned(debug, heap_size=ruff_memory_usage::heap_size)] +struct ImplicitAttributeName<'db> { + #[returns(copy)] + class_body_scope: ScopeId<'db>, + #[returns(ref)] + name: Name, + #[returns(copy)] + target_method_decorator: MethodDecorator, +} + +// The Salsa heap is tracked separately. +impl get_size2::GetSize for ImplicitAttributeName<'_> {} + +/// Infer the value written by an attribute definition, including unpacked and iteration targets. +fn implicit_attribute_binding_type<'db>( + db: &'db dyn Db, + definition: Definition<'db>, +) -> Option> { + let program_file = definition.program_file(db); + let module = parsed_module(db, program_file.python_file(db)).load(db); + let index = semantic_index(db, program_file); + let env = ProgramEnvironment::from_file(program_file); + + match definition.kind(db) { + DefinitionKind::AnnotatedAssignment(_) => { + // Annotated assignments are handled before inferring ordinary attribute bindings. + None + } + DefinitionKind::Assignment(assignment) => match assignment.unpack() { + Some(unpack) => { + // (..., self.name, ...) = + let unpacked = infer_unpack_types(db, unpack); + Some(unpacked.expression_type(assignment.target(&module))) + } + None => { + // self.name = + Some(infer_expression_type( + db, + index.expression(assignment.value(&module)), + TypeContext::default(), + )) + } + }, + DefinitionKind::For(for_stmt) => match for_stmt.target_kind() { + TargetKind::Sequence(_, unpack) => { + // for ..., self.name, ... in : + let unpacked = infer_unpack_types(db, unpack); + Some(unpacked.expression_type(for_stmt.target(&module))) + } + TargetKind::Single => { + // for self.name in : + let iterable_ty = infer_expression_type( + db, + index.expression(for_stmt.iterable(&module)), + TypeContext::default(), + ); + // TODO: Potential diagnostics resulting from the iterable are not reported. + Some( + iterable_ty + .iterate(db, &env) + .homogeneous_element_type(db, &env), + ) + } + }, + DefinitionKind::WithItem(with_item) => match with_item.target_kind() { + TargetKind::Sequence(_, unpack) => { + // with as ..., self.name, ...: + let unpacked = infer_unpack_types(db, unpack); + Some(unpacked.expression_type(with_item.target(&module))) + } + TargetKind::Single => { + // with as self.name: + let context_ty = infer_expression_type( + db, + index.expression(with_item.context_expr(&module)), + TypeContext::default(), + ); + Some(if with_item.is_async() { + context_ty.aenter(db, &env) + } else { + context_ty.enter(db, &env) + }) + } + }, + DefinitionKind::Comprehension(comprehension) => match comprehension.target_kind() { + TargetKind::Sequence(_, unpack) => { + // [... for ..., self.name, ... in ] + let unpacked = infer_unpack_types(db, unpack); + Some(unpacked.expression_type(comprehension.target(&module))) + } + TargetKind::Single => { + // [... for self.name in ] + let iterable_ty = infer_expression_type( + db, + index.expression(comprehension.iterable(&module)), + TypeContext::default(), + ); + // TODO: Potential diagnostics resulting from the iterable are not reported. + Some( + iterable_ty + .iterate(db, &env) + .homogeneous_element_type(db, &env), + ) + } + }, + // Named expressions cannot target attributes, and other definitions do not write one. + _ => None, + } +} + +#[salsa::tracked(returns(deref), heap_size=ruff_memory_usage::heap_size)] +pub(super) fn implicit_attribute_names<'db>( + db: &'db dyn Db, + class_body_scope: ScopeId<'db>, +) -> Box<[Name]> { + let index = semantic_index(db, class_body_scope.program_file(db)); + let mut names = Vec::new(); + + for function_scope_id in attribute_scopes(db, class_body_scope) { + names.extend( + index + .place_table(function_scope_id) + .members() + .filter_map(|member| member.as_instance_attribute().map(Name::new)), + ); + } + + names.sort_unstable(); + names.dedup(); + names.into_boxed_slice() +} + +fn implicit_attribute_cycle_recover<'db>( + db: &'db dyn Db, + cycle: &salsa::Cycle, + previous: &ImplicitAttribute<'db>, + attribute_member: ImplicitAttribute<'db>, + attribute: ImplicitAttributeName<'db>, +) -> ImplicitAttribute<'db> { + let env = ProgramEnvironment::from_scope(attribute.class_body_scope(db)); + let inner = + attribute_member + .member + .inner + .cycle_normalized(db, &env, previous.member.inner, cycle); + ImplicitAttribute { + member: Member { inner }, + ..attribute_member + } +} diff --git a/crates/ty_python_semantic/src/types/class/static_literal.rs b/crates/ty_python_semantic/src/types/class/static_literal.rs index 691fb6236f..c4c475ab09 100644 --- a/crates/ty_python_semantic/src/types/class/static_literal.rs +++ b/crates/ty_python_semantic/src/types/class/static_literal.rs @@ -11,23 +11,22 @@ use ruff_python_ast::{PythonVersion, name::Name}; use ruff_text_size::{Ranged, TextRange}; use std::cell::RefCell; +use super::implicit_attributes::implicit_attribute_names; use crate::{ Db, FxIndexMap, FxIndexSet, TypeQualifiers, place::{ - DefinedPlace, Definedness, Place, PlaceAndQualifiers, Provenance, PublicTypePolicy, - TypeOrigin, place_from_bindings, place_from_declarations, - }, - reachability::{ - DeclarationsIteratorExtension, ReachabilityConstraintsExtension, binding_reachability, + DefinedPlace, Definedness, Place, PlaceAndQualifiers, PublicTypePolicy, TypeOrigin, + place_from_bindings, place_from_declarations, }, + reachability::{DeclarationsIteratorExtension, ReachabilityConstraintsExtension}, types::{ ApplyTypeMappingVisitor, BoundTypeVarIdentity, BoundTypeVarInstance, CallArguments, CallableType, ClassBase, ClassLiteral, ClassType, DATACLASS_FLAGS, DataclassFlags, DataclassParams, GenericAlias, GenericContext, KnownClass, KnownInstanceType, MaterializationKind, MemberLookupPolicy, MetaclassCandidate, MetaclassTransformInfo, Parameter, Parameters, PropertyInstanceType, Signature, SpecialFormType, StaticMroError, - SubclassOfType, Truthiness, Type, TypeContext, TypeMapping, TypeVarVariance, - TypedDictModule, UnionBuilder, UnionType, + SubclassOfType, Type, TypeContext, TypeMapping, TypeVarVariance, TypedDictModule, + UnionBuilder, UnionType, bound_super::BoundSuperType, call::{CallError, CallErrorKind}, callable::{CallableFunctionProvenance, CallableTypeKind}, @@ -43,13 +42,9 @@ use crate::{ definition_expression_type, determine_upper_bound, diagnostic::INVALID_DATACLASS_OVERRIDE, enums::{enum_metadata, is_enum_class_by_inheritance, try_unwrap_nonmember_value}, - function::{ - DataclassTransformerParams, KnownFunction, is_implicit_classmethod, - is_implicit_staticmethod, - }, + function::{DataclassTransformerParams, KnownFunction}, generics::Specialization, - infer::infer_unpack_types, - infer_expression_type, inferred_declaration, + inferred_declaration, known_instance::DeprecatedInstance, member::{Member, class_member}, mro::{Mro, MroIterator}, @@ -60,12 +55,11 @@ use crate::{ visitor::{TypeCollector, TypeVisitor, walk_type_with_recursion_guard}, }, }; -use crate::{attribute_assignments, attribute_declarations}; use ty_python_core::{ ProgramFile, attribute_scopes, - definition::{Definition, DefinitionKind, DefinitionState, TargetKind}, + definition::{Definition, DefinitionKind, DefinitionState}, place_table, - scope::{Scope, ScopeId}, + scope::ScopeId, semantic_index, symbol::Symbol, use_def_map, @@ -1530,7 +1524,7 @@ impl<'db> StaticClassLiteral<'db> { return Member::definitely_declared(synthesized_member); } // The symbol was not found in the class scope. It might still be implicitly defined in `@classmethod`s. - return Self::implicit_attribute(db, body_scope, name, MethodDecorator::ClassMethod); + return self.implicit_attribute(db, name, MethodDecorator::ClassMethod); } // For dataclass-like classes, `KW_ONLY` sentinel fields are not real @@ -2847,414 +2841,6 @@ impl<'db> StaticClassLiteral<'db> { } } - /// Tries to find declarations/bindings of an attribute named `name` that are only - /// "implicitly" defined (`self.x = …`, `cls.x = …`) in a method of the class that - /// corresponds to `class_body_scope`. The `target_method_decorator` parameter is - /// used to skip methods that do not have the expected decorator. - fn implicit_attribute( - db: &'db dyn Db, - class_body_scope: ScopeId<'db>, - name: &str, - target_method_decorator: MethodDecorator, - ) -> Member<'db> { - Self::implicit_attribute_bindings(db, class_body_scope, name, target_method_decorator) - .member - } - - /// Separate assignments that establish an attribute from assignments that must first read it. - /// - /// ```python - /// class Counter: - /// def increment(self): - /// self.value += 1 - /// ``` - /// - /// Here, `value` remains undefined until MRO lookup finds an independent class or instance - /// attribute. The same rule applies to `cls.value` in a classmethod. - pub(super) fn implicit_attribute_bindings( - db: &'db dyn Db, - class_body_scope: ScopeId<'db>, - name: &str, - target_method_decorator: MethodDecorator, - ) -> ImplicitAttribute<'db> { - // Collect names in a tracked query so unrelated edits can preserve dependent member - // lookups, and avoid retaining query entries for names that no method can define. - let names = implicit_attribute_names(db, class_body_scope); - let Ok(name_index) = names.binary_search_by(|candidate| candidate.as_str().cmp(name)) - else { - return ImplicitAttribute { - member: Member::unbound(), - augmented_bindings: None, - }; - }; - - Self::implicit_attribute_inner( - db, - ImplicitAttributeName::new( - db, - class_body_scope, - &names[name_index], - target_method_decorator, - ), - ) - } - - #[salsa::tracked( - returns(copy), - cycle_fn=implicit_attribute_cycle_recover, - cycle_initial=|_, id, _| ImplicitAttribute { - member: Member { - inner: Place::bound(Type::divergent(id)).into(), - }, - augmented_bindings: None, - }, - heap_size=ruff_memory_usage::heap_size, - )] - fn implicit_attribute_inner( - db: &'db dyn Db, - attribute: ImplicitAttributeName<'db>, - ) -> ImplicitAttribute<'db> { - let class_body_scope = attribute.class_body_scope(db); - let name = attribute.name(db).as_str(); - let target_method_decorator = attribute.target_method_decorator(db); - let program_file = class_body_scope.program_file(db); - let python_file = program_file.python_file(db); - let env = &ProgramEnvironment::from_file(program_file); - - // If we do not see any declarations of an attribute, neither in the class body nor in - // any method, we build a union of the raw types inferred from all bindings of that - // attribute, then apply public-type promotion to the final union. - let mut union_of_inferred_types = UnionBuilder::new(db, env); - let mut qualifiers = TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE; - - let mut is_attribute_bound = false; - let mut augmented_bindings = Vec::new(); - let mut provenance = Provenance::Unknown; - - let module = parsed_module(db, python_file).load(db); - let index = semantic_index(db, program_file); - let class_map = use_def_map(db, class_body_scope); - let class_table = place_table(db, class_body_scope); - let is_valid_scope = |method_scope: &Scope| { - let Some(method_def) = method_scope.node().as_function() else { - return true; - }; - - // Check the decorators directly on the AST node to determine if this method - // is a classmethod or staticmethod. This is more reliable than checking the - // final evaluated type, which may be wrapped by other decorators like @cache. - let function_node = method_def.node(&module); - let definition = index.expect_single_definition(method_def); - - let mut is_classmethod = false; - let mut is_staticmethod = false; - - for decorator in &function_node.decorator_list { - let decorator_ty = - definition_expression_type(db, definition, &decorator.expression); - if let Type::ClassLiteral(class) = decorator_ty { - match class.known(db) { - Some(KnownClass::Classmethod) => is_classmethod = true, - Some(KnownClass::Staticmethod) => is_staticmethod = true, - _ => {} - } - } - } - - // Also check for implicit classmethods/staticmethods based on method name - let method_name = function_node.name.as_str(); - if is_implicit_classmethod(method_name) { - is_classmethod = true; - } - if is_implicit_staticmethod(method_name) { - is_staticmethod = true; - } - - match target_method_decorator { - MethodDecorator::None => !is_classmethod && !is_staticmethod, - MethodDecorator::ClassMethod => is_classmethod, - MethodDecorator::StaticMethod => is_staticmethod, - } - }; - - // First check declarations - for (attribute_declarations, method_scope_id) in - attribute_declarations(db, class_body_scope, name) - { - let method_scope = index.scope(method_scope_id); - if !is_valid_scope(method_scope) { - continue; - } - - for attribute_declaration in attribute_declarations { - let DefinitionState::Defined(declaration) = attribute_declaration.declaration - else { - continue; - }; - - let DefinitionKind::AnnotatedAssignment(assignment) = declaration.kind(db) else { - continue; - }; - - // We found an annotated assignment of one of the following forms (using 'self' in these - // examples, but we support arbitrary names for the first parameters of methods): - // - // self.name: - // self.name: = … - - let Some(annotation) = inferred_declaration(db, declaration).declared() else { - continue; - }; - let annotation = Place::declared(annotation.inner) - .with_definition(declaration) - .with_qualifiers( - annotation.qualifiers | TypeQualifiers::IMPLICIT_INSTANCE_ATTRIBUTE, - ); - - if let Some(all_qualifiers) = annotation.is_bare_final() { - if let Some(value) = assignment.value(&module) { - // If we see an annotated assignment with a bare `Final` as in - // `self.SOME_CONSTANT: Final = 1`, infer the type from the value - // on the right-hand side. - - let inferred_ty = infer_expression_type( - db, - index.expression(value), - TypeContext::default(), - ); - return ImplicitAttribute { - member: Member { - inner: Place::bound(inferred_ty) - .with_definition(declaration) - .with_qualifiers(all_qualifiers), - }, - augmented_bindings: None, - }; - } - - // If there is no right-hand side, just record that we saw a `Final` qualifier - qualifiers |= all_qualifiers; - continue; - } - - return ImplicitAttribute { - member: Member { inner: annotation }, - augmented_bindings: None, - }; - } - } - - for (attribute_assignments, attribute_binding_scope_id) in - attribute_assignments(db, class_body_scope, name) - { - let binding_scope = index.scope(attribute_binding_scope_id); - if !is_valid_scope(binding_scope) { - continue; - } - - let scope_for_reachability_analysis = { - if binding_scope.node().as_function().is_some() { - binding_scope - } else if binding_scope.is_eager() { - let mut eager_scope_parent = binding_scope; - while eager_scope_parent.is_eager() - && let Some(parent) = eager_scope_parent.parent() - { - eager_scope_parent = index.scope(parent); - } - eager_scope_parent - } else { - binding_scope - } - }; - - // The attribute assignment inherits the reachability of the method which contains it - let is_method_reachable = - if let Some(method_def) = scope_for_reachability_analysis.node().as_function() { - let method = index.expect_single_definition(method_def); - let method_place = class_table - .symbol_id(&method_def.node(&module).name) - .unwrap(); - class_map - .reachable_symbol_bindings(method_place) - .find_map(|bind| { - (bind.binding.is_defined_and(|def| def == method)) - .then(|| binding_reachability(db, class_map, &bind)) - }) - .unwrap_or(Truthiness::AlwaysFalse) - } else { - Truthiness::AlwaysFalse - }; - if is_method_reachable.is_always_false() { - continue; - } - - for attribute_assignment in attribute_assignments { - if let DefinitionState::Undefined = attribute_assignment.binding { - continue; - } - - let DefinitionState::Defined(binding) = attribute_assignment.binding else { - continue; - }; - - if matches!(binding.kind(db), DefinitionKind::AugmentedAssignment(_)) { - augmented_bindings.push(binding); - continue; - } - - if !is_method_reachable.is_always_false() { - is_attribute_bound = true; - } - - let inferred_ty = match binding.kind(db) { - DefinitionKind::AnnotatedAssignment(_) => { - // Annotated assignments were handled above. This branch is not - // unreachable (because of the `continue` above), but there is - // nothing to do here. - None - } - DefinitionKind::Assignment(assign) => match assign.unpack() { - Some(unpack) => { - // We found an unpacking assignment like: - // - // .., self.name, .. = - // (.., self.name, ..) = - // [.., self.name, ..] = - - let unpacked = infer_unpack_types(db, unpack); - Some(unpacked.expression_type(assign.target(&module))) - } - None => { - // We found an un-annotated attribute assignment of the form: - // - // self.name = - - Some(infer_expression_type( - db, - index.expression(assign.value(&module)), - TypeContext::default(), - )) - } - }, - DefinitionKind::For(for_stmt) => match for_stmt.target_kind() { - TargetKind::Sequence(_, unpack) => { - // We found an unpacking assignment like: - // - // for .., self.name, .. in : - - let unpacked = infer_unpack_types(db, unpack); - Some(unpacked.expression_type(for_stmt.target(&module))) - } - TargetKind::Single => { - // We found an attribute assignment like: - // - // for self.name in : - - let iterable_ty = infer_expression_type( - db, - index.expression(for_stmt.iterable(&module)), - TypeContext::default(), - ); - // TODO: Potential diagnostics resulting from the iterable are currently not reported. - Some( - iterable_ty - .iterate(db, env) - .homogeneous_element_type(db, env), - ) - } - }, - DefinitionKind::WithItem(with_item) => match with_item.target_kind() { - TargetKind::Sequence(_, unpack) => { - // We found an unpacking assignment like: - // - // with as .., self.name, ..: - - let unpacked = infer_unpack_types(db, unpack); - Some(unpacked.expression_type(with_item.target(&module))) - } - TargetKind::Single => { - // We found an attribute assignment like: - // - // with as self.name: - - let context_ty = infer_expression_type( - db, - index.expression(with_item.context_expr(&module)), - TypeContext::default(), - ); - Some(if with_item.is_async() { - context_ty.aenter(db, env) - } else { - context_ty.enter(db, env) - }) - } - }, - DefinitionKind::Comprehension(comprehension) => { - match comprehension.target_kind() { - TargetKind::Sequence(_, unpack) => { - // We found an unpacking assignment like: - // - // [... for .., self.name, .. in ] - - let unpacked = infer_unpack_types(db, unpack); - Some(unpacked.expression_type(comprehension.target(&module))) - } - TargetKind::Single => { - // We found an attribute assignment like: - // - // [... for self.name in ] - - let iterable_ty = infer_expression_type( - db, - index.expression(comprehension.iterable(&module)), - TypeContext::default(), - ); - // TODO: Potential diagnostics resulting from the iterable are currently not reported. - Some( - iterable_ty - .iterate(db, env) - .homogeneous_element_type(db, env), - ) - } - } - } - DefinitionKind::NamedExpression(_) => { - // A named expression whose target is an attribute is syntactically prohibited - None - } - _ => None, - }; - - if let Some(inferred_ty) = inferred_ty { - provenance = provenance.or(Provenance::SingleDefinition(binding)); - union_of_inferred_types = union_of_inferred_types.add(inferred_ty); - } - } - } - - let member = if is_attribute_bound { - Member { - inner: Place::bound( - union_of_inferred_types - .build() - .promote(db, env) - .promote_singletons(db, env), - ) - .with_provenance(provenance) - .with_qualifiers(qualifiers), - } - } else { - Member::unbound() - }; - - ImplicitAttribute { - member, - augmented_bindings: (!augmented_bindings.is_empty()) - .then(|| AugmentedBindings::new(db, augmented_bindings.into_boxed_slice())), - } - } - /// A helper function for `instance_member` that looks up the `name` attribute only on /// this class, not on its superclasses. pub(super) fn own_instance_member( @@ -3308,7 +2894,8 @@ impl<'db> StaticClassLiteral<'db> { if qualifiers.contains(TypeQualifiers::INIT_VAR) { // We ignore `InitVar` declarations on the class body, unless that attribute is overwritten // by an implicit assignment in a method - if Self::implicit_attribute(db, body_scope, name, MethodDecorator::None) + if self + .implicit_attribute(db, name, MethodDecorator::None) .is_undefined() { return Member::unbound(); @@ -3332,8 +2919,7 @@ impl<'db> StaticClassLiteral<'db> { if has_binding { // The attribute is declared and bound in the class body. - let implicit = - Self::implicit_attribute(db, body_scope, name, MethodDecorator::None); + let implicit = self.implicit_attribute(db, name, MethodDecorator::None); if let Place::Defined(DefinedPlace { ty: implicit_ty, provenance: implicit_provenance, @@ -3407,14 +2993,10 @@ impl<'db> StaticClassLiteral<'db> { ty: implicit_ty, provenance: implicit_provenance, .. - }) = Self::implicit_attribute( - db, - body_scope, - name, - MethodDecorator::None, - ) - .inner - .place + }) = self + .implicit_attribute(db, name, MethodDecorator::None) + .inner + .place { Member { inner: Place::Defined(DefinedPlace { @@ -3447,14 +3029,14 @@ impl<'db> StaticClassLiteral<'db> { // The attribute is not *declared* in the class body. It could still be declared/bound // in a method. - Self::implicit_attribute(db, body_scope, name, MethodDecorator::None) + self.implicit_attribute(db, name, MethodDecorator::None) } } } else { // This attribute is neither declared nor bound in the class body. // It could still be implicitly defined in a method. - Self::implicit_attribute(db, body_scope, name, MethodDecorator::None) + self.implicit_attribute(db, name, MethodDecorator::None) } } @@ -3934,77 +3516,3 @@ fn explicit_bases_cycle_fn<'db>( current } } - -/// Attributes assigned by instance methods or classmethods on a single class. -/// -/// Ordinary assignments such as `self.value = 1` or `cls.value = 1` establish an attribute -/// directly. Augmented assignments first require an existing instance or class attribute to supply -/// the value they read. -#[derive(Debug, Clone, Copy, PartialEq, Eq, get_size2::GetSize, salsa::SalsaValue)] -pub(super) struct ImplicitAttribute<'db> { - /// The attribute established by assignments that do not depend on an existing value. - pub(super) member: Member<'db>, - /// Augmented assignments that require an existing instance or class attribute. - pub(super) augmented_bindings: Option>, -} - -/// Augmented assignments deferred until MRO lookup finds the attribute they read. -#[salsa::interned(debug, heap_size=ruff_memory_usage::heap_size)] -pub(super) struct AugmentedBindings<'db> { - #[returns(deref)] - pub(super) definitions: Box<[Definition<'db>]>, -} - -// The Salsa heap is tracked separately. -impl get_size2::GetSize for AugmentedBindings<'_> {} - -#[salsa::interned(debug, heap_size=ruff_memory_usage::heap_size)] -struct ImplicitAttributeName<'db> { - #[returns(copy)] - class_body_scope: ScopeId<'db>, - #[returns(ref)] - name: Name, - #[returns(copy)] - target_method_decorator: MethodDecorator, -} - -// The Salsa heap is tracked separately. -impl get_size2::GetSize for ImplicitAttributeName<'_> {} - -#[salsa::tracked(returns(deref), heap_size=ruff_memory_usage::heap_size)] -fn implicit_attribute_names<'db>(db: &'db dyn Db, class_body_scope: ScopeId<'db>) -> Box<[Name]> { - let index = semantic_index(db, class_body_scope.program_file(db)); - let mut names = Vec::new(); - - for function_scope_id in attribute_scopes(db, class_body_scope) { - names.extend( - index - .place_table(function_scope_id) - .members() - .filter_map(|member| member.as_instance_attribute().map(Name::new)), - ); - } - - names.sort_unstable(); - names.dedup(); - names.into_boxed_slice() -} - -fn implicit_attribute_cycle_recover<'db>( - db: &'db dyn Db, - cycle: &salsa::Cycle, - previous: &ImplicitAttribute<'db>, - attribute_member: ImplicitAttribute<'db>, - attribute: ImplicitAttributeName<'db>, -) -> ImplicitAttribute<'db> { - let env = ProgramEnvironment::from_scope(attribute.class_body_scope(db)); - let inner = - attribute_member - .member - .inner - .cycle_normalized(db, &env, previous.member.inner, cycle); - ImplicitAttribute { - member: Member { inner }, - ..attribute_member - } -} From 672bb4edf04c84f8b0753346359daee4057158f2 Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Sun, 16 Aug 2026 08:29:02 +0200 Subject: [PATCH 062/371] [ty] Show error spans for errors originating in script metadata (#26693) ## Summary Preserve source ranges for PEP 723 script metadata by building a compact source map during extraction and applying it when deserializing ranged values. Closes https://github.com/astral-sh/ty/issues/4180 --- Cargo.lock | 1 + crates/ruff_python_ast/src/script.rs | 261 ++++++++++++++---- crates/ruff_ranged_value/Cargo.toml | 1 + crates/ruff_ranged_value/src/lib.rs | 45 ++- crates/ty/tests/cli/scripts.rs | 38 ++- crates/ty_project/src/script.rs | 6 +- ..._did_change_script_python_requirement.snap | 2 +- ...orts_inline_configuration_diagnostics.snap | 8 +- 8 files changed, 285 insertions(+), 77 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index 65d26aed2e..3f710cce15 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -3696,6 +3696,7 @@ version = "0.0.9" dependencies = [ "get-size2", "ruff_db", + "ruff_python_ast", "ruff_text_size", "schemars", "serde", diff --git a/crates/ruff_python_ast/src/script.rs b/crates/ruff_python_ast/src/script.rs index 00c0fc7c30..b9071ea7bc 100644 --- a/crates/ruff_python_ast/src/script.rs +++ b/crates/ruff_python_ast/src/script.rs @@ -1,6 +1,8 @@ use std::sync::LazyLock; use memchr::memmem::Finder; +use ruff_source_file::UniversalNewlineIterator; +use ruff_text_size::{Ranged, TextLen, TextRange, TextSize}; static FINDER: LazyLock = LazyLock::new(|| Finder::new(b"# /// script")); @@ -11,12 +13,12 @@ static FINDER: LazyLock = LazyLock::new(|| Finder::new(b"# /// script")) /// Vendored from: #[derive(Debug, Clone, Eq, PartialEq)] pub struct ScriptTag { - /// The content of the script before the metadata block. - prelude: String, /// The metadata block. metadata: String, - /// The content of the script after the metadata block. - postlude: String, + /// The source range of the metadata block, including its opening and closing delimiters. + range: TextRange, + /// Maps offsets in the extracted metadata to offsets in the original Python script. + source_map: ScriptSourceMap, } impl ScriptTag { @@ -25,9 +27,18 @@ impl ScriptTag { &self.metadata } + /// Returns the map from extracted TOML offsets to their original script offsets. + pub fn source_map(&self) -> &ScriptSourceMap { + &self.source_map + } + + /// Consumes this tag and returns its TOML together with its original-source mapping. + pub fn into_metadata_and_source_map(self) -> (String, ScriptSourceMap) { + (self.metadata, self.source_map) + } + /// Given the contents of a Python file, extract the `script` metadata block with leading - /// comment hashes removed, any preceding shebang or content (prelude), and the remaining Python - /// script. + /// comment hashes removed and map its offsets to the original Python script. /// /// Given the following input string representing the contents of a Python script: /// @@ -46,11 +57,14 @@ impl ScriptTag { /// print("Hello, World!") /// ``` /// - /// This function would return: - /// - /// - Preamble: `#!/usr/bin/env python3\n` - /// - Metadata: `requires-python = '>=3.11'\ndependencies = [\n 'requests<3',\n 'rich',\n]` - /// - Postlude: `import requests\n\nprint("Hello, World!")\n` + /// This function extracts the metadata: + /// ```toml + /// requires-python = '>=3.11' + /// dependencies = [ + /// 'requests<3', + /// 'rich', + /// ] + /// ``` /// /// See: pub fn parse(contents: &[u8]) -> Option { @@ -65,14 +79,11 @@ impl ScriptTag { return None; } - // Extract the preceding content. - let prelude = std::str::from_utf8(&contents[..index]).ok()?; - - // Decode as UTF-8. - let contents = &contents[index..]; let contents = std::str::from_utf8(contents).ok()?; + let contents = &contents[index..]; - let mut lines = contents.lines(); + let start = TextSize::try_from(index).ok()?; + let mut lines = UniversalNewlineIterator::with_offset(contents, start); // Ensure that the first line is exactly `# /// script`. if lines.next().is_none_or(|line| line != "# /// script") { @@ -84,37 +95,36 @@ impl ScriptTag { // > embedded content is formed by taking away the first two characters of each line if the // > second character is a space, otherwise just the first character (which means the line // > consists of only a single #). - let mut toml = vec![]; + let mut metadata = String::new(); + let mut source_map = ScriptSourceMap::default(); + let mut closing = None; - // Extract the content that follows the metadata block. - let mut python_script = vec![]; - - while let Some(line) = lines.next() { + for line in lines { // Remove the leading `#`. - let Some(line) = line.strip_prefix('#') else { - python_script.push(line); - python_script.extend(lines); + let Some(comment) = line.strip_prefix('#') else { break; }; - // If the line is empty, continue. - if line.is_empty() { - toml.push(""); - continue; - } - - // Otherwise, the line _must_ start with ` `. - let Some(line) = line.strip_prefix(' ') else { - python_script.push(line); - python_script.extend(lines); + let (content, indent_len) = if comment.is_empty() { + ("", TextSize::ZERO) + } else if let Some(content) = comment.strip_prefix(' ') { + (content, ' '.text_len()) + } else { break; }; - toml.push(line); + if content == "///" { + closing = Some((metadata.len(), source_map.markers.len(), line.range())); + } + + let prefix_length = '#'.text_len() + indent_len; + + source_map.push_marker(metadata.text_len(), line.start() + prefix_length); + metadata.push_str(content); + metadata.push('\n'); } - // Find the closing `# ///`. The precedence is such that we need to identify the _last_ such - // line. + // The last closing `# ///` wins, so discard that delimiter and everything after it. // // For example, given: // ```python @@ -126,32 +136,163 @@ impl ScriptTag { // ``` // // The latter `///` is the closing pragma - let index = toml.iter().rev().position(|line| *line == "///")?; - let index = toml.len() - index; - - // Discard any lines after the closing `# ///`. - // - // For example, given: - // ```python - // # /// script - // # - // # /// - // # - // # - // ``` - // - // We need to discard the last two lines. - toml.truncate(index - 1); + let (metadata_end, marker_count, closing_range) = closing?; + metadata.truncate(metadata_end); + source_map.truncate(marker_count); - // Join the lines into a single string. - let prelude = prelude.to_string(); - let metadata = toml.join("\n") + "\n"; - let postlude = python_script.join("\n") + "\n"; + if metadata.is_empty() { + metadata.push('\n'); + } else { + source_map.push_marker(metadata.text_len(), closing_range.start()); + } Some(Self { - prelude, metadata, - postlude, + range: TextRange::new(start, closing_range.end()), + source_map, }) } } + +impl Ranged for ScriptTag { + fn range(&self) -> TextRange { + self.range + } +} + +/// Maps offsets in extracted script metadata to offsets in the original Python source. +#[derive(Clone, Debug, Default, Eq, PartialEq)] +pub struct ScriptSourceMap { + markers: Vec, +} + +impl ScriptSourceMap { + /// Maps a metadata offset to the corresponding offset in the Python source. + pub fn map_offset(&self, offset: TextSize) -> TextSize { + let Some(index) = self + .markers + .partition_point(|marker| marker.metadata_offset <= offset) + .checked_sub(1) + else { + return offset; + }; + let marker = &self.markers[index]; + + marker.source_offset + (offset - marker.metadata_offset) + } + + /// Maps a metadata range to its corresponding range in the Python source. + pub fn map_range(&self, range: TextRange) -> TextRange { + TextRange::new(self.map_offset(range.start()), self.map_offset(range.end())) + } + + fn push_marker(&mut self, metadata_offset: TextSize, source_offset: TextSize) { + self.markers.push(ScriptSourceMarker { + metadata_offset, + source_offset, + }); + } + + fn truncate(&mut self, len: usize) { + self.markers.truncate(len); + } +} + +#[derive(Clone, Debug, Eq, PartialEq)] +struct ScriptSourceMarker { + metadata_offset: TextSize, + source_offset: TextSize, +} + +#[cfg(test)] +mod tests { + use ruff_text_size::{Ranged, TextLen, TextRange}; + + use super::ScriptTag; + + #[test] + fn carriage_return_line_endings() -> Result<(), &'static str> { + let tag = ScriptTag::parse(b"# /// script\r# value = true\r# ///\r") + .ok_or("Expected script metadata with carriage-return line endings")?; + + assert_eq!(tag.metadata(), "value = true\n"); + + Ok(()) + } + + #[test] + fn metadata_block_range_includes_both_delimiters() -> Result<(), &'static str> { + let prefix = "#!/usr/bin/env python3\n\n"; + let metadata = "# /// script\n# dependencies = []\n# ///"; + let source = format!("{prefix}{metadata}\n\nprint('hello')\n"); + let tag = ScriptTag::parse(source.as_bytes()).ok_or("Expected valid script metadata")?; + + assert_eq!( + tag.range(), + TextRange::at(prefix.text_len(), metadata.text_len()) + ); + + Ok(()) + } + + #[test] + fn metadata_range_accounts_for_unicode_crlf_and_multiline_values() -> Result<(), &'static str> { + let metadata_value = r#"""" +first + +last +""""#; + let source_value = r#"""" +# first +# +# last +# """"# + .replace('\n', "\r\n"); + let source = format!("π\r\n# /// script\r\n# value = {source_value}\r\n# ///\r\n"); + let tag = ScriptTag::parse(source.as_bytes()).ok_or("Expected valid script metadata")?; + + assert_eq!(tag.metadata(), format!("value = {metadata_value}\n")); + + let metadata_range = TextRange::at("value = ".text_len(), metadata_value.text_len()); + let source_range = TextRange::at( + "π\r\n# /// script\r\n# value = ".text_len(), + source_value.text_len(), + ); + + assert_eq!(tag.source_map().map_range(metadata_range), source_range); + + Ok(()) + } + + #[test] + fn last_closing_delimiter_discards_following_comments() -> Result<(), &'static str> { + let source = r"# /// script +# first = true +# /// +# last = true +# /// +# ignored = true +"; + let tag = ScriptTag::parse(source.as_bytes()).ok_or("Expected valid script metadata")?; + + assert_eq!( + tag.metadata(), + r"first = true +/// +last = true +" + ); + + let closing_start = source + .rfind("# ///") + .map(|offset| source[..offset].text_len()) + .ok_or("Expected the final closing delimiter")?; + assert_eq!( + tag.source_map().map_offset(tag.metadata().text_len()), + closing_start, + ); + assert_eq!(tag.end(), closing_start + "# ///".text_len()); + + Ok(()) + } +} diff --git a/crates/ruff_ranged_value/Cargo.toml b/crates/ruff_ranged_value/Cargo.toml index 2c628f2c86..c34f06c2ba 100644 --- a/crates/ruff_ranged_value/Cargo.toml +++ b/crates/ruff_ranged_value/Cargo.toml @@ -16,6 +16,7 @@ doctest = false [dependencies] ruff_db = { workspace = true } +ruff_python_ast = { workspace = true } ruff_text_size = { workspace = true } get-size2 = { workspace = true, optional = true } diff --git a/crates/ruff_ranged_value/src/lib.rs b/crates/ruff_ranged_value/src/lib.rs index e8a4a2ccda..dc5443ab6f 100644 --- a/crates/ruff_ranged_value/src/lib.rs +++ b/crates/ruff_ranged_value/src/lib.rs @@ -11,6 +11,7 @@ use toml::Spanned; use ruff_db::Db; use ruff_db::files::{File, system_path_to_file}; use ruff_db::system::SystemPathBuf; +use ruff_python_ast::script::ScriptSourceMap; use ruff_text_size::{TextRange, TextSize}; #[derive(Clone, Debug, PartialEq)] @@ -63,19 +64,31 @@ thread_local! { /// Use the [`ValueSourceGuard`] to initialize the thread local before calling into any /// deserialization code. It ensures that the thread local variable gets cleaned up /// once deserialization is done (once the guard gets dropped). - static VALUE_SOURCE: RefCell> = const { RefCell::new(None) }; + static VALUE_SOURCE: RefCell> = const { RefCell::new(None) }; } /// Guard to safely change the [`ValueSource`] for the current thread. #[must_use] pub struct ValueSourceGuard { - prev_value: Option<(ValueSource, bool)>, + prev_value: Option, } impl ValueSourceGuard { pub fn new(source: ValueSource, is_toml: bool) -> Self { - let prev = VALUE_SOURCE.replace(Some((source, is_toml))); - Self { prev_value: prev } + Self::replace(ValueSourceContext { + source, + has_span: is_toml, + source_map: None, + }) + } + + /// Sets the source and maps deserialized TOML ranges into that source. + pub fn with_source_map(source: ValueSource, source_map: ScriptSourceMap) -> Self { + Self::replace(ValueSourceContext { + source, + has_span: true, + source_map: Some(source_map), + }) } pub fn without_spans() -> Self { @@ -83,11 +96,16 @@ impl ValueSourceGuard { current .as_ref() .expect("value source to be set before disabling spans") - .0 + .source .clone() }); Self::new(source, false) } + + fn replace(context: ValueSourceContext) -> Self { + let prev = VALUE_SOURCE.replace(Some(context)); + Self { prev_value: prev } + } } impl Drop for ValueSourceGuard { @@ -96,6 +114,12 @@ impl Drop for ValueSourceGuard { } } +struct ValueSourceContext { + source: ValueSource, + has_span: bool, + source_map: Option, +} + /// A value that "remembers" where it comes from (source) and its range in source. /// /// ## Equality, Hash, and Ordering @@ -319,9 +343,12 @@ where D: Deserializer<'de>, { VALUE_SOURCE.with_borrow(|source| { - let (source, has_span) = source.clone().unwrap(); + let context = source + .as_ref() + .expect("value source to be set before deserializing a ranged value"); + let source = context.source.clone(); - if has_span { + if context.has_span { let spanned: Spanned = Spanned::deserialize(deserializer)?; let span = spanned.span(); let range = TextRange::new( @@ -330,6 +357,10 @@ where TextSize::try_from(span.end) .expect("Configuration file to be smaller than 4GB"), ); + let range = context + .source_map + .as_ref() + .map_or(range, |source_map| source_map.map_range(range)); Ok(Self::with_range(spanned.into_inner(), source, range)) } else { diff --git a/crates/ty/tests/cli/scripts.rs b/crates/ty/tests/cli/scripts.rs index 725c9e839e..983fe5ca49 100644 --- a/crates/ty/tests/cli/scripts.rs +++ b/crates/ty/tests/cli/scripts.rs @@ -83,6 +83,36 @@ fn verbose_rule_diagnostics_identify_script_metadata() -> anyhow::Result<()> { Ok(()) } +#[test] +fn unknown_rule_diagnostics_point_to_script_metadata() -> anyhow::Result<()> { + let case = CliTest::with_file( + "script.py", + r#" + # /// script + # [tool.ty.rules] + # unknown-script-rule = "warn" + # /// + "#, + )?; + + assert_cmd_snapshot!(case.command(), @r#" + success: false + exit_code: 1 + ----- stdout ----- + warning[unknown-rule]: Unknown rule `unknown-script-rule` + --> script.py:4:3 + | + 4 | # unknown-script-rule = "warn" + | ^^^^^^^^^^^^^^^^^^^ + + Found 1 diagnostic + + ----- stderr ----- + "#); + + Ok(()) +} + #[test] fn python_version_diagnostics_identify_script_metadata() -> anyhow::Result<()> { let case = CliTest::with_file( @@ -96,7 +126,7 @@ fn python_version_diagnostics_identify_script_metadata() -> anyhow::Result<()> { "#, )?; - assert_cmd_snapshot!(case.command(), @" + assert_cmd_snapshot!(case.command(), @r#" success: false exit_code: 1 ----- stdout ----- @@ -107,11 +137,15 @@ fn python_version_diagnostics_identify_script_metadata() -> anyhow::Result<()> { | ^^^^^^^^^^^^^^^^^^^^^^^ info: `PythonFinalizationError` was added as a builtin in Python 3.13 info: Python 3.12 was assumed when resolving types because it was specified in script metadata + --> script.py:3:21 + | + 3 | # requires-python = ">=3.12" + | ^^^^^^^^ Python version configured here Found 1 diagnostic ----- stderr ----- - "); + "#); assert_cmd_snapshot!(case.command().arg("--output-format").arg("concise"), @" success: false exit_code: 1 diff --git a/crates/ty_project/src/script.rs b/crates/ty_project/src/script.rs index 19822dac86..aa76f7b641 100644 --- a/crates/ty_project/src/script.rs +++ b/crates/ty_project/src/script.rs @@ -180,10 +180,10 @@ pub(crate) fn script_metadata(db: &dyn SourceDb, file: File) -> Option Date: Sun, 16 Aug 2026 13:19:40 -0400 Subject: [PATCH 063/371] Add explicit dependency cooldowns (#27796) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Adds explicit 7d cooldowns to all of our PEP 723 scripts, as well as similar cooldowns via `.npmrc`. ## Test Plan NFC. --- Stack created with GitHub Stacks CLI • Give Feedback 💬 --- crates/ruff_python_ast/generate.py | 3 + crates/ruff_python_ast/generate.py.lock | 7 ++ crates/ty_python_semantic/mdtest.py | 2 +- playground/.npmrc | 1 + playground/api/.npmrc | 1 + pyproject.toml | 3 + python/py-fuzzer/pyproject.toml | 3 + python/py-fuzzer/uv.lock | 4 + scripts/benchmarks/pyproject.toml | 3 + scripts/benchmarks/uv.lock | 91 ++++++++++--------- scripts/build_ruff_pgo.py | 3 + scripts/build_ruff_pgo.py.lock | 7 ++ scripts/bump-workspace-crate-versions.py | 3 + scripts/bump-workspace-crate-versions.py.lock | 7 ++ scripts/collect_ty_ecosystem_run_metadata.py | 3 + .../collect_ty_ecosystem_run_metadata.py.lock | 7 ++ scripts/generate-crate-readmes.py | 3 + scripts/generate-crate-readmes.py.lock | 7 ++ scripts/pyproject.toml | 3 + scripts/setup-crates-io-publish.py | 3 + scripts/setup-crates-io-publish.py.lock | 4 + scripts/setup_primer_project.py | 2 +- scripts/ty_benchmark/.npmrc | 1 + scripts/ty_benchmark/pyproject.toml | 3 + scripts/ty_benchmark/uv.lock | 4 + scripts/uv.lock | 4 + uv.lock | 4 + 27 files changed, 139 insertions(+), 47 deletions(-) create mode 100644 crates/ruff_python_ast/generate.py.lock create mode 100644 playground/.npmrc create mode 100644 playground/api/.npmrc create mode 100644 scripts/build_ruff_pgo.py.lock create mode 100644 scripts/bump-workspace-crate-versions.py.lock create mode 100644 scripts/collect_ty_ecosystem_run_metadata.py.lock create mode 100644 scripts/generate-crate-readmes.py.lock create mode 100644 scripts/ty_benchmark/.npmrc diff --git a/crates/ruff_python_ast/generate.py b/crates/ruff_python_ast/generate.py index 4d11f3ba60..1d360e891c 100644 --- a/crates/ruff_python_ast/generate.py +++ b/crates/ruff_python_ast/generate.py @@ -2,6 +2,9 @@ # /// script # requires-python = ">=3.11" # dependencies = [] +# +# [tool.uv] +# exclude-newer = "P7D" # /// from __future__ import annotations diff --git a/crates/ruff_python_ast/generate.py.lock b/crates/ruff_python_ast/generate.py.lock new file mode 100644 index 0000000000..35fa400167 --- /dev/null +++ b/crates/ruff_python_ast/generate.py.lock @@ -0,0 +1,7 @@ +version = 1 +revision = 3 +requires-python = ">=3.11" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" diff --git a/crates/ty_python_semantic/mdtest.py b/crates/ty_python_semantic/mdtest.py index 2f0ed30b4b..4732aed5db 100644 --- a/crates/ty_python_semantic/mdtest.py +++ b/crates/ty_python_semantic/mdtest.py @@ -7,7 +7,7 @@ # ] # # [tool.uv] -# exclude-newer = "7 days" +# exclude-newer = "P7D" # /// from __future__ import annotations diff --git a/playground/.npmrc b/playground/.npmrc new file mode 100644 index 0000000000..b435032acd --- /dev/null +++ b/playground/.npmrc @@ -0,0 +1 @@ +min-release-age = 7 diff --git a/playground/api/.npmrc b/playground/api/.npmrc new file mode 100644 index 0000000000..b435032acd --- /dev/null +++ b/playground/api/.npmrc @@ -0,0 +1 @@ +min-release-age = 7 diff --git a/pyproject.toml b/pyproject.toml index 5595f75bb0..4f5317e0fc 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -63,6 +63,9 @@ release = [ "rooster==0.1.1", ] +[tool.uv] +exclude-newer = "P7D" + [tool.uv.dependency-groups] dev = { requires-python = ">=3.12" } release = { requires-python = ">=3.12" } diff --git a/python/py-fuzzer/pyproject.toml b/python/py-fuzzer/pyproject.toml index de1af7cc26..1c1d6ad098 100644 --- a/python/py-fuzzer/pyproject.toml +++ b/python/py-fuzzer/pyproject.toml @@ -78,3 +78,6 @@ unfixable = [ [tool.ruff.lint.isort] combine-as-imports = true split-on-trailing-comma = false + +[tool.uv] +exclude-newer = "P7D" diff --git a/python/py-fuzzer/uv.lock b/python/py-fuzzer/uv.lock index 096049a945..2b59a1b212 100644 --- a/python/py-fuzzer/uv.lock +++ b/python/py-fuzzer/uv.lock @@ -2,6 +2,10 @@ version = 1 revision = 3 requires-python = ">=3.12" +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" + [[package]] name = "markdown-it-py" version = "4.0.0" diff --git a/scripts/benchmarks/pyproject.toml b/scripts/benchmarks/pyproject.toml index 059e341502..a9c61da312 100644 --- a/scripts/benchmarks/pyproject.toml +++ b/scripts/benchmarks/pyproject.toml @@ -23,3 +23,6 @@ linter = [ "pylint", "isort", ] + +[tool.uv] +exclude-newer = "P7D" diff --git a/scripts/benchmarks/uv.lock b/scripts/benchmarks/uv.lock index 842ff01a1c..a8c3dd3a31 100644 --- a/scripts/benchmarks/uv.lock +++ b/scripts/benchmarks/uv.lock @@ -1,13 +1,18 @@ version = 1 -requires-python = ">=3.13" +revision = 3 +requires-python = ">=3.14" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" [[package]] name = "astroid" version = "3.3.7" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/20/f6/7725404e3dcaeafe695d4fe42ad99eefbcba4cad2c83ca122e6b439c9f96/astroid-3.3.7.tar.gz", hash = "sha256:29fe1df7ef64dc17a54dbfad67b40b445340fcdba7c4012e7ecc9270c9b2f5b6", size = 398091 } +sdist = { url = "https://files.pythonhosted.org/packages/20/f6/7725404e3dcaeafe695d4fe42ad99eefbcba4cad2c83ca122e6b439c9f96/astroid-3.3.7.tar.gz", hash = "sha256:29fe1df7ef64dc17a54dbfad67b40b445340fcdba7c4012e7ecc9270c9b2f5b6", size = 398091, upload-time = "2024-12-21T14:44:02.611Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/83/0b/ef3a51abbf2064ac50447a02d1cd14c1f008590e96f780042c21108b6b56/astroid-3.3.7-py3-none-any.whl", hash = "sha256:e1ea2c358a3c760ef583d4963e773100fa2c693b27ed158a1d0e81adb4436903", size = 275125 }, + { url = "https://files.pythonhosted.org/packages/83/0b/ef3a51abbf2064ac50447a02d1cd14c1f008590e96f780042c21108b6b56/astroid-3.3.7-py3-none-any.whl", hash = "sha256:e1ea2c358a3c760ef583d4963e773100fa2c693b27ed158a1d0e81adb4436903", size = 275125, upload-time = "2024-12-21T14:43:59.935Z" }, ] [[package]] @@ -17,9 +22,9 @@ source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "pyflakes" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/2a/cb/486f912d6171bc5748c311a2984a301f4e2d054833a1da78485866c71522/autoflake-2.3.1.tar.gz", hash = "sha256:c98b75dc5b0a86459c4f01a1d32ac7eb4338ec4317a4469515ff1e687ecd909e", size = 27642 } +sdist = { url = "https://files.pythonhosted.org/packages/2a/cb/486f912d6171bc5748c311a2984a301f4e2d054833a1da78485866c71522/autoflake-2.3.1.tar.gz", hash = "sha256:c98b75dc5b0a86459c4f01a1d32ac7eb4338ec4317a4469515ff1e687ecd909e", size = 27642, upload-time = "2024-03-13T03:41:28.977Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/a2/ee/3fd29bf416eb4f1c5579cf12bf393ae954099258abd7bde03c4f9716ef6b/autoflake-2.3.1-py3-none-any.whl", hash = "sha256:3ae7495db9084b7b32818b4140e6dc4fc280b712fb414f5b8fe57b0a8e85a840", size = 32483 }, + { url = "https://files.pythonhosted.org/packages/a2/ee/3fd29bf416eb4f1c5579cf12bf393ae954099258abd7bde03c4f9716ef6b/autoflake-2.3.1-py3-none-any.whl", hash = "sha256:3ae7495db9084b7b32818b4140e6dc4fc280b712fb414f5b8fe57b0a8e85a840", size = 32483, upload-time = "2024-03-13T03:41:26.969Z" }, ] [[package]] @@ -29,9 +34,9 @@ source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "pycodestyle" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/6c/52/65556a5f917a4b273fd1b705f98687a6bd721dbc45966f0f6687e90a18b0/autopep8-2.3.1.tar.gz", hash = "sha256:8d6c87eba648fdcfc83e29b788910b8643171c395d9c4bcf115ece035b9c9dda", size = 92064 } +sdist = { url = "https://files.pythonhosted.org/packages/6c/52/65556a5f917a4b273fd1b705f98687a6bd721dbc45966f0f6687e90a18b0/autopep8-2.3.1.tar.gz", hash = "sha256:8d6c87eba648fdcfc83e29b788910b8643171c395d9c4bcf115ece035b9c9dda", size = 92064, upload-time = "2024-06-23T05:15:55.401Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/ad/9e/f0beffe45b507dca9d7540fad42b316b2fd1076dc484c9b1f23d9da570d7/autopep8-2.3.1-py2.py3-none-any.whl", hash = "sha256:a203fe0fcad7939987422140ab17a930f684763bf7335bdb6709991dd7ef6c2d", size = 45667 }, + { url = "https://files.pythonhosted.org/packages/ad/9e/f0beffe45b507dca9d7540fad42b316b2fd1076dc484c9b1f23d9da570d7/autopep8-2.3.1-py2.py3-none-any.whl", hash = "sha256:a203fe0fcad7939987422140ab17a930f684763bf7335bdb6709991dd7ef6c2d", size = 45667, upload-time = "2024-06-23T05:15:51.29Z" }, ] [[package]] @@ -45,13 +50,9 @@ dependencies = [ { name = "pathspec" }, { name = "platformdirs" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/d8/0d/cc2fb42b8c50d80143221515dd7e4766995bd07c56c9a3ed30baf080b6dc/black-24.10.0.tar.gz", hash = "sha256:846ea64c97afe3bc677b761787993be4991810ecc7a4a937816dd6bddedc4875", size = 645813 } +sdist = { url = "https://files.pythonhosted.org/packages/d8/0d/cc2fb42b8c50d80143221515dd7e4766995bd07c56c9a3ed30baf080b6dc/black-24.10.0.tar.gz", hash = "sha256:846ea64c97afe3bc677b761787993be4991810ecc7a4a937816dd6bddedc4875", size = 645813, upload-time = "2024-10-07T19:20:50.361Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d0/a0/a993f58d4ecfba035e61fca4e9f64a2ecae838fc9f33ab798c62173ed75c/black-24.10.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:1cbacacb19e922a1d75ef2b6ccaefcd6e93a2c05ede32f06a21386a04cedb981", size = 1643986 }, - { url = "https://files.pythonhosted.org/packages/37/d5/602d0ef5dfcace3fb4f79c436762f130abd9ee8d950fa2abdbf8bbc555e0/black-24.10.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:1f93102e0c5bb3907451063e08b9876dbeac810e7da5a8bfb7aeb5a9ef89066b", size = 1448085 }, - { url = "https://files.pythonhosted.org/packages/47/6d/a3a239e938960df1a662b93d6230d4f3e9b4a22982d060fc38c42f45a56b/black-24.10.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ddacb691cdcdf77b96f549cf9591701d8db36b2f19519373d60d31746068dbf2", size = 1760928 }, - { url = "https://files.pythonhosted.org/packages/dd/cf/af018e13b0eddfb434df4d9cd1b2b7892bab119f7a20123e93f6910982e8/black-24.10.0-cp313-cp313-win_amd64.whl", hash = "sha256:680359d932801c76d2e9c9068d05c6b107f2584b2a5b88831c83962eb9984c1b", size = 1436875 }, - { url = "https://files.pythonhosted.org/packages/8d/a7/4b27c50537ebca8bec139b872861f9d2bf501c5ec51fcf897cb924d9e264/black-24.10.0-py3-none-any.whl", hash = "sha256:3bb2b7a1f7b685f85b11fed1ef10f8a9148bceb49853e47a294a3dd963c1dd7d", size = 206898 }, + { url = "https://files.pythonhosted.org/packages/8d/a7/4b27c50537ebca8bec139b872861f9d2bf501c5ec51fcf897cb924d9e264/black-24.10.0-py3-none-any.whl", hash = "sha256:3bb2b7a1f7b685f85b11fed1ef10f8a9148bceb49853e47a294a3dd963c1dd7d", size = 206898, upload-time = "2024-10-07T19:20:48.317Z" }, ] [[package]] @@ -59,29 +60,29 @@ name = "click" version = "8.1.8" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "colorama", marker = "platform_system == 'Windows'" }, + { name = "colorama", marker = "sys_platform == 'win32'" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/b9/2e/0090cbf739cee7d23781ad4b89a9894a41538e4fcf4c31dcdd705b78eb8b/click-8.1.8.tar.gz", hash = "sha256:ed53c9d8990d83c2a27deae68e4ee337473f6330c040a31d4225c9574d16096a", size = 226593 } +sdist = { url = "https://files.pythonhosted.org/packages/b9/2e/0090cbf739cee7d23781ad4b89a9894a41538e4fcf4c31dcdd705b78eb8b/click-8.1.8.tar.gz", hash = "sha256:ed53c9d8990d83c2a27deae68e4ee337473f6330c040a31d4225c9574d16096a", size = 226593, upload-time = "2024-12-21T18:38:44.339Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/7e/d4/7ebdbd03970677812aac39c869717059dbb71a4cfc033ca6e5221787892c/click-8.1.8-py3-none-any.whl", hash = "sha256:63c132bbbed01578a06712a2d1f497bb62d9c1c0d329b7903a866228027263b2", size = 98188 }, + { url = "https://files.pythonhosted.org/packages/7e/d4/7ebdbd03970677812aac39c869717059dbb71a4cfc033ca6e5221787892c/click-8.1.8-py3-none-any.whl", hash = "sha256:63c132bbbed01578a06712a2d1f497bb62d9c1c0d329b7903a866228027263b2", size = 98188, upload-time = "2024-12-21T18:38:41.666Z" }, ] [[package]] name = "colorama" version = "0.4.6" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697 } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335 }, + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, ] [[package]] name = "dill" version = "0.3.9" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/70/43/86fe3f9e130c4137b0f1b50784dd70a5087b911fe07fa81e53e0c4c47fea/dill-0.3.9.tar.gz", hash = "sha256:81aa267dddf68cbfe8029c42ca9ec6a4ab3b22371d1c450abc54422577b4512c", size = 187000 } +sdist = { url = "https://files.pythonhosted.org/packages/70/43/86fe3f9e130c4137b0f1b50784dd70a5087b911fe07fa81e53e0c4c47fea/dill-0.3.9.tar.gz", hash = "sha256:81aa267dddf68cbfe8029c42ca9ec6a4ab3b22371d1c450abc54422577b4512c", size = 187000, upload-time = "2024-09-29T00:03:20.958Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/46/d1/e73b6ad76f0b1fb7f23c35c6d95dbc506a9c8804f43dda8cb5b0fa6331fd/dill-0.3.9-py3-none-any.whl", hash = "sha256:468dff3b89520b474c0397703366b7b95eebe6303f108adf9b19da1f702be87a", size = 119418 }, + { url = "https://files.pythonhosted.org/packages/46/d1/e73b6ad76f0b1fb7f23c35c6d95dbc506a9c8804f43dda8cb5b0fa6331fd/dill-0.3.9-py3-none-any.whl", hash = "sha256:468dff3b89520b474c0397703366b7b95eebe6303f108adf9b19da1f702be87a", size = 119418, upload-time = "2024-09-29T00:03:19.344Z" }, ] [[package]] @@ -93,81 +94,81 @@ dependencies = [ { name = "pycodestyle" }, { name = "pyflakes" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/37/72/e8d66150c4fcace3c0a450466aa3480506ba2cae7b61e100a2613afc3907/flake8-7.1.1.tar.gz", hash = "sha256:049d058491e228e03e67b390f311bbf88fce2dbaa8fa673e7aea87b7198b8d38", size = 48054 } +sdist = { url = "https://files.pythonhosted.org/packages/37/72/e8d66150c4fcace3c0a450466aa3480506ba2cae7b61e100a2613afc3907/flake8-7.1.1.tar.gz", hash = "sha256:049d058491e228e03e67b390f311bbf88fce2dbaa8fa673e7aea87b7198b8d38", size = 48054, upload-time = "2024-08-04T20:32:44.311Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d9/42/65004373ac4617464f35ed15931b30d764f53cdd30cc78d5aea349c8c050/flake8-7.1.1-py2.py3-none-any.whl", hash = "sha256:597477df7860daa5aa0fdd84bf5208a043ab96b8e96ab708770ae0364dd03213", size = 57731 }, + { url = "https://files.pythonhosted.org/packages/d9/42/65004373ac4617464f35ed15931b30d764f53cdd30cc78d5aea349c8c050/flake8-7.1.1-py2.py3-none-any.whl", hash = "sha256:597477df7860daa5aa0fdd84bf5208a043ab96b8e96ab708770ae0364dd03213", size = 57731, upload-time = "2024-08-04T20:32:42.661Z" }, ] [[package]] name = "isort" version = "5.13.2" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/87/f9/c1eb8635a24e87ade2efce21e3ce8cd6b8630bb685ddc9cdaca1349b2eb5/isort-5.13.2.tar.gz", hash = "sha256:48fdfcb9face5d58a4f6dde2e72a1fb8dcaf8ab26f95ab49fab84c2ddefb0109", size = 175303 } +sdist = { url = "https://files.pythonhosted.org/packages/87/f9/c1eb8635a24e87ade2efce21e3ce8cd6b8630bb685ddc9cdaca1349b2eb5/isort-5.13.2.tar.gz", hash = "sha256:48fdfcb9face5d58a4f6dde2e72a1fb8dcaf8ab26f95ab49fab84c2ddefb0109", size = 175303, upload-time = "2023-12-13T20:37:26.124Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d1/b3/8def84f539e7d2289a02f0524b944b15d7c75dab7628bedf1c4f0992029c/isort-5.13.2-py3-none-any.whl", hash = "sha256:8ca5e72a8d85860d5a3fa69b8745237f2939afe12dbf656afbcb47fe72d947a6", size = 92310 }, + { url = "https://files.pythonhosted.org/packages/d1/b3/8def84f539e7d2289a02f0524b944b15d7c75dab7628bedf1c4f0992029c/isort-5.13.2-py3-none-any.whl", hash = "sha256:8ca5e72a8d85860d5a3fa69b8745237f2939afe12dbf656afbcb47fe72d947a6", size = 92310, upload-time = "2023-12-13T20:37:23.244Z" }, ] [[package]] name = "mccabe" version = "0.7.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/e7/ff/0ffefdcac38932a54d2b5eed4e0ba8a408f215002cd178ad1df0f2806ff8/mccabe-0.7.0.tar.gz", hash = "sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325", size = 9658 } +sdist = { url = "https://files.pythonhosted.org/packages/e7/ff/0ffefdcac38932a54d2b5eed4e0ba8a408f215002cd178ad1df0f2806ff8/mccabe-0.7.0.tar.gz", hash = "sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325", size = 9658, upload-time = "2022-01-24T01:14:51.113Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/27/1a/1f68f9ba0c207934b35b86a8ca3aad8395a3d6dd7921c0686e23853ff5a9/mccabe-0.7.0-py2.py3-none-any.whl", hash = "sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e", size = 7350 }, + { url = "https://files.pythonhosted.org/packages/27/1a/1f68f9ba0c207934b35b86a8ca3aad8395a3d6dd7921c0686e23853ff5a9/mccabe-0.7.0-py2.py3-none-any.whl", hash = "sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e", size = 7350, upload-time = "2022-01-24T01:14:49.62Z" }, ] [[package]] name = "mypy-extensions" version = "1.0.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/98/a4/1ab47638b92648243faf97a5aeb6ea83059cc3624972ab6b8d2316078d3f/mypy_extensions-1.0.0.tar.gz", hash = "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782", size = 4433 } +sdist = { url = "https://files.pythonhosted.org/packages/98/a4/1ab47638b92648243faf97a5aeb6ea83059cc3624972ab6b8d2316078d3f/mypy_extensions-1.0.0.tar.gz", hash = "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782", size = 4433, upload-time = "2023-02-04T12:11:27.157Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/2a/e2/5d3f6ada4297caebe1a2add3b126fe800c96f56dbe5d1988a2cbe0b267aa/mypy_extensions-1.0.0-py3-none-any.whl", hash = "sha256:4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d", size = 4695 }, + { url = "https://files.pythonhosted.org/packages/2a/e2/5d3f6ada4297caebe1a2add3b126fe800c96f56dbe5d1988a2cbe0b267aa/mypy_extensions-1.0.0-py3-none-any.whl", hash = "sha256:4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d", size = 4695, upload-time = "2023-02-04T12:11:25.002Z" }, ] [[package]] name = "packaging" version = "24.2" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/d0/63/68dbb6eb2de9cb10ee4c9c14a0148804425e13c4fb20d61cce69f53106da/packaging-24.2.tar.gz", hash = "sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f", size = 163950 } +sdist = { url = "https://files.pythonhosted.org/packages/d0/63/68dbb6eb2de9cb10ee4c9c14a0148804425e13c4fb20d61cce69f53106da/packaging-24.2.tar.gz", hash = "sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f", size = 163950, upload-time = "2024-11-08T09:47:47.202Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl", hash = "sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759", size = 65451 }, + { url = "https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl", hash = "sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759", size = 65451, upload-time = "2024-11-08T09:47:44.722Z" }, ] [[package]] name = "pathspec" version = "0.12.1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/ca/bc/f35b8446f4531a7cb215605d100cd88b7ac6f44ab3fc94870c120ab3adbf/pathspec-0.12.1.tar.gz", hash = "sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712", size = 51043 } +sdist = { url = "https://files.pythonhosted.org/packages/ca/bc/f35b8446f4531a7cb215605d100cd88b7ac6f44ab3fc94870c120ab3adbf/pathspec-0.12.1.tar.gz", hash = "sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712", size = 51043, upload-time = "2023-12-10T22:30:45Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/cc/20/ff623b09d963f88bfde16306a54e12ee5ea43e9b597108672ff3a408aad6/pathspec-0.12.1-py3-none-any.whl", hash = "sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08", size = 31191 }, + { url = "https://files.pythonhosted.org/packages/cc/20/ff623b09d963f88bfde16306a54e12ee5ea43e9b597108672ff3a408aad6/pathspec-0.12.1-py3-none-any.whl", hash = "sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08", size = 31191, upload-time = "2023-12-10T22:30:43.14Z" }, ] [[package]] name = "platformdirs" version = "4.3.6" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/13/fc/128cc9cb8f03208bdbf93d3aa862e16d376844a14f9a0ce5cf4507372de4/platformdirs-4.3.6.tar.gz", hash = "sha256:357fb2acbc885b0419afd3ce3ed34564c13c9b95c89360cd9563f73aa5e2b907", size = 21302 } +sdist = { url = "https://files.pythonhosted.org/packages/13/fc/128cc9cb8f03208bdbf93d3aa862e16d376844a14f9a0ce5cf4507372de4/platformdirs-4.3.6.tar.gz", hash = "sha256:357fb2acbc885b0419afd3ce3ed34564c13c9b95c89360cd9563f73aa5e2b907", size = 21302, upload-time = "2024-09-17T19:06:50.688Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/3c/a6/bc1012356d8ece4d66dd75c4b9fc6c1f6650ddd5991e421177d9f8f671be/platformdirs-4.3.6-py3-none-any.whl", hash = "sha256:73e575e1408ab8103900836b97580d5307456908a03e92031bab39e4554cc3fb", size = 18439 }, + { url = "https://files.pythonhosted.org/packages/3c/a6/bc1012356d8ece4d66dd75c4b9fc6c1f6650ddd5991e421177d9f8f671be/platformdirs-4.3.6-py3-none-any.whl", hash = "sha256:73e575e1408ab8103900836b97580d5307456908a03e92031bab39e4554cc3fb", size = 18439, upload-time = "2024-09-17T19:06:49.212Z" }, ] [[package]] name = "pycodestyle" version = "2.12.1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/43/aa/210b2c9aedd8c1cbeea31a50e42050ad56187754b34eb214c46709445801/pycodestyle-2.12.1.tar.gz", hash = "sha256:6838eae08bbce4f6accd5d5572075c63626a15ee3e6f842df996bf62f6d73521", size = 39232 } +sdist = { url = "https://files.pythonhosted.org/packages/43/aa/210b2c9aedd8c1cbeea31a50e42050ad56187754b34eb214c46709445801/pycodestyle-2.12.1.tar.gz", hash = "sha256:6838eae08bbce4f6accd5d5572075c63626a15ee3e6f842df996bf62f6d73521", size = 39232, upload-time = "2024-08-04T20:26:54.576Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/3a/d8/a211b3f85e99a0daa2ddec96c949cac6824bd305b040571b82a03dd62636/pycodestyle-2.12.1-py2.py3-none-any.whl", hash = "sha256:46f0fb92069a7c28ab7bb558f05bfc0110dac69a0cd23c61ea0040283a9d78b3", size = 31284 }, + { url = "https://files.pythonhosted.org/packages/3a/d8/a211b3f85e99a0daa2ddec96c949cac6824bd305b040571b82a03dd62636/pycodestyle-2.12.1-py2.py3-none-any.whl", hash = "sha256:46f0fb92069a7c28ab7bb558f05bfc0110dac69a0cd23c61ea0040283a9d78b3", size = 31284, upload-time = "2024-08-04T20:26:53.173Z" }, ] [[package]] name = "pyflakes" version = "3.2.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/57/f9/669d8c9c86613c9d568757c7f5824bd3197d7b1c6c27553bc5618a27cce2/pyflakes-3.2.0.tar.gz", hash = "sha256:1c61603ff154621fb2a9172037d84dca3500def8c8b630657d1701f026f8af3f", size = 63788 } +sdist = { url = "https://files.pythonhosted.org/packages/57/f9/669d8c9c86613c9d568757c7f5824bd3197d7b1c6c27553bc5618a27cce2/pyflakes-3.2.0.tar.gz", hash = "sha256:1c61603ff154621fb2a9172037d84dca3500def8c8b630657d1701f026f8af3f", size = 63788, upload-time = "2024-01-05T00:28:47.703Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/d4/d7/f1b7db88d8e4417c5d47adad627a93547f44bdc9028372dbd2313f34a855/pyflakes-3.2.0-py2.py3-none-any.whl", hash = "sha256:84b5be138a2dfbb40689ca07e2152deb896a65c3a3e24c251c5c62489568074a", size = 62725 }, + { url = "https://files.pythonhosted.org/packages/d4/d7/f1b7db88d8e4417c5d47adad627a93547f44bdc9028372dbd2313f34a855/pyflakes-3.2.0-py2.py3-none-any.whl", hash = "sha256:84b5be138a2dfbb40689ca07e2152deb896a65c3a3e24c251c5c62489568074a", size = 62725, upload-time = "2024-01-05T00:28:45.903Z" }, ] [[package]] @@ -183,14 +184,14 @@ dependencies = [ { name = "platformdirs" }, { name = "tomlkit" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/81/d8/4471b2cb4ad18b4af717918c468209bd2bd5a02c52f60be5ee8a71b5af2c/pylint-3.3.2.tar.gz", hash = "sha256:9ec054ec992cd05ad30a6df1676229739a73f8feeabf3912c995d17601052b01", size = 1516485 } +sdist = { url = "https://files.pythonhosted.org/packages/81/d8/4471b2cb4ad18b4af717918c468209bd2bd5a02c52f60be5ee8a71b5af2c/pylint-3.3.2.tar.gz", hash = "sha256:9ec054ec992cd05ad30a6df1676229739a73f8feeabf3912c995d17601052b01", size = 1516485, upload-time = "2024-12-01T18:45:32.97Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/61/55/5eaf6c415f6ddb09b9b039278823a8e27fb81ea7a34ec80c6d9223b17f2e/pylint-3.3.2-py3-none-any.whl", hash = "sha256:77f068c287d49b8683cd7c6e624243c74f92890f767f106ffa1ddf3c0a54cb7a", size = 521873 }, + { url = "https://files.pythonhosted.org/packages/61/55/5eaf6c415f6ddb09b9b039278823a8e27fb81ea7a34ec80c6d9223b17f2e/pylint-3.3.2-py3-none-any.whl", hash = "sha256:77f068c287d49b8683cd7c6e624243c74f92890f767f106ffa1ddf3c0a54cb7a", size = 521873, upload-time = "2024-12-01T18:45:29.733Z" }, ] [[package]] name = "scripts" -version = "0.8.4" +version = "0.16.3" source = { virtual = "." } dependencies = [ { name = "autoflake" }, @@ -251,9 +252,9 @@ linter = [ name = "tomlkit" version = "0.13.2" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/b1/09/a439bec5888f00a54b8b9f05fa94d7f901d6735ef4e55dcec9bc37b5d8fa/tomlkit-0.13.2.tar.gz", hash = "sha256:fff5fe59a87295b278abd31bec92c15d9bc4a06885ab12bcea52c71119392e79", size = 192885 } +sdist = { url = "https://files.pythonhosted.org/packages/b1/09/a439bec5888f00a54b8b9f05fa94d7f901d6735ef4e55dcec9bc37b5d8fa/tomlkit-0.13.2.tar.gz", hash = "sha256:fff5fe59a87295b278abd31bec92c15d9bc4a06885ab12bcea52c71119392e79", size = 192885, upload-time = "2024-08-14T08:19:41.488Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/f9/b6/a447b5e4ec71e13871be01ba81f5dfc9d0af7e473da256ff46bc0e24026f/tomlkit-0.13.2-py3-none-any.whl", hash = "sha256:7a974427f6e119197f670fbbbeae7bef749a6c14e793db934baefc1b5f03efde", size = 37955 }, + { url = "https://files.pythonhosted.org/packages/f9/b6/a447b5e4ec71e13871be01ba81f5dfc9d0af7e473da256ff46bc0e24026f/tomlkit-0.13.2-py3-none-any.whl", hash = "sha256:7a974427f6e119197f670fbbbeae7bef749a6c14e793db934baefc1b5f03efde", size = 37955, upload-time = "2024-08-14T08:19:40.05Z" }, ] [[package]] @@ -263,7 +264,7 @@ source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "platformdirs" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/23/97/b6f296d1e9cc1ec25c7604178b48532fa5901f721bcf1b8d8148b13e5588/yapf-0.43.0.tar.gz", hash = "sha256:00d3aa24bfedff9420b2e0d5d9f5ab6d9d4268e72afbf59bb3fa542781d5218e", size = 254907 } +sdist = { url = "https://files.pythonhosted.org/packages/23/97/b6f296d1e9cc1ec25c7604178b48532fa5901f721bcf1b8d8148b13e5588/yapf-0.43.0.tar.gz", hash = "sha256:00d3aa24bfedff9420b2e0d5d9f5ab6d9d4268e72afbf59bb3fa542781d5218e", size = 254907, upload-time = "2024-11-14T00:11:41.584Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/37/81/6acd6601f61e31cfb8729d3da6d5df966f80f374b78eff83760714487338/yapf-0.43.0-py3-none-any.whl", hash = "sha256:224faffbc39c428cb095818cf6ef5511fdab6f7430a10783fdfb292ccf2852ca", size = 256158 }, + { url = "https://files.pythonhosted.org/packages/37/81/6acd6601f61e31cfb8729d3da6d5df966f80f374b78eff83760714487338/yapf-0.43.0-py3-none-any.whl", hash = "sha256:224faffbc39c428cb095818cf6ef5511fdab6f7430a10783fdfb292ccf2852ca", size = 256158, upload-time = "2024-11-14T00:11:39.37Z" }, ] diff --git a/scripts/build_ruff_pgo.py b/scripts/build_ruff_pgo.py index 9bcb7d533b..ae2072d41f 100644 --- a/scripts/build_ruff_pgo.py +++ b/scripts/build_ruff_pgo.py @@ -3,6 +3,9 @@ # /// script # requires-python = ">=3.11" # dependencies = [] +# +# [tool.uv] +# exclude-newer = "P7D" # /// from __future__ import annotations diff --git a/scripts/build_ruff_pgo.py.lock b/scripts/build_ruff_pgo.py.lock new file mode 100644 index 0000000000..35fa400167 --- /dev/null +++ b/scripts/build_ruff_pgo.py.lock @@ -0,0 +1,7 @@ +version = 1 +revision = 3 +requires-python = ">=3.11" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" diff --git a/scripts/bump-workspace-crate-versions.py b/scripts/bump-workspace-crate-versions.py index eb183685eb..33b024d6c3 100644 --- a/scripts/bump-workspace-crate-versions.py +++ b/scripts/bump-workspace-crate-versions.py @@ -8,6 +8,9 @@ # /// script # requires-python = ">=3.13" # dependencies = [] +# +# [tool.uv] +# exclude-newer = "P7D" # /// diff --git a/scripts/bump-workspace-crate-versions.py.lock b/scripts/bump-workspace-crate-versions.py.lock new file mode 100644 index 0000000000..8576417a3e --- /dev/null +++ b/scripts/bump-workspace-crate-versions.py.lock @@ -0,0 +1,7 @@ +version = 1 +revision = 3 +requires-python = ">=3.13" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" diff --git a/scripts/collect_ty_ecosystem_run_metadata.py b/scripts/collect_ty_ecosystem_run_metadata.py index aac9c4c52c..e0227f556b 100755 --- a/scripts/collect_ty_ecosystem_run_metadata.py +++ b/scripts/collect_ty_ecosystem_run_metadata.py @@ -3,6 +3,9 @@ # /// script # requires-python = ">=3.11" # dependencies = [] +# +# [tool.uv] +# exclude-newer = "P7D" # /// """Collect the exact inputs used by a Ruff ty ecosystem-analyzer run.""" diff --git a/scripts/collect_ty_ecosystem_run_metadata.py.lock b/scripts/collect_ty_ecosystem_run_metadata.py.lock new file mode 100644 index 0000000000..35fa400167 --- /dev/null +++ b/scripts/collect_ty_ecosystem_run_metadata.py.lock @@ -0,0 +1,7 @@ +version = 1 +revision = 3 +requires-python = ">=3.11" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" diff --git a/scripts/generate-crate-readmes.py b/scripts/generate-crate-readmes.py index 02f88709e4..38327db218 100644 --- a/scripts/generate-crate-readmes.py +++ b/scripts/generate-crate-readmes.py @@ -1,6 +1,9 @@ # /// script # requires-python = ">=3.13" # dependencies = [] +# +# [tool.uv] +# exclude-newer = "P7D" # /// from __future__ import annotations diff --git a/scripts/generate-crate-readmes.py.lock b/scripts/generate-crate-readmes.py.lock new file mode 100644 index 0000000000..8576417a3e --- /dev/null +++ b/scripts/generate-crate-readmes.py.lock @@ -0,0 +1,7 @@ +version = 1 +revision = 3 +requires-python = ">=3.13" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" diff --git a/scripts/pyproject.toml b/scripts/pyproject.toml index 2a64bb8c16..6993c23543 100644 --- a/scripts/pyproject.toml +++ b/scripts/pyproject.toml @@ -14,6 +14,9 @@ extend = "../pyproject.toml" # `ty_benchmark` is a standalone project with its own pyproject.toml files, search paths, etc. exclude = ["./ty_benchmark"] +[tool.uv] +exclude-newer = "P7D" + [tool.uv.sources] mypy-primer = { git = "https://github.com/hauntsaninja/mypy_primer" } diff --git a/scripts/setup-crates-io-publish.py b/scripts/setup-crates-io-publish.py index a62e6dbcea..ff7f317b46 100644 --- a/scripts/setup-crates-io-publish.py +++ b/scripts/setup-crates-io-publish.py @@ -20,6 +20,9 @@ # /// script # requires-python = ">=3.13" # dependencies = ["httpx"] +# +# [tool.uv] +# exclude-newer = "P7D" # /// from __future__ import annotations diff --git a/scripts/setup-crates-io-publish.py.lock b/scripts/setup-crates-io-publish.py.lock index 0e4bb17fde..99f0d0c41e 100644 --- a/scripts/setup-crates-io-publish.py.lock +++ b/scripts/setup-crates-io-publish.py.lock @@ -2,6 +2,10 @@ version = 1 revision = 3 requires-python = ">=3.13" +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" + [manifest] requirements = [{ name = "httpx" }] diff --git a/scripts/setup_primer_project.py b/scripts/setup_primer_project.py index f971d0a903..db5be5d8fd 100644 --- a/scripts/setup_primer_project.py +++ b/scripts/setup_primer_project.py @@ -9,7 +9,7 @@ # # bypass the adjacent lock and select ecosystem-analyzer's exact mypy-primer # # revision and project Python version, as shown in the module docstring. # # `exclude-newer` still constrains mypy-primer's registry dependencies. -# exclude-newer = "7 days" +# exclude-newer = "P7D" # # [tool.uv.sources] # # Keep this revision and the script's lockfile in sync with ecosystem-analyzer's diff --git a/scripts/ty_benchmark/.npmrc b/scripts/ty_benchmark/.npmrc new file mode 100644 index 0000000000..b435032acd --- /dev/null +++ b/scripts/ty_benchmark/.npmrc @@ -0,0 +1 @@ +min-release-age = 7 diff --git a/scripts/ty_benchmark/pyproject.toml b/scripts/ty_benchmark/pyproject.toml index 5bcb0a4ce9..d5e7e5a237 100644 --- a/scripts/ty_benchmark/pyproject.toml +++ b/scripts/ty_benchmark/pyproject.toml @@ -35,3 +35,6 @@ ignore = [ possibly-unresolved-reference = "error" division-by-zero = "error" unused-ignore-comment = "error" + +[tool.uv] +exclude-newer = "P7D" diff --git a/scripts/ty_benchmark/uv.lock b/scripts/ty_benchmark/uv.lock index dd30a3fb58..0214deffa3 100644 --- a/scripts/ty_benchmark/uv.lock +++ b/scripts/ty_benchmark/uv.lock @@ -2,6 +2,10 @@ version = 1 revision = 3 requires-python = ">=3.14" +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" + [[package]] name = "attrs" version = "26.1.0" diff --git a/scripts/uv.lock b/scripts/uv.lock index 09ab1d58fd..6461f0ce9b 100644 --- a/scripts/uv.lock +++ b/scripts/uv.lock @@ -2,6 +2,10 @@ version = 1 revision = 3 requires-python = ">=3.12" +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" + [[package]] name = "anyio" version = "4.14.0" diff --git a/uv.lock b/uv.lock index 489bb3460d..a9b3550aa3 100644 --- a/uv.lock +++ b/uv.lock @@ -7,6 +7,10 @@ resolution-markers = [ "python_full_version < '3.8'", ] +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" + [[package]] name = "annotated-doc" version = "0.0.4" From 8b623f337b59e2da5d9d12643cd62daec43fd727 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Sun, 16 Aug 2026 14:52:07 -0400 Subject: [PATCH 064/371] Address `uv audit` findings (#27797) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Addresses all `uv audit` findings. ## Test Plan NFC. --- Stack created with GitHub Stacks CLI • Give Feedback 💬 --- crates/ty_python_semantic/mdtest.py.lock | 6 ++-- python/py-fuzzer/uv.lock | 6 ++-- scripts/benchmarks/uv.lock | 43 +++++++++++++++++++----- 3 files changed, 40 insertions(+), 15 deletions(-) diff --git a/crates/ty_python_semantic/mdtest.py.lock b/crates/ty_python_semantic/mdtest.py.lock index de16515750..cf6c19687f 100644 --- a/crates/ty_python_semantic/mdtest.py.lock +++ b/crates/ty_python_semantic/mdtest.py.lock @@ -27,11 +27,11 @@ wheels = [ [[package]] name = "idna" -version = "3.11" +version = "3.15" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/0703ccc57f3a7233505399edb88de3cbd678da106337b9fcde432b65ed60/idna-3.11.tar.gz", hash = "sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902", size = 194582, upload-time = "2025-10-12T14:55:20.501Z" } +sdist = { url = "https://files.pythonhosted.org/packages/82/77/7b3966d0b9d1d31a36ddf1746926a11dface89a83409bf1483f0237aa758/idna-3.15.tar.gz", hash = "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc", size = 199245, upload-time = "2026-05-12T22:45:57.011Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/0e/61/66938bbb5fc52dbdf84594873d5b51fb1f7c7794e9c0f5bd885f30bc507b/idna-3.11-py3-none-any.whl", hash = "sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea", size = 71008, upload-time = "2025-10-12T14:55:18.883Z" }, + { url = "https://files.pythonhosted.org/packages/d2/23/408243171aa9aaba178d3e2559159c24c1171a641aa83b67bdd3394ead8e/idna-3.15-py3-none-any.whl", hash = "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", size = 72340, upload-time = "2026-05-12T22:45:55.733Z" }, ] [[package]] diff --git a/python/py-fuzzer/uv.lock b/python/py-fuzzer/uv.lock index 2b59a1b212..5c47195f74 100644 --- a/python/py-fuzzer/uv.lock +++ b/python/py-fuzzer/uv.lock @@ -114,11 +114,11 @@ dev = [ [[package]] name = "pygments" -version = "2.19.2" +version = "2.20.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz", hash = "sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", size = 4968631, upload-time = "2025-06-21T13:39:12.283Z" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl", hash = "sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b", size = 1225217, upload-time = "2025-06-21T13:39:07.939Z" }, + { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, ] [[package]] diff --git a/scripts/benchmarks/uv.lock b/scripts/benchmarks/uv.lock index a8c3dd3a31..8dc6816606 100644 --- a/scripts/benchmarks/uv.lock +++ b/scripts/benchmarks/uv.lock @@ -41,7 +41,7 @@ wheels = [ [[package]] name = "black" -version = "24.10.0" +version = "26.3.1" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "click" }, @@ -49,22 +49,28 @@ dependencies = [ { name = "packaging" }, { name = "pathspec" }, { name = "platformdirs" }, + { name = "pytokens" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/d8/0d/cc2fb42b8c50d80143221515dd7e4766995bd07c56c9a3ed30baf080b6dc/black-24.10.0.tar.gz", hash = "sha256:846ea64c97afe3bc677b761787993be4991810ecc7a4a937816dd6bddedc4875", size = 645813, upload-time = "2024-10-07T19:20:50.361Z" } +sdist = { url = "https://files.pythonhosted.org/packages/e1/c5/61175d618685d42b005847464b8fb4743a67b1b8fdb75e50e5a96c31a27a/black-26.3.1.tar.gz", hash = "sha256:2c50f5063a9641c7eed7795014ba37b0f5fa227f3d408b968936e24bc0566b07", size = 666155, upload-time = "2026-03-12T03:36:03.593Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/8d/a7/4b27c50537ebca8bec139b872861f9d2bf501c5ec51fcf897cb924d9e264/black-24.10.0-py3-none-any.whl", hash = "sha256:3bb2b7a1f7b685f85b11fed1ef10f8a9148bceb49853e47a294a3dd963c1dd7d", size = 206898, upload-time = "2024-10-07T19:20:48.317Z" }, + { url = "https://files.pythonhosted.org/packages/d5/da/e36e27c9cebc1311b7579210df6f1c86e50f2d7143ae4fcf8a5017dc8809/black-26.3.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:2d6bfaf7fd0993b420bed691f20f9492d53ce9a2bcccea4b797d34e947318a78", size = 1889234, upload-time = "2026-03-12T03:40:30.964Z" }, + { url = "https://files.pythonhosted.org/packages/0e/7b/9871acf393f64a5fa33668c19350ca87177b181f44bb3d0c33b2d534f22c/black-26.3.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:f89f2ab047c76a9c03f78d0d66ca519e389519902fa27e7a91117ef7611c0568", size = 1720522, upload-time = "2026-03-12T03:40:32.346Z" }, + { url = "https://files.pythonhosted.org/packages/03/87/e766c7f2e90c07fb7586cc787c9ae6462b1eedab390191f2b7fc7f6170a9/black-26.3.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b07fc0dab849d24a80a29cfab8d8a19187d1c4685d8a5e6385a5ce323c1f015f", size = 1787824, upload-time = "2026-03-12T03:40:33.636Z" }, + { url = "https://files.pythonhosted.org/packages/ac/94/2424338fb2d1875e9e83eed4c8e9c67f6905ec25afd826a911aea2b02535/black-26.3.1-cp314-cp314-win_amd64.whl", hash = "sha256:0126ae5b7c09957da2bdbd91a9ba1207453feada9e9fe51992848658c6c8e01c", size = 1445855, upload-time = "2026-03-12T03:40:35.442Z" }, + { url = "https://files.pythonhosted.org/packages/86/43/0c3338bd928afb8ee7471f1a4eec3bdbe2245ccb4a646092a222e8669840/black-26.3.1-cp314-cp314-win_arm64.whl", hash = "sha256:92c0ec1f2cc149551a2b7b47efc32c866406b6891b0ee4625e95967c8f4acfb1", size = 1258109, upload-time = "2026-03-12T03:40:36.832Z" }, + { url = "https://files.pythonhosted.org/packages/8e/0d/52d98722666d6fc6c3dd4c76df339501d6efd40e0ff95e6186a7b7f0befd/black-26.3.1-py3-none-any.whl", hash = "sha256:2bd5aa94fc267d38bb21a70d7410a89f1a1d318841855f698746f8e7f51acd1b", size = 207542, upload-time = "2026-03-12T03:36:01.668Z" }, ] [[package]] name = "click" -version = "8.1.8" +version = "8.3.3" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "colorama", marker = "sys_platform == 'win32'" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/b9/2e/0090cbf739cee7d23781ad4b89a9894a41538e4fcf4c31dcdd705b78eb8b/click-8.1.8.tar.gz", hash = "sha256:ed53c9d8990d83c2a27deae68e4ee337473f6330c040a31d4225c9574d16096a", size = 226593, upload-time = "2024-12-21T18:38:44.339Z" } +sdist = { url = "https://files.pythonhosted.org/packages/bb/63/f9e1ea081ce35720d8b92acde70daaedace594dc93b693c869e0d5910718/click-8.3.3.tar.gz", hash = "sha256:398329ad4837b2ff7cbe1dd166a4c0f8900c3ca3a218de04466f38f6497f18a2", size = 328061, upload-time = "2026-04-22T15:11:27.506Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/7e/d4/7ebdbd03970677812aac39c869717059dbb71a4cfc033ca6e5221787892c/click-8.1.8-py3-none-any.whl", hash = "sha256:63c132bbbed01578a06712a2d1f497bb62d9c1c0d329b7903a866228027263b2", size = 98188, upload-time = "2024-12-21T18:38:41.666Z" }, + { url = "https://files.pythonhosted.org/packages/ae/44/c1221527f6a71a01ec6fbad7fa78f1d50dfa02217385cf0fa3eec7087d59/click-8.3.3-py3-none-any.whl", hash = "sha256:a2bf429bb3033c89fa4936ffb35d5cb471e3719e1f3c8a7c3fff0b8314305613", size = 110502, upload-time = "2026-04-22T15:11:25.044Z" }, ] [[package]] @@ -137,11 +143,11 @@ wheels = [ [[package]] name = "pathspec" -version = "0.12.1" +version = "1.1.1" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/ca/bc/f35b8446f4531a7cb215605d100cd88b7ac6f44ab3fc94870c120ab3adbf/pathspec-0.12.1.tar.gz", hash = "sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712", size = 51043, upload-time = "2023-12-10T22:30:45Z" } +sdist = { url = "https://files.pythonhosted.org/packages/5a/82/42f767fc1c1143d6fd36efb827202a2d997a375e160a71eb2888a925aac1/pathspec-1.1.1.tar.gz", hash = "sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a", size = 135180, upload-time = "2026-04-27T01:46:08.907Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/cc/20/ff623b09d963f88bfde16306a54e12ee5ea43e9b597108672ff3a408aad6/pathspec-0.12.1-py3-none-any.whl", hash = "sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08", size = 31191, upload-time = "2023-12-10T22:30:43.14Z" }, + { url = "https://files.pythonhosted.org/packages/f1/d9/7fb5aa316bc299258e68c73ba3bddbc499654a07f151cba08f6153988714/pathspec-1.1.1-py3-none-any.whl", hash = "sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189", size = 57328, upload-time = "2026-04-27T01:46:07.06Z" }, ] [[package]] @@ -189,6 +195,25 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/61/55/5eaf6c415f6ddb09b9b039278823a8e27fb81ea7a34ec80c6d9223b17f2e/pylint-3.3.2-py3-none-any.whl", hash = "sha256:77f068c287d49b8683cd7c6e624243c74f92890f767f106ffa1ddf3c0a54cb7a", size = 521873, upload-time = "2024-12-01T18:45:29.733Z" }, ] +[[package]] +name = "pytokens" +version = "0.4.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b6/34/b4e015b99031667a7b960f888889c5bd34ef585c85e1cb56a594b92836ac/pytokens-0.4.1.tar.gz", hash = "sha256:292052fe80923aae2260c073f822ceba21f3872ced9a68bb7953b348e561179a", size = 23015, upload-time = "2026-01-30T01:03:45.924Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8f/a7/b470f672e6fc5fee0a01d9e75005a0e617e162381974213a945fcd274843/pytokens-0.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:4a14d5f5fc78ce85e426aa159489e2d5961acf0e47575e08f35584009178e321", size = 160821, upload-time = "2026-01-30T01:03:19.684Z" }, + { url = "https://files.pythonhosted.org/packages/80/98/e83a36fe8d170c911f864bfded690d2542bfcfacb9c649d11a9e6eb9dc41/pytokens-0.4.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:97f50fd18543be72da51dd505e2ed20d2228c74e0464e4262e4899797803d7fa", size = 254263, upload-time = "2026-01-30T01:03:20.834Z" }, + { url = "https://files.pythonhosted.org/packages/0f/95/70d7041273890f9f97a24234c00b746e8da86df462620194cef1d411ddeb/pytokens-0.4.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:dc74c035f9bfca0255c1af77ddd2d6ae8419012805453e4b0e7513e17904545d", size = 268071, upload-time = "2026-01-30T01:03:21.888Z" }, + { url = "https://files.pythonhosted.org/packages/da/79/76e6d09ae19c99404656d7db9c35dfd20f2086f3eb6ecb496b5b31163bad/pytokens-0.4.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:f66a6bbe741bd431f6d741e617e0f39ec7257ca1f89089593479347cc4d13324", size = 271716, upload-time = "2026-01-30T01:03:23.633Z" }, + { url = "https://files.pythonhosted.org/packages/79/37/482e55fa1602e0a7ff012661d8c946bafdc05e480ea5a32f4f7e336d4aa9/pytokens-0.4.1-cp314-cp314-win_amd64.whl", hash = "sha256:b35d7e5ad269804f6697727702da3c517bb8a5228afa450ab0fa787732055fc9", size = 104539, upload-time = "2026-01-30T01:03:24.788Z" }, + { url = "https://files.pythonhosted.org/packages/30/e8/20e7db907c23f3d63b0be3b8a4fd1927f6da2395f5bcc7f72242bb963dfe/pytokens-0.4.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:8fcb9ba3709ff77e77f1c7022ff11d13553f3c30299a9fe246a166903e9091eb", size = 168474, upload-time = "2026-01-30T01:03:26.428Z" }, + { url = "https://files.pythonhosted.org/packages/d6/81/88a95ee9fafdd8f5f3452107748fd04c24930d500b9aba9738f3ade642cc/pytokens-0.4.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:79fc6b8699564e1f9b521582c35435f1bd32dd06822322ec44afdeba666d8cb3", size = 290473, upload-time = "2026-01-30T01:03:27.415Z" }, + { url = "https://files.pythonhosted.org/packages/cf/35/3aa899645e29b6375b4aed9f8d21df219e7c958c4c186b465e42ee0a06bf/pytokens-0.4.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d31b97b3de0f61571a124a00ffe9a81fb9939146c122c11060725bd5aea79975", size = 303485, upload-time = "2026-01-30T01:03:28.558Z" }, + { url = "https://files.pythonhosted.org/packages/52/a0/07907b6ff512674d9b201859f7d212298c44933633c946703a20c25e9d81/pytokens-0.4.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:967cf6e3fd4adf7de8fc73cd3043754ae79c36475c1c11d514fc72cf5490094a", size = 306698, upload-time = "2026-01-30T01:03:29.653Z" }, + { url = "https://files.pythonhosted.org/packages/39/2a/cbbf9250020a4a8dd53ba83a46c097b69e5eb49dd14e708f496f548c6612/pytokens-0.4.1-cp314-cp314t-win_amd64.whl", hash = "sha256:584c80c24b078eec1e227079d56dc22ff755e0ba8654d8383b2c549107528918", size = 116287, upload-time = "2026-01-30T01:03:30.912Z" }, + { url = "https://files.pythonhosted.org/packages/c6/78/397db326746f0a342855b81216ae1f0a32965deccfd7c830a2dbc66d2483/pytokens-0.4.1-py3-none-any.whl", hash = "sha256:26cef14744a8385f35d0e095dc8b3a7583f6c953c2e3d269c7f82484bf5ad2de", size = 13729, upload-time = "2026-01-30T01:03:45.029Z" }, +] + [[package]] name = "scripts" version = "0.16.3" From 66418ef0d44fa149e884506ff5db447c78666782 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Sun, 16 Aug 2026 16:26:55 -0400 Subject: [PATCH 065/371] Use pinned forms for `uvx` (#27798) Signed-off-by: William Woodruff --- README.md | 4 ++-- docs/installation.md | 4 ++-- playground/README.md | 2 +- pyproject.toml | 1 + scripts/conformance.py | 13 +++++-------- 5 files changed, 11 insertions(+), 13 deletions(-) diff --git a/README.md b/README.md index bf3f8704d2..cb9190655f 100644 --- a/README.md +++ b/README.md @@ -124,8 +124,8 @@ Ruff is available as [`ruff`](https://pypi.org/project/ruff/) on PyPI. Invoke Ruff directly with [`uvx`](https://docs.astral.sh/uv/): ```shell -uvx ruff check # Lint all files in the current directory. -uvx ruff format # Format all files in the current directory. +uvx ruff@0.16.3 check # Lint all files in the current directory. +uvx ruff@0.16.3 format # Format all files in the current directory. ``` Or install Ruff with `uv` (recommended), `pip`, or `pipx`: diff --git a/docs/installation.md b/docs/installation.md index 12dc0849a1..95f4a688f4 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -5,8 +5,8 @@ Ruff is available as [`ruff`](https://pypi.org/project/ruff/) on PyPI. Ruff can be invoked directly with [`uvx`](https://docs.astral.sh/uv/): ```shell -uvx ruff check # Lint all files in the current directory. -uvx ruff format # Format all files in the current directory. +uvx ruff@0.16.3 check # Lint all files in the current directory. +uvx ruff@0.16.3 format # Format all files in the current directory. ``` Or installed with `uv` (recommended), `pip`, or `pipx`: diff --git a/playground/README.md b/playground/README.md index 4a5ce1c309..eb96b258b1 100644 --- a/playground/README.md +++ b/playground/README.md @@ -12,7 +12,7 @@ module. To run the datastore, which is based on [Workers KV](https://developers.cloudflare.com/workers/runtime-apis/kv/), install the [Wrangler CLI](https://developers.cloudflare.com/workers/wrangler/install-and-update/), -then run `npx wrangler dev --local` from the `./playground/api` directory. Note that the datastore +then run `npx wrangler@4.118.0 dev --local` from the `./playground/api` directory. Note that the datastore is only required to generate shareable URLs for code snippets. The development datastore does not require Cloudflare authentication or login, but in turn only persists data locally. diff --git a/pyproject.toml b/pyproject.toml index 4f5317e0fc..3da925242f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -136,6 +136,7 @@ version_files = [ "pyproject.toml", # Might become unneeded once Markdown formatting is stabilized. "docs/formatter.md", + "docs/installation.md", "docs/integrations.md", "docs/tutorial.md", "crates/ruff/Cargo.toml", diff --git a/scripts/conformance.py b/scripts/conformance.py index 8692ca2c22..011217a18b 100644 --- a/scripts/conformance.py +++ b/scripts/conformance.py @@ -1,18 +1,15 @@ """ Run typing conformance tests and compare results between two ty versions. -By default, this script will use `uv` to run the latest version of ty -as the new version with `uvx ty@latest`. This requires `uv` to be installed -and available in the system PATH. +ty versions can be supplied as `uvx ty` or `uvx ty@version` +for a specific version. This requires `uv` to be installed +and available on the system PATH. If CONFORMANCE_SUITE_COMMIT is set, the hash will be used to create links to the corresponding line in the conformance repository for each diagnostic. Otherwise, it will default to `main'. Examples: - # Compare an older version of ty to latest - %(prog)s --old-ty uvx ty@0.0.1a35 - # Compare two specific ty versions %(prog)s --old-ty uvx ty@0.0.1a35 --new-ty uvx ty@0.0.7 @@ -1019,8 +1016,8 @@ def parse_args(): parser.add_argument( "--new-ty", nargs="+", - default=["uvx", "ty@latest"], - help="Command to run new version of ty (default: uvx ty@latest)", + help="Command to run new version of ty", + required=True, ) parser.add_argument( From 3fd603988d4edd06afb94e1613b9120271d43695 Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Mon, 17 Aug 2026 08:58:39 +0200 Subject: [PATCH 066/371] Fix `InvalidInstruction` on windows CPUs that do not support `POPCNT` (#27803) --- Cargo.toml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/Cargo.toml b/Cargo.toml index 9c854d1227..058db6551a 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -134,7 +134,10 @@ lsp-server = { version = "0.10.0" } lsp-types = { package = "gen-lsp-types", version = "0.11.0", features = ["url"] } matchit = { version = "0.9.0" } memchr = { version = "2.7.1" } -mimalloc = { version = "0.1.52" } +# mimalloc v3.3.2 crashes on Windows CPUs that do not support POPCNT. +# Fixed in mimalloc v3.4.0: https://github.com/microsoft/mimalloc/issues/1291 +# Keep v2 until an updated Rust crate includes the fix. +mimalloc = { version = "0.1.52", features = ["v2"] } natord = { version = "1.0.9" } notify = { version = "8.0.0" } ordermap = { version = "1.0.0" } From 885bf66501b5e94047828694a4c373deec896cc1 Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Mon, 17 Aug 2026 11:55:48 +0200 Subject: [PATCH 067/371] [ty] Cache `py.typed` contents (#27805) --- crates/ty_module_resolver/src/path.rs | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/crates/ty_module_resolver/src/path.rs b/crates/ty_module_resolver/src/path.rs index a6fa6f897b..1b55a3bbbe 100644 --- a/crates/ty_module_resolver/src/path.rs +++ b/crates/ty_module_resolver/src/path.rs @@ -7,6 +7,7 @@ use camino::{Utf8Path, Utf8PathBuf}; use ruff_db::files::{ File, FilePath, directory_listing, system_path_to_file, vendored_path_to_file, }; +use ruff_db::source::source_text; use ruff_db::system::{System, SystemPath, SystemPathBuf}; use ruff_db::vendored::{VendoredPath, VendoredPathBuf}; @@ -172,18 +173,25 @@ impl ModulePath { /// Get the `py.typed` info for this package (not considering parent packages) pub(super) fn py_typed(&self, resolver: &ResolverContext) -> PyTyped { - let Some(py_typed_contents) = self.to_system_path().and_then(|path| { + let Some(py_typed_file) = self.to_system_path().and_then(|path| { if !directory_contains_file(resolver.db, &path, &["py.typed"]) { return None; } let py_typed_path = path.join("py.typed"); - let py_typed_file = system_path_to_file(resolver.db, py_typed_path).ok()?; - // If we fail to read it let's say that's like it doesn't exist - // (right now the difference between Untyped and Full is academic) - py_typed_file.read_to_string(resolver.db).ok() + system_path_to_file(resolver.db, py_typed_path).ok() }) else { return PyTyped::Untyped; }; + + // Different module names revisit the same package. Share the tracked contents instead of + // reading its marker from disk again for every module resolution. + let py_typed_contents = source_text(resolver.db, py_typed_file); + // If we fail to read it let's say that's like it doesn't exist + // (right now the difference between Untyped and Full is academic) + if py_typed_contents.read_error().is_some() { + return PyTyped::Untyped; + } + // The python typing spec says to look for "partial\n" but in the wild we've seen: // // * PARTIAL\n From dc46d7f6344e06dbff5a7e3afe64282bbb397a98 Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Mon, 17 Aug 2026 15:17:45 +0200 Subject: [PATCH 068/371] [ty] Fix signature help in trailing whitespace (#27784) --- crates/ty_ide/src/signature_help.rs | 46 +++++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 6 deletions(-) diff --git a/crates/ty_ide/src/signature_help.rs b/crates/ty_ide/src/signature_help.rs index ea0725d8b1..e01aa74689 100644 --- a/crates/ty_ide/src/signature_help.rs +++ b/crates/ty_ide/src/signature_help.rs @@ -11,9 +11,12 @@ use crate::FxIndexMap; use crate::docstring::Docstring; use crate::goto::docstring_for_call_definition; use ruff_db::parsed::parsed_module; +use ruff_db::source::source_text; use ruff_python_ast::find_node::covering_node; use ruff_python_ast::token::TokenKind; use ruff_python_ast::{self as ast, AnyNodeRef}; +use ruff_python_trivia::PythonWhitespace; +use ruff_source_file::LineRanges; use ruff_text_size::{Ranged, TextSize}; use ty_python_core::ProgramFile; use ty_python_semantic::SemanticModel; @@ -82,7 +85,7 @@ pub fn signature_help<'db>( let parsed = parsed_module(db, file.python_file(db)).load(db); // Get the call expression at the given position. - let (call_expr, current_arg_index) = get_call_expr(&parsed, offset)?; + let (call_expr, current_arg_index) = get_call_expr(db, &parsed, offset)?; let model = SemanticModel::new(db, file); @@ -113,16 +116,22 @@ pub fn signature_help<'db>( /// Returns the innermost call expression that contains the specified offset /// and the index of the argument that the offset maps to. -fn get_call_expr( - parsed: &ruff_db::parsed::ParsedModuleRef, +fn get_call_expr<'ast>( + db: &dyn Db, + parsed: &'ast ruff_db::parsed::ParsedModuleRef, offset: TextSize, -) -> Option<(&ast::ExprCall, usize)> { +) -> Option<(&'ast ast::ExprCall, usize)> { let root_node: AnyNodeRef = parsed.syntax().into(); + let source = source_text(db, parsed.module().file()); + let line_range = source.line_range(offset); + let line = &source[line_range]; + let line_end = line_range.start() + TextSize::of(line.trim_whitespace_end()); + let token_offset = offset.min(line_end); // Find the token under the cursor and use its offset to find the node let token = parsed .tokens() - .at_offset(offset) + .at_offset(token_offset) .max_by_key(|token| match token.kind() { TokenKind::Name | TokenKind::String @@ -147,7 +156,9 @@ fn get_call_expr( } // Close the signature help if the cursor is at the closing parenthesis - if token.kind() == TokenKind::Rpar && node.end() == token.end() && offset == token.end() + if token.kind() == TokenKind::Rpar + && node.end() == token.end() + && token_offset == token.end() { return false; } @@ -1284,6 +1295,29 @@ def ab(a: int, *, c: int): assert_eq!(result.signatures[0].active_parameter, Some(1)); } + #[test] + fn signature_help_in_trailing_whitespace() { + for whitespace in [" ", "\t", "\u{000c}"] { + let source = format!( + "def func(first: int, second: str) -> None: ...\n\nfunc(1,{whitespace}" + ); + let test = cursor_test(&source); + + let result = test.signature_help().expect("Should have signature help"); + assert_eq!(result.signatures[0].active_parameter, Some(1)); + } + } + + #[test] + fn signature_help_in_trailing_whitespace_before_newline() { + let test = cursor_test( + "def func(first: int, second: str) -> None: ...\n\nfunc(1, \n \"value\")", + ); + + let result = test.signature_help().expect("Should have signature help"); + assert_eq!(result.signatures[0].active_parameter, Some(1)); + } + #[test] fn signature_help_after_closing_paren_at_end_of_file() { let test = cursor_test( From 51c476a791541a20440fb16c97bbef056ab545b7 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Mon, 17 Aug 2026 18:22:02 +0100 Subject: [PATCH 069/371] [ty] Disable GitHub CLI telemetry in ecosystem skills (#27809) --- .agents/skills/minimizing-ty-ecosystem-changes/SKILL.md | 8 +++++--- .agents/skills/summarise-ecosystem-results/SKILL.md | 6 +++++- .../references/evidence-acquisition.md | 8 ++++---- .../references/subagent-handoff.md | 1 + 4 files changed, 15 insertions(+), 8 deletions(-) diff --git a/.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md b/.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md index ea77b4fe73..cb61ef9122 100644 --- a/.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md +++ b/.agents/skills/minimizing-ty-ecosystem-changes/SKILL.md @@ -15,6 +15,8 @@ description: Use when a user says "minimize this ty ecosystem change", "reproduc Start each investigation from fresh artifacts. Do not trust retained memories, previous minimizations, current upstream project state, or the helper script's default lockfile. +Prefix every direct or indirect `gh` invocation with `GH_TELEMETRY=false`; each Codex tool call may start a new shell. + ## Collect Exact-Run Metadata If the primary agent supplied an immutable `TY_ECOSYSTEM_RUN_METADATA` manifest, verify that its run ID and attempt match the frozen report and that it contains each assigned project. All subagents reuse the same read-only manifest; never modify it or generate another shared manifest. @@ -22,7 +24,7 @@ If the primary agent supplied an immutable `TY_ECOSYSTEM_RUN_METADATA` manifest, Otherwise, run the bundled helper once with the Actions run ID or URL, matching attempt, and every affected mypy-primer project name: ```bash -scripts/collect_ty_ecosystem_run_metadata.py \ +GH_TELEMETRY=false uv run --script scripts/collect_ty_ecosystem_run_metadata.py \ ... \ --attempt \ --output target/ty-ecosystem-run.json @@ -45,7 +47,7 @@ set -euo pipefail test -z "$(git status --short)" || { git status --short; exit 1; } original_ref="$(git symbolic-ref --quiet --short HEAD || git rev-parse HEAD)" -git fetch https://github.com/astral-sh/ruff.git +GH_TELEMETRY=false git fetch https://github.com/astral-sh/ruff.git mkdir -p target/ty-ecosystem-bins trap 'git checkout "$original_ref"' EXIT @@ -70,7 +72,7 @@ After restoring the original ref, inspect vendored definitions and Rust implemen Create a unique temporary directory for each project and use its absolute path. Read its Python version and the pinned mypy-primer revision from the shared manifest. Obtain the project revision from the `/blob//` component of the original diagnostic's source permalink, and check that links for the same project agree. If no diagnostic permalink exists, inspect the matching diagnostics shard or Actions logs; if the exact revision cannot be recovered, explicitly report that limitation. Then bypass the adjacent script lockfile: ```bash -uv run \ +GH_TELEMETRY=false uv run \ --python \ --with "mypy-primer @ git+https://github.com/hauntsaninja/mypy_primer@" \ --no-project \ diff --git a/.agents/skills/summarise-ecosystem-results/SKILL.md b/.agents/skills/summarise-ecosystem-results/SKILL.md index 1136e97397..a6e52e3b2b 100644 --- a/.agents/skills/summarise-ecosystem-results/SKILL.md +++ b/.agents/skills/summarise-ecosystem-results/SKILL.md @@ -13,6 +13,10 @@ description: Use when a user says "summarise ecosystem results", "summarize this 4. Lead the report with new or meaningfully changed project failures, including intermittent severe failures, then cover stable diagnostic changes and fully minimized examples. 5. Keep execution, audit, and traceability bookkeeping out of the report. +## GitHub CLI Telemetry + +Prefix every direct or indirect `gh` invocation with `GH_TELEMETRY=false`, including `GH_TELEMETRY=false uv run --script scripts/collect_ty_ecosystem_run_metadata.py ...`. Require the same of subagents. Codex tool calls may start separate shells, so an `export` in an earlier call is insufficient. + ## Deliverable Create `PR__ECOSYSTEM_SUMMARY.md` at the repository root by adapting [assets/report-template.md](assets/report-template.md). The finished artifact must be GitHub-flavored Markdown suitable for a GitHub comment, with each prose paragraph and list item on one source line. @@ -35,7 +39,7 @@ If summarising an ecosystem report is the only thing you're asked to do in a Cod 4. **Minimize to completion with provenance.** For each distinct source-attributable behavior change, follow the complete advanced-minimization workflow until an exhaustive pass finds no further reduction. Derive the reproducer from a cited ecosystem entry through a verified reduction chain; never replace that entry with an independently invented example demonstrating superficially similar behavior. Before accepting a reproducer, attempt to remove every import, inline every third-party definition, and inline relevant standard-library definitions. Retain a third-party import only when identified ty behavior depends on that library's identity or third-party search-path classification and neither removing the import nor inlining its definitions preserves the underlying behavior. If a genuine external blocker prevents completion, report that blocker to the user and identify the task as incomplete. Do not silently substitute an unminimized excerpt or present a partially minimized report as finished. 5. **Group by cause.** Group entries only when the same base-to-PR behavior, underlying trigger, explanation, and reproducer account for every entry. Identical diagnostic text or displayed `@Todo` types do not establish equivalence. 6. **Find existing ty issues.** When a diagnostic change exposes a pre-existing shortcoming in ty, search the `astral-sh/ty` issue tracker for the precise underlying behavior. Link matching issues directly from the relevant report section; do not mistake incorrect or incomplete third-party annotations for ty shortcomings. -7. **Write and verify.** Fill the report template and verify that every source-attributable behavior change has a fully minimized, provenance-preserving reproducer. Check every change number, link, diagnostic, retained import, reproducer's source provenance, and causal fingerprint when required. Verify that every retained third-party import is essential to identified ty behavior that depends on that library's identity or third-party search-path classification, that no avoidable standard-library import remains, and that no source-attributable section contains an unminimized excerpt. Then run `uv run --only-group dev --locked prek run --files PR__ECOSYSTEM_SUMMARY.md`. Present the Markdown file as the finished product only after these checks pass. +7. **Write and verify.** Fill the report template and verify that every source-attributable behavior change has a fully minimized, provenance-preserving reproducer. Check every change number, link, diagnostic, retained import, reproducer's source provenance, and causal fingerprint when required. Verify that every retained third-party import is essential to identified ty behavior that depends on that library's identity or third-party search-path classification, that no avoidable standard-library import remains, and that no source-attributable section contains an unminimized excerpt. Then run `GH_TELEMETRY=false uv run --only-group dev --locked prek run --files PR__ECOSYSTEM_SUMMARY.md`. Present the Markdown file as the finished product only after these checks pass. ## Parallel execution diff --git a/.agents/skills/summarise-ecosystem-results/references/evidence-acquisition.md b/.agents/skills/summarise-ecosystem-results/references/evidence-acquisition.md index f68ae78d53..c5ca836eda 100644 --- a/.agents/skills/summarise-ecosystem-results/references/evidence-acquisition.md +++ b/.agents/skills/summarise-ecosystem-results/references/evidence-acquisition.md @@ -12,11 +12,11 @@ set -euo pipefail snapshot_dir="$(mktemp -d "${TMPDIR:-/tmp}/ty-ecosystem-report.XXXXXX")" ecosystem_comment_id="" if [[ -n "$ecosystem_comment_id" ]]; then - gh api "repos/astral-sh/ruff/issues/comments/$ecosystem_comment_id" > "$snapshot_dir/comment.json" + GH_TELEMETRY=false gh api "repos/astral-sh/ruff/issues/comments/$ecosystem_comment_id" > "$snapshot_dir/comment.json" fi -gh run view --repo astral-sh/ruff --attempt \ +GH_TELEMETRY=false gh run view --repo astral-sh/ruff --attempt \ --json attempt,headSha,jobs,startedAt,updatedAt,url > "$snapshot_dir/run.json" -gh api --paginate --slurp \ +GH_TELEMETRY=false gh api --paginate --slurp \ "repos/astral-sh/ruff/actions/runs//artifacts?per_page=100" | jq '{artifacts: [.[].artifacts[]]}' > "$snapshot_dir/artifacts.json" printf 'TY_ECOSYSTEM_SNAPSHOT_DIR=%s\n' "$snapshot_dir" @@ -32,7 +32,7 @@ download_validated_artifact() { local destination="$2" mkdir -p "$destination" - gh api "repos/astral-sh/ruff/actions/artifacts/$artifact_id/zip" \ + GH_TELEMETRY=false gh api "repos/astral-sh/ruff/actions/artifacts/$artifact_id/zip" \ > "$snapshot_dir/artifact-$artifact_id.zip" unzip -q "$snapshot_dir/artifact-$artifact_id.zip" -d "$destination" } diff --git a/.agents/skills/summarise-ecosystem-results/references/subagent-handoff.md b/.agents/skills/summarise-ecosystem-results/references/subagent-handoff.md index aa945c91f2..8496df1de4 100644 --- a/.agents/skills/summarise-ecosystem-results/references/subagent-handoff.md +++ b/.agents/skills/summarise-ecosystem-results/references/subagent-handoff.md @@ -20,6 +20,7 @@ Give each subagent: - The paths to the frozen HTML report, optional matching structured JSON, and available diagnostics shards, plus the frozen comment path when available and the selected Actions run and attempt; use these captured inputs instead of refetching live evidence. - The exact assigned entries from the structured JSON when available, or from the frozen HTML report otherwise; distinguish source-attributable changes from outcomes without recoverable source, and provide the reported merge-base and PR run counts for intermittent severe failures. - The immutable `TY_ECOSYSTEM_RUN_METADATA`, `TY_ECOSYSTEM_BASE_BINARY`, `TY_ECOSYSTEM_PR_BINARY`, and `TY_ECOSYSTEM_CONFIG_HOME` absolute paths. +- The instruction to prefix every direct or indirect `gh` invocation with `GH_TELEMETRY=false`. - For assignments requiring reproduction, the instruction to use the `minimizing-ty-ecosystem-changes` skill with the shared manifest, copied profiling binaries, installed configuration, and a unique temporary directory; never generate another manifest. - For source-attributable assignments, the instruction to produce a fully minimized, provenance-preserving reproducer by exhausting the complete advanced-minimization workflow, including third-party dependency inlining, standard-library inlining, and an audit of every remaining import. - The instruction to inspect vendored definitions and Rust implementations with `git -C show :`, using the analyzed revisions from the immutable manifest rather than the restored working tree. From ad6afd5a7dc3703239c5892b69d1d484362d7107 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Mon, 17 Aug 2026 11:04:59 -0700 Subject: [PATCH 070/371] [ty] Prevent unrelated quantified constraints from destabilizing recursive queries (#27737) ## Summary This fixes a "too many iterations" panic I saw when fixing astral-sh/ty#4246. This panic reproduces on main already with a slightly different example, so fix it first here. Irrelevant quantified-away constraints can pollute a source-ordering sidecar and cause fixpoint to never converge. - Drop quantified-away constraints from persisted source ordering when their complete support is unrelated to the remaining live constraint set. This prevents fresh type variables from keeping recursive Salsa queries from reaching a fixed point. - Preserve related quantified constraints and conservatively retain entries whenever either support is incomplete. - Stop collecting type-variable declaration bounds, value constraints, and defaults as constraint support, so unrelated eager metadata does not add dependencies and lazy metadata does not incorrectly mark support incomplete. Related to astral-sh/ty#4246 and #27732. ## Test plan - Add an overloaded generic-protocol classmethod mdtest that reproduces `too many cycle iterations` on main and verifies both receiver-bound overload signatures. - Add separate protocol receiver-binding mdtests for a bounded legacy type variable and a defaulted PEP 695 method type variable. - Cover removal of unrelated quantified constraints and preservation of ordering for related quantified constraints with focused constraint-set unit tests. - Cover eager type-variable defaults and lazy declaration bounds, value constraints, and defaults while preserving type variables actually present in structural constraint bounds. --- .../resources/mdtest/protocols.md | 82 +++++++++ .../src/types/constraints.rs | 163 +++++++++++++++++- 2 files changed, 238 insertions(+), 7 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/protocols.md b/crates/ty_python_semantic/resources/mdtest/protocols.md index e57b740e07..1c3b9aef2a 100644 --- a/crates/ty_python_semantic/resources/mdtest/protocols.md +++ b/crates/ty_python_semantic/resources/mdtest/protocols.md @@ -4044,6 +4044,88 @@ class Incompatible(SupportsMethod[T_co]): raise NotImplementedError ``` +## Recursive protocol receiver binding with an overloaded class method + +Accessing an overloaded class method on a generic protocol can recursively bind the protocol's +receiver. The receiver-binding query must converge and preserve both overload signatures. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Protocol, overload + +class Container[T](Protocol): + def value(self) -> T: ... + @overload + @classmethod + def from_value[Self, S](cls: type[Self], value: S) -> object: ... + @overload + @classmethod + def from_value[Self, S](cls: type[Self], value: S, flag: bool) -> object: ... + +reveal_type(Container.from_value) # revealed: Overload[[S](value: S) -> object, [S](value: S, flag: bool) -> object] +``` + +## Recursive protocol receiver binding with a bounded type variable + +A class method can recursively bind a protocol receiver through a type variable with a declared +upper bound. Its declared bound is metadata, not another part of the receiver constraint. + +```toml +[environment] +python-version = "3.11" +``` + +```py +from typing import Protocol, TypeVar + +T = TypeVar("T") +S = TypeVar("S", bound=object) + +class Box(Protocol[T]): + value: T + + @classmethod + def first(cls: type[S]) -> S: + return cls() + + def second(self) -> S: ... + @classmethod + def last(cls: type[S]) -> object: ... + +reveal_type(Box.first()) # revealed: Box[Unknown] +``` + +## Recursive protocol receiver binding with a defaulted type variable + +A method type variable with a declared default can also appear while recursively binding a protocol +receiver. Its default is declaration metadata, not a type-variable occurrence in the constraint. + +```toml +[environment] +python-version = "3.13" +``` + +```py +from typing import Protocol + +class Box[T](Protocol): + value: T + + @classmethod + def first[S = int](cls: type[S]) -> S: + return cls() + + def second[S = int](self) -> S: ... + @classmethod + def last[S = int](cls: type[S]) -> object: ... + +reveal_type(Box.first()) # revealed: Box[Unknown] +``` + ## Subtyping of protocols with generic method members Protocol method members can be generic. They can have generic contexts scoped to the class: diff --git a/crates/ty_python_semantic/src/types/constraints.rs b/crates/ty_python_semantic/src/types/constraints.rs index b89052ae48..a14ca6cc2a 100644 --- a/crates/ty_python_semantic/src/types/constraints.rs +++ b/crates/ty_python_semantic/src/types/constraints.rs @@ -107,7 +107,7 @@ use ty_static::EnvVars; use crate::types::class::GenericAlias; use crate::types::constraints::support::{Support, SupportId}; -use crate::types::typevar::{BoundTypeVarIdentity, TypeVarSet}; +use crate::types::typevar::{BoundTypeVarIdentity, TypeVarInstance, TypeVarSet}; use crate::types::variance::VarianceInferable; use crate::types::visitor::{ TypeCollector, TypeKind, TypeVisitor, any_over_type, walk_non_atomic_type, @@ -1107,10 +1107,10 @@ impl<'db> ConstraintSetBuilder<'db> { .expect("non-terminal BDD should have source_order"); // Combining constraint sets can allocate a new source-order tree even when the BDD is - // unchanged. Preserve each constraint's first source position, but rebuild the persisted - // sidecar densely so redundant combinations cannot affect its IDs or owned-set equality. - // Unlike node and constraint IDs, source-order IDs are not embedded in the BDD, so the - // sidecar can be rebuilt without remapping the BDD. + // unchanged. Preserve each relevant constraint's first source position, but rebuild the + // persisted sidecar densely so redundant combinations cannot affect its IDs or owned-set + // equality. Unlike node and constraint IDs, source-order IDs are not embedded in the BDD, + // so the sidecar can be rebuilt without remapping the BDD. let mut storage = self.storage.into_inner(); let source_constraints = storage.calculate_source_orders(Some(source_order)); @@ -1139,9 +1139,23 @@ impl<'db> ConstraintSetBuilder<'db> { let mut source_orders: IndexVec = IndexVec::with_capacity(source_constraints.len().saturating_mul(2).saturating_sub(1)); + let live_support = storage.node_support(node); let source_order = source_constraints .into_iter() .fold(None, |left, source_constraint| { + // Quantified-away constraints can still determine the order of related live + // solutions. An unrelated constraint cannot, and keeping its fresh type variables + // in a cached value can prevent recursive Salsa queries from reaching a fixed + // point. Incomplete supports may hide a relationship, so preserve those entries. + let constraint_support = storage.constraint_support(source_constraint); + if !used_constraints[source_constraint.index()] + && let Some(live_support) = live_support + && live_support.is_complete() + && constraint_support.is_complete() + && !constraint_support.overlaps_with(live_support) + { + return left; + } used_constraints.set(source_constraint.index(), true); let right = source_orders.push(SourceOrder::Constraint(source_constraint)); @@ -1276,6 +1290,11 @@ impl<'db> ConstraintSetStorage<'db> { self.support.borrow_mut().mark_incomplete(); } + fn visit_type_var_type(&self, _db: &'db dyn Db, _typevar: TypeVarInstance<'db>) { + // Declaration bounds, constraints, and defaults are not occurrences in the + // constraint itself and must not contribute to its support. + } + fn visit_generic_alias_type(&self, db: &'db dyn Db, alias: GenericAlias<'db>) { for ty in alias.specialization(db).types(db) { self.visit_type(db, *ty); @@ -6992,6 +7011,7 @@ mod tests { use crate::db::tests::{TestDb, setup_db}; use crate::types::generics::ApplySpecialization; + use crate::types::typevar::{TypeVarBoundOrConstraintsEvaluation, TypeVarDefaultEvaluation}; use crate::types::{BoundTypeVarInstance, KnownClass, SubclassOfType, TypeVarVariance}; use ruff_python_ast::name::Name; @@ -7049,6 +7069,94 @@ mod tests { ); } + #[test] + fn constraint_support_ignores_typevar_declaration_defaults() { + let db = setup_db(); + let db = &db; + let env = db.program_environment(); + let t = create_typevar(db, "T"); + let metadata = create_typevar(db, "Metadata"); + let u = create_typevar(db, "U"); + let declaration = TypeVarInstance::new( + db, + u.typevar(db).identity(db), + None, + Some(TypeVarVariance::Invariant), + Some(TypeVarDefaultEvaluation::Eager(Type::TypeVar(metadata))), + ); + let u = BoundTypeVarInstance::new( + db, + declaration, + u.binding_context(db), + u.paramspec_attr(db), + u.freshness(db), + ); + let actual_bound = KnownClass::List.to_specialized_instance(db, &env, &[Type::TypeVar(u)]); + let mut storage = ConstraintSetStorage::default(); + let support = storage.intern_constraint_typevars( + db, + &env, + t, + ConstraintBounds::new(None, Some(actual_bound)), + ); + let mentioned = support + .iter() + .map(|typevar| storage.typevar_data(typevar)) + .collect::>(); + + assert_eq!(mentioned, vec![t.identity(db), u.identity(db)]); + assert!(support.is_complete()); + } + + #[test] + fn constraint_support_is_complete_for_lazy_typevar_declaration_metadata() { + let db = setup_db(); + let db = &db; + let env = db.program_environment(); + let t = create_typevar(db, "T"); + let u = create_typevar(db, "U"); + for (bound_or_constraints, default) in [ + ( + Some(TypeVarBoundOrConstraintsEvaluation::LazyUpperBound), + None, + ), + ( + Some(TypeVarBoundOrConstraintsEvaluation::LazyConstraints), + None, + ), + (None, Some(TypeVarDefaultEvaluation::Lazy)), + ] { + let declaration = TypeVarInstance::new( + db, + u.typevar(db).identity(db), + bound_or_constraints, + Some(TypeVarVariance::Invariant), + default, + ); + let u = BoundTypeVarInstance::new( + db, + declaration, + u.binding_context(db), + u.paramspec_attr(db), + u.freshness(db), + ); + let mut storage = ConstraintSetStorage::default(); + let support = storage.intern_constraint_typevars( + db, + &env, + t, + ConstraintBounds::new(None, Some(Type::TypeVar(u))), + ); + let mentioned = support + .iter() + .map(|typevar| storage.typevar_data(typevar)) + .collect::>(); + + assert_eq!(mentioned, vec![t.identity(db), u.identity(db)]); + assert!(support.is_complete()); + } + } + #[test] fn type_mapping_evaluates_mapped_subjects() { // ((T = int) ∧ ¬(T = str))[T ↦ int] = true @@ -8666,7 +8774,7 @@ mod tests { } #[test] - fn owned_constraint_set_type_walk_excludes_quantified_constraints() { + fn owned_constraint_set_discards_unrelated_quantified_constraints() { let db = setup_db(); let db = &db; let env = db.program_environment(); @@ -8693,7 +8801,48 @@ mod tests { ); assert_eq!( owned.inner.as_ref().map(|inner| inner.source_orders.len()), - Some(3), + Some(1), + ); + } + + #[test] + fn owned_constraint_set_preserves_related_quantified_constraint_order() { + let db = setup_db(); + let db = &db; + let env = db.program_environment(); + let t = create_typevar(db, "T"); + let u = create_typevar(db, "U"); + let v = create_typevar(db, "V"); + + let owned = ConstraintSetBuilder::new().into_owned(|builder| { + let u_t = ConstraintSet::constrain_typevar_with_bounds( + db, + &env, + builder, + u, + None, + Some(Type::TypeVar(t)), + ); + let t_v = ConstraintSet::constrain_typevar( + db, + &env, + builder, + t, + Type::TypeVar(v), + Type::TypeVar(v), + ); + + u_t.and(db, builder, || t_v).reduce_inferable( + db, + &env, + builder, + TypeVarSet::from_typevars(db, [t]), + ) + }); + + assert_eq!( + owned.inner.as_ref().map(|inner| inner.source_orders.len()), + Some(5), ); } From d130926f9a0f9d59e7bc7dc2390749c92a9fe0f0 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Mon, 17 Aug 2026 19:19:19 +0100 Subject: [PATCH 071/371] [ty] Update ecosystem-analyzer and mypy-primer pins (#27811) --- .github/workflows/ty-ecosystem-analyzer.yaml | 2 +- .github/workflows/ty-ecosystem-report.yaml | 2 +- scripts/setup_primer_project.py | 2 +- scripts/setup_primer_project.py.lock | 4 ++-- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/.github/workflows/ty-ecosystem-analyzer.yaml b/.github/workflows/ty-ecosystem-analyzer.yaml index 13a9e53125..a6d0d4a029 100644 --- a/.github/workflows/ty-ecosystem-analyzer.yaml +++ b/.github/workflows/ty-ecosystem-analyzer.yaml @@ -44,7 +44,7 @@ env: CARGO_PROFILE_PROFILING_DEBUG: line-tables-only # TODO: Update the mypy-primer revision in scripts/setup_primer_project.py # and regenerate its lockfile when updating ecosystem-analyzer. - ECOSYSTEM_ANALYZER_COMMIT: 2b409ad30445f16b863919e0799b438c6b04c645 + ECOSYSTEM_ANALYZER_COMMIT: a602846d9c13d7c6d7268dff728442f68c47d923 jobs: build-ty: diff --git a/.github/workflows/ty-ecosystem-report.yaml b/.github/workflows/ty-ecosystem-report.yaml index b37d675328..651e478783 100644 --- a/.github/workflows/ty-ecosystem-report.yaml +++ b/.github/workflows/ty-ecosystem-report.yaml @@ -20,7 +20,7 @@ env: RUST_BACKTRACE: 1 # Line-tables-only debug info: faster builds, backtraces still work. CARGO_PROFILE_PROFILING_DEBUG: line-tables-only - ECOSYSTEM_ANALYZER_COMMIT: 2b409ad30445f16b863919e0799b438c6b04c645 + ECOSYSTEM_ANALYZER_COMMIT: a602846d9c13d7c6d7268dff728442f68c47d923 jobs: ty-ecosystem-report: diff --git a/scripts/setup_primer_project.py b/scripts/setup_primer_project.py index db5be5d8fd..444f612977 100644 --- a/scripts/setup_primer_project.py +++ b/scripts/setup_primer_project.py @@ -14,7 +14,7 @@ # [tool.uv.sources] # # Keep this revision and the script's lockfile in sync with ecosystem-analyzer's # # mypy-primer pin so memory reports and ecosystem jobs use the same project definitions. -# mypy-primer = { git = "https://github.com/hauntsaninja/mypy_primer", rev = "6d6eebd8d37c9b8931381e79aa99808d9378c988" } +# mypy-primer = { git = "https://github.com/hauntsaninja/mypy_primer", rev = "db37f8a384c45c02fc52544fd819f979d66e174a" } # /// """Clone a mypy-primer project and set up a virtualenv with its dependencies installed. diff --git a/scripts/setup_primer_project.py.lock b/scripts/setup_primer_project.py.lock index feea69058f..ccb767b3c3 100644 --- a/scripts/setup_primer_project.py.lock +++ b/scripts/setup_primer_project.py.lock @@ -7,9 +7,9 @@ exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for exclude-newer-span = "P7D" [manifest] -requirements = [{ name = "mypy-primer", git = "https://github.com/hauntsaninja/mypy_primer?rev=6d6eebd8d37c9b8931381e79aa99808d9378c988" }] +requirements = [{ name = "mypy-primer", git = "https://github.com/hauntsaninja/mypy_primer?rev=db37f8a384c45c02fc52544fd819f979d66e174a" }] [[package]] name = "mypy-primer" version = "0.1.0" -source = { git = "https://github.com/hauntsaninja/mypy_primer?rev=6d6eebd8d37c9b8931381e79aa99808d9378c988#6d6eebd8d37c9b8931381e79aa99808d9378c988" } +source = { git = "https://github.com/hauntsaninja/mypy_primer?rev=db37f8a384c45c02fc52544fd819f979d66e174a#db37f8a384c45c02fc52544fd819f979d66e174a" } From 6b450f08273c42a85eca6554a28e6a32f2ddd752 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Mon, 17 Aug 2026 11:49:24 -0700 Subject: [PATCH 072/371] [ty] Preserve explicit globals after conditional rebinding (#27786) A nested class or comprehension inside a function could infer only the conditionally reassigned value of a `global`, dropping its existing module-level value. This let ty accept code that could fail at runtime, and could cause false-positive `possibly-unresolved-reference` errors. Resolve forwarded global snapshots against real module-level bindings and declarations before falling back to implicit globals or builtins. Exclude synthetic bindings from nested `global` assignments so existing implicit-global and builtin behavior is preserved. Closes https://github.com/astral-sh/ty/issues/4273. ## Test plan Added scope mdtests covering: - A nested class reading a global with an existing module-level binding after conditional reassignment. - A nested class reading a global that has only a module-level type declaration. - An eager comprehension reading a conditionally reassigned global. Existing neighboring mdtests also cover fallback to implicit globals and builtins. --- .../resources/mdtest/scopes/global.md | 63 +++++++++++++++++++ crates/ty_python_semantic/src/place_load.rs | 18 +++++- 2 files changed, 80 insertions(+), 1 deletion(-) diff --git a/crates/ty_python_semantic/resources/mdtest/scopes/global.md b/crates/ty_python_semantic/resources/mdtest/scopes/global.md index 3e025cf78f..4ba7169258 100644 --- a/crates/ty_python_semantic/resources/mdtest/scopes/global.md +++ b/crates/ty_python_semantic/resources/mdtest/scopes/global.md @@ -299,6 +299,53 @@ def factory(): reveal_type(x) # revealed: Literal[1] ``` +An explicit module-level binding remains visible when the enclosing function only conditionally +rebinds that global: + +```py +value = 0 + +def conditional_global_factory(flag: bool): + global value + if flag: + value = "updated" + + class Nested: + reveal_type(value) # revealed: Literal["updated", 0] +``` + +If the condition is known to be false, the nested class should see only the original module-level +binding and should not report an unresolved reference: + +```py +from typing import Literal + +known_false_value = 0 + +def known_false_global_factory(flag: Literal[False]): + global known_false_value + if flag: + known_false_value = "updated" + + class Nested: + reveal_type(known_false_value) # revealed: Literal[0] +``` + +A module-level declaration also remains visible when the enclosing function only conditionally binds +that global: + +```py +declared_value: int + +def conditional_declared_global_factory(flag: bool): + global declared_value + if flag: + declared_value = 1 + + class Nested: + reveal_type(declared_value) # revealed: int +``` + If the rebinding is conditional, an unbound enclosing snapshot continues to the implicit global: ```py @@ -323,6 +370,22 @@ def conditional_builtin_factory(flag: bool): reveal_type(len) # revealed: Literal[1] | (def len(obj: Sized, /) -> int) ``` +## Comprehension after global rebinding + +A comprehension is also an eager nested scope, so it should see both the original module-level +binding and a conditional global rebinding: + +```py +value = 0 + +def factory(flag: bool): + global value + if flag: + value = "updated" + + [reveal_type(value) for _ in [0]] # revealed: Literal["updated", 0] +``` + ## References to variables before they are defined within a class scope are considered global If we try to access a variable in a class before it has been defined, the lookup will fall back to diff --git a/crates/ty_python_semantic/src/place_load.rs b/crates/ty_python_semantic/src/place_load.rs index 37c8d0591f..8d9cc2e3b3 100644 --- a/crates/ty_python_semantic/src/place_load.rs +++ b/crates/ty_python_semantic/src/place_load.rs @@ -248,7 +248,23 @@ impl<'db> Iterator for PlaceLoadResolution<'db, '_> { bindings, enclosing_scope, } = snapshot; - self.next_node = Some(PlaceLoadResolutionNode::ImplicitGlobalSource); + let global_place_table = self.context.index.place_table(FileScopeId::global()); + let has_explicit_global = self + .loaded_symbol_name() + .and_then(|name| global_place_table.symbol_id(name)) + .is_some_and(|symbol_id| { + let symbol = global_place_table.symbol(symbol_id); + symbol.is_bound() || symbol.is_declared() + }); + + // Nested global assignments create synthetic module bindings even when the + // module never defines the name itself. Do not let those bindings hide an + // implicit global or builtin when the forwarded assignment did not run. + self.next_node = Some(if has_explicit_global { + PlaceLoadResolutionNode::ExplicitGlobalSource(PlaceLoadSourceRole::Ordinary) + } else { + PlaceLoadResolutionNode::ImplicitGlobalSource + }); let source = self.constraints.source( PlaceLoadSourceKind::Bindings(bindings), From 4f5cc6e4df4de31694aadf3fede81ea255e91563 Mon Sep 17 00:00:00 2001 From: Ibraheem Ahmed Date: Mon, 17 Aug 2026 18:14:58 -0400 Subject: [PATCH 073/371] [ty] Preserve static upper bounds of gradual solutions (#27664) We currently ignore concrete upper bounds when inferring a gradual solution. We should instead be intersecting the gradual type with its upper bound, e.g., ```py from typing import Any, Callable def infer[T](lower: T, upper: Callable[[T], None]) -> T: return lower def _(x: Any, upper: Callable[[int], None]): reveal_type(infer(x, upper)) # revealed: int & Any ``` --- .../resources/mdtest/bidirectional.md | 36 ++++ .../mdtest/generics/pep695/functions.md | 192 ++++++++++++++++++ .../ty_python_semantic/src/types/call/bind.rs | 7 +- .../src/types/constraints.rs | 100 +++++++++ .../ty_python_semantic/src/types/generics.rs | 2 +- .../src/types/infer/builder.rs | 2 +- 6 files changed, 335 insertions(+), 4 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/bidirectional.md b/crates/ty_python_semantic/resources/mdtest/bidirectional.md index 541568ba05..2f220cecff 100644 --- a/crates/ty_python_semantic/resources/mdtest/bidirectional.md +++ b/crates/ty_python_semantic/resources/mdtest/bidirectional.md @@ -1164,6 +1164,25 @@ def mean(data: DataFrame) -> float: x23: Mapping[Hashable, AggregateSpec] = {"col1": ["sum", mean], "col2": mean} ``` +## Recursive aliases remain stable in invariant collection contexts + +An invariant collection context can infer the same recursive type as both bounds. Because recursive +inference introduces `Divergent`, intersecting those bounds should not discard any element of the +union. + +```py +from collections.abc import MutableMapping, MutableSequence +from typing import TypeAlias, TypedDict + +class Leaf(TypedDict, total=False): + path: str + +RecursiveValue: TypeAlias = int | Leaf | MutableSequence["RecursiveValue | None"] | MutableMapping[str, "RecursiveValue | None"] +RecursiveMapping: TypeAlias = MutableMapping[str, RecursiveValue | None] + +recursive: RecursiveMapping = {} +``` + ## Implicit generic class specialization Callable type context is also used to inform the implicit specialization of a generic class: @@ -2099,6 +2118,23 @@ def _(callback: TakesInt) -> None: reveal_type(x2) # revealed: str ``` +A structural type context can infer a gradual lower bound and a static upper bound before dictionary +values contribute their constraints. The preliminary solution should retain the gradual lower bound. + +```py +T_co = TypeVar("T_co", covariant=True) + +class DictLike(Protocol[T_co]): + def __getitem__(self, key: str, /) -> T_co: ... + def __setitem__(self, key: str, value: Any, /) -> None: ... + +class Command: ... + +def _(command: Any): + # revealed: dict[str, Any] + mapping: DictLike[type[Command]] = reveal_type({"command": command}) +``` + Note that long chains of callables with constraint dependencies in reverse source-order may require multiple fixpoint iterations. diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md index f6d917c67f..924622934c 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md @@ -1346,6 +1346,198 @@ def g[T: A](b: B[T]): return f(b.x) # Fine ``` +## Inferred upper bounds restrict the range of gradual solutions + +Gradual lower bounds are intersected with their inferred upper bounds. + +```py +from collections.abc import Iterable +from typing import Any, Callable, TypeAlias +from ty_extensions._internal import Unknown + +def infer[T](lower: T, upper: Callable[[T], None]) -> T: + return lower + +def _(any_value: Any, unknown_value: Unknown, upper: Callable[[int], None]): + reveal_type(infer(any_value, upper)) # revealed: int & Any + reveal_type(infer(unknown_value, upper)) # revealed: int & Unknown +``` + +All inferred upper bounds contribute to the intersection, whether they are static or gradual: + +```py +def infer_multiple[T]( + value: T, + first: Callable[[T], None], + second: Callable[[T], None], +) -> T: + return value + +def _( + any_value: Any, + unknown_value: Unknown, + static: Callable[[int], None], + first: Callable[[int | list[Any]], None], + second: Callable[[int | dict[str, Any]], None], +): + reveal_type(infer_multiple(any_value, static, first)) # revealed: int & Any + reveal_type(infer_multiple(any_value, first, second)) # revealed: int & Any + reveal_type(infer_multiple(unknown_value, first, second)) # revealed: int & Unknown +``` + +An unsatisfiable gradual range falls back to unioning the inferred bounds for diagnostic recovery: + +```py +def _( + unknown_value: Unknown, + static: Callable[[int], None], + incompatible: Callable[[list[Any]], None], +): + result = infer_multiple( + unknown_value, + static, # error: [invalid-argument-type] + incompatible, # error: [invalid-argument-type] + ) + reveal_type(result) # revealed: Unknown | int | list[Any] +``` + +A gradual upper bound contributes its top materialization without replacing the gradual lower bound: + +```py +def _( + any_value: Any, + unknown_value: Unknown, + list_upper: Callable[[list[Any]], None], + tuple_upper: Callable[[tuple[Any, ...]], None], + callable_upper: Callable[[Callable[[Any], int]], None], +): + reveal_type(infer(any_value, list_upper)) # revealed: Top[list[Any]] & Any + reveal_type(infer(unknown_value, list_upper)) # revealed: Top[list[Any]] & Unknown + reveal_type(infer(any_value, tuple_upper)) # revealed: tuple[object, ...] & Any + reveal_type(infer(any_value, callable_upper)) # revealed: ((Never, /) -> int) & Any +``` + +The inferred upper bound is also retained when an invariant return type triggers promotion: + +```py +def infer_list[T](lower: T, upper: Callable[[T], None]) -> list[T]: + return [lower] + +def _(any_value: Any, upper: Callable[[int], None]): + reveal_type(infer_list(any_value, upper)) # revealed: list[int & Any] +``` + +Promotion must also preserve the upper bound when a gradual solution contains promotable literals: + +```py +def infer_promoted[T](static: T, gradual: T, upper: Callable[[T], None]) -> list[T]: + return [static, gradual] + +def _(any_value: Any, unknown_value: Unknown, upper: Callable[[int | str], None]): + reveal_type(infer_promoted(1, any_value, upper)) # revealed: list[int | (str & Any)] + reveal_type(infer_promoted(1, unknown_value, upper)) # revealed: list[int | (str & Unknown)] +``` + +The same restriction applies when a type variable occurs in a callable's parameter and return types: + +```py +class Base: ... +class Derived(Base): ... + +def predicate(value: Derived) -> bool: + return True + +def gradual_rule(value: Derived) -> Unknown: + raise NotImplementedError + +def condition[T](predicate: Callable[[T], bool], rule: Callable[[T], T]) -> Callable[[T], T]: + raise NotImplementedError + +reveal_type(condition(predicate, gradual_rule)) # revealed: (Derived & Unknown, /) -> Derived & Unknown +``` + +If the upper bound is a union, it is distributed across the gradual lower bound: + +```py +class A: ... +class B: ... +class Result(A): ... + +def reduce[T](function: Callable[[T, T], T], values: Iterable[T]) -> T: + raise NotImplementedError + +def combine(left: A | B, right: A | B) -> Result: + raise NotImplementedError + +def _(values: Iterable[Any]): + # revealed: Result | (A & Any) | (B & Any) + reveal_type(reduce(combine, values)) +``` + +Declared upper bounds validate a gradual solution but do not restrict its range on their own: + +```py +def bounded[T: A | B](value: T) -> T: + return value + +def bounded_with_upper[T: A | B](value: T, upper: Callable[[T], None]) -> T: + return value + +def _(any_value: Any, upper: Callable[[object], None]): + reveal_type(bounded(any_value)) # revealed: Any + reveal_type(bounded_with_upper(any_value, upper)) # revealed: Any +``` + +An inferred upper bound cannot introduce materializations outside the declared upper bound: + +```py +def bounded_range[T: int | str](value: T, upper: Callable[[T], None]) -> T: + return value + +def _(any_value: Any, unknown_value: Unknown, upper: Callable[[int | bytes], None]): + reveal_type(bounded_range(any_value, upper)) # revealed: int & Any + reveal_type(bounded_range(unknown_value, upper)) # revealed: int & Unknown + +def _(any_value: Any, upper: Callable[[bytes], None]): + reveal_type(bounded_range(any_value, upper)) # revealed: Any +``` + +Declared gradual bounds preserve the gradual type inferred from the lower bound: + +```py +def bounded_any[T: Any](value: T, upper: Callable[[T], None]) -> T: + return value + +def bounded_gradual[T: list[Any]](value: T, upper: Callable[[T], None]) -> T: + return value + +def _( + unknown_value: Unknown, + int_upper: Callable[[int], None], + list_upper: Callable[[list[int]], None], +): + reveal_type(bounded_any(unknown_value, int_upper)) # revealed: int & Unknown + reveal_type(bounded_gradual(unknown_value, list_upper)) # revealed: list[int] & Unknown +``` + +Recursive declared bounds do not introduce `Divergent` into a concrete solution: + +```py +Recursive: TypeAlias = int | list["Recursive"] + +def bounded_recursive[T: Recursive](value: T, upper: Callable[[T], None]) -> T: + return value + +def _(any_value: Any, unknown_value: Unknown, upper: Callable[[list[int]], None]): + any_result = bounded_recursive(any_value, upper) + unknown_result = bounded_recursive(unknown_value, upper) + + reveal_type(any_result) # revealed: list[int] & Any + reveal_type(any_result[0]) # revealed: int & Any + reveal_type(unknown_result) # revealed: list[int] & Unknown + reveal_type(unknown_result[0]) # revealed: int & Unknown +``` + ## Typevars in a union ```py diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index 59edb78e66..591dd1918d 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -5849,7 +5849,7 @@ impl<'a, 'db> ArgumentTypeChecker<'a, 'db> { .entry(identity) .and_modify(|current| *current = current.join(variance)) .or_insert(variance); - PathBounds::default_solve(db, self.env, constraints, path_bound) + PathBounds::preliminary_solve(db, self.env, constraints, path_bound) }); let Solutions::Constrained(solutions) = solutions else { @@ -7671,7 +7671,10 @@ impl<'db> Binding<'db> { generic_context.inferable_typevars(db), ); - if let Solutions::Constrained(solutions) = path_bounds.solve(db, env, constraints) { + let solutions = path_bounds.solve_with(|_variance, path_bound| { + PathBounds::preliminary_solve(db, env, constraints, path_bound) + }); + if let Solutions::Constrained(solutions) = solutions { for solution in solutions { for binding in solution { let identity = binding.bound_typevar.identity(db); diff --git a/crates/ty_python_semantic/src/types/constraints.rs b/crates/ty_python_semantic/src/types/constraints.rs index a14ca6cc2a..4ede8c84d7 100644 --- a/crates/ty_python_semantic/src/types/constraints.rs +++ b/crates/ty_python_semantic/src/types/constraints.rs @@ -3556,6 +3556,82 @@ impl<'db> PathBound<'db> { fn has_only_gradual_evidence(&self) -> bool { self.has_only_gradual_evidence } + + /// Restricts the range of a gradual solution by the upper bounds inferred for this constraint. + fn restrict_gradual_solution( + &self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + solution: Type<'db>, + ) -> Type<'db> { + if self.lower != Some(solution) + || !self.has_upper() + || solution.bottom_materialization(db, env) == solution.top_materialization(db, env) + { + return solution; + } + + // Unresolved type-variable relationships must not escape into the specialization. + if solution.has_typevar(db, env) || solution.has_unspecialized_type_var(db, env) { + return solution; + } + + // `Divergent` is not safely reflexive, so we cannot intersect identical bounds. + if self.upper.clauses.len() == 1 && self.upper.clauses.contains(&solution) { + return solution; + } + + // Gradual upper bounds are top-materialized, as the lower bound is already gradual. + let materialize_upper = |bound: Type<'db>| { + (!bound.has_typevar(db, env) && !bound.has_unspecialized_type_var(db, env)) + .then(|| bound.top_materialization(db, env)) + .filter(|bound| !bound.is_object()) + }; + + let declared_upper = match self.bound_typevar.typevar(db).bound_or_constraints(db, env) { + // Constrained type variables select solutions from their own set of constraints. + Some(TypeVarBoundOrConstraints::Constraints(_)) => return solution, + Some(TypeVarBoundOrConstraints::UpperBound(bound)) => materialize_upper(bound), + _ => None, + }; + + let mut upper_bounds = self + .upper + .clauses + .iter() + .copied() + .filter_map(materialize_upper); + let Some(first_upper) = upper_bounds.next() else { + return solution; + }; + + let Some(upper_bound) = IntersectionType::bounded_from_elements( + db, + env, + iter::once(first_upper) + .chain(upper_bounds) + .chain(declared_upper), + ) else { + return solution; + }; + + // Restrict the range of each gradual solution by the upper bound of this constraint. + let restrict_gradual = |element: Type<'db>| { + if element.bottom_materialization(db, env) == element.top_materialization(db, env) { + Some(element) + } else { + IntersectionType::bounded_from_elements(db, env, [upper_bound, element]) + } + }; + + let restricted = match solution { + Type::Union(union) => union.try_map(db, env, |element| restrict_gradual(*element)), + _ => restrict_gradual(solution), + }; + + // Keep the original gradual type if the intersection exceeds the DNF expansion limit. + restricted.unwrap_or(solution) + } } impl<'db> Type<'db> { @@ -3917,6 +3993,30 @@ impl<'db> PathBounds<'db> { env: &ProgramEnvironment<'db>, builder: &ConstraintSetBuilder<'db>, path_bound: &PathBound<'db>, + ) -> Result>, ()> { + let Some(solution) = Self::preliminary_solve(db, env, builder, path_bound)? else { + return Ok(None); + }; + + let restricted = path_bound.restrict_gradual_solution(db, env, solution); + + // An empty gradual range makes the constraint path unsatisfiable. + if restricted.is_never() && !solution.is_never() { + return Err(()); + } + + Ok(Some(restricted)) + } + + /// Selects a preliminary solution to use as type context during generic call inference. + /// + /// Unlike [`Self::default_solve`], the range of a gradual solution is not restricted by inferred + /// upper bounds, as the inferred types may not have stabilized yet. + pub(crate) fn preliminary_solve( + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + builder: &ConstraintSetBuilder<'db>, + path_bound: &PathBound<'db>, ) -> Result>, ()> { // Choose a solution type that satisfies the constraints on this path, as well as any upper // bound or constraints of the typevar itself. diff --git a/crates/ty_python_semantic/src/types/generics.rs b/crates/ty_python_semantic/src/types/generics.rs index 68a707b836..228fada6c1 100644 --- a/crates/ty_python_semantic/src/types/generics.rs +++ b/crates/ty_python_semantic/src/types/generics.rs @@ -2966,7 +2966,7 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { self.inferable, |_variance, path_bound| { let solution = - PathBounds::default_solve(db, self.env, self.constraints, path_bound); + PathBounds::preliminary_solve(db, self.env, self.constraints, path_bound); if solution.is_err() && first_error.is_none() { first_error = self.specialization_error_from_failed_bounds(path_bound); } diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index 8917ffa79e..4c6f513bf7 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -7352,7 +7352,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { .entry(identity) .and_modify(|current| *current = current.join(variance)) .or_insert(variance); - PathBounds::default_solve(db, env, &constraints, path_bound) + PathBounds::preliminary_solve(db, env, &constraints, path_bound) }); match solutions { From 01e19ac21d277663eb8867e19f004675f98e8fb9 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Mon, 17 Aug 2026 16:16:06 -0700 Subject: [PATCH 074/371] [ty] Specialize type variables determined by bound receivers (#27732) ## What was the problem? Checking an expression such as `pd.Series([1.0]) + np.array([1.0])` could hang indefinitely. The Series already determines one of the addition method's type variables, but ty tried to infer it together with the remaining arguments. That left too many possibilities to consider across NumPy's many operator overloads. ## How does this fix it? When a method explicitly annotates `self` or `cls`, first determine any method type variables fixed by that object. Use those known types when checking the remaining arguments and describing the bound method, while leaving unrelated variables available for normal argument inference. Only do this extra work when the variable also appears in another parameter or the return type. Variables used only in the receiver, variables belonging to the class, and `typing.Self` cannot help with argument checking. Skipping them avoids unnecessary work and preserves performance on `DateType` and `hydra-zen`. Type aliases are handled without expanding their definitions. Fixes astral-sh/ty#4246. ## Test plan - Cover a generic method where the receiver determines one type variable and another argument determines a separate variable. - Cover type aliases in receiver annotations, return types, and other parameters. - Cover type variables used only in the receiver, which should remain unspecialized. - Verify pandas Series addition and multiplication with NumPy arrays, with no performance regressions on `DateType` or `hydra-zen`. --- crates/ty_ide/src/completion.rs | 18 ++-- .../resources/mdtest/overloads.md | 59 +++++++++++- crates/ty_python_semantic/src/types.rs | 19 +++- crates/ty_python_semantic/src/types/method.rs | 18 ++-- .../src/types/signatures.rs | 93 ++++++++++++++++--- .../ty_python_semantic/src/types/typevar.rs | 36 +++++++ 6 files changed, 213 insertions(+), 30 deletions(-) diff --git a/crates/ty_ide/src/completion.rs b/crates/ty_ide/src/completion.rs index f9d3b2d4bf..ae8d24e73a 100644 --- a/crates/ty_ide/src/completion.rs +++ b/crates/ty_ide/src/completion.rs @@ -5003,13 +5003,13 @@ C. __name__ :: str __ne__ :: def __ne__(self, value: object, /) -> bool __new__ :: def __new__[Self](cls) -> Self - __or__ :: bound method .__or__[Self](value: Any, /) -> UnionType | Self + __or__ :: bound method .__or__(value: Any, /) -> UnionType | __prepare__ :: bound method .__prepare__(name: str, bases: tuple[type, ...], /, **kwds: Any) -> MutableMapping[str, object] __qualname__ :: str __reduce__ :: def __reduce__(self) -> str | tuple[Any, ...] __reduce_ex__ :: def __reduce_ex__(self, protocol: SupportsIndex, /) -> str | tuple[Any, ...] __repr__ :: def __repr__(self) -> str - __ror__ :: bound method .__ror__[Self](value: Any, /) -> UnionType | Self + __ror__ :: bound method .__ror__(value: Any, /) -> UnionType | __setattr__ :: def __setattr__(self, name: str, value: Any, /) -> None __sizeof__ :: def __sizeof__(self) -> int __str__ :: def __str__(self) -> str @@ -5202,13 +5202,13 @@ Quux. __name__ :: str __ne__ :: def __ne__(self, value: object, /) -> bool __new__ :: def __new__[Self](cls) -> Self - __or__ :: bound method .__or__[Self](value: Any, /) -> UnionType | Self + __or__ :: bound method .__or__(value: Any, /) -> UnionType | __prepare__ :: bound method .__prepare__(name: str, bases: tuple[type, ...], /, **kwds: Any) -> MutableMapping[str, object] __qualname__ :: str __reduce__ :: def __reduce__(self) -> str | tuple[Any, ...] __reduce_ex__ :: def __reduce_ex__(self, protocol: SupportsIndex, /) -> str | tuple[Any, ...] __repr__ :: def __repr__(self) -> str - __ror__ :: bound method .__ror__[Self](value: Any, /) -> UnionType | Self + __ror__ :: bound method .__ror__(value: Any, /) -> UnionType | __setattr__ :: def __setattr__(self, name: str, value: Any, /) -> None __sizeof__ :: def __sizeof__(self) -> int __str__ :: def __str__(self) -> str @@ -5265,14 +5265,14 @@ Answer. __flags__ :: int __format__ :: def __format__(self, format_spec: str) -> str __getattribute__ :: def __getattribute__(self, name: str, /) -> Any - __getitem__ :: bound method .__getitem__[_EnumMemberT](name: str) -> _EnumMemberT + __getitem__ :: bound method .__getitem__(name: str) -> Answer __getstate__ :: def __getstate__(self) -> object __hash__ :: def __hash__(self) -> int __init__ :: def __init__(self) -> None __init_subclass__ :: bound method .__init_subclass__() -> None __instancecheck__ :: bound method .__instancecheck__(instance: Any, /) -> bool __itemsize__ :: int - __iter__ :: bound method .__iter__[_EnumMemberT]() -> Iterator[_EnumMemberT] + __iter__ :: bound method .__iter__() -> Iterator[Answer] __len__ :: bound method .__len__() -> int __members__ :: MappingProxyType[str, Answer] __module__ :: str @@ -5280,14 +5280,14 @@ Answer. __name__ :: str __ne__ :: def __ne__(self, value: object, /) -> bool __new__ :: def __new__[Self](cls, value: object) -> Self - __or__ :: bound method .__or__[Self](value: Any, /) -> UnionType | Self + __or__ :: bound method .__or__(value: Any, /) -> UnionType | __order__ :: str __prepare__ :: bound method .__prepare__(cls: str, bases: tuple[type, ...], **kwds: Any) -> _EnumDict __qualname__ :: str __reduce__ :: def __reduce__(self) -> str | tuple[Any, ...] __repr__ :: def __repr__(self) -> str - __reversed__ :: bound method .__reversed__[_EnumMemberT]() -> Iterator[_EnumMemberT] - __ror__ :: bound method .__ror__[Self](value: Any, /) -> UnionType | Self + __reversed__ :: bound method .__reversed__() -> Iterator[Answer] + __ror__ :: bound method .__ror__(value: Any, /) -> UnionType | __setattr__ :: def __setattr__(self, name: str, value: Any, /) -> None __sizeof__ :: def __sizeof__(self) -> int __str__ :: def __str__(self) -> str diff --git a/crates/ty_python_semantic/resources/mdtest/overloads.md b/crates/ty_python_semantic/resources/mdtest/overloads.md index 8b3f9cdc04..f61101cf16 100644 --- a/crates/ty_python_semantic/resources/mdtest/overloads.md +++ b/crates/ty_python_semantic/resources/mdtest/overloads.md @@ -274,7 +274,7 @@ def union_receiver(reader: Reader[int | str]): ## Method type variables inferred from `self` -Binding an overload whose explicit receiver introduces a method type variable should infer that +Binding a method whose explicit receiver introduces a method type variable should infer that variable from the concrete receiver and apply it to the remainder of the signature. ```toml @@ -295,6 +295,9 @@ class ReceiverGeneric[T]: def method(self, value: object) -> object: return value + def single[S, U](self: "ReceiverGeneric[S]", value: U) -> tuple[S, U]: + return self.value, value + reveal_type(ReceiverGeneric[str]().method) # revealed: Overload[(value: str) -> str, (value: bytes) -> bytes] def takes_callable(fn: Callable[..., Any]) -> None: ... @@ -304,6 +307,60 @@ def use_generic_receiver[T](value: ReceiverGeneric[T]) -> None: takes_callable(value.method) ``` +Non-overloaded methods should also specialize receiver-determined type variables while preserving +other type variables for argument inference. + +```py +# revealed: bound method ReceiverGeneric[str].single[U](value: U) -> tuple[str, U] +reveal_type(ReceiverGeneric[str]().single) +reveal_type(ReceiverGeneric[str]().single(1)) # revealed: tuple[str, Literal[1]] +``` + +Type aliases in the receiver, return type, or another parameter must not conceal a method type +variable determined by the receiver. + +```py +type ReceiverAlias[T] = ReceiverGeneric[T] +type ValueAlias[T] = T + +class AliasedReceiver[T](ReceiverGeneric[T]): + def aliased_return[S](self: ReceiverAlias[S]) -> tuple[ValueAlias[S]]: + return (self.value,) + + def aliased_argument[S](self: ReceiverGeneric[S], value: ValueAlias[S]) -> None: ... + +value = AliasedReceiver[str]() + +# revealed: bound method AliasedReceiver[str].aliased_return() -> tuple[ValueAlias[str]] +reveal_type(value.aliased_return) + +# revealed: bound method AliasedReceiver[str].aliased_argument(value: ValueAlias[str]) -> None +reveal_type(value.aliased_argument) +# error: [invalid-argument-type] "Expected `ValueAlias[str]`, found `Literal[1]`" +value.aliased_argument(1) +``` + +## Method type variables used only in the receiver + +A method type variable that appears only in the receiver does not affect argument inference or the +return type, so binding the method does not need to specialize it. + +```toml +[environment] +python-version = "3.12" +``` + +```py +class Factory: + @classmethod + def describe[Receiver](cls: type[Receiver], value: int) -> str: + return str(value) + +# revealed: bound method .describe[Receiver](value: int) -> str +reveal_type(Factory.describe) +reveal_type(Factory.describe(1)) # revealed: str +``` + ## Constrained method type variables inferred from `self` Matching a receiver against a value-constrained method type variable must reject values outside that diff --git a/crates/ty_python_semantic/src/types.rs b/crates/ty_python_semantic/src/types.rs index 4827dad134..f53d43c7e9 100644 --- a/crates/ty_python_semantic/src/types.rs +++ b/crates/ty_python_semantic/src/types.rs @@ -5502,7 +5502,24 @@ impl<'db> Type<'db> { binding.bake_bound_type_into_overloads(db, env); binding.into() } else { - CallableBinding::from_overloads(self, signature.overloads.iter().cloned()) + // Solve exact receiver constraints before checking the other arguments, but + // retain the receiver itself for call inference and receiver diagnostics. + let overloads = signature.overloads.iter().map(|overload| { + if overload.has_receiver_determined_method_typevar(db, env) + && let Some(specialized) = overload.specialize_for_bound_receiver( + db, + env, + self_instance, + bound_method.typing_self_type(db), + ) + { + specialized + } else { + overload.clone() + } + }); + + CallableBinding::from_overloads(self, overloads) .with_bound_type(self_instance) .into() } diff --git a/crates/ty_python_semantic/src/types/method.rs b/crates/ty_python_semantic/src/types/method.rs index d38297fc99..e6b4a75cf2 100644 --- a/crates/ty_python_semantic/src/types/method.rs +++ b/crates/ty_python_semantic/src/types/method.rs @@ -153,12 +153,18 @@ impl<'db> BoundMethodType<'db> { ); }; - CallableSignature::single(signature.bind_self_with_receiver( - db, - env, - Some(receiver_type), - Some(typing_self_type), - )) + let specialized = if signature.has_receiver_determined_method_typevar(db, env) { + signature.specialize_for_bound_receiver(db, env, receiver_type, typing_self_type) + } else { + None + }; + + CallableSignature::single( + specialized + .as_ref() + .unwrap_or(signature) + .bind_self_with_receiver(db, env, Some(receiver_type), Some(typing_self_type)), + ) } pub(super) fn recursive_type_normalized_impl( diff --git a/crates/ty_python_semantic/src/types/signatures.rs b/crates/ty_python_semantic/src/types/signatures.rs index 2daf5afdc9..4cfaf65055 100644 --- a/crates/ty_python_semantic/src/types/signatures.rs +++ b/crates/ty_python_semantic/src/types/signatures.rs @@ -1270,13 +1270,13 @@ impl<'db> Signature<'db> { } } - /// Returns this signature bound to `receiver_type` if its explicit receiver annotation is - /// compatible with the bound receiver. + /// Specializes this signature using the type variables determined by its bound receiver. /// /// Matching the receiver can constrain type variables that occur elsewhere in the signature. - /// Exact bounds determine an unambiguous specialization; one-sided constraints remain attached - /// to the bound signature for later relation checks. - pub(crate) fn bind_self_if_compatible( + /// Exact bounds determine an unambiguous specialization; one-sided constraints remain + /// available to normal call inference. The receiver remains in the returned signature so + /// bound-method calls can still check it and report receiver-related diagnostics. + pub(crate) fn specialize_for_bound_receiver( &self, db: &'db dyn Db, env: &ProgramEnvironment<'db>, @@ -1290,7 +1290,7 @@ impl<'db> Signature<'db> { let bound_signature = self.bind_self_with_receiver(db, env, Some(receiver_type), Some(typing_self_type)); let Some(receiver_constraints) = bound_signature.receiver_constraints.as_ref() else { - return Some(bound_signature); + return Some(self.clone()); }; let constraints = ConstraintSetBuilder::new(); @@ -1299,17 +1299,17 @@ impl<'db> Signature<'db> { match when.solutions(db, env, &constraints, inferable) { Solutions::Unsatisfiable => return None, - Solutions::Unconstrained => return Some(bound_signature), + Solutions::Unconstrained => return Some(self.clone()), // Each receiver path can leave a different type variable unconstrained. Preserve the // original relation instead of combining those independent solutions. Solutions::Constrained(solutions) if solutions.len() > 1 => { - return Some(bound_signature); + return Some(self.clone()); } Solutions::Constrained(_) => {} } let Some(generic_context) = self.generic_context else { - return Some(bound_signature); + return Some(self.clone()); }; let mut builder = SpecializationBuilder::new(db, env, &constraints, inferable); @@ -1340,10 +1340,27 @@ impl<'db> Signature<'db> { Some(Type::TypeVar(typevar)) }); - Some( - self.apply_specialization(db, specialization) - .bind_self_with_receiver(db, env, Some(receiver_type), Some(typing_self_type)), - ) + Some(self.apply_specialization(db, specialization)) + } + + /// Returns this signature bound to `receiver_type` if its explicit receiver annotation is + /// compatible with the bound receiver. + pub(crate) fn bind_self_if_compatible( + &self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + receiver_type: Type<'db>, + typing_self_type: Type<'db>, + ) -> Option { + self.specialize_for_bound_receiver(db, env, receiver_type, typing_self_type) + .map(|signature| { + signature.bind_self_with_receiver( + db, + env, + Some(receiver_type), + Some(typing_self_type), + ) + }) } /// Returns `true` if this signature's first parameter can accept the bound `self` type. @@ -1425,6 +1442,56 @@ impl<'db> Signature<'db> { .is_some_and(|parameter| parameter.is_positional() && !parameter.inferred_annotation) } + /// Returns whether the receiver can determine a method type variable used elsewhere. + /// + /// Receiver-only variables cannot affect the rest of the signature, class type variables are + /// handled by class or constructor inference, and `typing.Self` is handled by receiver binding. + pub(crate) fn has_receiver_determined_method_typevar( + &self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + ) -> bool { + let Some(receiver) = self + .parameters + .get(0) + .filter(|parameter| parameter.is_positional() && !parameter.inferred_annotation) + else { + return false; + }; + let Some(generic_context) = self.generic_context else { + return false; + }; + let Some(definition) = self.definition else { + return false; + }; + let annotation = receiver.annotated_type(); + + let mut typevars = match annotation { + Type::TypeVar(typevar) => Either::Left(std::iter::once(typevar)), + Type::SubclassOf(subclass) if let Some(typevar) = subclass.into_type_var() => { + Either::Left(std::iter::once(typevar)) + } + _ => Either::Right(generic_context.variables(db)), + }; + + typevars.any(|typevar| { + let variable = typevar.typevar(db); + let identity = variable.identity(db); + + typevar.binding_context(db).definition() == Some(definition) + && !variable.is_self(db) + && annotation.references_typevar_through_aliases(db, env, identity) + && (self + .return_ty + .references_typevar_through_aliases(db, env, identity) + || self.parameters.iter().skip(1).any(|parameter| { + parameter + .annotated_type() + .references_typevar_through_aliases(db, env, identity) + })) + }) + } + pub(crate) fn has_implicit_positional_receiver_annotation(&self) -> bool { self.parameters .get(0) diff --git a/crates/ty_python_semantic/src/types/typevar.rs b/crates/ty_python_semantic/src/types/typevar.rs index 0fc00f7880..2541048bbf 100644 --- a/crates/ty_python_semantic/src/types/typevar.rs +++ b/crates/ty_python_semantic/src/types/typevar.rs @@ -61,6 +61,42 @@ impl<'db> Type<'db> { }) } + /// Returns whether this type might reference `typevar_id`, including type-alias arguments. + /// + /// Other non-lazy type-variable visitors stop at type aliases because inspecting an alias's + /// value can trigger lazy inference or expand a recursive definition. Receiver specialization + /// still needs to notice `T` in `Alias[T]`, so this visitor inspects the already-available + /// specialization arguments without evaluating the alias body. + /// + /// This deliberately over-approximates: `type Alias[T] = int` does not actually depend on + /// `T`, and specialization can also erase an argument. That can cause an unnecessary + /// receiver-specialization attempt, but actual receiver constraints are still solved before + /// changing the signature. Applying the same traversal to visitors that use type-variable + /// occurrences to drive inference or diagnostics can instead change behavior. + /// + /// TODO: Explore whether other type-variable visitors can safely inspect alias arguments, + /// accounting for unused parameters and arguments erased by specialization. + pub(crate) fn references_typevar_through_aliases( + self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + typevar_id: TypeVarIdentity<'db>, + ) -> bool { + any_over_type(db, env, self, false, |ty| match ty { + Type::TypeVar(typevar) => typevar_id == typevar.typevar(db).identity(db), + Type::KnownInstance(KnownInstanceType::TypeVar(typevar)) => { + typevar_id == typevar.identity(db) + } + Type::TypeAlias(alias) => alias.specialization(db).is_some_and(|specialization| { + specialization + .types(db) + .iter() + .any(|ty| ty.references_typevar_through_aliases(db, env, typevar_id)) + }), + _ => false, + }) + } + pub(crate) fn has_non_self_typevar( self, db: &'db dyn Db, From 14b29c048caf258b19d8e1f1605b586f7cff895a Mon Sep 17 00:00:00 2001 From: Charlie Marsh Date: Mon, 17 Aug 2026 19:54:38 -0400 Subject: [PATCH 075/371] [ty] Deduplicate exception checkpoints across equivalent branches (#27703) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary When analyzing a `try` block, we record which bindings an exception handler could see before each potentially raising operation. In a large function, copying and later merging these snapshots can be expensive. We already reuse checkpoints for consecutive calls when nothing relevant changes. However, harmless control flow can defeat that optimization: ```python def example(value: str, flag: bool) -> None: try: value.upper() if flag is True: pass value.upper() except Exception: pass ``` The `if` introduces branches, but after they rejoin, the second call exposes the same bindings to the handler. Previously, restoring or merging a branch always advanced the checkpoint's control-flow revision, so we retained another snapshot. We now recognize equivalent paths and reuse the checkpoint. Equivalent control flow does not necessarily mean equivalent bindings: nested exception handlers can merge paths with different possible values. To preserve all of those values, we track a restorable identity for the visible bindings as well as reachability, giving distinct merged states a fresh identity. We also preserve the call history needed to handle caught `NoReturn` calls correctly. If the reachability graph reaches its size limit, we fall back to conservative deduplication rather than retaining a snapshot for every subsequent call. Two supporting changes avoid redundant work when combining a reachability condition with itself and keep the larger flow snapshots out of the common recursive expression-visitor stack frame. This is complementary to #27787: that change makes evaluating narrowing and reachability histories cheaper, while this change avoids constructing redundant exception-state snapshots. ## Performance On a synthetic workload with 800 locals and 800 calls separated by equivalent `if` branches, this reduces CPU time by about 26% and peak memory by 59%. The ordinary-call control and the suppression-heavy workload are essentially unchanged: | Workload | CPU, before → after | Peak RSS, before → after | | --- | ---: | ---: | | Plain repeated calls | 148 → 148 ms | 123.4 → 123.3 MiB | | Calls separated by equivalent `if` branches | 268 → 199 ms | 102.5 → 42.1 MiB | | Conditional assignments under suppression | 819 → 816 ms | 111.2 → 109.9 MiB | --- crates/ruff_benchmark/benches/ty.rs | 49 +-- crates/ty_python_core/src/builder.rs | 285 +++++++++--------- .../src/builder/except_handlers.rs | 4 +- .../src/reachability_constraints.rs | 9 +- crates/ty_python_core/src/use_def.rs | 61 ++-- .../src/use_def/exception_checkpoint.rs | 128 ++++++++ .../mdtest/exception/control_flow.md | 76 +++++ 7 files changed, 431 insertions(+), 181 deletions(-) create mode 100644 crates/ty_python_core/src/use_def/exception_checkpoint.rs diff --git a/crates/ruff_benchmark/benches/ty.rs b/crates/ruff_benchmark/benches/ty.rs index f36a145da2..44cca34aef 100644 --- a/crates/ruff_benchmark/benches/ty.rs +++ b/crates/ruff_benchmark/benches/ty.rs @@ -1392,25 +1392,38 @@ fn benchmark_repeated_statement_calls(criterion: &mut Criterion) { }); } - let mut code = String::from("def f(value: str) -> None:\n"); - for index in 0..800 { - writeln!(&mut code, " local_{index} = {index}").ok(); - } - code.push_str(" try:\n"); - code.push_str(&" value.upper()\n".repeat(800)); - code.push_str(" except Exception:\n pass\n"); + for (name, parameters, statement) in [ + ( + "ty_micro[repeated_statement_calls_in_try]", + "value: str", + " value.upper()\n", + ), + ( + "ty_micro[repeated_statement_calls_in_try_with_if_branches]", + "value: str, flag: bool", + " if flag is True:\n pass\n value.upper()\n", + ), + ] { + let mut code = format!("def f({parameters}) -> None:\n"); + for index in 0..800 { + writeln!(&mut code, " local_{index} = {index}").ok(); + } + code.push_str(" try:\n"); + code.push_str(&statement.repeat(800)); + code.push_str(" except Exception:\n pass\n"); - criterion.bench_function("ty_micro[repeated_statement_calls_in_try]", |b| { - b.iter_batched_ref( - || setup_micro_case(&code), - |case| { - let Case { db } = case; - let result = db.check(); - assert_eq!(result.len(), 0); - }, - BatchSize::SmallInput, - ); - }); + criterion.bench_function(name, |b| { + b.iter_batched_ref( + || setup_micro_case(&code), + |case| { + let Case { db } = case; + let result = db.check(); + assert_eq!(result.len(), 0); + }, + BatchSize::SmallInput, + ); + }); + } } struct ProjectBenchmark<'a> { diff --git a/crates/ty_python_core/src/builder.rs b/crates/ty_python_core/src/builder.rs index a446878577..f0412c5740 100644 --- a/crates/ty_python_core/src/builder.rs +++ b/crates/ty_python_core/src/builder.rs @@ -3180,6 +3180,146 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { .get_or_init(|| source_text(self.db, self.file.file(self.db))) } + /// Visits a conditional expression without reserving its flow snapshots in every recursive + /// expression-visitor frame. This matters for deeply nested expressions in unoptimized builds. + fn visit_if_expression(&mut self, node: &'ast ast::ExprIf) { + let ast::ExprIf { + body, test, orelse, .. + } = node; + self.visit_expr(test); + let condition_flow_snapshot = self.flow_snapshot_for_condition(test); + let falsy = if let Some(snapshots) = condition_flow_snapshot.into_branches() { + self.flow_restore(snapshots.truthy); + snapshots.falsy + } else { + self.flow_snapshot() + }; + let (predicate, predicate_id) = self.record_expression_narrowing_constraint(test); + let reachability_constraint = self.record_reachability_constraint(predicate); + let in_type_checking_block = self.in_type_checking_block; + self.current_use_def_map_mut() + .record_range_reachability(body.range(), in_type_checking_block); + self.visit_expr(body); + let post_body = self.flow_snapshot(); + self.flow_restore(falsy); + + self.record_negated_narrowing_constraint(predicate, predicate_id); + self.record_negated_reachability_constraint(reachability_constraint); + let in_type_checking_block = self.in_type_checking_block; + self.current_use_def_map_mut() + .record_range_reachability(orelse.range(), in_type_checking_block); + self.visit_expr(orelse); + self.flow_merge(post_body); + } + + /// Keeps short-circuit flow snapshots out of the common recursive expression-visitor frame. + fn visit_bool_expression(&mut self, node: &'ast ast::ExprBoolOp) { + let ast::ExprBoolOp { values, op, .. } = node; + let mut snapshots = vec![]; + let mut reachability_constraints = vec![]; + let mut last_condition_flow_snapshots = None; + + for (index, value) in values.iter().enumerate() { + for id in &reachability_constraints { + self.current_use_def_map_mut() + .record_reachability_constraint(*id); // TODO: nicer API + } + + let in_type_checking_block = self.in_type_checking_block; + self.current_use_def_map_mut() + .record_range_reachability(value.range(), in_type_checking_block); + self.visit_expr(value); + + // Only non-final values can short-circuit this boolean operation. The final + // value can still have its own outcome-specific flow if it is nested. + if index < values.len() - 1 { + self.record_exception_checkpoint_if(!Self::condition_evaluation_is_known_safe( + value, + )); + let condition_flow_snapshots = self.take_condition_flow_snapshots(value); + let predicate = self.build_predicate(value); + let possibly_narrowed = self.compute_possibly_narrowed_places(&predicate); + let predicate_id = match op { + ast::BoolOp::And => self.add_predicate(predicate), + ast::BoolOp::Or => self.add_negated_predicate(predicate), + }; + let reachability_constraint = self + .current_reachability_constraints_mut() + .add_atom(predicate_id); + + let continuation = if let Some(condition_flow_snapshots) = condition_flow_snapshots + { + let (short_circuit, continuation) = + condition_flow_snapshots.into_short_circuit_and_continuation(*op); + self.flow_restore(short_circuit); + continuation + } else { + self.flow_snapshot() + }; + + // We first model the short-circuiting behavior. We take the short-circuit + // path here if all of the previous short-circuit paths were not taken, so + // we record all previously existing reachability constraints, and negate the + // one for the current expression. + + self.record_negated_reachability_constraint(reachability_constraint); + snapshots.push(self.flow_snapshot()); + + // Then we model the non-short-circuiting behavior. Here, we need to delay + // the application of the reachability constraint until after the expression + // has been evaluated, so we only push it onto the stack here. + self.flow_restore(continuation); + self.record_narrowing_constraint_id_for_places(predicate_id, &possibly_narrowed); + reachability_constraints.push(reachability_constraint); + } else { + last_condition_flow_snapshots = self.take_condition_flow_snapshots(value); + } + } + + let has_specialized_last = last_condition_flow_snapshots.is_some(); + let (last_short_circuit, no_short_circuit) = + if let Some(condition_flow_snapshots) = last_condition_flow_snapshots { + let (short_circuit, no_short_circuit) = + condition_flow_snapshots.into_short_circuit_and_continuation(*op); + (Some(short_circuit), Some(no_short_circuit)) + } else { + ( + None, + values + .iter() + .any(|value| any_over_expr(value, &ast::Expr::is_named_expr)) + .then(|| self.flow_snapshot()), + ) + }; + + if let Some(last_short_circuit) = last_short_circuit { + self.flow_restore(last_short_circuit); + } + + for snapshot in snapshots { + self.flow_merge(snapshot); + } + + if let Some(no_short_circuit) = no_short_circuit { + let bool_op_key = ExpressionNodeKey::from(ast::ExprRef::BoolOp(node)); + let maybe_short_circuit = self.flow_snapshot(); + + if has_specialized_last { + // Restore the merged post-expression flow after constructing the two + // outcome-specific snapshots. + self.flow_merge(no_short_circuit.clone()); + } + + let (truthy, falsy) = match op { + ast::BoolOp::And => (no_short_circuit, maybe_short_circuit), + ast::BoolOp::Or => (maybe_short_circuit, no_short_circuit), + }; + + self.condition_flow_snapshots_by_node + .insert(bool_op_key, ConditionFlowSnapshots { truthy, falsy }); + } + } + fn visit_stmt_impl(&mut self, stmt: &'ast ast::Stmt) { self.with_semantic_checker(|semantic, context| semantic.visit_stmt(stmt, context)); @@ -4348,6 +4488,8 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { guard_predicate_id, &possibly_narrowed, ); + self.current_use_def_map_mut() + .record_exception_checkpoint_binding_change(); let match_success_guard_failure = self.flow_snapshot(); self.flow_restore(truthy); self.current_use_def_map_mut() @@ -4355,6 +4497,8 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { guard_predicate_id, &possibly_narrowed, ); + self.current_use_def_map_mut() + .record_exception_checkpoint_binding_change(); match_success_guard_failure }); @@ -5185,34 +5329,7 @@ impl<'ast> Visitor<'ast> for SemanticIndexBuilder<'_, 'ast> { self.visit_expr(lambda.body.as_ref()); self.pop_scope(); } - ast::Expr::If(ast::ExprIf { - body, test, orelse, .. - }) => { - self.visit_expr(test); - let condition_flow_snapshot = self.flow_snapshot_for_condition(test); - let falsy = if let Some(snapshots) = condition_flow_snapshot.into_branches() { - self.flow_restore(snapshots.truthy); - snapshots.falsy - } else { - self.flow_snapshot() - }; - let (predicate, predicate_id) = self.record_expression_narrowing_constraint(test); - let reachability_constraint = self.record_reachability_constraint(predicate); - let in_type_checking_block = self.in_type_checking_block; - self.current_use_def_map_mut() - .record_range_reachability(body.range(), in_type_checking_block); - self.visit_expr(body); - let post_body = self.flow_snapshot(); - self.flow_restore(falsy); - - self.record_negated_narrowing_constraint(predicate, predicate_id); - self.record_negated_reachability_constraint(reachability_constraint); - let in_type_checking_block = self.in_type_checking_block; - self.current_use_def_map_mut() - .record_range_reachability(orelse.range(), in_type_checking_block); - self.visit_expr(orelse); - self.flow_merge(post_body); - } + ast::Expr::If(node) => self.visit_if_expression(node), ast::Expr::ListComp( list_comprehension @ ast::ExprListComp { elt, generators, .. @@ -5300,117 +5417,7 @@ impl<'ast> Visitor<'ast> for SemanticIndexBuilder<'_, 'ast> { )); } } - ast::Expr::BoolOp(ast::ExprBoolOp { - values, - range: _, - node_index: _, - op, - }) => { - let mut snapshots = vec![]; - let mut reachability_constraints = vec![]; - let mut last_condition_flow_snapshots = None; - - for (index, value) in values.iter().enumerate() { - for id in &reachability_constraints { - self.current_use_def_map_mut() - .record_reachability_constraint(*id); // TODO: nicer API - } - - let in_type_checking_block = self.in_type_checking_block; - self.current_use_def_map_mut() - .record_range_reachability(value.range(), in_type_checking_block); - self.visit_expr(value); - - // Only non-final values can short-circuit this boolean operation. The final - // value can still have its own outcome-specific flow if it is nested. - if index < values.len() - 1 { - self.record_exception_checkpoint_if( - !Self::condition_evaluation_is_known_safe(value), - ); - let condition_flow_snapshots = self.take_condition_flow_snapshots(value); - let predicate = self.build_predicate(value); - let possibly_narrowed = self.compute_possibly_narrowed_places(&predicate); - let predicate_id = match op { - ast::BoolOp::And => self.add_predicate(predicate), - ast::BoolOp::Or => self.add_negated_predicate(predicate), - }; - let reachability_constraint = self - .current_reachability_constraints_mut() - .add_atom(predicate_id); - - let continuation = - if let Some(condition_flow_snapshots) = condition_flow_snapshots { - let (short_circuit, continuation) = condition_flow_snapshots - .into_short_circuit_and_continuation(*op); - self.flow_restore(short_circuit); - continuation - } else { - self.flow_snapshot() - }; - - // We first model the short-circuiting behavior. We take the short-circuit - // path here if all of the previous short-circuit paths were not taken, so - // we record all previously existing reachability constraints, and negate the - // one for the current expression. - - self.record_negated_reachability_constraint(reachability_constraint); - snapshots.push(self.flow_snapshot()); - - // Then we model the non-short-circuiting behavior. Here, we need to delay - // the application of the reachability constraint until after the expression - // has been evaluated, so we only push it onto the stack here. - self.flow_restore(continuation); - self.record_narrowing_constraint_id_for_places( - predicate_id, - &possibly_narrowed, - ); - reachability_constraints.push(reachability_constraint); - } else { - last_condition_flow_snapshots = self.take_condition_flow_snapshots(value); - } - } - - let has_specialized_last = last_condition_flow_snapshots.is_some(); - let (last_short_circuit, no_short_circuit) = - if let Some(condition_flow_snapshots) = last_condition_flow_snapshots { - let (short_circuit, no_short_circuit) = - condition_flow_snapshots.into_short_circuit_and_continuation(*op); - (Some(short_circuit), Some(no_short_circuit)) - } else { - ( - None, - any_over_expr(expr, &ast::Expr::is_named_expr) - .then(|| self.flow_snapshot()), - ) - }; - - if let Some(last_short_circuit) = last_short_circuit { - self.flow_restore(last_short_circuit); - } - - for snapshot in snapshots { - self.flow_merge(snapshot); - } - - if let Some(no_short_circuit) = no_short_circuit { - let bool_op_key = ExpressionNodeKey::from(expr); - let maybe_short_circuit = self.flow_snapshot(); - - if has_specialized_last { - // Restore the merged post-expression flow after constructing the two - // outcome-specific snapshots. - self.flow_merge(no_short_circuit.clone()); - } - - let (truthy, falsy) = match op { - ast::BoolOp::And => (no_short_circuit, maybe_short_circuit), - ast::BoolOp::Or => (maybe_short_circuit, no_short_circuit), - }; - - self.condition_flow_snapshots_by_node - .insert(bool_op_key, ConditionFlowSnapshots { truthy, falsy }); - } - } + ast::Expr::BoolOp(node) => self.visit_bool_expression(node), ast::Expr::StringLiteral(_) => { walk_expr(self, expr); } diff --git a/crates/ty_python_core/src/builder/except_handlers.rs b/crates/ty_python_core/src/builder/except_handlers.rs index 750d213b24..7ce9cc6a4e 100644 --- a/crates/ty_python_core/src/builder/except_handlers.rs +++ b/crates/ty_python_core/src/builder/except_handlers.rs @@ -1,5 +1,5 @@ use crate::reachability_constraints::ScopedReachabilityConstraintId; -use crate::use_def::{ControlFlowRevision, FlowSnapshot, ScopedDefinitionId, UseDefMapBuilder}; +use crate::use_def::{ExceptionCheckpointKey, FlowSnapshot, UseDefMapBuilder}; use super::SemanticIndexBuilder; @@ -347,7 +347,7 @@ enum ExceptionContextKind { pub(super) struct ExceptionContext { exception_handlers: ExceptionHandlers, kind: ExceptionContextKind, - last_checkpoint_key: Option<(ScopedDefinitionId, ControlFlowRevision)>, + last_checkpoint_key: Option, /// Whether an exception escaped this suite and must also propagate after its cleanup. has_escaping_exception: bool, /// Whether apparently terminal control flow in a nested context-manager body, such as a diff --git a/crates/ty_python_core/src/reachability_constraints.rs b/crates/ty_python_core/src/reachability_constraints.rs index ad188fec73..4071841c6d 100644 --- a/crates/ty_python_core/src/reachability_constraints.rs +++ b/crates/ty_python_core/src/reachability_constraints.rs @@ -194,6 +194,11 @@ pub struct ReachabilityConstraintsBuilder { } impl ReachabilityConstraintsBuilder { + /// Returns whether new constraint combinations may lose precision at the arena limit. + pub(crate) fn is_saturated(&self) -> bool { + self.interiors.len() >= MAX_INTERIOR_NODES + } + pub(crate) fn build(self) -> ReachabilityConstraints { if self.interior_used.first_zero().is_none() { ReachabilityConstraints { @@ -355,7 +360,7 @@ impl ReachabilityConstraintsBuilder { match (a, b) { (ALWAYS_TRUE, _) | (_, ALWAYS_TRUE) => return ALWAYS_TRUE, (ALWAYS_FALSE, other) | (other, ALWAYS_FALSE) => return other, - (AMBIGUOUS, AMBIGUOUS) => return AMBIGUOUS, + _ if a == b => return a, _ => {} } @@ -425,7 +430,7 @@ impl ReachabilityConstraintsBuilder { match (a, b) { (ALWAYS_FALSE, _) | (_, ALWAYS_FALSE) => return ALWAYS_FALSE, (ALWAYS_TRUE, other) | (other, ALWAYS_TRUE) => return other, - (AMBIGUOUS, AMBIGUOUS) => return AMBIGUOUS, + _ if a == b => return a, _ => {} } diff --git a/crates/ty_python_core/src/use_def.rs b/crates/ty_python_core/src/use_def.rs index eab118ecfb..2e885d8589 100644 --- a/crates/ty_python_core/src/use_def.rs +++ b/crates/ty_python_core/src/use_def.rs @@ -272,8 +272,11 @@ use crate::{ BoundnessAnalysis, EnclosingSnapshotResult, LoopHeader, PossiblyNarrowedPlaces, SemanticIndex, }; +mod exception_checkpoint; mod place_state; +pub(super) use exception_checkpoint::ExceptionCheckpointKey; +use exception_checkpoint::{ExceptionCheckpointSnapshot, ExceptionCheckpointState}; pub use place_state::LiveBinding; pub use place_state::ScopedDefinitionId; pub(super) use place_state::{FutureDefinitions, PreviousDefinitions}; @@ -1373,6 +1376,8 @@ pub(super) struct FlowSnapshot { symbol_states: IndexVec, member_states: IndexVec, reachability: ScopedReachabilityConstraintId, + checkpoint_flow: ScopedReachabilityConstraintId, + checkpoint_state: ExceptionCheckpointSnapshot, pending_reachability: PendingReachabilityId, } @@ -1757,16 +1762,6 @@ pub(super) struct SingleSymbolSnapshot { associated_member_states: FxHashMap, } -/// Identifies a control-flow path within a single scope. -#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] -pub(super) struct ControlFlowRevision(u64); - -impl ControlFlowRevision { - fn advance(&mut self) { - self.0 += 1; - } -} - #[derive(Debug)] pub(super) struct UseDefMapBuilder<'db> { /// Append-only array of [`DefinitionState`]. @@ -1804,11 +1799,14 @@ pub(super) struct UseDefMapBuilder<'db> { /// keyed by their text range. range_reachability: Vec<(TextRange, RangeInfo)>, - /// Distinguishes control-flow paths that can have the same set of recorded definitions. + /// Identifies the current control-flow path for exception checkpoints. /// - /// This only moves forward when restoring switches paths or merging widens the current path. - /// Revisions are not restored from snapshots, so separate branches cannot appear equivalent. - control_flow_revision: ControlFlowRevision, + /// Unlike `reachability`, this excludes per-call gates so repeated calls with unchanged + /// bindings share a checkpoint. + checkpoint_flow: ScopedReachabilityConstraintId, + + /// Restorable identity of the bindings visible to exception handlers. + checkpoint_state: ExceptionCheckpointState, /// Live bindings for each so-far-recorded definition and, for binding-only definitions, the /// live declarations. @@ -1852,7 +1850,8 @@ impl<'db> UseDefMapBuilder<'db> { multi_bindings_by_use: FxHashMap::default(), reachability: ScopedReachabilityConstraintId::ALWAYS_TRUE, range_reachability: Vec::new(), - control_flow_revision: ControlFlowRevision::default(), + checkpoint_flow: ScopedReachabilityConstraintId::ALWAYS_TRUE, + checkpoint_state: ExceptionCheckpointState::default(), definitions_by_definition: FxHashMap::default(), symbol_states: IndexVec::new(), member_states: IndexVec::new(), @@ -1874,6 +1873,7 @@ impl<'db> UseDefMapBuilder<'db> { } fn push_definition(&mut self, state: DefinitionState<'db>) -> ScopedDefinitionId { + self.checkpoint_state.record_binding_change(); let def_id = self.all_definitions.push(state); let used_id = self.used_bindings.push(false); debug_assert_eq!(def_id, used_id); @@ -1889,6 +1889,7 @@ impl<'db> UseDefMapBuilder<'db> { } pub(super) fn add_place(&mut self, place: ScopedPlaceId) { + self.checkpoint_state.record_binding_change(); match place { ScopedPlaceId::Symbol(symbol) => { let new_place = self.symbol_states.push(PendingPlaceState::new( @@ -1925,9 +1926,16 @@ impl<'db> UseDefMapBuilder<'db> { self.all_definitions.next_index() } - /// Identifies the latest definitions and control-flow path observed by an exception handler. - pub(super) fn exception_checkpoint_key(&self) -> (ScopedDefinitionId, ControlFlowRevision) { - (self.next_definition_id(), self.control_flow_revision) + /// Identifies the visible bindings and control-flow path observed by an exception handler. + pub(super) fn exception_checkpoint_key(&self) -> ExceptionCheckpointKey { + self.checkpoint_state + .key((!self.reachability_constraints.is_saturated()).then_some(self.checkpoint_flow)) + } + + /// Invalidates checkpoint reuse for narrowing that has no corresponding reachability predicate. + /// Match guards use this because their success and failure are not modeled in reachability. + pub(super) fn record_exception_checkpoint_binding_change(&mut self) { + self.checkpoint_state.record_binding_change(); } pub(super) fn record_binding( @@ -2214,6 +2222,7 @@ impl<'db> UseDefMapBuilder<'db> { symbol: ScopedSymbolId, pre_definition: SingleSymbolSnapshot, ) { + self.checkpoint_state.record_binding_change(); let negated_reachability_id = self .reachability_constraints .add_not_constraint(reachability_id); @@ -2284,6 +2293,7 @@ impl<'db> UseDefMapBuilder<'db> { &mut self, constraint: ScopedNarrowingConstraint, ) { + self.checkpoint_state.record_call_gate(); let pending = self.pending_reachability.current; for state in self .symbol_states @@ -2304,6 +2314,9 @@ impl<'db> UseDefMapBuilder<'db> { &mut self, constraint: ScopedReachabilityConstraintId, ) { + self.checkpoint_flow = self + .reachability_constraints + .add_and_constraint(self.checkpoint_flow, constraint); self.record_reachability_constraint_impl( constraint, ScopedNarrowingConstraint::ALWAYS_TRUE, @@ -2319,6 +2332,7 @@ impl<'db> UseDefMapBuilder<'db> { reachability_constraint: ScopedReachabilityConstraintId, narrowing_constraint: ScopedNarrowingConstraint, ) { + self.checkpoint_state.record_call_gate(); self.record_reachability_constraint_impl(reachability_constraint, narrowing_constraint); } @@ -2675,6 +2689,8 @@ impl<'db> UseDefMapBuilder<'db> { symbol_states: self.symbol_states.clone(), member_states: self.member_states.clone(), reachability: self.reachability, + checkpoint_flow: self.checkpoint_flow, + checkpoint_state: self.checkpoint_state.snapshot(), pending_reachability: self.pending_reachability.current, } } @@ -2702,7 +2718,7 @@ impl<'db> UseDefMapBuilder<'db> { /// Restore the current builder places state to the given snapshot. pub(super) fn restore(&mut self, snapshot: FlowSnapshot) { - self.control_flow_revision.advance(); + self.checkpoint_state.restore(snapshot.checkpoint_state); // We never remove places from `place_states` (it's an IndexVec, and the place // IDs must line up), so the current number of known places must always be equal to or // greater than the number of known places in a previously-taken snapshot. @@ -2714,6 +2730,7 @@ impl<'db> UseDefMapBuilder<'db> { self.symbol_states = snapshot.symbol_states; self.member_states = snapshot.member_states; self.reachability = snapshot.reachability; + self.checkpoint_flow = snapshot.checkpoint_flow; self.pending_reachability.current = snapshot.pending_reachability; // If the snapshot we are restoring is missing some places we've recorded since, we need @@ -2746,7 +2763,8 @@ impl<'db> UseDefMapBuilder<'db> { return; } - self.control_flow_revision.advance(); + self.checkpoint_state.merge(snapshot.checkpoint_state); + // We never remove places from `place_states` (it's an IndexVec, and the place // IDs must line up), so the current number of known places must always be equal to or // greater than the number of known places in a previously-taken snapshot. @@ -2774,6 +2792,9 @@ impl<'db> UseDefMapBuilder<'db> { self.reachability = self .reachability_constraints .add_or_constraint(self.reachability, snapshot.reachability); + self.checkpoint_flow = self + .reachability_constraints + .add_or_constraint(self.checkpoint_flow, snapshot.checkpoint_flow); } pub(super) fn finish(mut self: Box) -> UseDefMap<'db> { diff --git a/crates/ty_python_core/src/use_def/exception_checkpoint.rs b/crates/ty_python_core/src/use_def/exception_checkpoint.rs new file mode 100644 index 0000000000..61a6956e3e --- /dev/null +++ b/crates/ty_python_core/src/use_def/exception_checkpoint.rs @@ -0,0 +1,128 @@ +use crate::reachability_constraints::ScopedReachabilityConstraintId; + +#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] +struct Revision(u64); + +/// The provenance of the visible bindings and the call gates applied to them. +#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] +pub(super) struct ExceptionCheckpointSnapshot { + bindings: Revision, + calls: Revision, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum CheckpointFlow { + Normalized(ScopedReachabilityConstraintId), + Conservative(Revision), +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub(crate) struct ExceptionCheckpointKey { + bindings: Revision, + flow: CheckpointFlow, +} + +/// Tracks state changes that normalized scope-wide reachability cannot distinguish. +/// +/// Binding identities are restored with flow snapshots. Joining different identities creates a +/// fresh one, even if both paths have ambiguous reachability. Calls do not immediately change the +/// checkpoint key, since a later straight-line call cannot expose additional bindings. Their +/// identities still matter when restoring or joining paths: catching an exception from a +/// `NoReturn` call can make previously unreachable bindings visible again. +#[derive(Debug, Default)] +pub(super) struct ExceptionCheckpointState { + current: ExceptionCheckpointSnapshot, + next_revision: Revision, + control_flow_revision: Revision, +} + +impl ExceptionCheckpointState { + fn fresh_revision(&mut self) -> Revision { + self.next_revision.0 += 1; + self.next_revision + } + + pub(super) fn record_binding_change(&mut self) { + self.current.bindings = self.fresh_revision(); + } + + pub(super) fn record_call_gate(&mut self) { + self.current.calls = self.fresh_revision(); + } + + pub(super) fn snapshot(&self) -> ExceptionCheckpointSnapshot { + self.current + } + + pub(super) fn restore(&mut self, snapshot: ExceptionCheckpointSnapshot) { + let calls_changed = self.current.calls != snapshot.calls; + self.control_flow_revision = self.fresh_revision(); + self.current = snapshot; + if calls_changed { + self.current.bindings = self.control_flow_revision; + } + } + + pub(super) fn merge(&mut self, snapshot: ExceptionCheckpointSnapshot) { + self.control_flow_revision = self.fresh_revision(); + if self.current != snapshot { + if self.current.calls != snapshot.calls { + self.current.calls = self.control_flow_revision; + } + self.current.bindings = self.control_flow_revision; + } + } + + /// Uses the conservative control-flow revision when the reachability arena is saturated. + pub(super) fn key( + &self, + normalized_flow: Option, + ) -> ExceptionCheckpointKey { + ExceptionCheckpointKey { + bindings: self.current.bindings, + flow: normalized_flow.map_or( + CheckpointFlow::Conservative(self.control_flow_revision), + CheckpointFlow::Normalized, + ), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const FLOW: Option = + Some(ScopedReachabilityConstraintId::AMBIGUOUS); + + #[test] + fn unchanged_branches_preserve_binding_identity() { + let mut state = ExceptionCheckpointState::default(); + state.record_binding_change(); + state.record_call_gate(); + let snapshot = state.snapshot(); + let key = state.key(FLOW); + + state.restore(snapshot); + state.merge(snapshot); + assert_eq!(state.key(FLOW), key); + } + + #[test] + fn conservative_keys_still_coalesce_straight_line_calls() { + let mut state = ExceptionCheckpointState::default(); + state.record_binding_change(); + let key = state.key(None); + assert_ne!(key, state.key(FLOW)); + state.record_call_gate(); + state.record_call_gate(); + assert_eq!(state.key(None), key); + + let snapshot = state.snapshot(); + state.restore(snapshot); + let restored_key = state.key(None); + assert_ne!(restored_key, key); + state.merge(snapshot); + assert_ne!(state.key(None), restored_key); + } +} diff --git a/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md b/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md index 96adadc938..1c47f64516 100644 --- a/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md +++ b/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md @@ -246,6 +246,20 @@ def repeated_calls() -> None: reveal_type(state) # revealed: Literal[0, "changed"] ``` +A branch that does not change any bindings preserves the state visible to the handler: + +```py +def unchanged_branch(flag: bool) -> None: + state = 0 + try: + may_raise() + if flag is True: + pass + may_raise() + except: + reveal_type(state) # revealed: Literal[0] +``` + Branch narrowing changes the state visible to the handler even when neither branch introduces a new binding: @@ -272,6 +286,20 @@ def restored_branches(value: int | None) -> None: reveal_type(value) # revealed: int | None ``` +Match guards also distinguish successful and failed branches without introducing a new binding: + +```py +def guarded_match_branches(value: int | None) -> None: + try: + match value: + case _ if value is not None: + may_raise() + case _: + may_raise() + except: + reveal_type(value) # revealed: int | None +``` + Deleting a binding changes the flow state even though the name remains present in the scope: ```py @@ -304,6 +332,54 @@ def call_never_returns() -> None: reveal_type(state) # revealed: Literal[0] ``` +## Nested handlers with merged bindings + +An inner handler can preserve the original binding while its `else` suite sees a later assignment. +After those paths merge, an exception must expose both bindings to the outer handler: + +```py +def may_raise() -> None: ... +def nested_try() -> None: + state = 0 + try: + try: + may_raise() + state = "changed" + except: + pass + else: + may_raise() + may_raise() + except: + reveal_type(state) # revealed: Literal[0, "changed"] +``` + +## Caught calls that never return + +Catching an exception from a `NoReturn` call makes the following code reachable again, even if no +bindings changed. The unreachable inner `else` suite must not hide the later exception: + +```py +from typing import NoReturn + +def may_raise() -> None: ... +def stop() -> NoReturn: + raise RuntimeError + +def nested_terminal() -> None: + state = 0 + try: + try: + stop() + except: + pass + else: + may_raise() + may_raise() + except: + reveal_type(state) # revealed: Literal[0] +``` + ## Operators and augmented assignments An arithmetic operator can raise after evaluating both operands: From 82e37beef9b39da7ad66cd27ab94edec8c187eb1 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 18 Aug 2026 00:55:41 +0100 Subject: [PATCH 076/371] [ty] Preserve TypeAliasType runtime origin (#27813) On Python 3.12, aliases declared with `type Alias = int` are instances of `typing.TypeAliasType`, but ty currently infers them as instances of `typing_extensions.TypeAliasType` because both classes share a `KnownClass` variant that resolves to the backport. - Distinguish the standard-library and backport `TypeAliasType` classes with separate `KnownClass` variants. - Generalize `TypedDictModule` into a shared `TypingModule` enum and retain the actual constructor origin on manually created aliases through specialization and materialization. - Update alias construction, runtime-class fallback, TypedDict handling, and IDE argument classification to use the correct module. - Cover statement-defined aliases and both direct constructors on Python 3.12, and update `__type_params__` inference to reflect the standard-library class. The fact that we fail to model this precisely right now appears to cause a surprising number of ecosystem diagnostics on bokeh. --- .../resources/mdtest/pep695_type_aliases.md | 29 ++++++++- crates/ty_python_semantic/src/types.rs | 59 ++++++++++++++++++- crates/ty_python_semantic/src/types/class.rs | 4 +- .../src/types/class/known.rs | 24 ++++++-- .../src/types/class/static_literal.rs | 4 +- .../src/types/class/typed_dict.rs | 18 +++--- .../src/types/class_base.rs | 10 ++-- .../ty_python_semantic/src/types/display.rs | 4 +- .../src/types/ide_support.rs | 9 ++- .../src/types/infer/builder.rs | 37 +++++++++--- .../src/types/infer/builder/class.rs | 4 +- .../builder/post_inference/static_class.rs | 4 +- .../src/types/infer/builder/typed_dict.rs | 8 +-- .../src/types/known_instance.rs | 2 +- .../types/property_tests/type_generation.rs | 2 +- .../src/types/special_form.rs | 58 +++--------------- .../src/types/type_alias.rs | 19 +++++- 17 files changed, 196 insertions(+), 99 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md b/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md index 882ceeb661..9423aa324f 100644 --- a/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md +++ b/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md @@ -23,6 +23,32 @@ def f() -> None: reveal_type(x) # revealed: int | str ``` +## Runtime classes + +On Python 3.12, aliases defined by a `type` statement or the `typing.TypeAliasType` constructor are +instances of the standard-library class, while aliases created with +`typing_extensions.TypeAliasType` are instances of the distinct backport class. + +```py +from typing import TypeAliasType as StdlibTypeAliasType +from typing_extensions import TypeAliasType as ExtensionsTypeAliasType +from ty_extensions import static_assert +from ty_extensions._internal import TypeOf, is_subtype_of + +type StatementAlias = int +StdlibAlias = StdlibTypeAliasType("StdlibAlias", int) +ExtensionsAlias = ExtensionsTypeAliasType("ExtensionsAlias", int) + +static_assert(is_subtype_of(TypeOf[StatementAlias], StdlibTypeAliasType)) +static_assert(not is_subtype_of(TypeOf[StatementAlias], ExtensionsTypeAliasType)) + +static_assert(is_subtype_of(TypeOf[StdlibAlias], StdlibTypeAliasType)) +static_assert(not is_subtype_of(TypeOf[StdlibAlias], ExtensionsTypeAliasType)) + +static_assert(is_subtype_of(TypeOf[ExtensionsAlias], ExtensionsTypeAliasType)) +static_assert(not is_subtype_of(TypeOf[ExtensionsAlias], StdlibTypeAliasType)) +``` + ## Type aliases in `type[...]` ```py @@ -203,7 +229,8 @@ def _(flag: bool): ```py type ListOrSet[T] = list[T] | set[T] -reveal_type(ListOrSet.__type_params__) # revealed: tuple[TypeVar | ParamSpec | TypeVarTuple, ...] +# revealed: tuple[typing.TypeVar | typing_extensions.TypeVar | typing.ParamSpec | typing_extensions.ParamSpec | typing.TypeVarTuple | typing_extensions.TypeVarTuple, ...] +reveal_type(ListOrSet.__type_params__) type Tuple1[T] = tuple[T] def _(cond: bool): diff --git a/crates/ty_python_semantic/src/types.rs b/crates/ty_python_semantic/src/types.rs index f53d43c7e9..9e1f3d0bf1 100644 --- a/crates/ty_python_semantic/src/types.rs +++ b/crates/ty_python_semantic/src/types.rs @@ -113,7 +113,6 @@ pub(crate) use literal::{ BytesLiteralType, EnumLiteralType, LiteralValueType, LiteralValueTypeKind, StringLiteralType, }; pub use special_form::SpecialFormType; -pub(crate) use special_form::TypedDictModule; use ty_python_core::definition::{Definition, DefinitionKind}; use ty_python_core::place::ScopedPlaceId; use ty_python_core::scope::ScopeId; @@ -494,6 +493,61 @@ pub(crate) struct FindLegacyTypeVars; type SpecializationVisitor<'db> = CycleDetector<'db, VisitSpecialization, Type<'db>, (), 3>; struct VisitSpecialization; +/// The standard-library `typing` module or its `typing_extensions` backport. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, get_size2::GetSize)] +pub enum TypingModule { + /// The standard-library `typing` module. + Typing, + /// The `typing_extensions` backport. + TypingExtensions, +} + +impl TypingModule { + /// Return the module for a `TypedDict` special form, including a union of the special forms + /// exported by `typing` and `typing_extensions`. + fn from_typed_dict_type<'db>(db: &'db dyn Db, ty: Type<'db>) -> Option { + match ty { + Type::SpecialForm(SpecialFormType::TypedDict(module)) => Some(module), + Type::Union(union) => { + let mut elements = union.elements(db).iter(); + let Type::SpecialForm(SpecialFormType::TypedDict(module)) = elements.next()? else { + return None; + }; + elements.try_fold(*module, |module, element| { + let Type::SpecialForm(SpecialFormType::TypedDict(element_module)) = element + else { + return None; + }; + // `typing_extensions.TypedDict` always offers strictly more functionality than `typing.TypedDict`. + // If any element is from `typing`, we therefore infer that the type is a `typing.TypedDict`, + // since an operation on a union is only valid if the operation is valid on all elements in the + // union. + Some(match (module, element_module) { + (Self::TypingExtensions, Self::TypingExtensions) => Self::TypingExtensions, + _ => Self::Typing, + }) + }) + } + _ => None, + } + } + + const fn from_type_alias_class(class: KnownClass) -> Option { + match class { + KnownClass::TypeAliasType => Some(Self::Typing), + KnownClass::ExtensionsTypeAliasType => Some(Self::TypingExtensions), + _ => None, + } + } + + const fn type_alias_class(self) -> KnownClass { + match self { + Self::Typing => KnownClass::TypeAliasType, + Self::TypingExtensions => KnownClass::ExtensionsTypeAliasType, + } + } +} + /// Whether a type represents the upper or lower bound of a gradual type. /// /// For generic specializations, this matters only if there is at least one invariant or constrained @@ -6034,7 +6088,7 @@ impl<'db> Type<'db> { ) } - KnownClass::TypeAliasType => { + KnownClass::TypeAliasType | KnownClass::ExtensionsTypeAliasType => { // ```py // def __new__( // cls, @@ -6330,6 +6384,7 @@ impl<'db> Type<'db> { | KnownClass::Property | KnownClass::Super | KnownClass::TypeAliasType + | KnownClass::ExtensionsTypeAliasType | KnownClass::Deprecated ) ) { diff --git a/crates/ty_python_semantic/src/types/class.rs b/crates/ty_python_semantic/src/types/class.rs index 7221d2a199..f796a3ae5d 100644 --- a/crates/ty_python_semantic/src/types/class.rs +++ b/crates/ty_python_semantic/src/types/class.rs @@ -45,7 +45,7 @@ use crate::types::tuple::TupleSpec; use crate::types::typevar::TypeVarSet; use crate::types::{ ApplyTypeMappingVisitor, CallableType, CallableTypes, DataclassParams, - FindLegacyTypeVarsVisitor, IntersectionType, TypeContext, TypeMapping, TypedDictModule, + FindLegacyTypeVarsVisitor, IntersectionType, TypeContext, TypeMapping, TypingModule, UnionBuilder, VarianceInferable, }; use crate::{ @@ -3130,7 +3130,7 @@ pub(super) enum ClassMemberResult<'db> { /// Found the member or exhausted the MRO. Done(CompletedMemberLookup<'db>), /// Encountered a `TypedDict` base. - TypedDict(TypedDictModule), + TypedDict(TypingModule), } pub(super) struct CompletedMemberLookup<'db> { diff --git a/crates/ty_python_semantic/src/types/class/known.rs b/crates/ty_python_semantic/src/types/class/known.rs index 480a792b8c..043f6a270f 100644 --- a/crates/ty_python_semantic/src/types/class/known.rs +++ b/crates/ty_python_semantic/src/types/class/known.rs @@ -112,6 +112,7 @@ pub enum KnownClass { TypeVarTuple, ExtensionsTypeVarTuple, // must be distinct from typing.TypeVarTuple, backports new features TypeAliasType, + ExtensionsTypeAliasType, // may be distinct from typing.TypeAliasType NoDefaultType, NewType, Hashable, @@ -193,6 +194,7 @@ impl KnownClass { | Self::FunctionType | Self::VersionInfo | Self::TypeAliasType + | Self::ExtensionsTypeAliasType | Self::TypeVar | Self::ExtensionsTypeVar | Self::ParamSpec @@ -373,6 +375,7 @@ impl KnownClass { | KnownClass::ExtensionsTypeVarTuple | KnownClass::Sentinel | KnownClass::TypeAliasType + | KnownClass::ExtensionsTypeAliasType | KnownClass::NoDefaultType | KnownClass::NewType | KnownClass::Hashable @@ -487,6 +490,7 @@ impl KnownClass { | KnownClass::ExtensionsTypeVarTuple | KnownClass::Sentinel | KnownClass::TypeAliasType + | KnownClass::ExtensionsTypeAliasType | KnownClass::NoDefaultType | KnownClass::NewType | KnownClass::Hashable @@ -602,6 +606,7 @@ impl KnownClass { | KnownClass::ExtensionsTypeVarTuple | KnownClass::Sentinel | KnownClass::TypeAliasType + | KnownClass::ExtensionsTypeAliasType | KnownClass::NoDefaultType | KnownClass::NewType | KnownClass::Hashable @@ -724,6 +729,7 @@ impl KnownClass { | Self::ExtensionsTypeVarTuple | Self::Sentinel | Self::TypeAliasType + | Self::ExtensionsTypeAliasType | Self::NoDefaultType | Self::NewType | Self::ChainMap @@ -849,6 +855,7 @@ impl KnownClass { | KnownClass::ExtensionsTypeVarTuple | KnownClass::Sentinel | KnownClass::TypeAliasType + | KnownClass::ExtensionsTypeAliasType | KnownClass::NoDefaultType | KnownClass::NewType | KnownClass::Hashable @@ -946,7 +953,7 @@ impl KnownClass { Self::TypeVarTuple => "TypeVarTuple", Self::ExtensionsTypeVarTuple => "TypeVarTuple", Self::Sentinel => "sentinel", - Self::TypeAliasType => "TypeAliasType", + Self::TypeAliasType | Self::ExtensionsTypeAliasType => "TypeAliasType", Self::NoDefaultType => "_NoDefaultType", Self::NewType => "NewType", Self::Hashable => "Hashable", @@ -1374,7 +1381,7 @@ impl KnownClass { | Self::ParamSpec | Self::Hashable | Self::SupportsIndex => KnownModule::Typing, - Self::TypeAliasType + Self::ExtensionsTypeAliasType | Self::ExtensionsTypeVar | Self::ExtensionsTypeVarTuple | Self::ExtensionsParamSpec @@ -1383,6 +1390,13 @@ impl KnownClass { | Self::Deprecated | Self::ExtensionTypedDictFallback | Self::NewType => KnownModule::TypingExtensions, + Self::TypeAliasType => { + if python_version >= PythonVersion::PY312 { + KnownModule::Typing + } else { + KnownModule::TypingExtensions + } + } Self::TypeVarTuple => { if python_version >= PythonVersion::PY311 { KnownModule::Typing @@ -1494,6 +1508,7 @@ impl KnownClass { | Self::AsyncGenerator | Self::Deprecated | Self::TypeAliasType + | Self::ExtensionsTypeAliasType | Self::TypeVar | Self::ExtensionsTypeVar | Self::ParamSpec @@ -1603,7 +1618,7 @@ impl KnownClass { "WrapperDescriptorType" => &[Self::WrapperDescriptorType], "BuiltinFunctionType" => &[Self::BuiltinFunctionType], "NewType" => &[Self::NewType], - "TypeAliasType" => &[Self::TypeAliasType], + "TypeAliasType" => &[Self::TypeAliasType, Self::ExtensionsTypeAliasType], "TypeVar" => &[Self::TypeVar, Self::ExtensionsTypeVar], "Iterable" => &[Self::Iterable, Self::TyExtensionsIterable], "Iterator" => &[Self::Iterator, Self::TyExtensionsIterator], @@ -1746,6 +1761,8 @@ impl KnownClass { | Self::ExtensionsParamSpec | Self::TypeVarTuple | Self::ExtensionsTypeVarTuple + | Self::TypeAliasType + | Self::ExtensionsTypeAliasType | Self::Sentinel | Self::NamedTupleLike | Self::ConstraintSet @@ -1774,7 +1791,6 @@ impl KnownClass { Self::NoneType => matches!(module, KnownModule::Typeshed | KnownModule::Types), Self::SpecialForm - | Self::TypeAliasType | Self::NoDefaultType | Self::Hashable | Self::SupportsIndex diff --git a/crates/ty_python_semantic/src/types/class/static_literal.rs b/crates/ty_python_semantic/src/types/class/static_literal.rs index c4c475ab09..d12bdc765b 100644 --- a/crates/ty_python_semantic/src/types/class/static_literal.rs +++ b/crates/ty_python_semantic/src/types/class/static_literal.rs @@ -25,7 +25,7 @@ use crate::{ DataclassParams, GenericAlias, GenericContext, KnownClass, KnownInstanceType, MaterializationKind, MemberLookupPolicy, MetaclassCandidate, MetaclassTransformInfo, Parameter, Parameters, PropertyInstanceType, Signature, SpecialFormType, StaticMroError, - SubclassOfType, Type, TypeContext, TypeMapping, TypeVarVariance, TypedDictModule, + SubclassOfType, Type, TypeContext, TypeMapping, TypeVarVariance, TypingModule, UnionBuilder, UnionType, bound_super::BoundSuperType, call::{CallError, CallErrorKind}, @@ -961,7 +961,7 @@ impl<'db> StaticClassLiteral<'db> { /// Return the module defining the `TypedDict` base of this class. #[salsa::tracked(returns(copy), cycle_initial=|_, _, _| None, heap_size=ruff_memory_usage::heap_size)] - pub(crate) fn typed_dict_module(self, db: &'db dyn Db) -> Option { + pub(crate) fn typed_dict_module(self, db: &'db dyn Db) -> Option { self.iter_mro(db, None) .find_map(ClassBase::typed_dict_module) } diff --git a/crates/ty_python_semantic/src/types/class/typed_dict.rs b/crates/ty_python_semantic/src/types/class/typed_dict.rs index fa2a1586fc..a17ff7e0de 100644 --- a/crates/ty_python_semantic/src/types/class/typed_dict.rs +++ b/crates/ty_python_semantic/src/types/class/typed_dict.rs @@ -22,8 +22,8 @@ use crate::types::typed_dict::{ }; use crate::types::{ BoundTypeVarInstance, CallableType, ClassBase, ClassLiteral, ClassType, KnownClass, - MemberLookupPolicy, Type, TypeContext, TypeMapping, TypeVarVariance, TypedDictModule, - TypedDictType, UnionType, determine_upper_bound, + MemberLookupPolicy, Type, TypeContext, TypeMapping, TypeVarVariance, TypedDictType, + TypingModule, UnionType, determine_upper_bound, }; use crate::{Db, FxIndexMap}; use ty_python_core::definition::Definition; @@ -900,7 +900,7 @@ pub struct DynamicTypedDictLiteral<'db> { pub(crate) anchor: DynamicTypedDictAnchor<'db>, #[returns(copy)] - pub(crate) typed_dict_module: TypedDictModule, + pub(crate) typed_dict_module: TypingModule, } impl get_size2::GetSize for DynamicTypedDictLiteral<'_> {} @@ -1060,7 +1060,7 @@ pub(in crate::types) fn synthesized_typed_dict_class_member<'db>( db, env, typed_dict, - TypedDictModule::Typing, + TypingModule::Typing, lookup_policy, name, || Type::TypedDict(typed_dict), @@ -1070,13 +1070,13 @@ pub(in crate::types) fn synthesized_typed_dict_class_member<'db>( pub(super) fn typed_dict_fallback_class_member<'db>( db: &'db dyn Db, env: &ProgramEnvironment<'db>, - module: TypedDictModule, + module: TypingModule, lookup_policy: MemberLookupPolicy, name: &str, ) -> PlaceAndQualifiers<'db> { let fallback = match module { - TypedDictModule::Typing => KnownClass::TypedDictFallback, - TypedDictModule::TypingExtensions => KnownClass::ExtensionTypedDictFallback, + TypingModule::Typing => KnownClass::TypedDictFallback, + TypingModule::TypingExtensions => KnownClass::ExtensionTypedDictFallback, }; fallback @@ -1089,7 +1089,7 @@ pub(super) fn typed_dict_class_member<'db>( db: &'db dyn Db, env: &ProgramEnvironment<'db>, class: ClassType<'db>, - module: TypedDictModule, + module: TypingModule, lookup_policy: MemberLookupPolicy, name: &str, ) -> PlaceAndQualifiers<'db> { @@ -1110,7 +1110,7 @@ fn typed_dict_inherited_class_member<'db>( db: &'db dyn Db, env: &ProgramEnvironment<'db>, typed_dict: TypedDictType<'db>, - module: TypedDictModule, + module: TypingModule, lookup_policy: MemberLookupPolicy, name: &str, new_upper_bound: impl FnOnce() -> Type<'db>, diff --git a/crates/ty_python_semantic/src/types/class_base.rs b/crates/ty_python_semantic/src/types/class_base.rs index f9796ac60a..4d9c6a6157 100644 --- a/crates/ty_python_semantic/src/types/class_base.rs +++ b/crates/ty_python_semantic/src/types/class_base.rs @@ -8,7 +8,7 @@ use crate::types::tuple::TupleType; use crate::types::{ ApplyTypeMappingVisitor, ClassLiteral, ClassType, DivergentType, DynamicType, KnownClass, KnownInstanceType, MaterializationKind, SpecialFormType, StaticMroError, Type, TypeContext, - TypeMapping, TypedDictModule, todo_type, + TypeMapping, TypingModule, todo_type, }; use crate::{Db, DisplaySettings}; @@ -37,7 +37,7 @@ pub enum ClassBase<'db> { /// but nonetheless appears in the MRO of classes that inherit from `Generic[T]`, /// `Protocol[T]`, or bare `Protocol`. Generic, - TypedDict(TypedDictModule), + TypedDict(TypingModule), } impl<'db> ClassBase<'db> { @@ -91,7 +91,7 @@ impl<'db> ClassBase<'db> { self.typed_dict_module().is_some() } - pub(super) const fn typed_dict_module(self) -> Option { + pub(super) const fn typed_dict_module(self) -> Option { match self { ClassBase::TypedDict(module) => Some(module), _ => None, @@ -104,7 +104,7 @@ impl<'db> ClassBase<'db> { /// pseudo-base when detecting duplicate or conflicting bases. pub(super) const fn mro_identity(self) -> Self { match self { - Self::TypedDict(_) => Self::TypedDict(TypedDictModule::Typing), + Self::TypedDict(_) => Self::TypedDict(TypingModule::Typing), _ => self, } } @@ -164,7 +164,7 @@ impl<'db> ClassBase<'db> { } } Type::Union(union) => { - if let Some(module) = TypedDictModule::from_type(db, ty) { + if let Some(module) = TypingModule::from_typed_dict_type(db, ty) { return Some(ClassBase::TypedDict(module)); } diff --git a/crates/ty_python_semantic/src/types/display.rs b/crates/ty_python_semantic/src/types/display.rs index b477932a5f..ba00df5b55 100644 --- a/crates/ty_python_semantic/src/types/display.rs +++ b/crates/ty_python_semantic/src/types/display.rs @@ -35,7 +35,7 @@ use crate::types::{ CallableType, IntersectionType, KnownBoundMethodType, KnownClass, KnownInstanceType, KnownUnion, LiteralValueType, LiteralValueTypeKind, MaterializationKind, PropertyInstanceType, Protocol, SpecialFormType, StringLiteralType, SubclassOfInner, SubclassOfType, Type, - TypeAliasType, TypeGuardLike, TypedDictModule, TypedDictType, UnionType, WrapperDescriptorKind, + TypeAliasType, TypeGuardLike, TypedDictType, TypingModule, UnionType, WrapperDescriptorKind, visitor, }; use ty_python_core::ProgramFile; @@ -1540,7 +1540,7 @@ impl<'db> FmtDetailed<'db> for DisplayRepresentation<'_, 'db> { f.set_invalid_type_annotation(); f.write_char('<')?; f.with_type(Type::SpecialForm(SpecialFormType::TypedDict( - TypedDictModule::Typing, + TypingModule::Typing, ))) .write_str("TypedDict")?; f.write_str(" with items ")?; diff --git a/crates/ty_python_semantic/src/types/ide_support.rs b/crates/ty_python_semantic/src/types/ide_support.rs index d69ed2a015..d9d516b293 100644 --- a/crates/ty_python_semantic/src/types/ide_support.rs +++ b/crates/ty_python_semantic/src/types/ide_support.rs @@ -1643,7 +1643,14 @@ fn known_type_form_parameter_index(db: &dyn Db, callable_type: Type<'_>) -> Opti Some(KnownFunction::AssertType) => Some(1), _ => None, }, - Type::ClassLiteral(class) if class.is_known(db, KnownClass::TypeAliasType) => Some(1), + Type::ClassLiteral(class) + if matches!( + class.known(db), + Some(KnownClass::TypeAliasType | KnownClass::ExtensionsTypeAliasType) + ) => + { + Some(1) + } _ => None, } } diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index 4c6f513bf7..e1bdf7548a 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -120,7 +120,7 @@ use crate::types::{ LiteralValueType, LiteralValueTypeKind, MemberLookupPolicy, ParamSpecAttrKind, Parameter, Parameters, ProgramEnvironment, SentinelInstance, Signature, SpecialFormType, SubclassOfType, Type, TypeAliasType, TypeAndQualifiers, TypeContext, TypeQualifiers, TypeVarBoundOrConstraints, - TypeVarKind, TypeVarVariance, TypedDictModule, UnionAccumulator, UnionBuilder, UnionType, + TypeVarKind, TypeVarVariance, TypingModule, UnionAccumulator, UnionBuilder, UnionType, any_over_type, binding_type, extract_fixed_length_iterable_element_types, infer_complete_scope_types, infer_scope_types, is_discarded_dict_key_assignment, todo_type, }; @@ -3299,7 +3299,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { namedtuple_kind, ) } else if let Some(typed_dict_module) = - TypedDictModule::from_type(self.db(), callable_type) + TypingModule::from_typed_dict_type(self.db(), callable_type) { self.infer_typeddict_call_expression( call_expr, @@ -3357,8 +3357,16 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // signalling that we must fall back to normal call inference. self.infer_builtins_type_call(call_expr, Some(definition)) } - Some(KnownClass::TypeAliasType) => { - self.infer_typealiastype_call(target, call_expr, definition) + Some(known_class) + if let Some(typing_module) = + TypingModule::from_type_alias_class(known_class) => + { + self.infer_typealiastype_call( + target, + call_expr, + definition, + typing_module, + ) } Some(KnownClass::Sentinel) => self .infer_sentinel_expression(target, call_expr, definition) @@ -3626,7 +3634,10 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { self.infer_newtype_assignment_deferred(arguments); return; } - (Some(KnownClass::TypeAliasType), InferenceRegion::Deferred(definition)) => { + ( + Some(KnownClass::TypeAliasType | KnownClass::ExtensionsTypeAliasType), + InferenceRegion::Deferred(definition), + ) => { self.infer_typealiastype_assignment_deferred(definition, arguments); return; } @@ -3636,7 +3647,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { } _ => {} } - if TypedDictModule::from_type(self.db(), func_ty).is_some() { + if TypingModule::from_typed_dict_type(self.db(), func_ty).is_some() { self.infer_functional_typeddict_deferred(arguments); return; } @@ -3778,6 +3789,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { target: &ast::Expr, call_expr: &ast::ExprCall, definition: Definition<'db>, + typing_module: TypingModule, ) -> Type<'db> { fn error<'db>( context: &InferContext<'db, '_>, @@ -3849,7 +3861,12 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { Type::KnownInstance(KnownInstanceType::TypeAliasType( TypeAliasType::ManualPEP695(ManualPEP695TypeAliasType::new( - db, name, definition, None, None, + db, + name, + definition, + typing_module, + None, + None, )), )) } @@ -8879,7 +8896,9 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { return ty; } - if let Some(typed_dict_module) = TypedDictModule::from_type(self.db(), callable_type) { + if let Some(typed_dict_module) = + TypingModule::from_typed_dict_type(self.db(), callable_type) + { return self.infer_typeddict_call_expression(call_expression, None, typed_dict_module); } @@ -9208,7 +9227,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { ); } } - Some(KnownClass::TypeAliasType) => { + Some(KnownClass::TypeAliasType | KnownClass::ExtensionsTypeAliasType) => { if let Some(builder) = self .context .report_lint(&INVALID_TYPE_ALIAS_TYPE, call_expression) diff --git a/crates/ty_python_semantic/src/types/infer/builder/class.rs b/crates/ty_python_semantic/src/types/infer/builder/class.rs index db731bfdb6..be1a97bf7c 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/class.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/class.rs @@ -3,7 +3,7 @@ use crate::ProgramEnvironment; use crate::place::Place; use crate::types::{ CallArguments, DataclassParams, KnownClass, KnownInstanceType, MemberLookupPolicy, - SpecialFormType, StaticClassLiteral, SubclassOfType, Type, TypeContext, TypedDictModule, + SpecialFormType, StaticClassLiteral, SubclassOfType, Type, TypeContext, TypingModule, call::CallError, callable::CallableFunctionProvenance, function::KnownFunction, @@ -55,7 +55,7 @@ impl<'db> TypeInferenceBuilder<'db, '_> { self.infer_expression(base, TypeContext::default()) }; is_typed_dict |= match ty { - ty if TypedDictModule::from_type(self.db(), ty).is_some() => true, + ty if TypingModule::from_typed_dict_type(self.db(), ty).is_some() => true, Type::ClassLiteral(class) => class.is_typed_dict(self.db()), Type::GenericAlias(alias) => alias.is_typed_dict(self.db()), _ => false, diff --git a/crates/ty_python_semantic/src/types/infer/builder/post_inference/static_class.rs b/crates/ty_python_semantic/src/types/infer/builder/post_inference/static_class.rs index 24059e69e6..82668ddbaa 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/post_inference/static_class.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/post_inference/static_class.rs @@ -16,7 +16,7 @@ use crate::{ types::{ CallArguments, ClassBase, ClassLiteral, ClassType, DataclassFlags, KnownClass, KnownInstanceType, MemberLookupPolicy, MetaclassCandidate, Parameters, Signature, - SpecialFormType, StaticClassLiteral, Type, TypeVarVariance, TypedDictModule, binding_type, + SpecialFormType, StaticClassLiteral, Type, TypeVarVariance, TypingModule, binding_type, call::Argument, class::{ AbstractMethod, CodeGeneratorKind, Field, FieldKind, MetaclassErrorKind, @@ -670,7 +670,7 @@ pub(crate) fn check_static_class_definitions<'db>( if let Some(args) = class_node.arguments.as_deref() { if class_kind == Some(CodeGeneratorKind::TypedDict) { let supports_pep_728 = context.in_stub() - || class.typed_dict_module(db) == Some(TypedDictModule::TypingExtensions) + || class.typed_dict_module(db) == Some(TypingModule::TypingExtensions) || env.python_version(db) >= PythonVersion::PY315; for keyword in &args.keywords { diff --git a/crates/ty_python_semantic/src/types/infer/builder/typed_dict.rs b/crates/ty_python_semantic/src/types/infer/builder/typed_dict.rs index 9abc4a8a92..5a00b7e76f 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/typed_dict.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/typed_dict.rs @@ -19,8 +19,8 @@ use crate::types::typed_dict::{ validate_typed_dict_constructor, validate_typed_dict_dict_literal, }; use crate::types::{ - ClassType, IntersectionType, KnownClass, Type, TypeAndQualifiers, TypeContext, TypedDictModule, - TypedDictType, any_over_type, + ClassType, IntersectionType, KnownClass, Type, TypeAndQualifiers, TypeContext, TypedDictType, + TypingModule, any_over_type, }; use crate::{Db, ProgramEnvironment, TypeQualifiers}; use ty_python_core::definition::Definition; @@ -99,7 +99,7 @@ impl<'db> TypeInferenceBuilder<'db, '_> { &mut self, call_expr: &ast::ExprCall, definition: Option>, - typed_dict_module: TypedDictModule, + typed_dict_module: TypingModule, ) -> Type<'db> { let env = self.program_environment(); let db = self.db(); @@ -173,7 +173,7 @@ impl<'db> TypeInferenceBuilder<'db, '_> { let mut closed = false; let mut extra_items = None; let supports_pep_728 = self.in_stub() - || typed_dict_module == TypedDictModule::TypingExtensions + || typed_dict_module == TypingModule::TypingExtensions || self.program_environment().python_version(db) >= PythonVersion::PY315; for kw in keywords { diff --git a/crates/ty_python_semantic/src/types/known_instance.rs b/crates/ty_python_semantic/src/types/known_instance.rs index 2d52894179..eed10ac11d 100644 --- a/crates/ty_python_semantic/src/types/known_instance.rs +++ b/crates/ty_python_semantic/src/types/known_instance.rs @@ -319,7 +319,7 @@ impl<'db> KnownInstanceType<'db> { Self::TypeAliasType(alias) if alias.specialization(db).is_some() => { KnownClass::GenericAlias } - Self::TypeAliasType(_) => KnownClass::TypeAliasType, + Self::TypeAliasType(alias) => alias.known_class(db), Self::Deprecated(_) => KnownClass::Deprecated, Self::Field(_) => KnownClass::Field, Self::ConstraintSet(_) => KnownClass::ConstraintSet, diff --git a/crates/ty_python_semantic/src/types/property_tests/type_generation.rs b/crates/ty_python_semantic/src/types/property_tests/type_generation.rs index fa14aa00f9..69d8fd6359 100644 --- a/crates/ty_python_semantic/src/types/property_tests/type_generation.rs +++ b/crates/ty_python_semantic/src/types/property_tests/type_generation.rs @@ -486,7 +486,7 @@ fn arbitrary_core_type(g: &mut Gen, fully_static: bool) -> Ty { Ty::KnownClassInstance(KnownClass::FunctionType), Ty::KnownClassInstance(KnownClass::SpecialForm), Ty::KnownClassInstance(KnownClass::TypeVar), - Ty::KnownClassInstance(KnownClass::TypeAliasType), + Ty::KnownClassInstance(KnownClass::ExtensionsTypeAliasType), Ty::KnownClassInstance(KnownClass::NoDefaultType), Ty::TypingLiteral, Ty::UnittestMockLiteral, diff --git a/crates/ty_python_semantic/src/types/special_form.rs b/crates/ty_python_semantic/src/types/special_form.rs index c9586d3893..cbefcd0838 100644 --- a/crates/ty_python_semantic/src/types/special_form.rs +++ b/crates/ty_python_semantic/src/types/special_form.rs @@ -1,7 +1,7 @@ //! An enumeration of special forms in the Python type system. //! Each of these is considered to inhabit a unique type in our model of the type system. -use super::{ClassType, Type, TypeFormType, class::KnownClass}; +use super::{ClassType, Type, TypeFormType, TypingModule, class::KnownClass}; use crate::ProgramEnvironment; use crate::db::Db; use crate::types::IntersectionType; @@ -110,7 +110,7 @@ pub enum SpecialFormType { /// The symbol `typing.TypeGuard` (which can also be found as `typing_extensions.TypeGuard`) TypeGuard, /// The symbol `typing.TypedDict` or `typing_extensions.TypedDict`. - TypedDict(TypedDictModule), + TypedDict(TypingModule), /// The symbol `typing.TypeIs` (which can also be found as `typing_extensions.TypeIs`) TypeIs, @@ -132,48 +132,6 @@ pub enum SpecialFormType { NamedTuple, } -/// The module or modules from which `TypedDict` may have been imported. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, get_size2::GetSize)] -pub enum TypedDictModule { - /// `typing.TypedDict`. - Typing, - /// `typing_extensions.TypedDict`. - TypingExtensions, -} - -impl TypedDictModule { - /// Return the module for a `TypedDict` special form, including a union of the special forms - /// exported by `typing` and `typing_extensions`. - pub(super) fn from_type<'db>(db: &'db dyn Db, ty: Type<'db>) -> Option { - match ty { - Type::SpecialForm(SpecialFormType::TypedDict(module)) => Some(module), - Type::Union(union) => { - let mut elements = union.elements(db).iter(); - let Type::SpecialForm(SpecialFormType::TypedDict(module)) = elements.next()? else { - return None; - }; - elements.try_fold(*module, |module, element| { - let Type::SpecialForm(SpecialFormType::TypedDict(element_module)) = element - else { - return None; - }; - // `typing_extensions.TypedDict` always offers strictly more functionality than `typing.TypedDict`. - // If any element is from `typing`, we therefore infer that the type is a `typing.TypedDict`, - // since an operation on a union is only valid if the operation is valid on all elements in the - // union. - Some(match (module, element_module) { - (TypedDictModule::TypingExtensions, TypedDictModule::TypingExtensions) => { - TypedDictModule::TypingExtensions - } - _ => TypedDictModule::Typing, - }) - }) - } - _ => None, - } - } -} - impl SpecialFormType { /// Return the [`KnownClass`] which this symbol is an instance of pub(crate) const fn class(self) -> KnownClass { @@ -489,8 +447,8 @@ impl SpecialFormType { SpecialFormTypeBuilder::Unpack => &[Self::Unpack], SpecialFormTypeBuilder::Tuple => &[Self::Tuple], SpecialFormTypeBuilder::TypedDict => &[ - Self::TypedDict(TypedDictModule::Typing), - Self::TypedDict(TypedDictModule::TypingExtensions), + Self::TypedDict(TypingModule::Typing), + Self::TypedDict(TypingModule::TypingExtensions), ], SpecialFormTypeBuilder::TypeOf => &[Self::TypeOf], SpecialFormTypeBuilder::List => { @@ -555,7 +513,7 @@ impl SpecialFormType { | Self::Tuple | Self::Type | Self::Generic - | Self::TypedDict(TypedDictModule::Typing) + | Self::TypedDict(TypingModule::Typing) | Self::TypingCallable => module.is_typing(), Self::Annotated @@ -594,7 +552,7 @@ impl SpecialFormType { KnownModule::CollectionsAbc | KnownModule::CollectionsAbcInternal ), - Self::TypedDict(TypedDictModule::TypingExtensions) => module.is_typing_extensions(), + Self::TypedDict(TypingModule::TypingExtensions) => module.is_typing_extensions(), } } @@ -799,8 +757,8 @@ impl SpecialFormType { &[KnownModule::Typing, KnownModule::TypingExtensions] } - SpecialFormType::TypedDict(TypedDictModule::Typing) => &[KnownModule::Typing], - SpecialFormType::TypedDict(TypedDictModule::TypingExtensions) => { + SpecialFormType::TypedDict(TypingModule::Typing) => &[KnownModule::Typing], + SpecialFormType::TypedDict(TypingModule::TypingExtensions) => { &[KnownModule::TypingExtensions] } diff --git a/crates/ty_python_semantic/src/types/type_alias.rs b/crates/ty_python_semantic/src/types/type_alias.rs index 6dbdc3b67e..a48bf167af 100644 --- a/crates/ty_python_semantic/src/types/type_alias.rs +++ b/crates/ty_python_semantic/src/types/type_alias.rs @@ -4,9 +4,9 @@ use std::fmt::Write; use crate::{ Db, FxOrderSet, types::{ - ApplyTypeMappingVisitor, BindingContext, BoundTypeVarIdentity, GenericContext, + ApplyTypeMappingVisitor, BindingContext, BoundTypeVarIdentity, GenericContext, KnownClass, KnownInstanceType, MaterializationKind, Type, TypeContext, TypeMapping, TypeVarVariance, - definition_expression_type, + TypingModule, definition_expression_type, display::qualified_name_components_from_scope, generics::{ApplySpecialization, Specialization, bind_typevar}, variance::VarianceInferable, @@ -147,6 +147,9 @@ pub struct ManualPEP695TypeAliasType<'db> { #[returns(copy)] pub definition: Definition<'db>, + #[returns(copy)] + pub(super) typing_module: TypingModule, + #[returns(copy)] pub(super) specialization: Option>, @@ -223,6 +226,7 @@ impl<'db> ManualPEP695TypeAliasType<'db> { db, self.name(db), self.definition(db), + self.typing_module(db), Some(f(generic_context)), self.materialization_kind(db), ) @@ -326,6 +330,15 @@ pub(super) fn walk_type_alias_type<'db, V: visitor::TypeVisitor<'db> + ?Sized>( #[salsa::tracked] impl<'db> TypeAliasType<'db> { + pub(super) fn known_class(self, db: &'db dyn Db) -> KnownClass { + match self { + TypeAliasType::PEP695(_) => KnownClass::TypeAliasType, + TypeAliasType::ManualPEP695(type_alias) => { + type_alias.typing_module(db).type_alias_class() + } + } + } + pub(crate) fn name(self, db: &'db dyn Db) -> &'db str { match self { TypeAliasType::PEP695(type_alias) => type_alias.name(db), @@ -397,6 +410,7 @@ impl<'db> TypeAliasType<'db> { db, alias.name(db), alias.definition(db), + alias.typing_module(db), None, None, )) @@ -433,6 +447,7 @@ impl<'db> TypeAliasType<'db> { db, alias.name(db), alias.definition(db), + alias.typing_module(db), alias.specialization(db), materialization_kind, )) From 2b6180ae6c6b21e2000f9b4a0d97f40741869f1d Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Mon, 17 Aug 2026 17:44:18 -0700 Subject: [PATCH 077/371] [ty] Preserve class objects in lazy protocol checks (#27815) Lazy protocol checks currently replace `type[T]` and `type[Any]` with plain `type`. This loses the class object's interface before generic inference can use it. For example, collecting an iterable class together with an empty fallback can lose `Self` and infer `Unknown` instead. Allow lazy assignability to use the same structural protocol check as eager assignability. Strict subtyping keeps its existing behavior. This is a prerequisite for #27812 and is related to astral-sh/ty#4291. ## Test plan - Add a public mdtest using a generic metaclass iterator and a classmethod that collects `frozenset[Self]` through an empty fallback. - Cover a gradual class object combined with a concrete iterable, ensuring the concrete element type still contributes to inference. --- .../resources/mdtest/protocols.md | 38 +++++++++++++++++++ .../ty_python_semantic/src/types/relation.rs | 2 +- 2 files changed, 39 insertions(+), 1 deletion(-) diff --git a/crates/ty_python_semantic/resources/mdtest/protocols.md b/crates/ty_python_semantic/resources/mdtest/protocols.md index 1c3b9aef2a..5f00ed8eee 100644 --- a/crates/ty_python_semantic/resources/mdtest/protocols.md +++ b/crates/ty_python_semantic/resources/mdtest/protocols.md @@ -4687,6 +4687,44 @@ static_assert(not is_assignable_to(TypeOf[StringMembership], Container[int])) static_assert(not is_assignable_to(TypeOf[NonBooleanMembership], Container[int])) ``` +## Class objects with bounded type-variable receivers + +An iterable class combined with an empty fallback must contribute its member type to generic call +inference. A classmethod can therefore collect its own instances without losing `Self`. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from collections.abc import Iterator +from typing import Self + +class IterableMeta(type): + def __iter__[T](self: type[T]) -> Iterator[T]: + raise NotImplementedError + +class Item(metaclass=IterableMeta): + @classmethod + def all(cls, enabled: bool) -> frozenset[Self]: + items = frozenset(cls if enabled else ()) + reveal_type(items) # revealed: frozenset[Self@all] + return items +``` + +## Generic inference from gradual class objects + +`type[Any]` can be assigned to an iterable protocol. A concrete fallback must still contribute its +element type to generic inference. + +```py +from typing import Any + +def collect(cls: type[Any], enabled: bool) -> None: + reveal_type(list(cls if enabled else (1,))) # revealed: list[int] +``` + ## Subtyping of protocols with `@classmethod` or `@staticmethod` members The typing spec states that protocols may have `@classmethod` or `@staticmethod` method members. diff --git a/crates/ty_python_semantic/src/types/relation.rs b/crates/ty_python_semantic/src/types/relation.rs index 4535c0e345..79f0634d80 100644 --- a/crates/ty_python_semantic/src/types/relation.rs +++ b/crates/ty_python_semantic/src/types/relation.rs @@ -2219,7 +2219,7 @@ impl<'a, 'c, 'db> TypeRelationChecker<'a, 'c, 'db> { // if `type` is a subtype of that protocol. (Type::SubclassOf(source_subclass_ty), Type::ProtocolInstance(_)) if (source_subclass_ty.is_dynamic() || source_subclass_ty.is_type_var()) - && !self.is_eager_assignability() => + && !self.relation.is_assignability() => { self.check_type_pair(db, KnownClass::Type.to_instance(db, env), target) } From 2668df9b6a47afb53d0c7e64caa9c2c298ae26cf Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Mon, 17 Aug 2026 20:09:13 -0700 Subject: [PATCH 078/371] [ty] Preserve declared types in exception handlers (#27817) An annotated assignment could lose its declared type in an exception handler, when the path to the exception handler is through an exception evaluating its right-hand side. Assignments in the handler were then inferred without that type context, so a fallback assignment could acquire an incompatible value type, without the benefit of type context from the declared type. This is arguably "expected behavior" in our flow-sensitive declared-types model, but intuitively it feels like the declaration should "take effect" before the RHS "executes". Modeling it that way requires a bit of new machinery in the use-def map to allow splitting the declaration and binding from a single assignment in control flow, but it's not too bad and even allows some simplifications. Record the declaration before visiting the annotation and right-hand side, then record the value binding only after the right-hand side completes. The two control-flow entries retain their execution order, but only the binding participates in usage analysis. Keeping usage state in those entries also removes the parallel usage vector. This preserves ty's existing exception-point model. Fixes astral-sh/ty#4293. ## Test plan - Exception-flow mdtests cover contextual dictionary inference, incompatible fallback assignments, previous or unbound values, reannotations, declarations after an earlier exception checkpoint, and assignments made inside the right-hand side. - Unused-binding tests cover annotated loop-carried values, shadowed annotated bindings, and later bindings captured by closures. --- crates/ty_python_core/src/builder.rs | 189 +++++++++++++---- crates/ty_python_core/src/use_def.rs | 198 ++++++++---------- .../ty_python_core/src/use_def/place_state.rs | 3 +- .../mdtest/exception/control_flow.md | 107 +++++++++- .../src/types/ide_support/unused_bindings.rs | 67 +++++- 5 files changed, 399 insertions(+), 165 deletions(-) diff --git a/crates/ty_python_core/src/builder.rs b/crates/ty_python_core/src/builder.rs index f0412c5740..9483a5a355 100644 --- a/crates/ty_python_core/src/builder.rs +++ b/crates/ty_python_core/src/builder.rs @@ -1416,12 +1416,19 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { self.add_dict_key_assignment_definitions(&node.targets, &node.value, assignment); } - Some(CurrentAssignment::AnnAssign(ann_assign)) => { + Some(CurrentAssignment::AnnAssign { + node: ann_assign, + pending, + }) => { self.add_standalone_type_expression(&ann_assign.annotation); - let assignment = self.add_definition( - place_id, - AnnotatedAssignmentDefinitionNodeRef { node: ann_assign }, - ); + let assignment = if let Some(pending) = pending { + self.finish_annotated_assignment(pending) + } else { + self.add_definition( + place_id, + AnnotatedAssignmentDefinitionNodeRef { node: ann_assign }, + ) + }; if let Some(value) = ann_assign.value.as_deref() { self.add_dict_key_assignment_definitions( @@ -1498,7 +1505,19 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { place: ScopedPlaceId, definition_node: impl Into> + std::fmt::Debug + Copy, ) -> Definition<'db> { - let (definition, num_definitions) = self.push_additional_definition(place, definition_node); + let definition = self.create_definition(place, definition_node); + self.record_definition(place, definition, None); + definition + } + + /// Create a definition without making its declaration or binding visible in control flow. + fn create_definition( + &mut self, + place: ScopedPlaceId, + definition_node: impl Into> + std::fmt::Debug + Copy, + ) -> Definition<'db> { + let (definition, num_definitions) = + self.create_additional_definition(place, definition_node); debug_assert_eq!( num_definitions, 1, "Attempted to create multiple `Definition`s associated with AST node {definition_node:?}" @@ -1528,10 +1547,6 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { /// Push a new [`Definition`] onto the list of definitions /// associated with the `definition_node` AST node. /// - /// Returns a 2-element tuple, where the first element is the newly created [`Definition`] - /// and the second element is the number of definitions that are now associated with - /// `definition_node`. - /// /// Most AST nodes can only be associated with at most one [`Definition`]. Generally prefer /// `add_definition` above, which enforces that. This method should currently only be used with /// `*` imports and loop headers. @@ -1539,6 +1554,20 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { &mut self, place: ScopedPlaceId, definition_node: impl Into>, + ) { + let (definition, _) = self.create_additional_definition(place, definition_node); + self.record_definition(place, definition, None); + } + + /// Create a [`Definition`] without recording it in control flow. + /// + /// Returns the new definition and the number of definitions now associated with its AST + /// node. Loop headers are not stored by AST node, so their count is zero. Prefer + /// [`Self::create_definition`] when the node must have exactly one definition. + fn create_additional_definition( + &mut self, + place: ScopedPlaceId, + definition_node: impl Into>, ) -> (Definition<'db>, usize) { let definition_node: DefinitionNodeRef<'ast, 'db> = definition_node.into(); @@ -1560,8 +1589,6 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { definitions.len() }; - self.record_definition(place, definition, None); - (definition, num_definitions) } @@ -1578,8 +1605,82 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { previous_definitions: Option, ) { let kind = definition.kind(self.db); - let is_loop_header = kind.is_loop_header(); let category = kind.category(self.source_type.is_stub(), self.module); + match category { + DefinitionCategory::Declaration => { + self.mark_place_declared(place); + self.current_use_def_map_mut() + .record_declaration(place, definition); + } + DefinitionCategory::DeclarationAndBinding => { + self.mark_place_declared(place); + self.record_binding_with(definition, |use_def, place| { + use_def.record_combined_definition(place, definition, category); + }); + } + DefinitionCategory::Binding => { + let previous = previous_definitions.unwrap_or(if kind.is_loop_header() { + PreviousDefinitions::AreKept + } else { + PreviousDefinitions::AreShadowed + }); + self.record_binding_with(definition, |use_def, place| { + use_def.record_binding( + place, + definition, + previous, + FutureDefinitions::ShadowThisOne, + ); + }); + } + } + } + + /// Declare an annotated name assignment whose value will be bound after visiting its RHS. + /// Other targets and annotations without a RHS are recorded in full by `add_definition`. + fn begin_annotated_assignment( + &mut self, + node: &'ast ast::StmtAnnAssign, + ) -> Option> { + let ast::Expr::Name(name) = &*node.target else { + return None; + }; + node.value.as_ref()?; + + let place = self.add_symbol(name.id.clone()).into(); + let definition = + self.create_definition(place, AnnotatedAssignmentDefinitionNodeRef { node }); + self.mark_place_declared(place); + self.current_use_def_map_mut().record_combined_definition( + place, + definition, + DefinitionCategory::Declaration, + ); + Some(PendingAnnotatedAssignment { definition }) + } + + /// Bind the value of an annotated assignment whose declaration was recorded before its RHS. + fn finish_annotated_assignment( + &mut self, + pending: PendingAnnotatedAssignment<'db>, + ) -> Definition<'db> { + let definition = pending.definition; + self.record_binding_with(definition, |use_def, place| { + use_def.record_combined_definition(place, definition, DefinitionCategory::Binding); + }); + definition + } + + /// Record one binding while keeping aliases, captures, and lazy snapshots in sync. + /// The callback receives the definition's place and must append that binding to the current + /// use-def map. + fn record_binding_with( + &mut self, + definition: Definition<'db>, + record: impl FnOnce(&mut UseDefMapBuilder<'db>, ScopedPlaceId), + ) { + let place = definition.place(self.db); + let is_loop_header = definition.kind(self.db).is_loop_header(); // We need to avoid marking places as bound as soon as we encounter a loop header // definition for them, because that would lead to false-positive semantic syntax errors in @@ -1589,43 +1690,19 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { // global x # [invalid-syntax] if `x` is already used or bound // x = 1 // ``` - if category.is_binding() && !is_loop_header { + if !is_loop_header { self.mark_place_bound(place); self.invalidate_narrowing_aliases_for(place); } - if category.is_declaration() { - self.mark_place_declared(place); - } let definition_id = self.current_use_def_map().next_definition_id(); - let use_def = self.current_use_def_map_mut(); - match category { - DefinitionCategory::DeclarationAndBinding => { - use_def.record_declaration_and_binding(place, definition); - self.delete_associated_bindings(place); - } - DefinitionCategory::Declaration => use_def.record_declaration(place, definition), - DefinitionCategory::Binding => { - let previous = previous_definitions.unwrap_or(if is_loop_header { - PreviousDefinitions::AreKept - } else { - PreviousDefinitions::AreShadowed - }); - use_def.record_binding( - place, - definition, - previous, - FutureDefinitions::ShadowThisOne, - ); - if !is_loop_header { - self.delete_associated_bindings(place); - } - } + record(self.current_use_def_map_mut(), place); + + if !is_loop_header { + self.delete_associated_bindings(place); } - if category.is_binding() - && let Some(id) = place.as_symbol() - { + if let Some(id) = place.as_symbol() { self.record_pending_capture_binding(id, definition_id); self.update_lazy_snapshots(id); } @@ -2236,6 +2313,10 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { /// Records the current flow state immediately before an operation that may raise an exception. /// + /// This models exceptions from ordinary operations, not every possible interruption. In + /// particular, we do not add arbitrary exception points for asynchronously raised exceptions + /// such as those originating in signal handlers. + /// /// Child expressions must already have been visited, so their completed assignments are /// visible if the parent operation fails: /// @@ -3848,6 +3929,10 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { } ast::Stmt::AnnAssign(node) => { debug_assert_eq!(&self.current_assignments, &[]); + // For an assignment with a value, an exception from the annotation or RHS must + // not discard the declared type. The value is still bound only after the RHS + // completes, so a handler can observe an earlier binding (or an unbound name). + let pending = self.begin_annotated_assignment(node); self.visit_expr(&node.annotation); if let Some(value) = &node.value { self.visit_expr(value); @@ -3888,7 +3973,7 @@ impl<'db, 'ast> SemanticIndexBuilder<'db, 'ast> { *node.target, ast::Expr::Attribute(_) | ast::Expr::Subscript(_) | ast::Expr::Name(_) ) { - self.push_assignment(CurrentAssignment::AnnAssign(node)); + self.push_assignment(CurrentAssignment::AnnAssign { node, pending }); self.visit_expr(&node.target); self.pop_assignment(); @@ -5705,13 +5790,23 @@ impl SemanticSyntaxContext for SemanticIndexBuilder<'_, '_> { } } +/// A simple-name annotated assignment with an RHS whose declaration is already recorded. +/// Created only by `begin_annotated_assignment`; finishing it records the value binding. +#[derive(Copy, Clone, Debug, PartialEq)] +struct PendingAnnotatedAssignment<'db> { + definition: Definition<'db>, +} + #[derive(Copy, Clone, Debug, PartialEq)] enum CurrentAssignment<'ast, 'db> { Assign { node: &'ast ast::StmtAssign, unpack: Option>, }, - AnnAssign(&'ast ast::StmtAnnAssign), + AnnAssign { + node: &'ast ast::StmtAnnAssign, + pending: Option>, + }, AugAssign(&'ast ast::StmtAugAssign), For { node: &'ast ast::StmtFor, @@ -5736,7 +5831,9 @@ impl CurrentAssignment<'_, '_> { Self::For { unpack, .. } | Self::WithItem { unpack, .. } | Self::Comprehension { unpack, .. } => unpack.as_mut().map(|(position, _)| position), - Self::Assign { .. } | Self::AnnAssign(_) | Self::AugAssign(_) | Self::Named(_) => None, + Self::Assign { .. } | Self::AnnAssign { .. } | Self::AugAssign(_) | Self::Named(_) => { + None + } } } } diff --git a/crates/ty_python_core/src/use_def.rs b/crates/ty_python_core/src/use_def.rs index 2e885d8589..f6334594c0 100644 --- a/crates/ty_python_core/src/use_def.rs +++ b/crates/ty_python_core/src/use_def.rs @@ -251,7 +251,7 @@ use smallvec::SmallVec; use thin_vec::ThinVec; use crate::ast_ids::ScopedUseId; -use crate::definition::{Definition, DefinitionState}; +use crate::definition::{Definition, DefinitionCategory, DefinitionState}; use crate::frozen::FrozenMap; use crate::member::ScopedMemberId; use crate::narrowing_constraints::{ @@ -645,79 +645,56 @@ static ALWAYS_UNBOUND_BINDINGS: LazyLock = static ALWAYS_UNDECLARED_DECLARATIONS: LazyLock = LazyLock::new(|| Declarations::undeclared(ScopedReachabilityConstraintId::ALWAYS_TRUE)); +/// One event in a scope's use-def history. #[derive(Clone, Copy, Debug, PartialEq, Eq, get_size2::GetSize, salsa::SalsaValue)] -enum RetainedDefinitionState<'db> { +enum DefinitionEntry<'db> { + /// The early declaration of a combined definition whose binding is recorded separately. + /// It participates in declaration lookup, but not in binding-usage analysis. + DeclarationPart(Definition<'db>), + /// A binding or standalone declaration with no recorded use. Unused(Definition<'db>), Used(Definition<'db>), Undefined, Deleted, } -impl<'db> RetainedDefinitionState<'db> { - fn new(state: DefinitionState<'db>, used: bool) -> Self { - match state { - DefinitionState::Defined(definition) if used => Self::Used(definition), - DefinitionState::Defined(definition) => Self::Unused(definition), - DefinitionState::Undefined => { - debug_assert!(!used); - Self::Undefined - } - DefinitionState::Deleted => { - debug_assert!(!used); - Self::Deleted - } - } - } - +impl<'db> DefinitionEntry<'db> { fn state(self) -> DefinitionState<'db> { match self { - Self::Unused(definition) | Self::Used(definition) => { - DefinitionState::Defined(definition) - } + Self::DeclarationPart(definition) + | Self::Unused(definition) + | Self::Used(definition) => DefinitionState::Defined(definition), Self::Undefined => DefinitionState::Undefined, Self::Deleted => DefinitionState::Deleted, } } - - fn is_used(self) -> bool { - matches!(self, Self::Used(_)) - } } -static_assertions::assert_eq_size!(RetainedDefinitionState<'static>, DefinitionState<'static>); +static_assertions::assert_eq_size!(DefinitionEntry<'static>, DefinitionState<'static>); /// Retained definition states, excluding the implicit unbound definition at index zero. #[derive(Debug, PartialEq, Eq, get_size2::GetSize, salsa::SalsaValue)] struct RetainedDefinitions<'db> { - states: Box<[RetainedDefinitionState<'db>]>, + states: Box<[DefinitionEntry<'db>]>, } impl<'db> RetainedDefinitions<'db> { - fn new( - states: IndexVec>, - used: IndexVec, - ) -> Self { + fn new(states: IndexVec>) -> Self { let mut states = states.into_iter(); - let mut used = used.into_iter(); let unbound_state = states.next(); - let unbound_used = used.next(); - debug_assert_eq!(unbound_state, Some(DefinitionState::Undefined)); - debug_assert_eq!(unbound_used, Some(false)); + debug_assert_eq!(unbound_state, Some(DefinitionEntry::Undefined)); Self { - states: states - .zip(used) - .map(|(state, used)| RetainedDefinitionState::new(state, used)) - .collect(), + states: states.collect(), } } #[inline] - fn get(&self, id: ScopedDefinitionId) -> RetainedDefinitionState<'db> { + fn get(&self, id: ScopedDefinitionId) -> DefinitionEntry<'db> { let index = id.index(); if index == 0 { - RetainedDefinitionState::Undefined + DefinitionEntry::Undefined } else { self.states[index - 1] } @@ -725,18 +702,12 @@ impl<'db> RetainedDefinitions<'db> { fn iter_enumerated( &self, - ) -> impl Iterator)> + '_ { - std::iter::once(( - ScopedDefinitionId::UNBOUND, - RetainedDefinitionState::Undefined, - )) - .chain( - self.states - .iter() - .copied() - .enumerate() - .map(|(index, state)| (ScopedDefinitionId::new(index + 1), state)), - ) + ) -> impl Iterator)> + '_ { + self.states + .iter() + .copied() + .enumerate() + .map(|(index, entry)| (ScopedDefinitionId::new(index + 1), entry)) } } @@ -888,12 +859,22 @@ impl<'db> UseDefMap<'db> { self.end_of_scope_reachability } - pub fn all_definitions_with_usage( + /// Definitions relevant to usage analysis, including standalone declarations. + /// + /// The early declaration part of a combined definition is omitted: its later binding entry + /// carries the usage information for that definition. + pub fn definitions_with_usage( &self, - ) -> impl Iterator, bool)> + '_ { + ) -> impl Iterator, bool)> + '_ { self.all_definitions .iter_enumerated() - .map(|(id, state)| (id, state.state(), state.is_used())) + .filter_map(|(id, entry)| match entry { + DefinitionEntry::Unused(definition) => Some((id, definition, false)), + DefinitionEntry::Used(definition) => Some((id, definition, true)), + DefinitionEntry::DeclarationPart(_) + | DefinitionEntry::Undefined + | DefinitionEntry::Deleted => None, + }) } pub fn bindings_at_use(&self, use_id: ScopedUseId) -> BindingWithConstraintsIterator<'_, 'db> { @@ -1764,13 +1745,8 @@ pub(super) struct SingleSymbolSnapshot { #[derive(Debug)] pub(super) struct UseDefMapBuilder<'db> { - /// Append-only array of [`DefinitionState`]. - all_definitions: IndexVec>, - - /// Tracks whether each binding definition has at least one use. - /// - /// Uses the same index as `all_definitions`. - used_bindings: IndexVec, + /// Append-only history of declarations and bindings, including their usage state. + all_definitions: IndexVec>, /// Builder of predicates. predicates: PredicatesBuilder<'db>, @@ -1841,8 +1817,7 @@ pub(super) struct UseDefMapBuilder<'db> { impl<'db> UseDefMapBuilder<'db> { pub(super) fn new(is_class_scope: bool) -> Self { Self { - all_definitions: IndexVec::from_iter([DefinitionState::Undefined]), - used_bindings: IndexVec::from_iter([false]), + all_definitions: IndexVec::from_iter([DefinitionEntry::Undefined]), predicates: PredicatesBuilder::default(), reachability_constraints: ReachabilityConstraintsBuilder::default(), narrowing_constraints: NarrowingConstraintsBuilder::default(), @@ -1872,16 +1847,14 @@ impl<'db> UseDefMapBuilder<'db> { self.loop_headers[id] = header; } - fn push_definition(&mut self, state: DefinitionState<'db>) -> ScopedDefinitionId { + fn push_definition(&mut self, entry: DefinitionEntry<'db>) -> ScopedDefinitionId { + // Declaration-only entries also change the type visible to an exception handler. self.checkpoint_state.record_binding_change(); - let def_id = self.all_definitions.push(state); - let used_id = self.used_bindings.push(false); - debug_assert_eq!(def_id, used_id); - def_id + self.all_definitions.push(entry) } pub(super) fn definition(&self, def_id: ScopedDefinitionId) -> DefinitionState<'db> { - self.all_definitions[def_id] + self.all_definitions[def_id].state() } pub(super) fn mark_unreachable(&mut self) { @@ -1946,7 +1919,7 @@ impl<'db> UseDefMapBuilder<'db> { can_be_shadowed: FutureDefinitions, ) { let pending = self.pending_reachability.current; - let def_id = self.push_definition(DefinitionState::Defined(binding)); + let def_id = self.push_definition(DefinitionEntry::Unused(binding)); let place_state = pending_place_state_mut(place, &mut self.symbol_states, &mut self.member_states); let place_state = self.pending_reachability.materialize( @@ -2353,7 +2326,7 @@ impl<'db> UseDefMapBuilder<'db> { place: ScopedPlaceId, declaration: Definition<'db>, ) { - let def_id = self.push_definition(DefinitionState::Defined(declaration)); + let def_id = self.push_definition(DefinitionEntry::Unused(declaration)); let pending = self.pending_reachability.current; let place_state = pending_place_state_mut(place, &mut self.symbol_states, &mut self.member_states); @@ -2385,14 +2358,24 @@ impl<'db> UseDefMapBuilder<'db> { ); } - pub(super) fn record_declaration_and_binding( + /// Record some or all of a definition that both declares a type and binds a value. + /// + /// Annotated assignments can declare before their RHS and bind afterward. Each phase gets a + /// fresh scoped ID, so definitions created by the RHS remain in execution order. + pub(super) fn record_combined_definition( &mut self, place: ScopedPlaceId, definition: Definition<'db>, + part: DefinitionCategory, ) { // We don't need to store prior state for a definition that is both a declaration and a // binding. - let def_id = self.push_definition(DefinitionState::Defined(definition)); + let entry = if part.is_binding() { + DefinitionEntry::Unused(definition) + } else { + DefinitionEntry::DeclarationPart(definition) + }; + let def_id = self.push_definition(entry); let pending = self.pending_reachability.current; let place_state = pending_place_state_mut(place, &mut self.symbol_states, &mut self.member_states); @@ -2402,38 +2385,41 @@ impl<'db> UseDefMapBuilder<'db> { &mut self.narrowing_constraints, &mut self.reachability_constraints, ); - place_state.record_declaration(def_id, self.reachability); - place_state.record_binding( - def_id, - self.reachability, - self.is_class_scope, - place.is_symbol(), - PreviousDefinitions::AreShadowed, - FutureDefinitions::ShadowThisOne, - ); - let reachable_definitions = match place { ScopedPlaceId::Symbol(symbol) => &mut self.reachable_symbol_definitions[symbol], ScopedPlaceId::Member(member) => &mut self.reachable_member_definitions[member], }; - reachable_definitions.declarations.record_declaration( - def_id, - self.reachability, - PreviousDefinitions::AreKept, - ); - reachable_definitions.bindings.record_binding( - def_id, - self.reachability, - self.is_class_scope, - place.is_symbol(), - PreviousDefinitions::AreKept, - FutureDefinitions::ShadowThisOne, - ); + if part.is_declaration() { + place_state.record_declaration(def_id, self.reachability); + reachable_definitions.declarations.record_declaration( + def_id, + self.reachability, + PreviousDefinitions::AreKept, + ); + } + if part.is_binding() { + place_state.record_binding( + def_id, + self.reachability, + self.is_class_scope, + place.is_symbol(), + PreviousDefinitions::AreShadowed, + FutureDefinitions::ShadowThisOne, + ); + reachable_definitions.bindings.record_binding( + def_id, + self.reachability, + self.is_class_scope, + place.is_symbol(), + PreviousDefinitions::AreKept, + FutureDefinitions::ShadowThisOne, + ); + } } pub(super) fn delete_binding(&mut self, place: ScopedPlaceId) { - let def_id = self.push_definition(DefinitionState::Deleted); + let def_id = self.push_definition(DefinitionEntry::Deleted); let pending = self.pending_reachability.current; let place_state = pending_place_state_mut(place, &mut self.symbol_states, &mut self.member_states); @@ -2671,15 +2657,9 @@ impl<'db> UseDefMapBuilder<'db> { } fn mark_definition_used(&mut self, definition_id: ScopedDefinitionId) { - if definition_id.is_unbound() { - return; - } - - if matches!( - self.all_definitions[definition_id], - DefinitionState::Defined(_) - ) { - self.used_bindings[definition_id] = true; + let entry = &mut self.all_definitions[definition_id]; + if let DefinitionEntry::Unused(definition) = *entry { + *entry = DefinitionEntry::Used(definition); } } @@ -2937,7 +2917,7 @@ impl<'db> UseDefMapBuilder<'db> { narrowing_constraints, }) }); - let all_definitions = RetainedDefinitions::new(self.all_definitions, self.used_bindings); + let all_definitions = RetainedDefinitions::new(self.all_definitions); UseDefMap { all_definitions, diff --git a/crates/ty_python_core/src/use_def/place_state.rs b/crates/ty_python_core/src/use_def/place_state.rs index 01d0494bdc..24d8206af5 100644 --- a/crates/ty_python_core/src/use_def/place_state.rs +++ b/crates/ty_python_core/src/use_def/place_state.rs @@ -50,7 +50,8 @@ use crate::ReachabilityConstraintsBuilder; use crate::narrowing_constraints::{NarrowingConstraintsBuilder, ScopedNarrowingConstraint}; use crate::reachability_constraints::ScopedReachabilityConstraintId; -/// A newtype-index for a definition in a particular scope. +/// An index into a scope's use-def history. A combined definition can have separate declaration +/// and binding entries when they take effect at different points in control flow. #[newtype_index] #[derive(Ord, PartialOrd, get_size2::GetSize)] pub struct ScopedDefinitionId; diff --git a/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md b/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md index 1c47f64516..ae36f031ba 100644 --- a/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md +++ b/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md @@ -3,6 +3,9 @@ These tests describe which names are defined and what types they have in the branches of a `try`/`except`/`else`/`finally` statement. +The analysis models exceptions from ordinary Python operations. It intentionally does not treat +every possible interruption, such as an exception raised by a signal handler, as an exception point. + For a full writeup on the semantics of exception handlers, see [this document][1]. Functions whose names start with `could_raise_` make it clear that a call may raise an exception @@ -10,8 +13,8 @@ before an assignment completes. Any other function call can raise as well. ## Operations that cannot raise -An exception handler can run only if the `try` block contains an operation that can raise. Assigning -a literal to a local name cannot raise: +Under this model, an exception handler is reachable only if the `try` block contains an operation +that can raise. Assigning a literal to a local name does not introduce an exception point: ```py x = 1 @@ -46,6 +49,106 @@ def known_safe_conditions(value: int | None) -> None: reveal_type(state) # revealed: Literal[1] ``` +## Annotated assignments that can raise + +An annotation applies to assignments in the exception handler even if evaluating the annotated +assignment's right-hand side raises. In particular, it provides type context for a collection +literal in the handler. + +```py +from typing import Any + +def could_raise_dict() -> dict[str, Any]: + return {} + +def requires_str(value: str) -> None: ... +def fallback() -> None: + try: + result: dict[str, Any] = could_raise_dict() + except Exception: + result = {"correct": False, "message": "fallback"} + reveal_type(result) # revealed: dict[str, Any] + + reveal_type(result) # revealed: dict[str, Any] + requires_str(result["message"]) +``` + +The declaration also rejects an incompatible assignment in the handler. + +```py +def could_raise_int() -> int: + return 1 + +def incompatible_fallback() -> None: + try: + value: int = could_raise_int() + except Exception: + value = "wrong" # error: [invalid-assignment] +``` + +An earlier call in the `try` block does not hide a declaration reached before a later call raises. + +```py +def declaration_after_call() -> None: + value = int() + try: + could_raise_int() + value: int = could_raise_int() + except Exception: + value = "wrong" # error: [invalid-assignment] +``` + +The declaration does not make the new value available before the assignment completes. A handler +still sees the previous value, or an unbound name if there was no previous binding. + +```py +def previous_binding() -> None: + value = 0 + try: + value: int = could_raise_int() + except Exception: + reveal_type(value) # revealed: Literal[0] + +def no_previous_binding() -> None: + try: + value: int = could_raise_int() + except Exception: + # error: [unresolved-reference] + reveal_type(value) # revealed: Unknown +``` + +A new annotation replaces an earlier declared type even if its right-hand side raises. + +```py +def reannotated() -> None: + value: object = None + try: + value: int = could_raise_int() + except Exception: + value = 1 + + reveal_type(value) # revealed: int +``` + +Assignments made while evaluating the right-hand side still reach the handler. When the call +returns, its result replaces the value assigned by the walrus expression on the successful path. + +```py +from collections.abc import Callable +from typing import Literal + +def assignment_in_rhs(could_raise_after: Callable[[int], Literal[3]]) -> None: + value = 0 + try: + value: int = could_raise_after(value := 2) + except Exception: + reveal_type(value) # revealed: Literal[0, 2] + else: + reveal_type(value) # revealed: Literal[3] + + reveal_type(value) # revealed: Literal[0, 2, 3] +``` + ## Looking up an undefined name An undefined name raises `NameError`, so an exception handler can provide its value: diff --git a/crates/ty_python_semantic/src/types/ide_support/unused_bindings.rs b/crates/ty_python_semantic/src/types/ide_support/unused_bindings.rs index 1597d2a040..02a8783a2a 100644 --- a/crates/ty_python_semantic/src/types/ide_support/unused_bindings.rs +++ b/crates/ty_python_semantic/src/types/ide_support/unused_bindings.rs @@ -7,7 +7,7 @@ use ruff_db::parsed::parsed_module; use ruff_python_ast::name::Name; use ruff_text_size::TextRange; use rustc_hash::FxHashSet; -use ty_python_core::definition::{DefinitionCategory, DefinitionKind, DefinitionState}; +use ty_python_core::definition::{DefinitionCategory, DefinitionKind}; use ty_python_core::place::ScopedPlaceId; use ty_python_core::scope::{FileScopeId, ScopeKind}; use ty_python_core::{ProgramFile, SemanticIndex, semantic_index}; @@ -112,8 +112,8 @@ pub fn unused_bindings(db: &dyn Db, file: ProgramFile<'_>) -> Box<[UnusedBinding let used_definitions = index.scope_ids().flat_map(|scope_id| { index .use_def_map(scope_id.file_scope_id(db)) - .all_definitions_with_usage() - .filter_map(|(_, state, is_used)| is_used.then_some(state.definition()).flatten()) + .definitions_with_usage() + .filter_map(|(_, definition, is_used)| is_used.then_some(definition)) }); let used_user_visible_definitions = super::user_visible_definitions(db, used_definitions); @@ -142,10 +142,7 @@ pub fn unused_bindings(db: &dyn Db, file: ProgramFile<'_>) -> Box<[UnusedBinding // track used IDs as we go. let mut loop_header_used_definition_ids = FxHashSet::default(); - for (definition_id, state, is_used) in use_def_map.all_definitions_with_usage() { - let DefinitionState::Defined(definition) = state else { - continue; - }; + for (definition_id, definition, is_used) in use_def_map.definitions_with_usage() { let is_used = is_used || used_user_visible_definitions.contains(&definition); if is_used { @@ -822,6 +819,23 @@ mod tests { Ok(()) } + #[test] + fn closure_uses_later_annotated_binding() -> anyhow::Result<()> { + let source = dedent( + " + def outer(): + def inner(): + return value + + value: int = 1 + return inner + ", + ); + + assert!(collect_unused_names(&source)?.is_empty()); + Ok(()) + } + #[test] fn nested_comprehension_capture_uses_intermediate_rebindings() -> anyhow::Result<()> { let source = dedent( @@ -951,6 +965,45 @@ mod tests { Ok(()) } + #[test] + fn skips_annotated_loop_carried_rebinding() -> anyhow::Result<()> { + let source = dedent( + " + def f(items: list[int]) -> None: + value = 0 + for item in items: + print(value) + value: int = item + ", + ); + + assert!(collect_unused_names(&source)?.is_empty()); + Ok(()) + } + + #[test] + fn reports_shadowed_annotated_binding() -> anyhow::Result<()> { + let source = dedent( + " + def f() -> int: + value: int = 1 + value: int = 2 + return value + ", + ); + + let bindings = collect_unused_bindings(&source)?; + let start = TextSize::try_from(source.find("value: int = 1").unwrap()).unwrap(); + assert_eq!( + bindings, + vec![UnusedBinding { + range: TextRange::new(start, start + TextSize::new(5)), + name: Name::new("value"), + }] + ); + Ok(()) + } + #[test] fn skips_annotation_only_declaration_before_reassignment() -> anyhow::Result<()> { let source = dedent( From 95d14fda9071ce830009597c1148bd5764b8044b Mon Sep 17 00:00:00 2001 From: Dhruv Manilawala Date: Tue, 18 Aug 2026 15:09:30 +0530 Subject: [PATCH 079/371] [ty] Support unpacking tuple type aliases (#27825) ## Summary Resolve type aliases before validating `*Alias` and `Unpack[Alias]`. The unpack checker previously rejected PEP 695 alias wrappers instead of inspecting their underlying tuple types, producing false positives and losing argument checking. ## Tests Add focused coverage for both unpack spellings, argument types and arity, specialized alias chains, and invalid non-tuple aliases. --- .../resources/mdtest/pep695_type_aliases.md | 44 +++++++++++++++++++ .../types/infer/builder/type_expression.rs | 4 +- 2 files changed, 47 insertions(+), 1 deletion(-) diff --git a/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md b/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md index 9423aa324f..137b607d80 100644 --- a/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md +++ b/crates/ty_python_semantic/resources/mdtest/pep695_type_aliases.md @@ -256,6 +256,50 @@ def f(x: Foo[int]): reveal_type(x.foo()) # revealed: int ``` +## Unpacking tuple aliases + +Both unpack spellings accept a tuple alias and preserve positional argument types and arity. + +```py +from typing import Unpack + +type Pair = tuple[int, str] + +def starred(*args: *Pair) -> None: + reveal_type(args) # revealed: tuple[int, str] + +def explicit(*args: Unpack[Pair]) -> None: + reveal_type(args) # revealed: tuple[int, str] + +starred(1, "a") +starred(1) # error: [missing-argument] +starred(1, 2) # error: [invalid-argument-type] +explicit(1, "a") +explicit(1, "a", 3) # error: [too-many-positional-arguments] +``` + +Unpacking also follows alias chains and applies generic substitutions. + +```py +type GenericPair[T] = tuple[T, str] +type SpecializedPair = GenericPair[bytes] + +def specialized(*args: *SpecializedPair) -> None: + reveal_type(args) # revealed: tuple[bytes, str] + +specialized(b"a", "b") +specialized(1, "a") # error: [invalid-argument-type] +``` + +Non-tuple aliases remain invalid. + +```py +type NotTuple = list[int] + +def invalid_starred(*args: *NotTuple) -> None: ... # error: [invalid-type-form] +def invalid_explicit(*args: Unpack[NotTuple]) -> None: ... # error: [invalid-type-form] +``` + ## Stringified values Stringifying the right-hand side of a type alias is redundant, but allowed: diff --git a/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs b/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs index 8a5bef5d47..290e0252b3 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/type_expression.rs @@ -970,7 +970,7 @@ impl<'db> TypeInferenceBuilder<'db, '_> { .context .inference_flags .replace(InferenceFlags::IN_UNPACK_TYPE_ARGUMENT, true); - let starred_type = self.infer_type_expression(value); + let starred_type = self.infer_type_expression(value).resolve_type_alias(db); self.context.inference_flags.set( InferenceFlags::IN_UNPACK_TYPE_ARGUMENT, previously_in_unpack_type_argument, @@ -2543,6 +2543,8 @@ impl<'db> TypeInferenceBuilder<'db, '_> { return inner_ty; } + let inner_ty = inner_ty.resolve_type_alias(db); + // Preserve valid unpack targets so that `Unpack[...]` follows the same // argument-binding path as an equivalent starred annotation. if inner_ty.exact_tuple_instance_spec(self.db()).is_some() From dbf935d74b8229070d6b19cb2d19e19e7c0e6c58 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 18 Aug 2026 10:41:43 +0100 Subject: [PATCH 080/371] [ty] Disambiguate same-named types in several diagnostics (#27814) --- .../mdtest/diagnostics/same_names.md | 373 ++++++++++++++++++ .../ty_python_semantic/src/types/call/bind.rs | 70 +++- .../src/types/diagnostic.rs | 31 +- .../ty_python_semantic/src/types/display.rs | 20 +- .../ty_python_semantic/src/types/function.rs | 21 +- .../src/types/infer/builder.rs | 13 +- .../infer/builder/attribute_assignment.rs | 13 +- .../builder/post_inference/static_class.rs | 26 +- .../src/types/infer/builder/subscript.rs | 16 +- 9 files changed, 519 insertions(+), 64 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/diagnostics/same_names.md b/crates/ty_python_semantic/resources/mdtest/diagnostics/same_names.md index 92d6e33eb6..dc5fd099e7 100644 --- a/crates/ty_python_semantic/resources/mdtest/diagnostics/same_names.md +++ b/crates/ty_python_semantic/resources/mdtest/diagnostics/same_names.md @@ -304,3 +304,376 @@ def get_models_tuple() -> tuple[Model]: # error: [invalid-return-type] "Return type does not match returned value: expected `tuple[mdtest_snippet.Model]`, found `tuple[module.Model]`" return (Model(),) ``` + +## Callable special forms + +ty distinguishes same-named classes nested in the signatures of two callable special forms. + +`first.py`: + +```py +from typing import Callable + +class StartResponse: ... + +Application = Callable[[StartResponse], int] +``` + +```py +from typing import Callable + +try: + from first import Application, StartResponse +except ImportError: + class StartResponse: ... + + # error: [invalid-assignment] "Object of type ` int'>` is not assignable to ` int'>`" + Application = Callable[[StartResponse], int] +``` + +## Method and constructor descriptions + +ty distinguishes the defining class of a bound method, unbound method, or constructor from a +same-named argument type. Method owners with no visible ambiguity remain unqualified. + +`first.py`: + +```py +class Model: ... +``` + +`second.py`: + +```py +import first + +class Model: + def __init__(self, value: first.Model) -> None: ... + def method(self, value: first.Model) -> None: ... + +class Other: + def method(self, value: first.Model) -> None: ... +``` + +```py +import second + +def calls(value: second.Model, other: second.Other) -> None: + # error: [invalid-argument-type] "Argument to bound method `second.Model.method` is incorrect: Expected `first.Model`, found `Literal[1]`" + value.method(1) + + # error: [invalid-argument-type] "Argument to function `second.Model.method` is incorrect: Expected `first.Model`, found `Literal[1]`" + second.Model.method(value, 1) + + # error: [invalid-argument-type] "Argument to `second.Model.__init__` is incorrect: Expected `first.Model`, found `Literal[1]`" + second.Model(1) + + # No competing type named `Other` appears in this diagnostic, so its method owner stays unqualified. + # error: [invalid-argument-type] "Argument to bound method `Other.method` is incorrect: Expected `Model`, found `Literal[1]`" + other.method(1) +``` + +## Builtin class descriptions + +ty distinguishes a builtin class used as a callable from a same-named argument type. + +```py +import builtins + +class tuple: ... + +def convert(value: tuple) -> None: + # error: [invalid-argument-type] "Argument to class `builtins.tuple` is incorrect: Expected `Iterable[Unknown]`, found `mdtest_snippet.tuple`" + builtins.tuple(value) +``` + +## Identifying union members + +ty uses the same qualification for a union member missing an attribute as for the complete union. + +`first.py`: + +```py +class Model: + present: int +``` + +`second.py`: + +```py +class Model: ... +``` + +```py +import first +import second + +def missing_attribute(value: first.Model | second.Model) -> int: + # error: [unresolved-attribute] "Attribute `present` is not defined on `second.Model` in union `first.Model | second.Model`" + return value.present +``` + +## Aliased union members + +ty distinguishes a union's type alias from a same-named member that does not define an attribute. + +```toml +[environment] +python-version = "3.12" +``` + +`first.py`: + +```py +class Present: + present: int +``` + +`second.py`: + +```py +class Model: ... +``` + +`alias.py`: + +```py +import first +import second + +type Model = first.Present | second.Model +``` + +```py +from alias import Model + +def missing_attribute(value: Model) -> int: + # error: [unresolved-attribute] "Attribute `present` is not defined on `second.Model` in union `alias.Model`" + return value.present +``` + +## Redefined union members + +When distinct union members have the same name in the same module, ty identifies the missing member +using both its source location and its module name. + +`test.py`: + +```py +def coinflip() -> bool: + return True + +if coinflip(): + class Model: + present: int + +else: + class Model: ... + +# error: [unresolved-attribute] "Attribute `present` is not defined on `test.Model @ src/test.py:9:11` in union `test.Model @ src/test.py:5:11 | test.Model @ src/test.py:9:11`" +Model().present +``` + +## Attribute assignments + +For ordinary and union attribute assignments, ty distinguishes the assigned class from a same-named +class appearing elsewhere in the diagnostic. + +`first.py`: + +```py +class Model: ... +``` + +`second.py`: + +```py +class Model: ... +``` + +```py +import first +import second + +class Owner: + item: first.Model + +class Other: + item: int + +def assign_attribute(owner: Owner, value: second.Model) -> None: + # error: [invalid-assignment] "Object of type `second.Model` is not assignable to attribute `item` of type `first.Model`" + owner.item = value + +def assign_union_attribute(owner: first.Model | Other, value: second.Model) -> None: + # error: [invalid-assignment] "Object of type `second.Model` is not assignable to attribute `item` on type `first.Model | Other`" + owner.item = value +``` + +## Subscript assignments + +ty distinguishes an incompatible assigned value or subscript key from a same-named class nested in +the subscripted object's type. + +`first.py`: + +```py +class Model: ... +``` + +`second.py`: + +```py +class Model: ... +``` + +```py +import first +import second + +def assign_value(values: list[first.Model], value: second.Model) -> None: + # error: [invalid-assignment] "Invalid subscript assignment with key of type `Literal[0]` and value of type `second.Model` on object of type `list[first.Model]`" + values[0] = value + +def assign_key(values: dict[first.Model, int], key: second.Model) -> None: + # error: [invalid-assignment] "Invalid subscript assignment with key of type `second.Model` and value of type `Literal[1]` on object of type `dict[first.Model, int]`" + values[key] = 1 +``` + +## Type assertions + +ty distinguishes an asserted class from a same-named inferred class. + +```toml +[environment] +python-version = "3.11" +``` + +`first.py`: + +```py +class Model: ... +``` + +`second.py`: + +```py +class Model: ... +``` + +```py +from typing import assert_type + +import first +import second + +def invalid_assertion(value: second.Model) -> None: + assert_type(value, first.Model) # snapshot: type-assertion-failure +``` + +```snapshot +error[type-assertion-failure]: Argument does not have asserted type `first.Model` + --> src/mdtest_snippet.py:7:5 + | +7 | assert_type(value, first.Model) # snapshot: type-assertion-failure + | ^^^^^^^^^^^^-----^^^^^^^^^^^^^^ + | | + | Inferred type is `second.Model` +info: `first.Model` and `second.Model` are not equivalent types +``` + +## Unspellable subtype assertions + +ty distinguishes same-named classes throughout a type assertion about an unspellable intersection. + +```toml +[environment] +python-version = "3.11" +``` + +`first.py`: + +```py +class Model: ... +``` + +`second.py`: + +```py +class Model: ... +``` + +```py +from typing import assert_type + +import first +import second + +def invalid_subtype_assertion(value: first.Model) -> None: + if isinstance(value, second.Model): + assert_type(value, second.Model) # snapshot: assert-type-unspellable-subtype +``` + +```snapshot +error[assert-type-unspellable-subtype]: Argument does not have asserted type `second.Model` + --> src/mdtest_snippet.py:8:9 + | +8 | assert_type(value, second.Model) # snapshot: assert-type-unspellable-subtype + | ^^^^^^^^^^^^-----^^^^^^^^^^^^^^^ + | | + | Inferred type is `first.Model & second.Model` +info: `first.Model & second.Model` is a subtype of `second.Model`, but they are not equivalent +``` + +## Incompatible inherited methods + +ty distinguishes a derived class from its same-named base when their inherited methods are +incompatible. + +`first.py`: + +```py +class Model: + def method(self, value: int) -> int: + return value +``` + +`second.py`: + +```py +class Different: + def method(self, value: str) -> str: + return value +``` + +```py +import first +import second + +# error: [invalid-method-override] "Base classes for class `mdtest_snippet.Model` define method `method` incompatibly: `first.Model.method` is incompatible with `Different.method`" +class Model(first.Model, second.Different): ... +``` + +## Conflicting metaclasses + +ty distinguishes same-named classes and metaclasses throughout a metaclass-conflict diagnostic. + +`first.py`: + +```py +class Meta(type): ... +class Model(metaclass=Meta): ... +``` + +```py +import first + +class OtherMeta(type): ... + +# error: [conflicting-metaclass] "derived class (`mdtest_snippet.Model`) must be a subclass of the metaclasses of all its bases, but `OtherMeta` (metaclass of `mdtest_snippet.Model`) and `Meta` (metaclass of base class `first.Model`) have no subclass relationship" +class Model(first.Model, metaclass=OtherMeta): ... +class Meta(type): ... + +# error: [conflicting-metaclass] "derived class (`Other`) must be a subclass of the metaclasses of all its bases, but `mdtest_snippet.Meta` (metaclass of `Other`) and `first.Meta` (metaclass of base class `Model`) have no subclass relationship" +class Other(first.Model, metaclass=Meta): ... +``` diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index 591dd1918d..2ebe9c7bb0 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -8293,21 +8293,52 @@ pub(crate) struct CallableDescription<'a> { } impl<'db> CallableDescription<'db> { + fn defining_class(db: &'db dyn Db, callable_type: Type<'db>) -> Option> { + let function = match callable_type { + Type::FunctionLiteral(function) => function, + Type::BoundMethod(method) => method.function(db), + Type::ClassLiteral(class) => return Some(class), + _ => return None, + }; + + let semantic_index = semantic_index(db, function.program_file(db)); + let enclosing_scope = semantic_index.scope(function.definition(db).file_scope(db)); + let class_node = enclosing_scope.node().as_class()?; + + original_class_type(db, semantic_index.expect_single_definition(class_node)) + } + pub(crate) fn new( db: &'db dyn Db, callable_type: Type<'db>, + ) -> Option> { + Self::new_with_settings(db, callable_type, None) + } + + fn new_with_settings( + db: &'db dyn Db, + callable_type: Type<'db>, + settings: Option<&DisplaySettings<'db>>, ) -> Option> { fn qualified_function_name<'db>( db: &'db dyn Db, function: FunctionType<'db>, + settings: Option<&DisplaySettings<'db>>, ) -> Cow<'db, str> { - let semantic_index = semantic_index(db, function.program_file(db)); - let enclosing_scope = semantic_index.scope(function.definition(db).file_scope(db)); - if let Some(class_node) = enclosing_scope.node().as_class() - && let Some(class) = - original_class_type(db, semantic_index.expect_single_definition(class_node)) + if let Some(class) = + CallableDescription::defining_class(db, Type::FunctionLiteral(function)) { - Cow::Owned(format!("{}.{}", class.name(db), function.name(db))) + settings + .map(|settings| { + Cow::Owned(format!( + "{}.{}", + class.display_with(db, settings.clone()), + function.name(db) + )) + }) + .unwrap_or_else(|| { + Cow::Owned(format!("{}.{}", class.name(db), function.name(db))) + }) } else { Cow::Borrowed(function.name(db)) } @@ -8320,11 +8351,15 @@ impl<'db> CallableDescription<'db> { } else { "function" }), - name: qualified_function_name(db, function), + name: qualified_function_name(db, function, settings), }), Type::ClassLiteral(class_type) => Some(CallableDescription { kind: Some("class"), - name: Cow::Borrowed(class_type.name(db)), + name: settings + .map(|settings| { + Cow::Owned(class_type.display_with(db, settings.clone()).to_string()) + }) + .unwrap_or_else(|| Cow::Borrowed(class_type.name(db).as_str())), }), Type::SubclassOf(subclass) if let Some(typevar) = subclass.into_type_var() => { Some(CallableDescription { @@ -8341,7 +8376,7 @@ impl<'db> CallableDescription<'db> { }; CallableDescription { kind, - name: qualified_function_name(db, function), + name: qualified_function_name(db, function, settings), } }), Type::KnownBoundMethod(KnownBoundMethodType::FunctionTypeDunderGet(function)) => { @@ -8846,10 +8881,17 @@ impl<'db> BindingError<'db> { return; }; - let display_settings = DisplaySettings::from_possibly_ambiguous_types( + let defining_class = + CallableDescription::defining_class(db, callable_ty).map(Type::ClassLiteral); + let types = [*provided_ty, *expected_ty] + .into_iter() + .chain(defining_class); + let display_settings = + DisplaySettings::from_possibly_ambiguous_types(db, env, types); + let qualified_callable_description = CallableDescription::new_with_settings( db, - env, - [provided_ty, expected_ty], + callable_ty, + Some(&display_settings), ); let provided_ty_display = provided_ty.display_with(db, env, display_settings.clone()); @@ -8857,7 +8899,9 @@ impl<'db> BindingError<'db> { let mut diag = builder.into_diagnostic(format_args!( "Argument{} is incorrect", - callable_description + qualified_callable_description + .as_ref() + .or(callable_description) .map(|description| format!(" to {description}")) .unwrap_or_default() )); diff --git a/crates/ty_python_semantic/src/types/diagnostic.rs b/crates/ty_python_semantic/src/types/diagnostic.rs index 635a02743b..04029d9098 100644 --- a/crates/ty_python_semantic/src/types/diagnostic.rs +++ b/crates/ty_python_semantic/src/types/diagnostic.rs @@ -1763,13 +1763,14 @@ pub(super) fn report_invalid_attribute_assignment( // diagnostic being emitted here. let env = &context.program_environment(); + let settings = DisplaySettings::from_possibly_ambiguous_types(db, env, [source_ty, target_ty]); let Some(mut diag) = report_invalid_assignment_with_message( context, range, format_args!( "Object of type `{}` is not assignable to attribute `{attribute_name}` of type `{}`", - source_ty.display(db, env), - target_ty.display(db, env), + source_ty.display_with(db, env, settings.clone()), + target_ty.display_with(db, env, settings), ), ) else { return; @@ -4365,20 +4366,20 @@ pub(super) fn report_incompatible_base_method<'db>( let (selected_owner, selected_definition, selected_decorator) = selected; let (contract_owner, contract_definition, contract_decorator) = contract; - let (selected_name, contract_name) = if selected_owner.name(db) == contract_owner.name(db) { - ( - selected_owner.qualified_name(db).to_string(), - contract_owner.qualified_name(db).to_string(), - ) - } else { - ( - selected_owner.name(db).to_string(), - contract_owner.name(db).to_string(), - ) - }; + let types = [ + Type::from(class), + Type::from(selected_owner), + Type::from(contract_owner), + ]; + let settings = + DisplaySettings::from_possibly_ambiguous_types(db, context.program_environment(), types); + let class_name = ClassLiteral::Static(class).display_with(db, settings.clone()); + let selected_name = selected_owner + .class_literal(db) + .display_with(db, settings.clone()); + let contract_name = contract_owner.class_literal(db).display_with(db, settings); let mut diagnostic = builder.into_diagnostic(format_args!( - "Base classes for class `{}` define method `{member}` incompatibly", - class.name(db) + "Base classes for class `{class_name}` define method `{member}` incompatibly", )); diagnostic.set_primary_annotation_message(format_args!( "`{selected_name}.{member}` is incompatible with `{contract_name}.{member}`" diff --git a/crates/ty_python_semantic/src/types/display.rs b/crates/ty_python_semantic/src/types/display.rs index ba00df5b55..a6f006b42e 100644 --- a/crates/ty_python_semantic/src/types/display.rs +++ b/crates/ty_python_semantic/src/types/display.rs @@ -824,7 +824,11 @@ pub(super) fn qualified_name_components_from_scope( } impl<'db> ClassLiteral<'db> { - fn display_with(self, db: &'db dyn Db, settings: DisplaySettings<'db>) -> ClassDisplay<'db> { + pub(crate) fn display_with( + self, + db: &'db dyn Db, + settings: DisplaySettings<'db>, + ) -> ClassDisplay<'db> { ClassDisplay { db, class: self, @@ -833,7 +837,7 @@ impl<'db> ClassLiteral<'db> { } } -struct ClassDisplay<'db> { +pub(crate) struct ClassDisplay<'db> { db: &'db dyn Db, class: ClassLiteral<'db>, settings: DisplaySettings<'db>, @@ -2244,14 +2248,6 @@ impl TupleSpecialization { } impl<'db> CallableType<'db> { - fn display<'a>( - &'a self, - db: &'db dyn Db, - env: &'a ProgramEnvironment<'db>, - ) -> DisplayCallableType<'a, 'db> { - Self::display_with(self, db, env, DisplaySettings::default()) - } - fn display_with<'a>( &'a self, db: &'db dyn Db, @@ -3768,7 +3764,9 @@ impl<'db> FmtDetailed<'db> for DisplayKnownInstanceRepr<'_, 'db> { f.with_type(Type::SpecialForm(SpecialFormType::TypingCallable)) .write_str("Callable")?; f.write_str(" special-form '")?; - callable.display(db, self.env).fmt_detailed(f)?; + callable + .display_with(db, self.env, self.settings.clone()) + .fmt_detailed(f)?; f.write_str("'>") } KnownInstanceType::TypeGenericAlias(inner) => { diff --git a/crates/ty_python_semantic/src/types/function.rs b/crates/ty_python_semantic/src/types/function.rs index 7f3907a086..b1237a88eb 100644 --- a/crates/ty_python_semantic/src/types/function.rs +++ b/crates/ty_python_semantic/src/types/function.rs @@ -2518,9 +2518,14 @@ impl KnownFunction { &ASSERT_TYPE_UNSPELLABLE_SUBTYPE }; if let Some(builder) = context.report_lint(diagnostic, call_expression) { + let settings = DisplaySettings::from_possibly_ambiguous_types( + db, + env, + [*actual_ty, asserted_ty], + ); let mut diagnostic = builder.into_diagnostic(format_args!( "Argument does not have asserted type `{}`", - asserted_ty.display(db, env), + asserted_ty.display_with(db, env, settings.clone()), )); diagnostic.annotate( @@ -2532,28 +2537,28 @@ impl KnownFunction { ) .message(format_args!( "Inferred type is `{}`", - actual_ty.display(db, env) + actual_ty.display_with(db, env, settings.clone()) )), ); if actual_ty.is_subtype_of(db, env, asserted_ty) { diagnostic.info(format_args!( "`{inferred_type}` is a subtype of `{asserted_type}`, but they are not equivalent", - asserted_type = asserted_ty.display(db, env), - inferred_type = actual_ty.display(db, env), + asserted_type = asserted_ty.display_with(db, env, settings.clone()), + inferred_type = actual_ty.display_with(db, env, settings.clone()), )); } else { diagnostic.info(format_args!( "`{asserted_type}` and `{inferred_type}` are not equivalent types", - asserted_type = asserted_ty.display(db, env), - inferred_type = actual_ty.display(db, env), + asserted_type = asserted_ty.display_with(db, env, settings.clone()), + inferred_type = actual_ty.display_with(db, env, settings.clone()), )); } diagnostic.set_concise_message(format_args!( "Type `{}` does not match asserted type `{}`", - actual_ty.display(db, env), - asserted_ty.display(db, env), + actual_ty.display_with(db, env, settings.clone()), + asserted_ty.display_with(db, env, settings), )); } } diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index e1bdf7548a..c57dc662a1 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -124,7 +124,7 @@ use crate::types::{ any_over_type, binding_type, extract_fixed_length_iterable_element_types, infer_complete_scope_types, infer_scope_types, is_discarded_dict_key_assignment, todo_type, }; -use crate::{AnalysisSettings, Db, FxIndexSet, FxOrderSet}; +use crate::{AnalysisSettings, Db, DisplaySettings, FxIndexSet, FxOrderSet}; use ty_python_core::definition::{ AnnotatedAssignmentDefinitionKind, AssignmentDefinitionKind, ComprehensionDefinitionKind, Definition, DefinitionKind, DefinitionNodeKey, DefinitionState, ExceptHandlerDefinitionKind, @@ -10449,9 +10449,15 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { if let Some(builder) = self.context.report_lint(&UNRESOLVED_ATTRIBUTE, attribute) { + let types = std::iter::once(union_like_type) + .chain(elements_missing_the_attribute.iter().copied()); + let settings = + DisplaySettings::from_possibly_ambiguous_types(db, env, types); let missing_types = elements_missing_the_attribute .iter() - .map(|ty| format!("`{}`", ty.display(db, env))) + .map(|ty| { + format!("`{}`", ty.display_with(db, env, settings.clone())) + }) .collect::>() .join(", "); @@ -10459,7 +10465,8 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { "Attribute `{attr_name}` is not defined on {} \ in union `{union_like_type}`", missing_types, - union_like_type = union_like_type.display(db, env), + union_like_type = + union_like_type.display_with(db, env, settings), )); } return type_when_bound; diff --git a/crates/ty_python_semantic/src/types/infer/builder/attribute_assignment.rs b/crates/ty_python_semantic/src/types/infer/builder/attribute_assignment.rs index 8511760946..8bb0de3a1b 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/attribute_assignment.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/attribute_assignment.rs @@ -15,7 +15,9 @@ use crate::types::diagnostic::{ INVALID_ASSIGNMENT, INVALID_ATTRIBUTE_ACCESS, UNRESOLVED_ATTRIBUTE, report_bad_dunder_set_call, report_invalid_attribute_assignment, report_possibly_missing_attribute, }; -use crate::types::{CallDunderError, MemberLookupPolicy, Type, TypeContext, TypeQualifiers}; +use crate::types::{ + CallDunderError, DisplaySettings, MemberLookupPolicy, Type, TypeContext, TypeQualifiers, +}; impl<'db> TypeInferenceBuilder<'db, '_> { /// Make sure that the attribute assignment `obj.attribute = value` is valid. @@ -808,11 +810,16 @@ impl<'db> AssignmentAttributeWriteEvaluator<'_, 'db, '_, '_> { .context .report_lint(&INVALID_ASSIGNMENT, self.target) { + let settings = DisplaySettings::from_possibly_ambiguous_types( + db, + env, + [value_ty, object_ty], + ); builder.into_diagnostic(format_args!( "Object of type `{}` is not assignable to attribute `{}` on type `{}`", - value_ty.display(db, env), + value_ty.display_with(db, env, settings.clone()), self.attribute, - object_ty.display(db, env), + object_ty.display_with(db, env, settings), )); } } diff --git a/crates/ty_python_semantic/src/types/infer/builder/post_inference/static_class.rs b/crates/ty_python_semantic/src/types/infer/builder/post_inference/static_class.rs index 82668ddbaa..d5e261261f 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/post_inference/static_class.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/post_inference/static_class.rs @@ -14,9 +14,10 @@ use crate::{ diagnostic::format_enumeration, place::{DefinedPlace, Place, TypeOrigin, place_from_bindings, place_from_declarations}, types::{ - CallArguments, ClassBase, ClassLiteral, ClassType, DataclassFlags, KnownClass, - KnownInstanceType, MemberLookupPolicy, MetaclassCandidate, Parameters, Signature, - SpecialFormType, StaticClassLiteral, Type, TypeVarVariance, TypingModule, binding_type, + CallArguments, ClassBase, ClassLiteral, ClassType, DataclassFlags, DisplaySettings, + KnownClass, KnownInstanceType, MemberLookupPolicy, MetaclassCandidate, Parameters, + Signature, SpecialFormType, StaticClassLiteral, Type, TypeVarVariance, TypingModule, + binding_type, call::Argument, class::{ AbstractMethod, CodeGeneratorKind, Field, FieldKind, MetaclassErrorKind, @@ -649,16 +650,27 @@ pub(crate) fn check_static_class_definitions<'db>( } else if let Some(builder) = context.report_lint(&CONFLICTING_METACLASS, class_node) { + let types = [ + Type::from(class), + Type::from(*metaclass1), + Type::from(*metaclass2), + Type::from(*class2), + ]; + let settings = DisplaySettings::from_possibly_ambiguous_types(db, env, types); builder.into_diagnostic(format_args!( "The metaclass of a derived class (`{class}`) \ must be a subclass of the metaclasses of all its bases, \ but `{metaclass_of_class}` (metaclass of `{class}`) \ and `{metaclass_of_base}` (metaclass of base class `{base}`) \ have no subclass relationship", - class = class.name(db), - metaclass_of_class = metaclass1.name(db), - metaclass_of_base = metaclass2.name(db), - base = class2.name(db), + class = ClassLiteral::Static(class).display_with(db, settings.clone()), + metaclass_of_class = metaclass1 + .class_literal(db) + .display_with(db, settings.clone()), + metaclass_of_base = metaclass2 + .class_literal(db) + .display_with(db, settings.clone()), + base = ClassLiteral::Static(*class2).display_with(db, settings), )); } } diff --git a/crates/ty_python_semantic/src/types/infer/builder/subscript.rs b/crates/ty_python_semantic/src/types/infer/builder/subscript.rs index 0461705187..38c7e3646e 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/subscript.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/subscript.rs @@ -29,7 +29,7 @@ use crate::types::typed_dict::{ use crate::types::typevar::TypeVarSet; use crate::types::{ BoundTypeVarInstance, CallArguments, CallDunderError, CallableBinding, CycleDetector, - DynamicType, InternedType, KnownClass, KnownInstanceType, LintDiagnosticGuard, + DisplaySettings, DynamicType, InternedType, KnownClass, KnownInstanceType, LintDiagnosticGuard, MemberLookupPolicy, Parameter, Parameters, SpecialFormType, StaticClassLiteral, Type, TypeAliasType, TypeAndQualifiers, TypeContext, TypeVarBoundOrConstraints, UnionType, UnionTypeInstance, any_over_type, todo_type, @@ -1999,14 +1999,22 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { target.range.cover(rhs_value_node.range()), ) { - let assigned_d = rhs_value_ty.display(db, env); - let object_d = object_ty.display(db, env); + let settings = + DisplaySettings::from_possibly_ambiguous_types( + db, + env, + [rhs_value_ty, object_ty, slice_ty], + ); + let assigned_d = + rhs_value_ty.display_with(db, env, settings.clone()); + let object_d = + object_ty.display_with(db, env, settings.clone()); let mut diagnostic = builder.into_diagnostic(format_args!( "Invalid subscript assignment with key of type `{}` \ and value of type `{assigned_d}` \ on object of type `{object_d}`", - slice_ty.display(db, env), + slice_ty.display_with(db, env, settings), )); // Special diagnostic for dictionaries From 938e41e7a9da0839b289198970b744a2860ea954 Mon Sep 17 00:00:00 2001 From: Micha Reiser Date: Tue, 18 Aug 2026 12:49:33 +0200 Subject: [PATCH 081/371] [ty] Do not prefer unsafe fixes in the language server (#27822) --- .../ty_server/src/server/api/diagnostics.rs | 2 + .../src/server/api/requests/code_action.rs | 2 +- crates/ty_server/tests/e2e/code_actions.rs | 35 ++++++ ..._code_actions__code_action_unsafe_fix.snap | 102 ++++++++++++++++++ 4 files changed, 140 insertions(+), 1 deletion(-) create mode 100644 crates/ty_server/tests/e2e/snapshots/e2e__code_actions__code_action_unsafe_fix.snap diff --git a/crates/ty_server/src/server/api/diagnostics.rs b/crates/ty_server/src/server/api/diagnostics.rs index 3d91096d13..d65156a239 100644 --- a/crates/ty_server/src/server/api/diagnostics.rs +++ b/crates/ty_server/src/server/api/diagnostics.rs @@ -647,6 +647,7 @@ pub(crate) struct FullDiagnosticData { pub(crate) struct DiagnosticFixData { pub(crate) fix_title: String, pub(crate) edits: HashMap>, + pub(crate) preferred: bool, } #[derive(Serialize, Deserialize)] @@ -722,6 +723,7 @@ impl DiagnosticData { .map(ToString::to_string) .unwrap_or_else(|| format!("Fix {}", diagnostic.id())), edits: lsp_edits, + preferred: fix.applies(Applicability::Safe), }) } } diff --git a/crates/ty_server/src/server/api/requests/code_action.rs b/crates/ty_server/src/server/api/requests/code_action.rs index d9526d0ba8..eed3531e2f 100644 --- a/crates/ty_server/src/server/api/requests/code_action.rs +++ b/crates/ty_server/src/server/api/requests/code_action.rs @@ -81,7 +81,7 @@ impl BackgroundDocumentRequestHandler for CodeActionRequestHandler { document_changes: None, change_annotations: None, }), - is_preferred: Some(true), + is_preferred: Some(fix.preferred), command: None, disabled: None, data: None, diff --git a/crates/ty_server/tests/e2e/code_actions.rs b/crates/ty_server/tests/e2e/code_actions.rs index 977fe8f146..e806e1c4b2 100644 --- a/crates/ty_server/tests/e2e/code_actions.rs +++ b/crates/ty_server/tests/e2e/code_actions.rs @@ -82,6 +82,41 @@ unused-ignore-comment = \"warn\" Ok(()) } +#[test] +fn code_action_unsafe_fix() -> Result<()> { + let workspace_root = SystemPath::new("src"); + let foo = SystemPath::new("src/foo.py"); + // Removing the suppression is unsafe because it would activate `fmt: off`. + let foo_content = "\ +# ty: ignore[division-by-zero] # fmt: off +x = 20 / 2 +"; + + let ty_toml = SystemPath::new("ty.toml"); + let ty_toml_content = "\ +[rules] +unused-ignore-comment = \"warn\" +"; + + let mut server = TestServerBuilder::new()? + .with_workspace(workspace_root, None)? + .with_file(ty_toml, ty_toml_content)? + .with_file(foo, foo_content)? + .build() + .wait_until_workspaces_are_initialized(); + + server.open_text_document(foo, foo_content, 1); + + let diagnostics = server.document_diagnostic_request(foo, None); + let code_action_params = code_actions_at(&server, diagnostics, foo, full_range(foo_content)); + let code_action_id = server.send_request::(code_action_params); + let code_actions = server.await_response::(&code_action_id); + + insta::assert_json_snapshot!(code_actions); + + Ok(()) +} + #[test] fn no_code_action_for_non_overlapping_range_on_same_line() -> Result<()> { let workspace_root = SystemPath::new("src"); diff --git a/crates/ty_server/tests/e2e/snapshots/e2e__code_actions__code_action_unsafe_fix.snap b/crates/ty_server/tests/e2e/snapshots/e2e__code_actions__code_action_unsafe_fix.snap new file mode 100644 index 0000000000..dd6e2c7a06 --- /dev/null +++ b/crates/ty_server/tests/e2e/snapshots/e2e__code_actions__code_action_unsafe_fix.snap @@ -0,0 +1,102 @@ +--- +source: crates/ty_server/tests/e2e/code_actions.rs +expression: code_actions +--- +[ + { + "title": "Remove the unused suppression comment", + "kind": "quickfix", + "diagnostics": [ + { + "range": { + "start": { + "line": 0, + "character": 0 + }, + "end": { + "line": 0, + "character": 31 + } + }, + "severity": 2, + "code": "unused-ignore-comment", + "codeDescription": { + "href": "https://ty.dev/rules#unused-ignore-comment" + }, + "source": "ty", + "message": "Unused `ty: ignore` directive\n\nhelp: Remove the unused suppression comment", + "tags": [ + 1 + ] + } + ], + "isPreferred": false, + "edit": { + "changes": { + "file:///src/foo.py": [ + { + "range": { + "start": { + "line": 0, + "character": 0 + }, + "end": { + "line": 0, + "character": 31 + } + }, + "newText": "" + } + ] + } + } + }, + { + "title": "Ignore 'unused-ignore-comment' for this line", + "kind": "quickfix", + "diagnostics": [ + { + "range": { + "start": { + "line": 0, + "character": 0 + }, + "end": { + "line": 0, + "character": 31 + } + }, + "severity": 2, + "code": "unused-ignore-comment", + "codeDescription": { + "href": "https://ty.dev/rules#unused-ignore-comment" + }, + "source": "ty", + "message": "Unused `ty: ignore` directive\n\nhelp: Remove the unused suppression comment", + "tags": [ + 1 + ] + } + ], + "isPreferred": false, + "edit": { + "changes": { + "file:///src/foo.py": [ + { + "range": { + "start": { + "line": 0, + "character": 41 + }, + "end": { + "line": 0, + "character": 41 + } + }, + "newText": " # ty: ignore[unused-ignore-comment]" + } + ] + } + } + } +] From 7812d6d133f0b92ec6895ac0715333f96273562f Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Tue, 18 Aug 2026 07:39:30 -0700 Subject: [PATCH 082/371] [ty] Specialize inherited members of unspecialized generic classes (#27658) ## Summary Fixes astral-sh/ty#4232. - Apply an unspecialized generic class's default type arguments after inherited class-member lookup, without changing subclass-to-base specialization or the structural MRO. - Preserve generic constructor inference through its existing identity specialization while correctly resolving explicit access to inherited `__new__` and `__init__` methods. - Align inherited synthesized dataclass initializer access with ordinary inherited member access. ## Test plan - Add legacy and PEP 695 mdtests covering inherited attributes, static methods, class methods, ordinary methods, method-scoped generics, generic descriptors, explicit and inferred constructor calls, declared type-parameter defaults, and partially specialized base classes. - Update the existing generic dataclass mdtest to verify that explicit inherited initializer access resolves unspecialized class type variables while construction continues to infer its type arguments. --- .../mdtest/generics/legacy/classes.md | 75 ++++++++++++++++++- .../mdtest/generics/pep695/classes.md | 36 +++++++++ .../src/types/class/static_literal.rs | 17 ++++- 3 files changed, 126 insertions(+), 2 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md index 7a5b5a0da7..24371ff8fd 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/classes.md @@ -1024,7 +1024,7 @@ When a generic subclass fills its superclass's type parameter with one of its ow propagate through: ```py -from typing_extensions import Generic, TypeVar +from typing_extensions import Generic, Self, TypeVar T = TypeVar("T") U = TypeVar("U") @@ -1034,6 +1034,17 @@ W = TypeVar("W") class Parent(Generic[T]): x: T + @staticmethod + def static(value: T) -> T: + return value + + @classmethod + def class_method(cls, value: T) -> T: + return value + + def method(self, value: T, other: U) -> U: + return other + class ExplicitlyGenericChild(Parent[U], Generic[U]): ... class ExplicitlyGenericGrandchild(ExplicitlyGenericChild[V], Generic[V]): ... class ExplicitlyGenericGreatgrandchild(ExplicitlyGenericGrandchild[W], Generic[W]): ... @@ -1050,6 +1061,68 @@ reveal_type(ExplicitlyGenericGreatgrandchild[int]().x) # revealed: int reveal_type(ImplicitlyGenericGreatgrandchild[int]().x) # revealed: int ``` +Implicitly generic subclasses, explicitly generic subclasses, and longer inheritance chains all +replace an unresolved class type variable with `Unknown`. + +```py +reveal_type(Parent.x) # revealed: Unknown +reveal_type(ExplicitlyGenericChild.x) # revealed: Unknown +reveal_type(ImplicitlyGenericChild.x) # revealed: Unknown +reveal_type(ImplicitlyGenericGrandchild.x) # revealed: Unknown +``` + +The same specialization applies to inherited static methods, class methods, and ordinary methods. +Type variables belonging to a method remain generic. + +```py +# revealed: def static(value: Unknown) -> Unknown +reveal_type(ImplicitlyGenericChild.static) +# revealed: bound method .class_method(value: Unknown) -> Unknown +reveal_type(ImplicitlyGenericChild.class_method) +# revealed: def method[U](self, value: Unknown, other: U) -> U +reveal_type(ImplicitlyGenericChild.method) + +ImplicitlyGenericChild.static(1) +ImplicitlyGenericChild.class_method(1) +reveal_type(ImplicitlyGenericChild[int].static(1)) # revealed: int +``` + +Constructor methods inherit their class's type variables into their own generic contexts, so they +remain generic when accessed explicitly. Calling the class itself also infers its type arguments. + +```py +class ConstructorParent(Generic[T]): + def __new__(cls, value: T) -> Self: + return super().__new__(cls) + + def __init__(self, value: T) -> None: ... + +class ConstructorChild(ConstructorParent[T]): ... + +# revealed: def __new__[Self, T](cls, value: T) -> Self +reveal_type(ConstructorChild.__new__) +# revealed: def __init__[T](self, value: T) -> None +reveal_type(ConstructorChild.__init__) +reveal_type(ConstructorChild(1)) # revealed: ConstructorChild[int] +``` + +A generic descriptor inherited from the parent also receives the receiver's specialization before +its `__get__` method is called. + +```py +class Descriptor(Generic[T]): + def __get__(self, instance: object | None, owner: type[object]) -> T: + raise NotImplementedError + +class DescriptorParent(Generic[T]): + descriptor: Descriptor[T] = Descriptor() + +class DescriptorChild(DescriptorParent[T]): ... + +reveal_type(DescriptorChild.descriptor) # revealed: Unknown +reveal_type(DescriptorChild[int].descriptor) # revealed: int +``` + ## Generic methods Generic classes can contain methods that are themselves generic. The generic methods can refer to diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/classes.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/classes.md index 48a9d0e383..8b67a47057 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/classes.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/classes.md @@ -730,6 +730,10 @@ propagate through: class Parent[T]: x: T + @staticmethod + def static(value: T) -> T: + return value + class Child[U](Parent[U]): ... class Grandchild[V](Child[V]): ... class Greatgrandchild[W](Child[W]): ... @@ -740,6 +744,38 @@ reveal_type(Grandchild[int]().x) # revealed: int reveal_type(Greatgrandchild[int]().x) # revealed: int ``` +Attributes and static methods inherited by an unspecialized generic subclass use its default type +arguments instead of exposing its class-scoped type variables. + +```py +reveal_type(Parent.x) # revealed: Unknown +reveal_type(Child.x) # revealed: Unknown +reveal_type(Grandchild.x) # revealed: Unknown + +# revealed: def static(value: Unknown) -> Unknown +reveal_type(Child.static) +Child.static(1) +reveal_type(Child[int].static(1)) # revealed: int +``` + +Declared defaults must be preserved, and concrete arguments in partially specialized bases must not +be replaced with `Unknown`. + +```py +class DefaultChild[T = int](Parent[T]): ... + +class PairParent[T, U]: + fixed: T + unresolved: U + +class PartiallyFixed[T](PairParent[int, T]): ... + +reveal_type(DefaultChild.x) # revealed: int +reveal_type(DefaultChild[str].x) # revealed: str +reveal_type(PartiallyFixed.fixed) # revealed: int +reveal_type(PartiallyFixed.unresolved) # revealed: Unknown +``` + ## Generic methods Generic classes can contain methods that are themselves generic. The generic methods can refer to diff --git a/crates/ty_python_semantic/src/types/class/static_literal.rs b/crates/ty_python_semantic/src/types/class/static_literal.rs index d12bdc765b..20ccc2d48d 100644 --- a/crates/ty_python_semantic/src/types/class/static_literal.rs +++ b/crates/ty_python_semantic/src/types/class/static_literal.rs @@ -1342,7 +1342,22 @@ impl<'db> StaticClassLiteral<'db> { name: &str, policy: MemberLookupPolicy, ) -> PlaceAndQualifiers<'db> { - self.class_member_from_mro(db, env, name, policy, self.iter_mro(db, specialization)) + let member = + self.class_member_from_mro(db, env, name, policy, self.iter_mro(db, specialization)); + + // An unspecialized MRO retains mappings such as `Parent[T@Child]`, so ordinary members + // accessed through `Child` must use its default arguments. Constructor methods are different: + // we add their class's type variables to the callable's generic context, so those variables + // are genuinely inferable and must remain generic instead of using the default arguments. + if specialization.is_none() + && !matches!(name, "__new__" | "__init__") + && let Some(generic_context) = self.generic_context(db) + { + let specialization = generic_context.default_specialization(db, self.known(db)); + member.map_type(|ty| ty.apply_specialization(db, specialization)) + } else { + member + } } pub(crate) fn class_member_from_mro( From 1d5d4c52dad72c86c11ebaaa4f29e70ffd433ba5 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Tue, 18 Aug 2026 08:17:46 -0700 Subject: [PATCH 083/371] [ty] Infer generic protocols from class objects (#27812) ## Summary Passing a class object to a generic protocol parameter could leave the protocol's type arguments as `Unknown`, even when its attributes or methods supplied enough information to infer them. Ordinary instance arguments and unions already used structural constraints, but individual class objects fell through to inference that only considered `__call__`. Use the existing structural constraint machinery for class literals, specialized generic classes, and `type[C]` arguments. This preserves the class object's own interface and also fixes `list(EnumClass)` inferring `list[Unknown]`. Fixes https://github.com/astral-sh/ty/issues/4291 ## Test plan Added mdtests for inference through plain class attributes, class methods, static methods, ordinary-method protocols, specialized generic class objects, and unions of class types. Callback regressions cover preserving `T` when passing `type[T]` and inferring from a data member alongside `__call__`. The tests also retain instance behavior and rejection of an incompatible return type. Updated the enum iteration test to expect the precise member type. ### Ecosystem The ecosystem report includes a new `invalid-assignment` diagnostic in [Scrapy's telnet test](https://github.com/scrapy/scrapy/blob/488a762d7173922190ea027e0fb6f6d9d3b59880/tests/test_extension_telnet.py#L46), which replaces a zero-argument bound method with `dict`. This PR correctly infers `TelnetConsole` where the result was previously `Unknown`, revealing an orthogonal, pre-existing limitation in ty's instance-method assignment checking. The directly typed assignment is rejected on both the base and PR revisions; the inference change does not introduce that restriction. Compatible class-level method replacement was addressed in [#26158](https://github.com/astral-sh/ruff/pull/26158), while instance-method replacement remains separate work related to [ty#350](https://github.com/astral-sh/ty/issues/350). This is just better type inference exposing an unrelated issue, so we accept it for this PR. Other ecosystem changes are desirable improved inference. --- .../resources/mdtest/enums.md | 3 +- .../resources/mdtest/protocols.md | 164 ++++++++++++++++++ .../ty_python_semantic/src/types/generics.rs | 10 +- 3 files changed, 174 insertions(+), 3 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/enums.md b/crates/ty_python_semantic/resources/mdtest/enums.md index acce0c731e..b9c1e053ba 100644 --- a/crates/ty_python_semantic/resources/mdtest/enums.md +++ b/crates/ty_python_semantic/resources/mdtest/enums.md @@ -2064,8 +2064,7 @@ class Color(Enum): for color in Color: reveal_type(color) # revealed: Color -# TODO: Should be `list[Color]` -reveal_type(list(Color)) # revealed: list[Unknown] +reveal_type(list(Color)) # revealed: list[Color] ``` ## Methods / non-member attributes diff --git a/crates/ty_python_semantic/resources/mdtest/protocols.md b/crates/ty_python_semantic/resources/mdtest/protocols.md index 5f00ed8eee..b581e33b0e 100644 --- a/crates/ty_python_semantic/resources/mdtest/protocols.md +++ b/crates/ty_python_semantic/resources/mdtest/protocols.md @@ -4418,6 +4418,119 @@ class IntParser: parser: Parser = IntParser ``` +## Generic protocol inference from class attributes + +A class object can supply the type argument of a protocol through an ordinary attribute. Passing the +class itself should infer the same type as passing an instance. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Protocol + +class HasValue[T](Protocol): + value: T + +class IntValue: + value: int = 1 + +def get_value[T](obj: HasValue[T]) -> T: + return obj.value + +reveal_type(get_value(IntValue)) # revealed: int +reveal_type(get_value(IntValue())) # revealed: int + +def _(cls: type[IntValue]) -> None: + reveal_type(get_value(cls)) # revealed: int +``` + +## Generic protocol inference from class methods + +The return type of a class method can determine a protocol's type argument, including when the +argument is the class object itself. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Protocol, TypeVar + +T_co = TypeVar("T_co", covariant=True) +T = TypeVar("T") + +class Factory(Protocol[T_co]): + @classmethod + def make(cls) -> T_co: ... + +class Concrete: + @classmethod + def make(cls) -> "Concrete": + return cls() + +def from_protocol(factory: Factory[T]) -> T: + return factory.make() + +reveal_type(from_protocol(Concrete)) # revealed: Concrete +reveal_type(from_protocol(Concrete())) # revealed: Concrete + +bad: Factory[str] = Concrete # error: [invalid-assignment] +``` + +Specialized generic class objects and unions of class objects also contribute their method return +types to inference. + +```py +class GenericFactory[T]: + @classmethod + def make(cls) -> T: + raise NotImplementedError + +reveal_type(from_protocol(GenericFactory[int])) # revealed: int + +def _(cls: type[GenericFactory[int]] | type[GenericFactory[str]]) -> None: + reveal_type(from_protocol(cls)) # revealed: int | str +``` + +## Generic protocol inference from static methods + +A static method on a class object can satisfy either a static-method or an instance-method protocol +member. Both forms should contribute the method's return type to inference. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Protocol + +class StaticFactory[T](Protocol): + @staticmethod + def make() -> T: ... + +class InstanceFactory[T](Protocol): + def make(self) -> T: ... + +class IntFactory: + @staticmethod + def make() -> int: + return 1 + +def from_static[T](factory: StaticFactory[T]) -> T: + return factory.make() + +def from_instance[T](factory: InstanceFactory[T]) -> T: + return factory.make() + +reveal_type(from_static(IntFactory)) # revealed: int +reveal_type(from_instance(IntFactory)) # revealed: int +``` + ## Class objects and `Self`-returning class-method protocol members When a class object is checked against a class-method protocol member, `Self` in the protocol @@ -5578,6 +5691,57 @@ class Constructor(Protocol): constructor: Constructor = Product ``` +## Generic constructor callback inference + +Passing `type[T]` to a generic callback protocol must preserve the type variable returned by the +class's constructor. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Protocol + +class Callback[T](Protocol): + def __call__(self) -> T: ... + +def invoke[T](callback: Callback[T]) -> T: + return callback() + +def create[T](cls: type[T]) -> T: + reveal_type(invoke(cls)) # revealed: T@create + return invoke(cls) +``` + +## Generic callback inference from other members + +A callable protocol can infer a type argument from another member. Comparing only its `__call__` +signature would miss the class attribute that determines `T` here. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Protocol + +class ConstructorWithValue[T](Protocol): + value: T + + def __call__(self) -> object: ... + +class Product: + value: int = 1 + +def get_value[T](factory: ConstructorWithValue[T]) -> T: + return factory.value + +reveal_type(get_value(Product)) # revealed: int +``` + ## Generic protocols and union arguments When a union is passed to a parameter annotated as a generic protocol, each union element can diff --git a/crates/ty_python_semantic/src/types/generics.rs b/crates/ty_python_semantic/src/types/generics.rs index 228fada6c1..3fd64c1aea 100644 --- a/crates/ty_python_semantic/src/types/generics.rs +++ b/crates/ty_python_semantic/src/types/generics.rs @@ -4073,7 +4073,15 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { return self.infer_from_constraint_set(when); } - (formal @ Type::ProtocolInstance(_), actual @ Type::TypedDict(_)) => { + ( + formal @ Type::ProtocolInstance(_), + actual @ (Type::ClassLiteral(_) + | Type::GenericAlias(_) + | Type::SubclassOf(_) + | Type::TypedDict(_)), + ) => { + // A class object can itself implement a protocol. Compare its members directly; + // converting it to its instance type would infer from a different interface. let when = self.constraint_for_relation(formal, actual, relation_polarity); return self.infer_from_constraint_set(when); } From ce46a36be796faff60d1ad5734e571bb1223c0a7 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Tue, 18 Aug 2026 08:17:46 -0700 Subject: [PATCH 084/371] [ty] Simplify generic protocol inference (#27819) Generic protocol inference still treated some arguments as callable signatures even after class objects gained structural inference in #27812. That duplicated the protocol logic and could miss type-variable evidence from members other than `__call__`. Use one structural-constraint branch for protocol formals, preserving the TypedDict-union optimization. Remove the protocol-specific `__call__` path and its unused helper. This also allows module attributes and ordinary function attributes to contribute to generic protocol inference, and preserves type-variable evidence hidden behind type aliases. Stacked on #27812. ## Test plan Added mdtests for inferring a protocol type argument from a module attribute and from a function's `__name__` alongside `__call__`. Added coverage for inference and type-variable bounds through both PEP 695 and `TypeAliasType` aliases. Simplified the class-method regression fixture to use PEP 695 syntax. Existing coverage continues to exercise class objects, callable overloads, ParamSpec, TypeVarTuple, TypedDict unions, and enum iteration. --------- Co-authored-by: Alex Waygood --- .../resources/mdtest/protocols.md | 117 ++++++++++++++++-- .../ty_python_semantic/src/types/generics.rs | 68 +--------- .../src/types/protocol_class.rs | 25 ---- 3 files changed, 116 insertions(+), 94 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/protocols.md b/crates/ty_python_semantic/resources/mdtest/protocols.md index b581e33b0e..69d6219438 100644 --- a/crates/ty_python_semantic/resources/mdtest/protocols.md +++ b/crates/ty_python_semantic/resources/mdtest/protocols.md @@ -4398,6 +4398,86 @@ factory_object: FactoryObject = factory factory_module: FactoryModule = factory ``` +## Generic protocol inference from module objects + +A module attribute can determine a protocol's type argument when the module itself is passed to a +generic function. + +```toml +[environment] +python-version = "3.12" +``` + +`values.py`: + +```py +value: int = 1 +``` + +`main.py`: + +```py +from typing import Protocol + +import values + +class HasValue[T](Protocol): + value: T + +def get_value[T](obj: HasValue[T]) -> T: + return obj.value + +reveal_type(get_value(values)) # revealed: int +``` + +## Generic protocol inference through type aliases + +An alias for an instance type does not obscure the members that determine a protocol's type +arguments. Inference sees through these aliases and checks the bounds of these arguments. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Protocol + +class HasValue[T](Protocol): + def get(self) -> T: ... + +class Box[T]: + def get(self) -> T: + raise NotImplementedError + +type Alias[T] = Box[T] + +def get_value[T](value: HasValue[T]) -> T: + return value.get() + +def require_str[T: str](value: HasValue[T]) -> T: + return value.get() + +def check_alias(value: Alias[int]): + reveal_type(get_value(value)) # revealed: int + # error: [invalid-argument-type] "Argument type `int` does not satisfy upper bound `str` of type variable `T`" + require_str(value) +``` + +An equivalent generic alias constructed with `TypeAliasType` preserves the same information. + +```py +from typing import TypeAliasType, TypeVar + +U = TypeVar("U") +ConstructedAlias = TypeAliasType("ConstructedAlias", Box[U], type_params=(U,)) + +def check_constructed_alias(value: ConstructedAlias[int]): + reveal_type(get_value(value)) # revealed: int + # error: [invalid-argument-type] "Argument type `int` does not satisfy upper bound `str` of type variable `T`" + require_str(value) +``` + ## Class objects with class-method protocol members A class object implements a protocol when its directly accessible members have compatible types. The @@ -4458,21 +4538,18 @@ python-version = "3.12" ``` ```py -from typing import Protocol, TypeVar - -T_co = TypeVar("T_co", covariant=True) -T = TypeVar("T") +from typing import Protocol -class Factory(Protocol[T_co]): +class Factory[T](Protocol): @classmethod - def make(cls) -> T_co: ... + def make(cls) -> T: ... class Concrete: @classmethod def make(cls) -> "Concrete": return cls() -def from_protocol(factory: Factory[T]) -> T: +def from_protocol[T](factory: Factory[T]) -> T: return factory.make() reveal_type(from_protocol(Concrete)) # revealed: Concrete @@ -5742,6 +5819,32 @@ def get_value[T](factory: ConstructorWithValue[T]) -> T: reveal_type(get_value(Product)) # revealed: int ``` +## Generic callback inference from function attributes + +A function object's attributes also contribute to protocol inference. Here, the type argument comes +from `__name__`, not the callback's return type. + +```toml +[environment] +python-version = "3.12" +``` + +```py +from typing import Protocol + +class NamedCallback[T](Protocol): + __name__: T + + def __call__(self) -> object: ... + +def get_name[T](callback: NamedCallback[T]) -> T: + return callback.__name__ + +def callback() -> None: ... + +reveal_type(get_name(callback)) # revealed: str +``` + ## Generic protocols and union arguments When a union is passed to a parameter annotated as a generic protocol, each union element can diff --git a/crates/ty_python_semantic/src/types/generics.rs b/crates/ty_python_semantic/src/types/generics.rs index 3fd64c1aea..4e1d1328c4 100644 --- a/crates/ty_python_semantic/src/types/generics.rs +++ b/crates/ty_python_semantic/src/types/generics.rs @@ -4006,29 +4006,9 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { return Ok(()); } - ( - formal @ (Type::NominalInstance(_) | Type::ProtocolInstance(_)), - Type::NominalInstance(actual_nominal), - ) => { - // Extract formal_alias if this is a generic class - let formal_alias = match formal { - Type::NominalInstance(formal_nominal) => { - formal_nominal.class(db, self.env).into_generic_alias() - } - - Type::ProtocolInstance(_) => { - // TODO: For protocols, we use the new constraint set implementation, which - // will handle implicitly implemented protocols and generic protocols. We - // eventually want this logic to be used for _all_ nominal instances - // (replacing the logic below). - let when = self.constraint_for_relation(formal, actual, relation_polarity); - return self.infer_from_constraint_set(when); - } - - _ => None, - }; - - if let Some(formal_alias) = formal_alias { + (Type::NominalInstance(formal_nominal), Type::NominalInstance(actual_nominal)) => { + if let Some(formal_alias) = formal_nominal.class(db, self.env).into_generic_alias() + { let formal_origin = formal_alias.origin(db); for base in actual_nominal.class(db, self.env).iter_mro(db) { let ClassBase::Class(ClassType::Generic(base_alias)) = base else { @@ -4056,13 +4036,11 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { } } - // TODO: in principle this could be a generalized Union-actual arm that maps over the - // union, but the old solver isn't well-equipped to handle that (due to side effects - // from even failed matches), so for now we handle this particular case. - (formal @ Type::ProtocolInstance(_), actual @ Type::Union(actual_union)) => { + (formal @ Type::ProtocolInstance(_), actual) => { // Common TypedDict constraints prove only `actual <= formal`. Contravariance // reverses that relation, while invariance additionally requires the reverse. - let when = if matches!(relation_polarity, TypeVarVariance::Covariant) + let when = if let Type::Union(actual_union) = actual + && matches!(relation_polarity, TypeVarVariance::Covariant) && let Some(common) = self.common_typed_dict_protocol_constraints(formal, actual_union) { @@ -4073,40 +4051,6 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { return self.infer_from_constraint_set(when); } - ( - formal @ Type::ProtocolInstance(_), - actual @ (Type::ClassLiteral(_) - | Type::GenericAlias(_) - | Type::SubclassOf(_) - | Type::TypedDict(_)), - ) => { - // A class object can itself implement a protocol. Compare its members directly; - // converting it to its instance type would infer from a different interface. - let when = self.constraint_for_relation(formal, actual, relation_polarity); - return self.infer_from_constraint_set(when); - } - - // When the formal type is a protocol with a `__call__` method, infer the specialization - // from matching the actual type's callable signature against the protocol's `__call__` - // method signature. - (Type::ProtocolInstance(formal_protocol), _) => { - let Some(call_method) = formal_protocol.interface(db).call_method(db, self.env) - else { - return Ok(()); - }; - let Some(actual_callables) = actual.try_upcast_to_callable(db, self.env) else { - return Ok(()); - }; - - // The protocol interface exposes the callable signature already bound for - // instance access. - self.infer_from_callable_signature( - call_method, - actual_callables, - relation_polarity, - )?; - } - (Type::Callable(formal_callable), _) => { let Some(actual_callables) = actual.try_upcast_to_callable(db, self.env) else { return Ok(()); diff --git a/crates/ty_python_semantic/src/types/protocol_class.rs b/crates/ty_python_semantic/src/types/protocol_class.rs index be49967184..cf83df1ad8 100644 --- a/crates/ty_python_semantic/src/types/protocol_class.rs +++ b/crates/ty_python_semantic/src/types/protocol_class.rs @@ -526,31 +526,6 @@ impl<'db> ProtocolInterfaceView<'db> { }) } - /// Returns the callable signature exposed by instance access to a protocol's `__call__` - /// method. - /// - /// The callable is already in its instance-bound form, so callers must not bind it again. - pub(super) fn call_method( - self, - db: &'db dyn Db, - env: &ProgramEnvironment<'db>, - ) -> Option> { - self.member_by_name(db, "__call__").and_then(|member| { - if !member.is_method() { - return None; - } - match member - .access(db, env, ProtocolMemberAccessMode::Instance) - .read - .and_then(|read| read.resolve(db, env)) - .map(ProtocolMemberType::ty) - { - Some(Type::Callable(callable)) => Some(callable), - _ => None, - } - }) - } - pub(super) fn instance_member( self, db: &'db dyn Db, From 83df79510e5fbbef139ccd24beeec6f3991a8553 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Tue, 18 Aug 2026 09:06:25 -0700 Subject: [PATCH 085/371] [ty] Preserve property subclass types and accessors (#27833) ## Summary Subclasses of `property` lost their getter type during construction, and applying an inherited `.setter` reduced subsequent attribute access to `Unknown`. Preserve the descriptor's nominal class and accessor types when it inherits the standard property implementation. Generalize the existing `enum.property` handling to ordinary property subclasses, retain generic specializations through accessor replacement, and recognize inherited descriptor methods without overriding custom behavior. Subclasses with custom constructors, descriptor methods, or accessor attributes continue through ordinary inference. Closes https://github.com/astral-sh/ty/issues/4302 ## Test plan Added mdtests for getter and setter inference, invalid setter assignments, direct construction, all three accessor-copy methods, generic and `enum.property` subclasses, custom accessor methods and attributes, custom descriptor behavior and constructors, and subclass truthiness. A variance regression verifies that mutable state on a generic property subclass makes its owning class invariant and prevents unsafe writes through a widened owner. Existing incrementality tests also cover the method-recognition path. --- .../resources/mdtest/enums.md | 33 +++ .../mdtest/generics/pep695/variance.md | 29 +++ .../resources/mdtest/properties.md | 205 ++++++++++++++++++ crates/ty_python_semantic/src/types.rs | 193 +++++++++++++---- crates/ty_python_semantic/src/types/bool.rs | 17 +- .../ty_python_semantic/src/types/call/bind.rs | 16 +- .../src/types/call/bind/enum_property.rs | 60 ----- .../src/types/call/bind/property.rs | 103 +++++++++ .../ty_python_semantic/src/types/display.rs | 40 ++-- 9 files changed, 571 insertions(+), 125 deletions(-) delete mode 100644 crates/ty_python_semantic/src/types/call/bind/enum_property.rs create mode 100644 crates/ty_python_semantic/src/types/call/bind/property.rs diff --git a/crates/ty_python_semantic/resources/mdtest/enums.md b/crates/ty_python_semantic/resources/mdtest/enums.md index b9c1e053ba..c348c36af2 100644 --- a/crates/ty_python_semantic/resources/mdtest/enums.md +++ b/crates/ty_python_semantic/resources/mdtest/enums.md @@ -1214,6 +1214,39 @@ class InheritedChoices(BaseChoices): reveal_type(InheritedChoices.A.value) # revealed: str ``` +### Subclasses of `enum.property` + +An inherited property initializer and accessor-copy methods retain the descriptor's subclass. + +```toml +[environment] +python-version = "3.11" +``` + +```py +from enum import Enum, property as enum_property + +class CustomProperty(enum_property): ... + +def get_value(obj: object) -> int: + return 1 + +def set_value(obj: object, value: str) -> None: + pass + +descriptor = CustomProperty(get_value).setter(set_value) +reveal_type(descriptor) # revealed: CustomProperty +retained: CustomProperty = descriptor + +class Choice(Enum): + A = 1 + value = descriptor + +reveal_type(Choice.A.value) # revealed: int +Choice.A.value = "new" +Choice.A.value = 1 # error: [invalid-assignment] +``` + ### `types.DynamicClassAttribute` Attributes defined using `types.DynamicClassAttribute` are not considered members: diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/variance.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/variance.md index 7a0a7f8c5e..072e9c1af8 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/variance.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/variance.md @@ -727,6 +727,35 @@ static_assert(not is_subtype_of(D[B], D[A])) static_assert(not is_subtype_of(D[A], D[B])) ``` +### Property subclasses + +A property subclass can carry mutable state in its own type parameters. That state makes the owning +class invariant even when the property's getter does not mention the type parameter. + +```py +from ty_extensions import static_assert +from ty_extensions._internal import is_subtype_of + +def get_value(obj: object) -> int: + return 1 + +class CustomProperty[T](property): + metadata: T + +class Owner[T]: + value = CustomProperty[T](get_value) + +static_assert(not is_subtype_of(Owner[str], Owner[object])) +static_assert(not is_subtype_of(Owner[object], Owner[str])) + +def overwrite(owner: Owner[object]) -> None: + type(owner).value.metadata = object() + +def misuse(owner: Owner[str]) -> str: + overwrite(owner) # error: [invalid-argument-type] + return type(owner).value.metadata +``` + ### Implicit Attributes Implicit attributes work like normal ones diff --git a/crates/ty_python_semantic/resources/mdtest/properties.md b/crates/ty_python_semantic/resources/mdtest/properties.md index 85e66e5098..71087becf0 100644 --- a/crates/ty_python_semantic/resources/mdtest/properties.md +++ b/crates/ty_python_semantic/resources/mdtest/properties.md @@ -49,6 +49,211 @@ c.my_property = 2 c.my_property = "a" ``` +## Property subclasses + +A subclass that inherits the built-in property implementation retains both its nominal type and its +accessors. Adding a setter does not discard the getter's return type. + +```py +class CustomProperty(property): + def description(self) -> str: + return "custom" + +class C: + @CustomProperty + def value(self) -> int: + return 1 + + @value.setter + def value(self, value: str) -> None: + pass + +reveal_type(C.value) # revealed: CustomProperty +reveal_type(C.value.description()) # revealed: str +reveal_type(C().value) # revealed: int +C().value = "new" +C().value = 1 # error: [invalid-assignment] +``` + +## Replacing subclass accessors + +Direct construction and each accessor decorator preserve the subclass. Replacing one accessor also +preserves the other accessors. + +```py +class CustomProperty(property): ... + +def get_value(obj: object) -> int: + return 1 + +def set_value(obj: object, value: str) -> None: + pass + +def delete_value(obj: object) -> None: + pass + +def get_text(obj: object) -> str: + return "value" + +original = CustomProperty(get_value) +updated = original.setter(set_value).deleter(delete_value).getter(get_text) +reveal_type(original) # revealed: CustomProperty +reveal_type(updated) # revealed: CustomProperty +reveal_type(original.fget) # revealed: def get_value(obj: object) -> int +reveal_type(updated.fget) # revealed: def get_text(obj: object) -> str +reveal_type(updated.fset) # revealed: def set_value(obj: object, value: str) -> None +reveal_type(updated.fdel) # revealed: def delete_value(obj: object) -> None + +class C: + before = original + after = updated + +reveal_type(C.before) # revealed: CustomProperty +reveal_type(C().before) # revealed: int +reveal_type(C().after) # revealed: str +reveal_type(updated.__get__(C(), C)) # revealed: str +reveal_type(type(updated).__get__(updated, C(), C)) # revealed: str +C().after = "new" +C().after = 1 # error: [invalid-assignment] +del C().after +``` + +## Generic property subclasses + +The nominal class specialization is retained when an accessor is replaced. + +```py +from typing import Generic, TypeVar + +T = TypeVar("T") + +class CustomProperty(property, Generic[T]): + metadata: T + +def get_value(obj: object) -> int: + return 1 + +def set_value(obj: object, value: str) -> None: + pass + +descriptor = CustomProperty[bytes](get_value).setter(set_value) +reveal_type(descriptor) # revealed: CustomProperty[bytes] +reveal_type(descriptor.metadata) # revealed: bytes +``` + +Specializing the class that owns the property also specializes the descriptor's nominal type. + +```py +class Owner(Generic[T]): + value = CustomProperty[T](get_value) + +reveal_type(Owner[str].value) # revealed: CustomProperty[str] +reveal_type(Owner[str].value.metadata) # revealed: str +reveal_type(Owner[str]().value) # revealed: int +``` + +## Overridden property accessor methods + +A subclass can replace an accessor-copy method. Its declared return type takes precedence over the +built-in copy behavior. + +```py +from typing import Any, Callable + +class ReplacementProperty(property): ... + +class CustomProperty(property): + def setter(self, fset: Callable[[Any, Any], None], /) -> ReplacementProperty: + return ReplacementProperty() + +def set_value(obj: object, value: str) -> None: + pass + +reveal_type(CustomProperty().setter(set_value)) # revealed: ReplacementProperty +``` + +## Overridden property descriptor methods + +Subclasses that change the descriptor protocol are checked as ordinary descriptors. Their `__get__` +annotations must not be replaced with the stored getter's return type. + +```py +from typing import overload +from typing_extensions import Self + +def get_value(obj: object) -> int: + return 1 + +class CustomGetter(property): + @overload + def __get__(self, instance: None, owner: type, /) -> Self: ... + @overload + def __get__(self, instance: object, owner: type | None = None, /) -> str: ... + def __get__(self, instance: object, owner: type | None = None, /) -> Self | str: + return self if instance is None else "custom" + +class C: + value = CustomGetter(get_value) + +reveal_type(C.value) # revealed: CustomGetter +reveal_type(C().value) # revealed: str +``` + +## Overridden accessor attributes + +A subclass may hide an accessor attribute without changing the getter that the descriptor calls. The +stored callable must not replace that explicitly defined attribute. + +```py +class HiddenGetter(property): + fget: None = None + +def get_value(obj: object) -> int: + return 1 + +descriptor = HiddenGetter(get_value) +reveal_type(descriptor.fget) # revealed: None +``` + +## Custom property constructors + +A custom initializer can give its arguments a different meaning. We must not interpret those +arguments as the built-in getter, setter, and deleter parameters. + +```py +from typing import Any, Callable + +class CustomProperty(property): + def __init__(self, description: str, getter: Callable[[Any], Any]) -> None: + super().__init__(getter) + +def get_value(obj: object) -> int: + return 1 + +descriptor = CustomProperty("value", get_value) +reveal_type(descriptor) # revealed: CustomProperty + +class C: + value = descriptor + +reveal_type(C.value) # revealed: CustomProperty +reveal_type(C().value) # revealed: Unknown +``` + +## Property subclass truthiness + +Tracking the accessors does not make a subclass with a custom `__bool__` unconditionally truthy. + +```py +from typing import Literal + +class FalsyProperty(property): + def __bool__(self) -> Literal[False]: + return False + +reveal_type(bool(FalsyProperty())) # revealed: Literal[False] +``` + ## Properties returning `Self` A property that returns `Self` refers to an instance of the class: diff --git a/crates/ty_python_semantic/src/types.rs b/crates/ty_python_semantic/src/types.rs index 9e1f3d0bf1..d622acb27e 100644 --- a/crates/ty_python_semantic/src/types.rs +++ b/crates/ty_python_semantic/src/types.rs @@ -20,7 +20,9 @@ use ruff_python_ast as ast; use ruff_python_ast::name::Name; use ruff_text_size::Ranged; use smallvec::smallvec_inline; -use ty_module_resolver::{ImportingFile, KnownModule, Module, ModuleName, resolve_module}; +use ty_module_resolver::{ + ImportingFile, KnownModule, Module, ModuleName, file_to_module, resolve_module, +}; pub(crate) use self::callable::UpcastPolicy; use self::class::ClassInstanceFlags; @@ -1108,7 +1110,92 @@ pub enum PropertyAccessorRole { Deleter, } -/// Represents an instance of `builtins.property` or `enum.property`. +/// The nominal class of a precise property. Known classes remain lazy so synthesized properties +/// do not need to resolve typeshed just to record their class. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, get_size2::GetSize, salsa::SalsaValue)] +pub enum PropertyInstanceClass<'db> { + Builtin, + Enum, + Subclass(ClassType<'db>), +} + +impl<'db> PropertyInstanceClass<'db> { + fn from_class(db: &'db dyn Db, class: ClassType<'db>) -> Self { + match class.known(db) { + Some(KnownClass::Property) => Self::Builtin, + Some(KnownClass::EnumProperty) => Self::Enum, + _ => Self::Subclass(class), + } + } + + fn to_class_literal(self, db: &'db dyn Db, env: &ProgramEnvironment<'db>) -> Type<'db> { + match self { + Self::Builtin => KnownClass::Property.to_class_literal(db, env), + Self::Enum => KnownClass::EnumProperty.to_class_literal(db, env), + Self::Subclass(class) => class.into(), + } + } + + fn to_instance(self, db: &'db dyn Db, env: &ProgramEnvironment<'db>) -> Type<'db> { + match self { + Self::Builtin => KnownClass::Property.to_instance(db, env), + Self::Enum => KnownClass::EnumProperty.to_instance(db, env), + Self::Subclass(class) => Type::instance(db, env, class), + } + } +} + +/// Identifies the actual implementation, rather than a method with the same name on a subclass. +fn is_property_method<'db>( + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + function: FunctionType<'db>, +) -> bool { + let class = match file_to_module(db, function.program_file(db).resolver_file(db)) + .and_then(|module| module.known(db)) + { + Some(KnownModule::Builtins) => KnownClass::Property, + Some(KnownModule::Enum | KnownModule::Types) => KnownClass::EnumProperty, + _ => return false, + }; + + class + .try_to_class_literal(db, env) + .and_then(|class| { + ClassLiteral::Static(class) + .class_member(db, env, function.name(db), MemberLookupPolicy::default()) + .place + .ignore_possibly_undefined() + }) + .and_then(Type::as_function_literal) + // Comparing literals avoids the cross-module AST dependency of `FunctionType::definition`. + .is_some_and(|original| original.literal(db) == function.literal(db)) +} + +/// Recognizes inherited property descriptor methods without replacing subclass overrides. +fn property_wrapper_descriptor<'db>( + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + name: &str, + member: Type<'db>, +) -> Type<'db> { + let wrapper = match name { + "__get__" => WrapperDescriptorKind::PropertyDunderGet, + "__set__" => WrapperDescriptorKind::PropertyDunderSet, + "__delete__" => WrapperDescriptorKind::PropertyDunderDelete, + _ => return member, + }; + if member + .as_function_literal() + .is_some_and(|function| is_property_method(db, env, function)) + { + Type::WrapperDescriptor(wrapper) + } else { + member + } +} + +/// Represents a property with known accessors and the standard descriptor behavior. #[salsa::interned(debug, constructor=new_internal, heap_size=ruff_memory_usage::heap_size)] pub struct PropertyInstanceType<'db> { #[returns(copy)] @@ -1118,7 +1205,7 @@ pub struct PropertyInstanceType<'db> { #[returns(copy)] pub deleter: Option>, #[returns(copy)] - instance_class: KnownClass, + instance_class: PropertyInstanceClass<'db>, } fn walk_property_instance_type<'db, V: visitor::TypeVisitor<'db> + ?Sized>( @@ -1126,6 +1213,9 @@ fn walk_property_instance_type<'db, V: visitor::TypeVisitor<'db> + ?Sized>( property: PropertyInstanceType<'db>, visitor: &V, ) { + if let PropertyInstanceClass::Subclass(class) = property.instance_class(db) { + visitor.visit_type(db, class.into()); + } if let Some(getter) = property.getter(db) { visitor.visit_type(db, getter); } @@ -1147,16 +1237,23 @@ impl<'db> PropertyInstanceType<'db> { setter: Option>, deleter: Option>, ) -> Self { - Self::new_internal(db, getter, setter, deleter, KnownClass::Property) + Self::new_internal(db, getter, setter, deleter, PropertyInstanceClass::Builtin) } - fn new_enum_property( + fn new_with_class( db: &'db dyn Db, + class: ClassType<'db>, getter: Option>, setter: Option>, deleter: Option>, ) -> Self { - Self::new_internal(db, getter, setter, deleter, KnownClass::EnumProperty) + Self::new_internal( + db, + getter, + setter, + deleter, + PropertyInstanceClass::from_class(db, class), + ) } fn with_accessors( @@ -1220,7 +1317,13 @@ impl<'db> PropertyInstanceType<'db> { let deleter = self .deleter(db) .map(|ty| ty.apply_type_mapping_impl(db, type_mapping, tcx, visitor)); - self.with_accessors(db, getter, setter, deleter) + let instance_class = match self.instance_class(db) { + PropertyInstanceClass::Subclass(class) => PropertyInstanceClass::Subclass( + class.apply_type_mapping_impl(db, type_mapping, tcx, visitor), + ), + class => class, + }; + Self::new_internal(db, getter, setter, deleter, instance_class) } fn recursive_type_normalized_impl( @@ -1254,7 +1357,19 @@ impl<'db> PropertyInstanceType<'db> { ), None => None, }; - Some(self.with_accessors(db, getter, setter, deleter)) + let instance_class = match self.instance_class(db) { + PropertyInstanceClass::Subclass(class) => PropertyInstanceClass::Subclass( + class.recursive_type_normalized_impl(db, env, div, nested)?, + ), + class => class, + }; + Some(Self::new_internal( + db, + getter, + setter, + deleter, + instance_class, + )) } fn find_legacy_typevars_impl( @@ -1265,6 +1380,9 @@ impl<'db> PropertyInstanceType<'db> { typevars: &mut FxOrderSet>, visitor: &FindLegacyTypeVarsVisitor<'db>, ) { + if let PropertyInstanceClass::Subclass(class) = self.instance_class(db) { + class.find_legacy_typevars_impl(db, env, binding_context, typevars, visitor); + } if let Some(ty) = self.getter(db) { ty.find_legacy_typevars_impl(db, env, binding_context, typevars, visitor); } @@ -3211,7 +3329,11 @@ impl<'db> Type<'db> { .into(), ), - _ => Some(class.class_member(db, env, name, policy)), + _ => Some( + class + .class_member(db, env, name, policy) + .map_type(|member| property_wrapper_descriptor(db, env, name, member)), + ), } } @@ -3225,9 +3347,11 @@ impl<'db> Type<'db> { )) } - Type::GenericAlias(alias) => { - Some(ClassType::from(*alias).class_member(db, env, name, policy)) - } + Type::GenericAlias(alias) => Some( + ClassType::from(*alias) + .class_member(db, env, name, policy) + .map_type(|member| property_wrapper_descriptor(db, env, name, member)), + ), Type::SubclassOf(subclass_of_ty) => { subclass_of_ty.find_name_in_mro_with_policy(db, env, name, policy) @@ -4943,29 +5067,13 @@ impl<'db> Type<'db> { )) .into() } - Type::ClassLiteral(class) - if name == "__get__" && class.is_known(db, KnownClass::Property) => + Type::ClassLiteral(_) | Type::GenericAlias(_) + if matches!(name_str, "__get__" | "__set__" | "__delete__") + && let Some(wrapper @ Type::WrapperDescriptor(_)) = this + .find_name_in_mro_with_policy(db, env, name_str, policy) + .and_then(|member| member.place.ignore_possibly_undefined()) => { - Place::bound(Type::WrapperDescriptor( - WrapperDescriptorKind::PropertyDunderGet, - )) - .into() - } - Type::ClassLiteral(class) - if name == "__set__" && class.is_known(db, KnownClass::Property) => - { - Place::bound(Type::WrapperDescriptor( - WrapperDescriptorKind::PropertyDunderSet, - )) - .into() - } - Type::ClassLiteral(class) - if name == "__delete__" && class.is_known(db, KnownClass::Property) => - { - Place::bound(Type::WrapperDescriptor( - WrapperDescriptorKind::PropertyDunderDelete, - )) - .into() + Place::bound(wrapper).into() } Type::BoundMethod(bound_method) => match name_str { "__self__" => Place::bound(bound_method.self_instance(db)).into(), @@ -9033,13 +9141,16 @@ impl<'db> VarianceInferable<'db> for Type<'db> { Type::EnumComplement(complement) => complement .to_intersection(db, env) .variance_of(db, env, typevar), - Type::PropertyInstance(property_instance_type) => property_instance_type - .getter(db) - .iter() - .chain(&property_instance_type.setter(db)) - .chain(&property_instance_type.deleter(db)) - .map(|ty| ty.variance_of(db, env, typevar)) - .collect(), + Type::PropertyInstance(property_instance_type) => [ + Some(property_instance_type.instance_fallback(db, env)), + property_instance_type.getter(db), + property_instance_type.setter(db), + property_instance_type.deleter(db), + ] + .into_iter() + .flatten() + .map(|ty| ty.variance_of(db, env, typevar)) + .collect(), Type::SubclassOf(subclass_of_type) => subclass_of_type.variance_of(db, env, typevar), Type::TypeIs(type_is_type) => type_is_type.variance_of(db, env, typevar), Type::TypeGuard(type_guard_type) => type_guard_type.variance_of(db, env, typevar), diff --git a/crates/ty_python_semantic/src/types/bool.rs b/crates/ty_python_semantic/src/types/bool.rs index 557fb9f91c..7d537e571d 100644 --- a/crates/ty_python_semantic/src/types/bool.rs +++ b/crates/ty_python_semantic/src/types/bool.rs @@ -5,9 +5,9 @@ use ruff_text_size::{Ranged, TextRange}; use crate::types::{ CallArguments, CallDunderError, ClassType, CycleDetector, KnownClass, KnownInstanceType, - LiteralValueTypeKind, SubclassOfInner, Type, TypeContext, TypeVarBoundOrConstraints, UnionType, - call::CallErrorKind, constraints::ConstraintSetBuilder, context::InferContext, - diagnostic::UNSUPPORTED_BOOL_CONVERSION, typed_dict::TypedDictField, + LiteralValueTypeKind, PropertyInstanceClass, SubclassOfInner, Type, TypeContext, + TypeVarBoundOrConstraints, UnionType, call::CallErrorKind, constraints::ConstraintSetBuilder, + context::InferContext, diagnostic::UNSUPPORTED_BOOL_CONVERSION, typed_dict::TypedDictField, }; use ty_python_core::Truthiness; @@ -256,6 +256,17 @@ impl<'db> Type<'db> { Truthiness::from(*is_non_empty) } + Type::PropertyInstance(property) + if let PropertyInstanceClass::Subclass(class) = property.instance_class(db) => + { + Type::instance(db, env, class).try_bool_impl( + db, + env, + allow_short_circuit, + visitor, + )? + } + Type::FunctionLiteral(_) | Type::BoundMethod(_) | Type::WrapperDescriptor(_) diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index 2ebe9c7bb0..b0aae99f51 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -9,7 +9,7 @@ //! `ty_python_semantic::types::call::bind`. mod constructor; -mod enum_property; +mod property; use std::borrow::Cow; use std::cell::{Cell, RefCell}; @@ -72,7 +72,7 @@ use crate::types::{ InternedConstraintSet, IntersectionType, KnownBoundMethodType, KnownClass, KnownInstanceType, LiteralValueTypeKind, NominalInstanceType, PropertyInstanceType, SpecialFormType, TypeContext, TypeMapping, TypeVarBoundOrConstraints, TypeVarVariance, UnionAccumulator, UnionBuilder, - UnionType, WrapperDescriptorKind, enums, list_members, + UnionType, WrapperDescriptorKind, enums, is_property_method, list_members, }; use crate::{DisplaySettings, FxOrderSet}; use ruff_db::diagnostic::{Annotation, Diagnostic, Span, SubDiagnostic, SubDiagnosticSeverity}; @@ -1724,6 +1724,13 @@ impl<'db> Bindings<'db> { overload.set_return_type(Type::Never); } } + [ + Some(property @ Type::NominalInstance(_)), + Some(instance), + .., + ] if instance.is_none(db) => { + overload.set_return_type(*property); + } _ => {} } } @@ -1885,7 +1892,8 @@ impl<'db> Bindings<'db> { Type::BoundMethod(bound_method) if let Type::PropertyInstance(property) = - bound_method.self_instance(db) => + bound_method.self_instance(db) + && is_property_method(db, env, bound_method.function(db)) => { match bound_method.function(db).name(db).as_str() { "setter" => { @@ -3197,7 +3205,7 @@ impl<'db> Bindings<'db> { } } - self.evaluate_enum_property_calls(db, call_arguments); + self.evaluate_property_calls(db, env, call_arguments); } } diff --git a/crates/ty_python_semantic/src/types/call/bind/enum_property.rs b/crates/ty_python_semantic/src/types/call/bind/enum_property.rs deleted file mode 100644 index b897cb7205..0000000000 --- a/crates/ty_python_semantic/src/types/call/bind/enum_property.rs +++ /dev/null @@ -1,60 +0,0 @@ -use super::Bindings; -use crate::db::Db; -use crate::types::call::CallArguments; -use crate::types::{KnownClass, PropertyInstanceType, Type}; -use itertools::Itertools; - -impl<'db> Bindings<'db> { - /// Replaces constructed `enum.property` instances with the property type derived from their - /// accessor arguments. - pub(super) fn evaluate_enum_property_calls( - &mut self, - db: &'db dyn Db, - call_arguments: &CallArguments<'_, 'db>, - ) { - let property_instance = - |getter: Option>, setter: Option>, deleter: Option>| { - Type::PropertyInstance(PropertyInstanceType::new_enum_property( - db, - getter.filter(|ty| !ty.is_none(db)), - setter.filter(|ty| !ty.is_none(db)), - deleter.filter(|ty| !ty.is_none(db)), - )) - }; - - // TODO: Preserve subclasses of `enum.property`. `PropertyInstanceType` currently records - // only a known property class, so this rewrite collapses subclass instances to - // `enum.property`. - for constructor in self.iter_constructor_items_mut() { - if !constructor - .constructed_instance_type() - .is_instance_of(db, KnownClass::EnumProperty) - { - continue; - } - - let property = { - let Ok((_, overload)) = constructor.callable().matching_overloads().exactly_one() - else { - continue; - }; - let accessor = |parameter_index| { - call_arguments - .iter() - .zip(overload.argument_matches()) - .find_map(|((_, argument_types), argument_matches)| { - let parameter = argument_matches - .parameters - .iter() - .find(|parameter| parameter.index == parameter_index)?; - parameter - .argument_type - .or_else(|| argument_types.get_default()) - }) - }; - property_instance(accessor(0), accessor(1), accessor(2)) - }; - constructor.set_constructed_instance_type(property); - } - } -} diff --git a/crates/ty_python_semantic/src/types/call/bind/property.rs b/crates/ty_python_semantic/src/types/call/bind/property.rs new file mode 100644 index 0000000000..c0ad4ac5c2 --- /dev/null +++ b/crates/ty_python_semantic/src/types/call/bind/property.rs @@ -0,0 +1,103 @@ +use super::{Bindings, ConstructorCallableKind}; +use crate::db::Db; +use crate::types::call::CallArguments; +use crate::types::{ + ClassBase, KnownClass, MemberLookupPolicy, ProgramEnvironment, PropertyInstanceType, Type, + is_property_method, +}; +use itertools::Itertools; + +impl<'db> Bindings<'db> { + /// Retains the accessors and nominal class when the property initializer is inherited. + pub(super) fn evaluate_property_calls( + &mut self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + call_arguments: &CallArguments<'_, 'db>, + ) { + for constructor in self.iter_constructor_items_mut() { + if constructor.context().kind() != ConstructorCallableKind::Init { + continue; + } + let function = match constructor.callable().callable_type { + Type::BoundMethod(method) => method.function(db), + Type::FunctionLiteral(function) => function, + _ => continue, + }; + if function.name(db) != "__init__" || !is_property_method(db, env, function) { + continue; + } + let Some(instance) = constructor + .constructed_instance_type() + .as_nominal_instance() + else { + continue; + }; + let class = instance.class(db, env); + // The first class that defines the accessor storage must be a known property class. + let inherits_accessors = class + .iter_mro(db) + .filter_map(ClassBase::into_class) + .find_map(|base| { + if matches!( + base.known(db), + Some(KnownClass::Property | KnownClass::EnumProperty) + ) { + Some(true) + } else if ["fget", "fset", "fdel"] + .into_iter() + .any(|name| !base.own_class_member(db, env, None, name).is_undefined()) + { + Some(false) + } else { + None + } + }); + if inherits_accessors != Some(true) { + continue; + } + // Property-specific protocol and override checks use the stored accessors directly. + // A subclass that changes descriptor behavior must instead use ordinary descriptors. + if ["__get__", "__set__", "__delete__"] + .into_iter() + .any(|name| { + class + .class_member(db, env, name, MemberLookupPolicy::default()) + .place + .ignore_possibly_undefined() + .and_then(Type::as_function_literal) + .is_none_or(|function| !is_property_method(db, env, function)) + }) + { + continue; + } + let Ok((_, overload)) = constructor.callable().matching_overloads().exactly_one() + else { + continue; + }; + let accessor = |parameter_index| { + call_arguments + .iter() + .zip(overload.argument_matches()) + .find_map(|((_, argument_types), argument_matches)| { + let parameter = argument_matches + .parameters + .iter() + .find(|parameter| parameter.index == parameter_index)?; + parameter + .argument_type + .or_else(|| argument_types.get_default()) + }) + .filter(|ty| !ty.is_none(db)) + }; + let property = Type::PropertyInstance(PropertyInstanceType::new_with_class( + db, + class, + accessor(0), + accessor(1), + accessor(2), + )); + constructor.set_constructed_instance_type(property); + } + } +} diff --git a/crates/ty_python_semantic/src/types/display.rs b/crates/ty_python_semantic/src/types/display.rs index a6f006b42e..9e7592265f 100644 --- a/crates/ty_python_semantic/src/types/display.rs +++ b/crates/ty_python_semantic/src/types/display.rs @@ -33,10 +33,10 @@ use crate::types::typevar::BoundTypeVarIdentity; use crate::types::visitor::TypeVisitor; use crate::types::{ CallableType, IntersectionType, KnownBoundMethodType, KnownClass, KnownInstanceType, - KnownUnion, LiteralValueType, LiteralValueTypeKind, MaterializationKind, PropertyInstanceType, - Protocol, SpecialFormType, StringLiteralType, SubclassOfInner, SubclassOfType, Type, - TypeAliasType, TypeGuardLike, TypedDictType, TypingModule, UnionType, WrapperDescriptorKind, - visitor, + KnownUnion, LiteralValueType, LiteralValueTypeKind, MaterializationKind, PropertyInstanceClass, + PropertyInstanceType, Protocol, SpecialFormType, StringLiteralType, SubclassOfInner, + SubclassOfType, Type, TypeAliasType, TypeGuardLike, TypedDictType, TypingModule, UnionType, + WrapperDescriptorKind, visitor, }; use ty_python_core::ProgramFile; use ty_python_core::definition::Definition; @@ -1017,11 +1017,11 @@ struct DisplayRepresentation<'env, 'db> { settings: DisplaySettings<'db>, } -fn property_display_name(db: &dyn Db, property: PropertyInstanceType<'_>) -> &'static str { - if property.instance_class(db) == KnownClass::EnumProperty { - "enum.property" - } else { - "property" +fn property_display_name<'db>(db: &'db dyn Db, property: PropertyInstanceType<'db>) -> &'db str { + match property.instance_class(db) { + PropertyInstanceClass::Builtin => "property", + PropertyInstanceClass::Enum => "enum.property", + PropertyInstanceClass::Subclass(class) => class.name(db), } } @@ -1129,6 +1129,13 @@ impl<'db> FmtDetailed<'db> for DisplayRepresentation<'_, 'db> { f.write_char('>') } }, + Type::PropertyInstance(property) + if let PropertyInstanceClass::Subclass(class) = property.instance_class(db) => + { + Type::instance(db, self.env, class) + .display_with(db, self.env, self.settings.clone()) + .fmt_detailed(f) + } Type::PropertyInstance(property) => f .with_type(self.ty) .write_str(property_display_name(db, property)), @@ -1279,23 +1286,23 @@ impl<'db> FmtDetailed<'db> for DisplayRepresentation<'_, 'db> { } Type::KnownBoundMethod(method_type) => { f.set_invalid_type_annotation(); - let (cls, member_name, cls_name, ty, ty_name) = match method_type { + let (class_ty, member_name, cls_name, ty, ty_name) = match method_type { KnownBoundMethodType::FunctionTypeDunderGet(function) => ( - KnownClass::FunctionType, + KnownClass::FunctionType.to_class_literal(db, self.env), "__get__", "function", Type::FunctionLiteral(function), Some(&**function.name(db)), ), KnownBoundMethodType::FunctionTypeDunderCall(function) => ( - KnownClass::FunctionType, + KnownClass::FunctionType.to_class_literal(db, self.env), "__call__", "function", Type::FunctionLiteral(function), Some(&**function.name(db)), ), KnownBoundMethodType::PropertyDunderGet(property) => ( - property.instance_class(db), + property.instance_class(db).to_class_literal(db, self.env), "__get__", property_display_name(db, property), Type::PropertyInstance(property), @@ -1305,7 +1312,7 @@ impl<'db> FmtDetailed<'db> for DisplayRepresentation<'_, 'db> { .map(|getter| &**getter.name(db)), ), KnownBoundMethodType::PropertyDunderSet(property) => ( - property.instance_class(db), + property.instance_class(db).to_class_literal(db, self.env), "__set__", property_display_name(db, property), Type::PropertyInstance(property), @@ -1315,7 +1322,7 @@ impl<'db> FmtDetailed<'db> for DisplayRepresentation<'_, 'db> { .map(|setter| &**setter.name(db)), ), KnownBoundMethodType::PropertyDunderDelete(property) => ( - property.instance_class(db), + property.instance_class(db).to_class_literal(db, self.env), "__delete__", property_display_name(db, property), Type::PropertyInstance(property), @@ -1325,7 +1332,7 @@ impl<'db> FmtDetailed<'db> for DisplayRepresentation<'_, 'db> { .map(|deleter| &**deleter.name(db)), ), KnownBoundMethodType::StrStartswith(literal) => ( - KnownClass::Property, + KnownClass::Property.to_class_literal(db, self.env), "startswith", "string", Type::LiteralValue(LiteralValueType::promotable( @@ -1374,7 +1381,6 @@ impl<'db> FmtDetailed<'db> for DisplayRepresentation<'_, 'db> { } }; - let class_ty = cls.to_class_literal(db, self.env); f.write_char('<')?; f.with_type(KnownClass::MethodWrapperType.to_class_literal(db, self.env)) .write_str("method-wrapper")?; From 9b9b1dee36d2f52e93c230edef8bccf784842f12 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Tue, 18 Aug 2026 09:53:23 -0700 Subject: [PATCH 086/371] [ty] Implement intersection meta-type projection (#27660) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Implement `to_meta_type()` for intersections, removing a Todo type. This allows removing a special-cased workaround for intersections that was introduced in https://github.com/astral-sh/ruff/pull/27644 - Implement intersection meta-type projection by combining positive class constraints, preserving bounded type variables and `Self`, discarding value-only refinements, and retaining exact enum alternatives. - Pass the complete receiver meta-type to descriptors, removing the intersection-specific owner workaround introduced for enum attribute inference. - Preserve nominal class information when an optional type-variable bound excludes `None`, alongside structural protocol behavior and the runtime `dict` class of narrowed `TypedDict` values. ## Test plan Added and updated mdtests cover: - Nominal class intersections, `type()`/`__class__`, and constructor round trips. - Bounded type variables intersected with unrelated classes, optional bounds narrowed by excluding `None`, and truthiness-narrowed `Self`. - Positive and negative truthiness constraints, excluded literals, pure negations, and intersections without a positive class bound. - Descriptor owners requiring both sides of an intersection. - Narrowed `TypedDict` values retaining `dict` as their runtime class. - Callable qualified-name lookup and named-tuple classmethods preserving their complete intersection owner. ## Ecosystem impact Most new ecosystem diagnostics expose existing constructor, signature, or attribute-checking behavior that was previously hidden by `@Todo`: SciPy includes inherited `object.__init__` errors and duplicate diagnostics; rotki exposes superclass-constructor selection and strict generic narrowing; pandas, Sphinx, and SymPy expose existing constructor or flow-analysis limitations; and Pydantic assigns an undeclared class attribute. All these same underlying limitations reproduce on `main`, if you express the class-type intersection directly instead of relying on `to_meta_type()` to create it. So I consider these out-of-scope for this PR. One known false positive remains in koda-validate: a `NamedTuple`-bounded class passed to `inspect.signature` is rejected. General callable-intersection support is intentionally deferred to keep this change focused. In [Porcupine’s lexer comparison](https://github.com/Akuli/porcupine/blob/8e34077345317427aa4664d6b1ec556c3c6812e7/porcupine/plugins/highlight/pygments_highlighter.py#L51), the more precise class type also exposes an existing method-identity limitation. ty treats `type(self._lexer).get_tokens_unprocessed == RegexLexer.get_tokens_unprocessed` as always false, even though a `PythonLexer` makes it true at runtime. This incorrectly removes an `unsound-return-statement` warning in the reachable branch. The same behavior reproduces on the merge base when the class-object intersection is written explicitly. Handling method identity correctly when subclasses can override a method is tracked in [astral-sh/ty#2428](https://github.com/astral-sh/ty/issues/2428) and is deferred to a separate change. --- .../resources/mdtest/annotations/callable.md | 9 +- .../resources/mdtest/descriptor_protocol.md | 31 ++++ .../resources/mdtest/intersection_types.md | 147 ++++++++++++++++++ .../resources/mdtest/named_tuple.md | 3 +- .../resources/mdtest/typed_dict.md | 25 +++ crates/ty_python_semantic/src/types.rs | 52 +++++-- .../src/types/set_theoretic.rs | 6 +- 7 files changed, 247 insertions(+), 26 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/annotations/callable.md b/crates/ty_python_semantic/resources/mdtest/annotations/callable.md index 25a5f044c8..6eb3680b52 100644 --- a/crates/ty_python_semantic/resources/mdtest/annotations/callable.md +++ b/crates/ty_python_semantic/resources/mdtest/annotations/callable.md @@ -491,9 +491,8 @@ def f_okay(c: Callable[[], None]): if hasattr(c, "__qualname__"): reveal_type(c.__qualname__) # revealed: object - # TODO: should be `property` - # (or complain that we don't know that `type(c)` has the attribute at all!) - reveal_type(type(c).__qualname__) # revealed: @Todo(Intersection meta-type) + # This is the class object's own qualified name, not the instance's descriptor. + reveal_type(type(c).__qualname__) # revealed: str # `hasattr` only guarantees that an attribute is readable. # @@ -504,8 +503,8 @@ def f_okay(c: Callable[[], None]): # into a writable attribute...? What would that look like? Something like this? if ( hasattr(type(c), "__qualname__") - and isinstance(type(c).__qualname__, property) - and type(c).__qualname__.fset is not None + and isinstance(descriptor := type(c).__qualname__, property) + and descriptor.fset is not None ): c.__qualname__ = "my_callable" # error: [invalid-assignment] ``` diff --git a/crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md b/crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md index 82614f9966..d3065d9655 100644 --- a/crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md +++ b/crates/ty_python_semantic/resources/mdtest/descriptor_protocol.md @@ -1358,6 +1358,37 @@ def descriptor_value(descriptor: Descriptor) -> None: C().value ``` +### Intersection receivers preserve their complete owner type + +A descriptor can require its owner to satisfy both classes in an intersection. + +```py +from __future__ import annotations + +class Descriptor: + def __get__(self, instance: object, owner: type[Left] & type[Right]) -> int: + return 1 + +class Left: + value = Descriptor() + +class Right: ... + +def receiver(value: Left & Right) -> None: + # Only `Left` supplies the descriptor, but its owner must retain `Right` too. + # Passing `type[Left]` instead of `type[Left] & type[Right]` would cause an + # `invalid-attribute-access` error. + reveal_type(value.value) # revealed: int +``` + +A receiver known only to be `Left` does not satisfy the descriptor's owner type. + +```py +def incomplete_owner(value: Left) -> None: + # error: [invalid-attribute-access] "Expected `type[Left] & type[Right]`, found `type[Left]`" + value.value +``` + ### Every `__get__` definition must accept the call A conditionally defined method can have several callable signatures. The access is invalid if any diff --git a/crates/ty_python_semantic/resources/mdtest/intersection_types.md b/crates/ty_python_semantic/resources/mdtest/intersection_types.md index 7c2a89df85..d29334f8af 100644 --- a/crates/ty_python_semantic/resources/mdtest/intersection_types.md +++ b/crates/ty_python_semantic/resources/mdtest/intersection_types.md @@ -1401,6 +1401,153 @@ def f(c: C): reveal_type(c.x) # revealed: ~AlwaysFalsy ``` +## Meta-types of intersections + +### Positive class constraints + +The class of an intersection must satisfy the class constraints supplied by every positive element. +Instantiating the resulting class intersection recovers the corresponding instance intersection. + +```py +class Left: ... +class Right: ... + +def positive(value: Left & Right) -> None: + reveal_type(value.__class__) # revealed: type[Left] & type[Right] + reveal_type(type(value)) # revealed: type[Left] & type[Right] + reveal_type(type(value)()) # revealed: Left & Right +``` + +### Bounded type variables + +Projecting an intersection into its class type preserves a bounded type variable instead of +replacing it with its upper bound. An unrelated positive class constraint is preserved too. + +```py +class Bound: ... +class Other: ... + +def preserve[T: Bound](value: T & Other) -> None: + reveal_type(type(value)) # revealed: type[T@preserve] & type[Other] + reveal_type(type(value)()) # revealed: T@preserve & Other +``` + +### Excluded alternatives in type-variable bounds + +Excluding an alternative from a type variable's union bound can reveal a definite class. Preserve +both that class constraint and the original type variable in the resulting class type. + +```py +class Bound: + label = "bound" + +def exclude_none[T: Bound | None](value: T) -> None: + if value is not None: + reveal_type(type(value)) # revealed: type[T@exclude_none] & type[Bound] + reveal_type(type(value).label) # revealed: str +``` + +### Excluded alternatives in class-object bounds + +If the remaining bound is a class object, its class is its metaclass. Preserve that metaclass +constraint alongside the original type variable. + +```py +class Meta(type): ... +class Bound(metaclass=Meta): ... + +def accepts_meta(value: type[Meta]) -> None: ... +def exclude_none[T: type[Bound] | None](value: T) -> None: + if value is not None: + reveal_type(type(value)) # revealed: type[T@exclude_none] & type[Meta] + accepts_meta(type(value)) +``` + +For a final class, the metaclass is known exactly. This also holds for a specialized generic class. + +```py +from typing import final + +@final +class FinalBound(metaclass=Meta): ... + +@final +class FinalGenericBound[U](metaclass=Meta): ... + +def exclude_none_final[T: type[FinalBound] | None](value: T) -> None: + if value is not None: + reveal_type(type(value)) # revealed: type[T@exclude_none_final] & + accepts_meta(type(value)) + +def exclude_none_generic[T: type[FinalGenericBound[int]] | None](value: T) -> None: + if value is not None: + reveal_type(type(value)) # revealed: type[T@exclude_none_generic] & + accepts_meta(type(value)) +``` + +### Truthiness refinements + +Whether an individual object is truthy or falsy does not constrain its runtime class. Both positive +and negative truthiness refinements must therefore disappear from its meta-type. + +```py +from ty_extensions import AlwaysFalsy + +class Base: ... + +def truthiness(falsy: Base & AlwaysFalsy, not_falsy: Base & ~AlwaysFalsy) -> None: + reveal_type(type(falsy)) # revealed: type[Base] + reveal_type(type(not_falsy)) # revealed: type[Base] +``` + +### Truthiness-narrowed `Self` + +Truthiness describes an individual instance, not its class. Narrowing `Self` by truthiness must +therefore preserve `type[Self]` while discarding the value-only refinement. + +```py +from typing import Self + +class Base: + def __bool__(self) -> bool: + return True + + def clone(self: Self) -> Self: + if not self: + return self + + reveal_type(self) # revealed: Self@clone & ~AlwaysFalsy + reveal_type(type(self)) # revealed: type[Self@clone] + return type(self)() +``` + +### Negative value constraints + +Excluding particular instance values does not exclude their classes: a nonzero integer can still +have class `int`. + +```py +from typing import Literal + +def nonzero(value: int & ~Literal[0]) -> None: + reveal_type(type(value)) # revealed: type[int] +``` + +### Intersections without a positive class constraint + +A pure negation supplies no positive class bound, and a truthiness constraint describes only an +instance value. Both conservatively project to the unconstrained class type. + +```py +from ty_extensions import AlwaysTruthy + +class Excluded: ... + +def unconstrained(negative: ~Excluded, truthy: AlwaysTruthy & ~Excluded) -> None: + reveal_type(type(negative)) # revealed: type + reveal_type(type(truthy)) # revealed: type +``` + ## Methods on intersections ### The same method from a common base diff --git a/crates/ty_python_semantic/resources/mdtest/named_tuple.md b/crates/ty_python_semantic/resources/mdtest/named_tuple.md index ccf335b9a6..a2c1bed9db 100644 --- a/crates/ty_python_semantic/resources/mdtest/named_tuple.md +++ b/crates/ty_python_semantic/resources/mdtest/named_tuple.md @@ -1475,7 +1475,8 @@ satisfy: ```py def expects_named_tuple(x: typing.NamedTuple): reveal_type(x) # revealed: tuple[object, ...] & NamedTupleLike - reveal_type(x._make) # revealed: bound method type[NamedTupleLike]._make(iterable: Iterable[Any]) -> NamedTupleLike + # revealed: bound method (type[tuple[object, ...]] & type[NamedTupleLike])._make(iterable: Iterable[Any]) -> tuple[object, ...] & NamedTupleLike + reveal_type(x._make) # revealed: bound method (tuple[object, ...] & NamedTupleLike)._replace(...) -> tuple[object, ...] & NamedTupleLike reveal_type(x._replace) # revealed: Overload[(value: tuple[object, ...], /) -> tuple[object, ...], [_T](value: tuple[_T, ...], /) -> tuple[object, ...]] diff --git a/crates/ty_python_semantic/resources/mdtest/typed_dict.md b/crates/ty_python_semantic/resources/mdtest/typed_dict.md index 582341ff04..cbf9f0250d 100644 --- a/crates/ty_python_semantic/resources/mdtest/typed_dict.md +++ b/crates/ty_python_semantic/resources/mdtest/typed_dict.md @@ -2816,6 +2816,31 @@ def _(p: Person) -> None: reveal_type(p.__class__) # revealed: ``` +Truthiness narrowing can give a `TypedDict` value an intersection type, but its runtime class is +still `dict`. + +```py +class OptionalPerson(TypedDict, total=False): + name: str + +def narrowed_class(person: OptionalPerson) -> None: + if person: + reveal_type(type(person)) # revealed: + reveal_type(person.__class__) # revealed: +``` + +Excluding `None` from a type variable's `TypedDict` bound should also identify `dict` as the runtime +class. This is difficult to represent while preserving the type variable: `type[Person]` describes +the `TypedDict` schema constructor, not the runtime `dict` class. + +```py +def exclude_none[T: Person | None](value: T) -> None: + if value is not None: + # TODO: Preserve the runtime class. Intersecting `type[T]` with the exact `dict` + # class is not sufficient: specializing `T` to `Person` makes that intersection `Never`. + reveal_type(type(value)) # revealed: type[T@exclude_none] +``` + Passing a `TypedDict` to `dict()` copies it into a regular dictionary: ```py diff --git a/crates/ty_python_semantic/src/types.rs b/crates/ty_python_semantic/src/types.rs index d622acb27e..5104d25e76 100644 --- a/crates/ty_python_semantic/src/types.rs +++ b/crates/ty_python_semantic/src/types.rs @@ -4542,20 +4542,8 @@ impl<'db> Type<'db> { policy: InstanceFallbackShadowsNonDataDescriptor, ) -> MemberLookupResult<'db> { let meta_attr_plain = Self::instance_lookup_class_member_with_policy(db, env, key); - // A TypeVar retains its class identity when lookup is delegated to its bound, including - // after narrowing. Narrowing can also add an unrelated class to a mixin's `Self`, in which - // case the TypeVar alone is not a valid owner for descriptors from that class. - let owner = match receiver { - Type::TypeVar(_) => receiver, - Type::Intersection(intersection) => intersection - .positive(db) - .iter() - .copied() - .find(|element| element.is_type_var() && element.is_subtype_of(db, env, key.ty(db))) - .unwrap_or(key.ty(db)), - _ => key.ty(db), - } - .to_meta_type(db, env); + // Preserve the receiver's type variables and all its narrowed class constraints. + let owner = receiver.to_meta_type(db, env); let ( PlaceAndQualifiers { place: meta_attr, @@ -7642,13 +7630,43 @@ impl<'db> Type<'db> { SubclassOfType::from(db, env, SubclassOfInner::Dynamic(dynamic)) } Type::Divergent(_) => self, - // TODO intersections Type::Intersection(intersection) => { if let Some(alternatives) = intersection.finite_alternative_union(db, env) { alternatives.to_meta_type(db, env) } else { - SubclassOfType::try_from_type(db, env, todo_type!("Intersection meta-type")) - .expect("Type::Todo should be a valid `SubclassOfInner`") + // Negative constraints do not generally constrain classes: `int & ~Literal[0]` + // still has meta-type `type[int]`. Pure negations are bounded by `object`. + let mut builder = IntersectionBuilder::new(db, env); + for positive in intersection.positive_elements_or_object(db) { + builder.add_positive_in_place(positive.to_meta_type(db, env)); + } + + // An exclusion can narrow a type variable's union bound to a definite class: + // `(T: C | None) & ~None` has meta-type `type[T] & type[C]`. + // If the remaining bound is a class object, retain its metaclass instead. + // Structural bounds need separate runtime-class handling (see `dunder_class`). + if !intersection.negative(db).is_empty() + && intersection + .iter_positive(db) + .any(|positive| matches!(positive, Type::TypeVar(_))) + && let Some(narrowed_bound) = match intersection + .with_expanded_typevars_and_newtypes(db, env) + { + bound @ (Type::NominalInstance(_) + | Type::ClassLiteral(_) + | Type::GenericAlias(_)) => Some(bound), + bound @ Type::SubclassOf(subclass_of) + if let SubclassOfInner::Class(_) = subclass_of.subclass_of() => + { + Some(bound) + } + _ => None, + } + { + builder.add_positive_in_place(narrowed_bound.to_meta_type(db, env)); + } + + builder.build() } } Type::EnumComplement(complement) => complement diff --git a/crates/ty_python_semantic/src/types/set_theoretic.rs b/crates/ty_python_semantic/src/types/set_theoretic.rs index dfed4e5d24..b7f4391d43 100644 --- a/crates/ty_python_semantic/src/types/set_theoretic.rs +++ b/crates/ty_python_semantic/src/types/set_theoretic.rs @@ -1038,8 +1038,8 @@ impl<'db> IntersectionType<'db> { builder.build() } - /// Compute the `__class__` type when this intersection contains a positive class-backed - /// protocol constraint. + /// Compute the `__class__` type for class-backed protocols and `TypedDict` instances, + /// whose runtime classes differ from their internal meta-types. /// /// Negative instance constraints are not transferred: an object not satisfying `P` does not /// imply that other instances of its class cannot satisfy `P`. @@ -1052,7 +1052,7 @@ impl<'db> IntersectionType<'db> { matches!( positive, Type::ProtocolInstance(protocol) if protocol.class_origin(db).is_some() - ) + ) || positive.is_typed_dict() }) { return None; } From b458a048858f455a1326bfb251fdd15ffaf5169e Mon Sep 17 00:00:00 2001 From: Brent Westbrook <36778786+ntBre@users.noreply.github.com> Date: Tue, 18 Aug 2026 13:38:30 -0400 Subject: [PATCH 087/371] [`pyflakes`] Emit semantic syntax errors in string type definitions as `F722` (#27835) Summary -- As uncovered in #17804, `F722` previously applied only to parse errors and not to semantic syntax errors. More confusingly, we were emitting semantic syntax errors, and even other lint rule errors that correspond to syntax errors, on string type definitions, even when `F722` was disabled. This PR adds a check to `Checker::report_semantic_error` for the semantic model's `in_string_type_definition` flag that converts these semantic errors into regular `F722` diagnostics, when the rule is enabled, and returns without reporting an error when the rule is disabled. Test Plan -- New mdtests covering semantic errors in forward annotations --- .../forward-annotation-syntax-error.md | 51 +++++++++++++++++++ crates/ruff_linter/src/checkers/ast/mod.rs | 13 +++++ 2 files changed, 64 insertions(+) create mode 100644 crates/ruff_linter/resources/mdtest/pyflakes/forward-annotation-syntax-error.md diff --git a/crates/ruff_linter/resources/mdtest/pyflakes/forward-annotation-syntax-error.md b/crates/ruff_linter/resources/mdtest/pyflakes/forward-annotation-syntax-error.md new file mode 100644 index 0000000000..7c325f1477 --- /dev/null +++ b/crates/ruff_linter/resources/mdtest/pyflakes/forward-annotation-syntax-error.md @@ -0,0 +1,51 @@ +# `forward-annotation-syntax-error` (`F722`) + +```toml +target-version = "py312" + +[lint] +select = ["F722"] +``` + +## Parse errors + +Quoted annotations must parse as Python expressions. + +```py +# error: [forward-annotation-syntax-error] "Expected an expression" +invalid: "/" +``` + +## Semantic syntax errors + +An expression can parse successfully but still contain a semantic syntax error. + +```py +# error: [forward-annotation-syntax-error] "Duplicate parameter" +invalid: "(lambda x, x: 0)" +``` + +## Semantic syntax errors currently mapped to disabled lint rules + +`F722` reports semantic syntax errors even when their overlapping lint rules are disabled, in this +case `yield-outside-function` (`F704`). + +```py +# error: [forward-annotation-syntax-error] "`yield` statement outside of a function" +invalid: "(yield 1)" +``` + +## Semantic syntax errors currently mapped to enabled lint rules + +Disabling `F722` suppresses the semantic syntax error even when `F704` remains enabled. + +```toml +target-version = "py312" + +[lint] +select = ["F704"] +``` + +```py +invalid: "(yield 1)" +``` diff --git a/crates/ruff_linter/src/checkers/ast/mod.rs b/crates/ruff_linter/src/checkers/ast/mod.rs index abb8ffd40a..822474365a 100644 --- a/crates/ruff_linter/src/checkers/ast/mod.rs +++ b/crates/ruff_linter/src/checkers/ast/mod.rs @@ -708,6 +708,19 @@ impl SemanticSyntaxContext for Checker<'_> { } fn report_semantic_error(&self, error: SemanticSyntaxError) { + // F722 + if self.semantic.in_string_type_definition() { + if self.is_rule_enabled(Rule::ForwardAnnotationSyntaxError) { + self.report_type_diagnostic( + pyflakes::rules::ForwardAnnotationSyntaxError { + parse_error: error.to_string(), + }, + error.range, + ); + } + return; + } + match error.kind { SemanticSyntaxErrorKind::LateFutureImport => { // F404 From e96caa26b709a3bff8eea43e9d47b4335f1bfddc Mon Sep 17 00:00:00 2001 From: Eduardo Rittner Coelho <116819854+eduardorittner@users.noreply.github.com> Date: Tue, 18 Aug 2026 14:39:19 -0300 Subject: [PATCH 088/371] [syntax-errors] Detect duplicate keyword arguments (#17804) ## Summary Part of https://github.com/astral-sh/ruff/issues/17412 Detects duplicate keyword arguments in function call. ## Test Plan Added inline tests to validate intended behavior --------- Co-authored-by: Brent Westbrook --- .../pylint/repeated_keyword_argument.py | 4 + crates/ruff_linter/src/checkers/ast/mod.rs | 1 + .../pylint/rules/repeated_keyword_argument.rs | 27 +- ..._PLE1132_repeated_keyword_argument.py.snap | 14 + .../inline/err/duplicate_keyword_args.py | 4 + .../inline/ok/non_duplicate_keyword_args.py | 4 + crates/ruff_python_parser/src/error.rs | 6 - .../src/parser/expression.rs | 21 +- .../ruff_python_parser/src/semantic_errors.rs | 66 +++- ...alid_syntax@duplicate_keyword_args.py.snap | 333 ++++++++++++++++++ ...ments__duplicate_keyword_arguments.py.snap | 6 +- ..._syntax@non_duplicate_keyword_args.py.snap | 298 ++++++++++++++++ .../diagnostics/semantic_syntax_errors.md | 27 ++ .../ty_python_semantic/src/types/call/bind.rs | 23 +- 14 files changed, 784 insertions(+), 50 deletions(-) create mode 100644 crates/ruff_python_parser/resources/inline/err/duplicate_keyword_args.py create mode 100644 crates/ruff_python_parser/resources/inline/ok/non_duplicate_keyword_args.py create mode 100644 crates/ruff_python_parser/tests/snapshots/invalid_syntax@duplicate_keyword_args.py.snap create mode 100644 crates/ruff_python_parser/tests/snapshots/valid_syntax@non_duplicate_keyword_args.py.snap diff --git a/crates/ruff_linter/resources/test/fixtures/pylint/repeated_keyword_argument.py b/crates/ruff_linter/resources/test/fixtures/pylint/repeated_keyword_argument.py index b7bb0d7e54..9c97918bfb 100644 --- a/crates/ruff_linter/resources/test/fixtures/pylint/repeated_keyword_argument.py +++ b/crates/ruff_linter/resources/test/fixtures/pylint/repeated_keyword_argument.py @@ -18,3 +18,7 @@ def func(a=10, b=20, c=30): func(a=11, b=21, c=31, **{"b": 22, "c": 41, "a": 51}) func(a=11, b=21, **{"c": 31}, **{"c": 32}) func(a=11, b=21, **{"c": 31, "c": 32}) +func(**{"a": 11}, a=21) + +# Duplicate explicit keywords are syntax errors, not PLE1132 diagnostics. +func(a=11, a=21) diff --git a/crates/ruff_linter/src/checkers/ast/mod.rs b/crates/ruff_linter/src/checkers/ast/mod.rs index 822474365a..7b77bde0cb 100644 --- a/crates/ruff_linter/src/checkers/ast/mod.rs +++ b/crates/ruff_linter/src/checkers/ast/mod.rs @@ -829,6 +829,7 @@ impl SemanticSyntaxContext for Checker<'_> { | SemanticSyntaxErrorKind::InvalidStarExpression | SemanticSyntaxErrorKind::AsyncComprehensionInSyncComprehension(_) | SemanticSyntaxErrorKind::DuplicateParameter(_) + | SemanticSyntaxErrorKind::DuplicateKeywordArgument(_) | SemanticSyntaxErrorKind::NonlocalDeclarationAtModuleLevel | SemanticSyntaxErrorKind::LoadBeforeNonlocalDeclaration { .. } | SemanticSyntaxErrorKind::NonlocalAndGlobal(_) diff --git a/crates/ruff_linter/src/rules/pylint/rules/repeated_keyword_argument.rs b/crates/ruff_linter/src/rules/pylint/rules/repeated_keyword_argument.rs index ab2db3f95d..7c18f8c28f 100644 --- a/crates/ruff_linter/src/rules/pylint/rules/repeated_keyword_argument.rs +++ b/crates/ruff_linter/src/rules/pylint/rules/repeated_keyword_argument.rs @@ -40,21 +40,26 @@ impl Violation for RepeatedKeywordArgument { pub(crate) fn repeated_keyword_argument(checker: &Checker, call: &ExprCall) { let ExprCall { arguments, .. } = call; + // Avoid allocating if there's only one non-unpacked keyword argument, or the unpacked value is + // not a dict literal. + if let [keyword] = &*arguments.keywords { + if keyword.arg.is_some() || !keyword.value.is_dict_expr() { + return; + } + } + let mut seen = FxHashSet::with_capacity_and_hasher(arguments.keywords.len(), FxBuildHasher); for keyword in &*arguments.keywords { if let Some(id) = &keyword.arg { - // Ex) `func(a=1, a=2)` - if !seen.insert(id.as_str()) { - checker.report_diagnostic( - RepeatedKeywordArgument { - duplicate_keyword: id.to_string(), - }, - keyword.range(), - ); - } - } else if let Expr::Dict(dict) = &keyword.value { - // Ex) `func(**{"a": 1, "a": 2})` + seen.insert(id.as_str()); + } + } + + for keyword in &*arguments.keywords { + if keyword.arg.is_none() + && let Expr::Dict(dict) = &keyword.value + { for key in dict.iter_keys().flatten() { if let Expr::StringLiteral(ExprStringLiteral { value, .. }) = key { if !seen.insert(value.to_str()) { diff --git a/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLE1132_repeated_keyword_argument.py.snap b/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLE1132_repeated_keyword_argument.py.snap index 0c08b9d9c2..d7ef675f09 100644 --- a/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLE1132_repeated_keyword_argument.py.snap +++ b/crates/ruff_linter/src/rules/pylint/snapshots/ruff_linter__rules__pylint__tests__PLE1132_repeated_keyword_argument.py.snap @@ -74,6 +74,7 @@ PLE1132 Repeated keyword argument: `c` 19 | func(a=11, b=21, **{"c": 31}, **{"c": 32}) | ^^^ 20 | func(a=11, b=21, **{"c": 31, "c": 32}) +21 | func(**{"a": 11}, a=21) | PLE1132 Repeated keyword argument: `c` @@ -83,3 +84,16 @@ PLE1132 Repeated keyword argument: `c` 19 | func(a=11, b=21, **{"c": 31}, **{"c": 32}) 20 | func(a=11, b=21, **{"c": 31, "c": 32}) | ^^^ +21 | func(**{"a": 11}, a=21) + | + +PLE1132 Repeated keyword argument: `a` + --> repeated_keyword_argument.py:21:9 + | +19 | func(a=11, b=21, **{"c": 31}, **{"c": 32}) +20 | func(a=11, b=21, **{"c": 31, "c": 32}) +21 | func(**{"a": 11}, a=21) + | ^^^ +22 | +23 | # Duplicate explicit keywords are syntax errors, not PLE1132 diagnostics. + | diff --git a/crates/ruff_python_parser/resources/inline/err/duplicate_keyword_args.py b/crates/ruff_python_parser/resources/inline/err/duplicate_keyword_args.py new file mode 100644 index 0000000000..f599841eb4 --- /dev/null +++ b/crates/ruff_python_parser/resources/inline/err/duplicate_keyword_args.py @@ -0,0 +1,4 @@ +def foo(x): ... +foo(x=1, x=2) +def baz(x, y, z): ... +baz(x, y=1, z=3, y=4) diff --git a/crates/ruff_python_parser/resources/inline/ok/non_duplicate_keyword_args.py b/crates/ruff_python_parser/resources/inline/ok/non_duplicate_keyword_args.py new file mode 100644 index 0000000000..d42953792a --- /dev/null +++ b/crates/ruff_python_parser/resources/inline/ok/non_duplicate_keyword_args.py @@ -0,0 +1,4 @@ +def foo(x): ... +foo(x=1) +def bar(x, y, z): ... +foo(x="a", y=1, z=True) diff --git a/crates/ruff_python_parser/src/error.rs b/crates/ruff_python_parser/src/error.rs index 13e4b5a628..09036f4361 100644 --- a/crates/ruff_python_parser/src/error.rs +++ b/crates/ruff_python_parser/src/error.rs @@ -148,9 +148,6 @@ pub enum ParseErrorType { /// A default value was found for a `*` or `**` parameter. VarParameterWithDefault, - /// A keyword argument was repeated. - DuplicateKeywordArgumentError(String), - /// An invalid expression was found in the assignment target. InvalidAssignmentTarget, /// An invalid expression was found in the named assignment target. @@ -321,9 +318,6 @@ impl std::fmt::Display for ParseErrorType { f.write_str("Invalid augmented assignment target") } ParseErrorType::InvalidDeleteTarget => f.write_str("Invalid delete target"), - ParseErrorType::DuplicateKeywordArgumentError(arg_name) => { - write!(f, "Duplicate keyword argument {arg_name:?}") - } ParseErrorType::UnexpectedIpythonEscapeCommand => { f.write_str("IPython escape commands are only allowed in `Mode::Ipython`") } diff --git a/crates/ruff_python_parser/src/parser/expression.rs b/crates/ruff_python_parser/src/parser/expression.rs index c74d55e782..2b59e1c3b8 100644 --- a/crates/ruff_python_parser/src/parser/expression.rs +++ b/crates/ruff_python_parser/src/parser/expression.rs @@ -1,7 +1,6 @@ use std::ops::Deref; use bitflags::bitflags; -use rustc_hash::{FxBuildHasher, FxHashSet}; use thin_vec::ThinVec; use ruff_python_ast::name::Name; @@ -2974,31 +2973,13 @@ impl<'src> Parser<'src> { } /// Performs the following validations on the arguments: - /// 1. There aren't any duplicate keyword argument - /// 2. Generator expressions are parenthesized when required by the argument context. + /// - Generator expressions are parenthesized when required by the argument context. fn validate_arguments( &mut self, arguments: &ast::Arguments, has_trailing_comma: bool, context: ArgumentsContext, ) { - let mut all_arg_names = - FxHashSet::with_capacity_and_hasher(arguments.keywords.len(), FxBuildHasher); - - for (name, range) in arguments - .keywords - .iter() - .filter_map(|argument| argument.arg.as_ref().map(|arg| (arg, argument.range))) - { - let arg_name = name.as_str(); - if !all_arg_names.insert(arg_name) { - self.add_error( - ParseErrorType::DuplicateKeywordArgumentError(arg_name.to_string()), - range, - ); - } - } - let generator_must_be_parenthesized = match context { ArgumentsContext::Call => has_trailing_comma || arguments.len() > 1, // CPython rejects an unparenthesized generator expression as a class base even though diff --git a/crates/ruff_python_parser/src/semantic_errors.rs b/crates/ruff_python_parser/src/semantic_errors.rs index 4d41444c82..a8f1a8ddf9 100644 --- a/crates/ruff_python_parser/src/semantic_errors.rs +++ b/crates/ruff_python_parser/src/semantic_errors.rs @@ -227,10 +227,19 @@ impl SemanticSyntaxChecker { } } Stmt::ClassDef(ast::StmtClassDef { - type_params: Some(type_params), + type_params, + arguments, .. - }) - | Stmt::TypeAlias(ast::StmtTypeAlias { + }) => { + if let Some(type_params) = type_params { + Self::duplicate_type_parameter_name(type_params, ctx); + Self::type_parameter_default_order(type_params, ctx); + } + if let Some(arguments) = arguments { + Self::duplicate_keyword_args(arguments, ctx); + } + } + Stmt::TypeAlias(ast::StmtTypeAlias { type_params: Some(type_params), .. }) => { @@ -775,6 +784,40 @@ impl SemanticSyntaxChecker { } } + fn duplicate_keyword_args(args: &ast::Arguments, ctx: &Ctx) { + if args.keywords.len() < 2 { + return; + } + + let mut all_arg_names = + FxHashSet::with_capacity_and_hasher(args.keywords.len(), FxBuildHasher); + + for (ident, range) in args + .keywords + .iter() + .filter_map(|keyword| keyword.arg.as_ref().map(|arg| (arg, keyword.range))) + { + if !all_arg_names.insert(ident.as_str()) { + // test_err duplicate_keyword_args + // def foo(x): ... + // foo(x=1, x=2) + // def baz(x, y, z): ... + // baz(x, y=1, z=3, y=4) + + // test_ok non_duplicate_keyword_args + // def foo(x): ... + // foo(x=1) + // def bar(x, y, z): ... + // foo(x="a", y=1, z=True) + Self::add_error( + ctx, + SemanticSyntaxErrorKind::DuplicateKeywordArgument(ident.to_string()), + range, + ); + } + } + } + fn irrefutable_match_case(stmt: &ast::StmtMatch, ctx: &Ctx) { // test_ok irrefutable_case_pattern_at_end // match x: @@ -1024,6 +1067,9 @@ impl SemanticSyntaxChecker { } Self::duplicate_parameter_name(parameters, ctx); } + Expr::Call(ast::ExprCall { arguments, .. }) => { + Self::duplicate_keyword_args(arguments, ctx); + } _ => {} } } @@ -1391,6 +1437,9 @@ impl Display for SemanticSyntaxError { SemanticSyntaxErrorKind::NonlocalDeclarationAtModuleLevel => { write!(f, "nonlocal declaration not allowed at module level") } + SemanticSyntaxErrorKind::DuplicateKeywordArgument(name) => { + write!(f, "Duplicate keyword argument `{name}`") + } SemanticSyntaxErrorKind::NonlocalAndGlobal(name) => { write!(f, "name `{name}` is nonlocal and global") } @@ -1842,6 +1891,17 @@ pub enum SemanticSyntaxErrorKind { /// ``` DuplicateParameter(String), + /// Represents duplicated keyword arguments in a function call or class definition. + /// + /// ## Examples + /// + /// ```python + /// def f(x): ... + /// f(x=1, x=2) + /// class C(metaclass=type, metaclass=type): ... + /// ``` + DuplicateKeywordArgument(String), + /// Represents a nonlocal declaration at module level NonlocalDeclarationAtModuleLevel, diff --git a/crates/ruff_python_parser/tests/snapshots/invalid_syntax@duplicate_keyword_args.py.snap b/crates/ruff_python_parser/tests/snapshots/invalid_syntax@duplicate_keyword_args.py.snap new file mode 100644 index 0000000000..b43bfde962 --- /dev/null +++ b/crates/ruff_python_parser/tests/snapshots/invalid_syntax@duplicate_keyword_args.py.snap @@ -0,0 +1,333 @@ +--- +source: crates/ruff_python_parser/tests/fixtures.rs +input_file: crates/ruff_python_parser/resources/inline/err/duplicate_keyword_args.py +--- +## AST + +``` +Module( + ModModule { + node_index: NodeIndex(None), + range: 0..74, + body: [ + FunctionDef( + StmtFunctionDef { + node_index: NodeIndex(None), + range: 0..15, + is_async: false, + decorator_list: [], + name: Identifier { + id: Name("foo"), + range: 4..7, + node_index: NodeIndex(None), + }, + type_params: None, + parameters: Parameters { + range: 7..10, + node_index: NodeIndex(None), + posonlyargs: [], + args: [ + ParameterWithDefault { + range: 8..9, + node_index: NodeIndex(None), + parameter: Parameter { + range: 8..9, + node_index: NodeIndex(None), + name: Identifier { + id: Name("x"), + range: 8..9, + node_index: NodeIndex(None), + }, + annotation: None, + }, + default: None, + }, + ], + vararg: None, + kwonlyargs: [], + kwarg: None, + }, + returns: None, + body: [ + Expr( + StmtExpr { + node_index: NodeIndex(None), + range: 12..15, + value: EllipsisLiteral( + ExprEllipsisLiteral { + node_index: NodeIndex(None), + range: 12..15, + }, + ), + }, + ), + ], + }, + ), + Expr( + StmtExpr { + node_index: NodeIndex(None), + range: 16..29, + value: Call( + ExprCall { + node_index: NodeIndex(None), + range: 16..29, + func: Name( + ExprName { + node_index: NodeIndex(None), + range: 16..19, + id: Name("foo"), + ctx: Load, + }, + ), + arguments: Arguments { + range: 19..29, + node_index: NodeIndex(None), + args: [], + keywords: [ + Keyword { + range: 20..23, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("x"), + range: 20..21, + node_index: NodeIndex(None), + }, + ), + value: NumberLiteral( + ExprNumberLiteral { + node_index: NodeIndex(None), + range: 22..23, + value: Int( + 1, + ), + }, + ), + }, + Keyword { + range: 25..28, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("x"), + range: 25..26, + node_index: NodeIndex(None), + }, + ), + value: NumberLiteral( + ExprNumberLiteral { + node_index: NodeIndex(None), + range: 27..28, + value: Int( + 2, + ), + }, + ), + }, + ], + }, + }, + ), + }, + ), + FunctionDef( + StmtFunctionDef { + node_index: NodeIndex(None), + range: 30..51, + is_async: false, + decorator_list: [], + name: Identifier { + id: Name("baz"), + range: 34..37, + node_index: NodeIndex(None), + }, + type_params: None, + parameters: Parameters { + range: 37..46, + node_index: NodeIndex(None), + posonlyargs: [], + args: [ + ParameterWithDefault { + range: 38..39, + node_index: NodeIndex(None), + parameter: Parameter { + range: 38..39, + node_index: NodeIndex(None), + name: Identifier { + id: Name("x"), + range: 38..39, + node_index: NodeIndex(None), + }, + annotation: None, + }, + default: None, + }, + ParameterWithDefault { + range: 41..42, + node_index: NodeIndex(None), + parameter: Parameter { + range: 41..42, + node_index: NodeIndex(None), + name: Identifier { + id: Name("y"), + range: 41..42, + node_index: NodeIndex(None), + }, + annotation: None, + }, + default: None, + }, + ParameterWithDefault { + range: 44..45, + node_index: NodeIndex(None), + parameter: Parameter { + range: 44..45, + node_index: NodeIndex(None), + name: Identifier { + id: Name("z"), + range: 44..45, + node_index: NodeIndex(None), + }, + annotation: None, + }, + default: None, + }, + ], + vararg: None, + kwonlyargs: [], + kwarg: None, + }, + returns: None, + body: [ + Expr( + StmtExpr { + node_index: NodeIndex(None), + range: 48..51, + value: EllipsisLiteral( + ExprEllipsisLiteral { + node_index: NodeIndex(None), + range: 48..51, + }, + ), + }, + ), + ], + }, + ), + Expr( + StmtExpr { + node_index: NodeIndex(None), + range: 52..73, + value: Call( + ExprCall { + node_index: NodeIndex(None), + range: 52..73, + func: Name( + ExprName { + node_index: NodeIndex(None), + range: 52..55, + id: Name("baz"), + ctx: Load, + }, + ), + arguments: Arguments { + range: 55..73, + node_index: NodeIndex(None), + args: [ + Name( + ExprName { + node_index: NodeIndex(None), + range: 56..57, + id: Name("x"), + ctx: Load, + }, + ), + ], + keywords: [ + Keyword { + range: 59..62, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("y"), + range: 59..60, + node_index: NodeIndex(None), + }, + ), + value: NumberLiteral( + ExprNumberLiteral { + node_index: NodeIndex(None), + range: 61..62, + value: Int( + 1, + ), + }, + ), + }, + Keyword { + range: 64..67, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("z"), + range: 64..65, + node_index: NodeIndex(None), + }, + ), + value: NumberLiteral( + ExprNumberLiteral { + node_index: NodeIndex(None), + range: 66..67, + value: Int( + 3, + ), + }, + ), + }, + Keyword { + range: 69..72, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("y"), + range: 69..70, + node_index: NodeIndex(None), + }, + ), + value: NumberLiteral( + ExprNumberLiteral { + node_index: NodeIndex(None), + range: 71..72, + value: Int( + 4, + ), + }, + ), + }, + ], + }, + }, + ), + }, + ), + ], + }, +) +``` +## Semantic Syntax Errors + + | +1 | def foo(x): ... +2 | foo(x=1, x=2) + | ^^^ Syntax Error: Duplicate keyword argument `x` +3 | def baz(x, y, z): ... +4 | baz(x, y=1, z=3, y=4) + | + + + | +2 | foo(x=1, x=2) +3 | def baz(x, y, z): ... +4 | baz(x, y=1, z=3, y=4) + | ^^^ Syntax Error: Duplicate keyword argument `y` diff --git a/crates/ruff_python_parser/tests/snapshots/invalid_syntax@expressions__arguments__duplicate_keyword_arguments.py.snap b/crates/ruff_python_parser/tests/snapshots/invalid_syntax@expressions__arguments__duplicate_keyword_arguments.py.snap index 61195ea44c..182ca169e0 100644 --- a/crates/ruff_python_parser/tests/snapshots/invalid_syntax@expressions__arguments__duplicate_keyword_arguments.py.snap +++ b/crates/ruff_python_parser/tests/snapshots/invalid_syntax@expressions__arguments__duplicate_keyword_arguments.py.snap @@ -141,13 +141,13 @@ Module( }, ) ``` -## Errors +## Semantic Syntax Errors | 1 | foo(a=1, b=2, c=3, b=4, a=5) - | ^^^ Syntax Error: Duplicate keyword argument "b" + | ^^^ Syntax Error: Duplicate keyword argument `b` | 1 | foo(a=1, b=2, c=3, b=4, a=5) - | ^^^ Syntax Error: Duplicate keyword argument "a" + | ^^^ Syntax Error: Duplicate keyword argument `a` diff --git a/crates/ruff_python_parser/tests/snapshots/valid_syntax@non_duplicate_keyword_args.py.snap b/crates/ruff_python_parser/tests/snapshots/valid_syntax@non_duplicate_keyword_args.py.snap new file mode 100644 index 0000000000..244f8063cb --- /dev/null +++ b/crates/ruff_python_parser/tests/snapshots/valid_syntax@non_duplicate_keyword_args.py.snap @@ -0,0 +1,298 @@ +--- +source: crates/ruff_python_parser/tests/fixtures.rs +input_file: crates/ruff_python_parser/resources/inline/ok/non_duplicate_keyword_args.py +--- +## AST + +``` +Module( + ModModule { + node_index: NodeIndex(None), + range: 0..71, + body: [ + FunctionDef( + StmtFunctionDef { + node_index: NodeIndex(None), + range: 0..15, + is_async: false, + decorator_list: [], + name: Identifier { + id: Name("foo"), + range: 4..7, + node_index: NodeIndex(None), + }, + type_params: None, + parameters: Parameters { + range: 7..10, + node_index: NodeIndex(None), + posonlyargs: [], + args: [ + ParameterWithDefault { + range: 8..9, + node_index: NodeIndex(None), + parameter: Parameter { + range: 8..9, + node_index: NodeIndex(None), + name: Identifier { + id: Name("x"), + range: 8..9, + node_index: NodeIndex(None), + }, + annotation: None, + }, + default: None, + }, + ], + vararg: None, + kwonlyargs: [], + kwarg: None, + }, + returns: None, + body: [ + Expr( + StmtExpr { + node_index: NodeIndex(None), + range: 12..15, + value: EllipsisLiteral( + ExprEllipsisLiteral { + node_index: NodeIndex(None), + range: 12..15, + }, + ), + }, + ), + ], + }, + ), + Expr( + StmtExpr { + node_index: NodeIndex(None), + range: 16..24, + value: Call( + ExprCall { + node_index: NodeIndex(None), + range: 16..24, + func: Name( + ExprName { + node_index: NodeIndex(None), + range: 16..19, + id: Name("foo"), + ctx: Load, + }, + ), + arguments: Arguments { + range: 19..24, + node_index: NodeIndex(None), + args: [], + keywords: [ + Keyword { + range: 20..23, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("x"), + range: 20..21, + node_index: NodeIndex(None), + }, + ), + value: NumberLiteral( + ExprNumberLiteral { + node_index: NodeIndex(None), + range: 22..23, + value: Int( + 1, + ), + }, + ), + }, + ], + }, + }, + ), + }, + ), + FunctionDef( + StmtFunctionDef { + node_index: NodeIndex(None), + range: 25..46, + is_async: false, + decorator_list: [], + name: Identifier { + id: Name("bar"), + range: 29..32, + node_index: NodeIndex(None), + }, + type_params: None, + parameters: Parameters { + range: 32..41, + node_index: NodeIndex(None), + posonlyargs: [], + args: [ + ParameterWithDefault { + range: 33..34, + node_index: NodeIndex(None), + parameter: Parameter { + range: 33..34, + node_index: NodeIndex(None), + name: Identifier { + id: Name("x"), + range: 33..34, + node_index: NodeIndex(None), + }, + annotation: None, + }, + default: None, + }, + ParameterWithDefault { + range: 36..37, + node_index: NodeIndex(None), + parameter: Parameter { + range: 36..37, + node_index: NodeIndex(None), + name: Identifier { + id: Name("y"), + range: 36..37, + node_index: NodeIndex(None), + }, + annotation: None, + }, + default: None, + }, + ParameterWithDefault { + range: 39..40, + node_index: NodeIndex(None), + parameter: Parameter { + range: 39..40, + node_index: NodeIndex(None), + name: Identifier { + id: Name("z"), + range: 39..40, + node_index: NodeIndex(None), + }, + annotation: None, + }, + default: None, + }, + ], + vararg: None, + kwonlyargs: [], + kwarg: None, + }, + returns: None, + body: [ + Expr( + StmtExpr { + node_index: NodeIndex(None), + range: 43..46, + value: EllipsisLiteral( + ExprEllipsisLiteral { + node_index: NodeIndex(None), + range: 43..46, + }, + ), + }, + ), + ], + }, + ), + Expr( + StmtExpr { + node_index: NodeIndex(None), + range: 47..70, + value: Call( + ExprCall { + node_index: NodeIndex(None), + range: 47..70, + func: Name( + ExprName { + node_index: NodeIndex(None), + range: 47..50, + id: Name("foo"), + ctx: Load, + }, + ), + arguments: Arguments { + range: 50..70, + node_index: NodeIndex(None), + args: [], + keywords: [ + Keyword { + range: 51..56, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("x"), + range: 51..52, + node_index: NodeIndex(None), + }, + ), + value: StringLiteral( + ExprStringLiteral { + node_index: NodeIndex(None), + range: 53..56, + value: StringLiteralValue { + inner: Single( + StringLiteral { + range: 53..56, + node_index: NodeIndex(None), + value: "a", + flags: StringLiteralFlags { + quote_style: Double, + prefix: Empty, + triple_quoted: false, + unclosed: false, + }, + }, + ), + }, + }, + ), + }, + Keyword { + range: 58..61, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("y"), + range: 58..59, + node_index: NodeIndex(None), + }, + ), + value: NumberLiteral( + ExprNumberLiteral { + node_index: NodeIndex(None), + range: 60..61, + value: Int( + 1, + ), + }, + ), + }, + Keyword { + range: 63..69, + node_index: NodeIndex(None), + arg: Some( + Identifier { + id: Name("z"), + range: 63..64, + node_index: NodeIndex(None), + }, + ), + value: BooleanLiteral( + ExprBooleanLiteral { + node_index: NodeIndex(None), + range: 65..69, + value: true, + }, + ), + }, + ], + }, + }, + ), + }, + ), + ], + }, +) +``` diff --git a/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md b/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md index 433d0b08bb..7d51b886db 100644 --- a/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md +++ b/crates/ty_python_semantic/resources/mdtest/diagnostics/semantic_syntax_errors.md @@ -151,6 +151,33 @@ match obj: pass ``` +## Duplicate keyword arguments + +```toml +[environment] +python-version = "3.12" +``` + +```py +def f(x: int) -> None: ... + +# error: [invalid-syntax] "Duplicate keyword argument `x`" +f(x=1, x=2) + +# error: [parameter-already-assigned] "Multiple values provided for parameter `x` of function `f`" +f(1, x=2) +``` + +Duplicate keywords are also invalid in class definitions: + +```py +# error: [invalid-syntax] "Duplicate keyword argument `metaclass`" +class C(metaclass=type, metaclass=type): ... + +# error: [invalid-syntax] "Duplicate keyword argument `metaclass`" +class Generic[T](metaclass=type, metaclass=type): ... +``` + ## `return`, `yield`, `yield from`, and `await` outside function ```py diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index b0aae99f51..b50442a49e 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -4785,13 +4785,22 @@ impl<'a, 'db> ArgumentMatcher<'a, 'db> { positional: bool, variable_argument_length: bool, ) { - if self.parameter_info[parameter_index].matched { - if !parameter.is_variadic() && !parameter.is_keyword_variadic() { - self.errors.push(BindingError::ParameterAlreadyAssigned { - argument_index: self.get_argument_index(argument_index), - parameter: ParameterContext::new(parameter, parameter_index, positional), - }); - } + if self.parameter_info[parameter_index].matched + && !parameter.is_variadic() + && !parameter.is_keyword_variadic() + // Repeated explicit keywords are already reported as syntax errors. + && !matches!( + argument, + Argument::Keyword(name) + if self.arguments.iter().take(argument_index).any(|(previous, _)| { + matches!(previous, Argument::Keyword(previous_name) if previous_name == name) + }) + ) + { + self.errors.push(BindingError::ParameterAlreadyAssigned { + argument_index: self.get_argument_index(argument_index), + parameter: ParameterContext::new(parameter, parameter_index, positional), + }); } if variable_argument_length && matches!( From a5a1cba51b2cb4efdbb65ab66e062870401e4f78 Mon Sep 17 00:00:00 2001 From: Carl Meyer Date: Tue, 18 Aug 2026 12:06:38 -0700 Subject: [PATCH 089/371] [ty] Separate generic constraint accumulation from legacy projection (#27743) ## Summary This is step 3 in rebooting #26712 as a series of focused PRs (astral-sh/ty#3557). `SpecializationBuilder` currently mixes two operations: 1. Collecting the constraints for a generic call. 2. Copying their solutions into the legacy type-variable mapping. Copying solutions too early loses relationships between alternatives. This PR separates the work into three explicit steps: 1. Analyze a constraint set and collect any useful diagnostic information. 2. Record the constraint set for the whole call. 3. Update the legacy mapping only when a caller needs it. This lets overloaded callbacks examine every alternative before accepted alternatives update the mapping. Rejected overloads cannot affect valid specializations, and the existing first-rejection diagnostic behavior is preserved. Existing fallbacks for contextual preferences, `ParamSpec`, `TypeVarTuple`, and recursive specialization remain in place. ## Test plan Added mdtests cover: - Generic overloaded callbacks passed to constructors. - Bounded and constrained type variables when an invalid callback overload appears first or last. - A `ParamSpec`-forwarding callable passed through an unpacked `TypeVarTuple`. Rust unit tests also cover independent analysis, recording, and projection; valid and rejected inference paths; lower-bound classification; and grouped declaration failures. --- .../resources/mdtest/call/function.md | 24 ++ .../mdtest/generics/legacy/callables.md | 38 ++ .../mdtest/generics/legacy/unpack.md | 18 + .../ty_python_semantic/src/types/generics.rs | 377 ++++++++++++++---- 4 files changed, 380 insertions(+), 77 deletions(-) diff --git a/crates/ty_python_semantic/resources/mdtest/call/function.md b/crates/ty_python_semantic/resources/mdtest/call/function.md index 2809aec140..0def1b3710 100644 --- a/crates/ty_python_semantic/resources/mdtest/call/function.md +++ b/crates/ty_python_semantic/resources/mdtest/call/function.md @@ -181,6 +181,30 @@ dynamic: Any = [] reveal_type(map(operator.add, ints, dynamic)) # revealed: map[Unknown] ``` +## Generic overloaded callable constraints in constructors + +An overloaded callback can have a type variable of its own. An overload rejected by the constructor +must not leave a mapping for that variable that causes the accepted overload to be rejected. + +```py +from typing import Generic, TypeVar, overload + +T = TypeVar("T", str, bytes) + +class Result(Generic[T]): ... + +@overload +def convert(value: T) -> Result[T]: ... +@overload +def convert(value: Result[T]) -> Result[T]: ... +def convert(value: T | Result[T]) -> Result[T]: + raise NotImplementedError + +# TODO: Preserve correlated overloaded-callback solutions (astral-sh/ty#2799) to infer +# `map[Result[str]]`. +reveal_type(map(convert, ["a"])) # revealed: map[Unknown] +``` + ## Decorated ```py diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/callables.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/callables.md index 06eab72da0..61f87dcc84 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/callables.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/callables.md @@ -296,6 +296,44 @@ def f(val: str | bytes) -> None: reveal_type(accepts_callable(f)) # revealed: str | bytes ``` +## Rejected overloaded callbacks preserve valid specializations + +An overloaded callback may contain one alternative whose return type violates a type variable's +upper bound or declared constraints. The valid alternative must determine the specialization +regardless of the order in which the overloads appear. + +```py +from typing import Callable, TypeVar, overload + +Bounded = TypeVar("Bounded", bound=int) +Constrained = TypeVar("Constrained", int, bytes) + +@overload +def invalid_first(value: str) -> str: ... +@overload +def invalid_first(value: int) -> int: ... +def invalid_first(value: str | int) -> str | int: + return value + +@overload +def invalid_last(value: int) -> int: ... +@overload +def invalid_last(value: str) -> str: ... +def invalid_last(value: str | int) -> str | int: + return value + +def infer_bound(callback: Callable[..., Bounded]) -> Bounded: + raise NotImplementedError + +def infer_constrained(callback: Callable[..., Constrained]) -> Constrained: + raise NotImplementedError + +reveal_type(infer_bound(invalid_first)) # revealed: int +reveal_type(infer_bound(invalid_last)) # revealed: int +reveal_type(infer_constrained(invalid_first)) # revealed: int +reveal_type(infer_constrained(invalid_last)) # revealed: int +``` + ## Overloaded callable with a constrained type variable When `T` is constrained to a union by other arguments, the overloaded callable must still be treated diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/unpack.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/unpack.md index 64b26e03fc..95335ce8df 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/unpack.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/unpack.md @@ -82,6 +82,24 @@ reveal_type(invoke(format_value, 1, "value")) # revealed: str reveal_type(invoke(format_value, 1)) # revealed: str ``` +## Forwarding a `ParamSpec` through an unpacked type variable tuple + +A callable that forwards a parameter specification can itself be passed, with its arguments, to a +callable whose positional parameters are described by an unpacked type variable tuple. + +```py +from typing import Callable, ParamSpec, TypeVarTuple, Unpack + +P = ParamSpec("P") +Ts = TypeVarTuple("Ts") + +def invoke(callback: Callable[[Unpack[Ts]], None], *args: Unpack[Ts]) -> None: ... +def forward(callback: Callable[P, None], *args: P.args, **kwargs: P.kwargs) -> None: ... +def one_arg(value: int) -> None: ... + +invoke(forward, one_arg, 1) +``` + ## Type aliases A legacy alias can use `Unpack[Ts]` and accept either individual types or an unpacked tuple type. diff --git a/crates/ty_python_semantic/src/types/generics.rs b/crates/ty_python_semantic/src/types/generics.rs index 4e1d1328c4..9976475a2a 100644 --- a/crates/ty_python_semantic/src/types/generics.rs +++ b/crates/ty_python_semantic/src/types/generics.rs @@ -6,13 +6,14 @@ use std::collections::hash_map::Entry; use itertools::Itertools; use ruff_python_ast as ast; use rustc_hash::{FxHashMap, FxHashSet}; +use smallvec::SmallVec; use crate::types::callable::walk_callable_type; use crate::types::class::ClassType; use crate::types::class_base::ClassBase; use crate::types::constraints::{ ConstraintBounds, ConstraintSet, ConstraintSetBuilder, IteratorConstraintsExtension, PathBound, - PathBounds, Solutions, + PathBounds, Solution, Solutions, }; use crate::types::infer::original_class_type; use crate::types::relation::{ @@ -2436,14 +2437,100 @@ impl<'db> TypeVarInference<'db> { } } -/// A failure to project a constraint set into the legacy type-mapping representation. +/// The valid specializations of a constraint set, or evidence for why it is unsatisfiable. /// -/// A type-variable declaration failure can be reported immediately. Other unsatisfiable -/// relations must remain in the pending constraint set so that they invalidate the call-wide -/// solution without producing a misleading bound diagnostic. -enum ConstraintSetInferenceError<'db> { - InvalidTypeVar(SpecializationError<'db>), - Unsatisfiable, +/// Failed paths can occur alongside valid paths, but their declaration failures matter only when +/// every path is rejected. Preserve that evidence exclusively for unsatisfiable constraint sets. +enum ConstraintSetAnalysis<'db> { + Unsatisfiable(SmallVec<[ConstraintFailure<'db>; 1]>), + Unconstrained, + Constrained(Vec>), +} + +impl<'db> ConstraintSetAnalysis<'db> { + /// Reports why a type variable's declared bound or constraints cannot be satisfied. + /// + /// Multiple rejected paths describe one failure when their lower bounds violate the same type + /// variable's declaration in a contravariant position. Their argument types are combined into + /// an intersection. For example, paths rejecting `int` and `bool` for `T: bytes` report + /// `bool`, the intersection of `int` and `bool`. + /// + /// The inference API returns at most one declaration error per relation. Failures involving + /// different declarations, variances, or type variables cannot be combined meaningfully, so + /// only the first is reported. + fn specialization_error( + &self, + db: &'db dyn Db, + env: &ProgramEnvironment<'db>, + ) -> Option> { + let Self::Unsatisfiable(failures) = self else { + return None; + }; + + let first = failures.first()?; + // A single failure needs no aggregation; failures for different declarations, type + // variables, or variances cannot be combined, so they also return the first failure. + // TODO: Rank incompatible failures by their diagnostic usefulness, or report them + // separately. + if failures.len() < 2 + || failures.iter().any(|failure| { + failure.error.bound_typevar() != first.error.bound_typevar() + || failure.variance != first.variance + || !matches!( + (&first.error, &failure.error), + ( + SpecializationError::MismatchedBound { .. }, + SpecializationError::MismatchedBound { .. } + ) | ( + SpecializationError::MismatchedConstraint { .. }, + SpecializationError::MismatchedConstraint { .. } + ) + ) + }) + { + return Some(first.error.clone()); + } + + let arguments = failures.iter().map(|failure| failure.error.argument_type()); + let argument = match first.variance { + ConstraintFailureVariance::Contravariant => { + IntersectionType::from_elements(db, env, arguments) + } + ConstraintFailureVariance::Invariant => { + // TODO: Combine invariant failures without losing their lower- or upper-bound + // evidence. + return Some(first.error.clone()); + } + }; + + let mut error = first.error.clone(); + let (SpecializationError::MismatchedBound { + argument: existing, .. + } + | SpecializationError::MismatchedConstraint { + argument: existing, .. + }) = &mut error; + *existing = argument; + Some(error) + } +} + +/// A declared type-variable bound or constraint rejected while solving one alternative. +/// +/// The variance identifies whether the rejected lower bound also has an upper bound, so multiple +/// failures from the same relation can be combined into one diagnostic. +struct ConstraintFailure<'db> { + error: SpecializationError<'db>, + variance: ConstraintFailureVariance, +} + +/// The possible variances for a path with a lower bound that violates a declaration. +/// +/// Covariant and bivariant paths have no lower bound, so they cannot produce declaration failures. +#[derive(Clone, Copy, Eq, PartialEq)] +enum ConstraintFailureVariance { + Contravariant, + Invariant, } /// Returns the directional comparisons required by this comparison's polarity. @@ -2551,7 +2638,8 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { for (formal, actual) in argument_relations { let when = actual.when_constraint_set_assignable_to(db, self.env, formal, self.constraints); - let _ = self.add_type_mappings_from_constraint_set(when); + let analysis = self.analyze_constraint_set(when); + self.project_for_legacy_fallback(&analysis); } let types = self.solve_hash_map_with(generic_context, &mut choose); @@ -2945,21 +3033,11 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { self.intersect_pending_typevar_constraint(bound_typevar, bounds); } - /// Finds all of the valid specializations of a constraint set, and adds their type mappings to - /// the specialization that this builder is building up. - /// - /// TODO: This is a stopgap! Eventually, the builder will maintain a single constraint set for - /// the main specialization that we are building, and [`build_with`][Self::build_with] will - /// build the specialization directly from that constraint set. This method lets us migrate to - /// that brave new world incrementally, by using the new constraint set mechanism piecemeal for - /// certain type comparisons. - fn add_type_mappings_from_constraint_set( - &mut self, - set: ConstraintSet<'db, 'c>, - ) -> Result<(), ConstraintSetInferenceError<'db>> { + /// Solves one relation without recording it or changing the legacy type mappings. + fn analyze_constraint_set(&self, set: ConstraintSet<'db, 'c>) -> ConstraintSetAnalysis<'db> { let db = self.db; - let mut first_error = None; - let solutions = match set.solutions_with( + let mut failures = SmallVec::new(); + let solutions = set.solutions_with( db, self.env, self.constraints, @@ -2967,21 +3045,34 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { |_variance, path_bound| { let solution = PathBounds::preliminary_solve(db, self.env, self.constraints, path_bound); - if solution.is_err() && first_error.is_none() { - first_error = self.specialization_error_from_failed_bounds(path_bound); + if solution.is_err() + && let Some(failure) = self.constraint_failure_from_failed_bounds(path_bound) + { + failures.push(failure); } solution }, - ) { - Solutions::Unsatisfiable => { - return Err(first_error.map_or( - ConstraintSetInferenceError::Unsatisfiable, - ConstraintSetInferenceError::InvalidTypeVar, - )); - } - Solutions::Unconstrained => return Ok(()), - Solutions::Constrained(solutions) => solutions, + ); + + match solutions { + Solutions::Unsatisfiable => ConstraintSetAnalysis::Unsatisfiable(failures), + Solutions::Unconstrained => ConstraintSetAnalysis::Unconstrained, + Solutions::Constrained(solutions) => ConstraintSetAnalysis::Constrained(solutions), + } + } + + /// Adds valid solutions to the compatibility mapping used by legacy inference consumers. + /// + /// This projection loses correlations between alternatives, so callers must only request it + /// after they have accepted the corresponding relation. + /// + /// TODO: Remove this compatibility path once [`build_with`][Self::build_with] and all other + /// inference consumers can build specializations solely from the call-wide constraint set. + fn project_for_legacy_fallback(&mut self, analysis: &ConstraintSetAnalysis<'db>) { + let ConstraintSetAnalysis::Constrained(solutions) = analysis else { + return; }; + for solution in solutions { for binding in solution { let solution = self.remove_inferable_typevar_artifacts_from_solution( @@ -2991,21 +3082,25 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { self.insert_hash_map_type_mapping(binding.bound_typevar, solution); } } - Ok(()) } - /// Returns an actionable type-variable error for a failed projected path. + /// Classifies a failed path when its lower bound violates a type-variable declaration. /// /// Conflicting inferred lower and upper bounds are not necessarily violations of the type /// variable's declaration, so they remain generic unsatisfiable constraints. - fn specialization_error_from_failed_bounds( + fn constraint_failure_from_failed_bounds( &self, path_bound: &PathBound<'db>, - ) -> Option> { + ) -> Option> { let db = self.db; let bound_typevar = path_bound.bound_typevar; let argument = path_bound.lower?; - match bound_typevar + let variance = if path_bound.has_upper() { + ConstraintFailureVariance::Invariant + } else { + ConstraintFailureVariance::Contravariant + }; + let error = match bound_typevar .typevar(db) .bound_or_constraints(db, self.env)? { @@ -3022,24 +3117,34 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { argument, }) } - } + }?; + Some(ConstraintFailure { error, variance }) + } + + /// Records one relation in the call-wide constraint set. + /// + /// Generic unsatisfiability is retained in `pending` rather than reported as a misleading + /// type-variable declaration error. + fn record_constraint_set(&mut self, when: ConstraintSet<'db, 'c>) { + self.pending.intersect(self.db, self.constraints, when); } - /// Adds legacy type mappings from `when` and records it in the call-wide constraint set. + /// Records a relation and projects its solutions into the legacy type mapping. /// - /// Generic unsatisfiability is retained in `pending`; only failures against a type variable's - /// declared bound or constraints are returned for immediate diagnosis. + /// Contextual preference checks, variadic inference, and recursive-specialization recovery + /// require the projected mapping while processing the call. fn infer_from_constraint_set( &mut self, when: ConstraintSet<'db, 'c>, ) -> Result<(), SpecializationError<'db>> { let db = self.db; - let result = self.add_type_mappings_from_constraint_set(when); - self.pending.intersect(db, self.constraints, when); - match result { - Ok(()) | Err(ConstraintSetInferenceError::Unsatisfiable) => Ok(()), - Err(ConstraintSetInferenceError::InvalidTypeVar(error)) => Err(error), + let analysis = self.analyze_constraint_set(when); + self.record_constraint_set(when); + if let Some(error) = analysis.specialization_error(db, self.env) { + return Err(error); } + self.project_for_legacy_fallback(&analysis); + Ok(()) } /// Returns the assignability constraints required by this comparison's polarity. @@ -3292,41 +3397,48 @@ impl<'db, 'c> SpecializationBuilder<'db, 'c> { ); self.infer_from_constraint_set(when)?; } else { - // An overloaded actual callable is compatible with the formal signature if at - // least one of its overloads is. We collect type mappings from all satisfiable - // overloads, and only report an error if none of them are satisfiable. + // An overloaded actual callable is compatible if at least one overload matches. + // Analyze every alternative without changing the builder; only accepted overloads + // contribute mappings after their combined relation has been committed. let env = self.env.clone(); let constraints = self.constraints; - let mut first_error = None; - let combined = actual_callable - .signatures(db) - .overloads - .iter() - .filter_map(|actual_signature| { - let when = actual_signature.when_constraint_set_assignable_to_signatures( - db, - &env, - formal_signature, - constraints, - ); - match self.add_type_mappings_from_constraint_set(when) { - Ok(()) => Some(when), - Err(error) => { - first_error.get_or_insert(error); - None - } + let mut first_rejection = None; + let mut accepted = + SmallVec::<[(ConstraintSet<'db, 'c>, ConstraintSetAnalysis<'db>); 1]>::new(); + for actual_signature in &actual_callable.signatures(db).overloads { + let when = actual_signature.when_constraint_set_assignable_to_signatures( + db, + &env, + formal_signature, + constraints, + ); + let analysis = self.analyze_constraint_set(when); + match analysis { + rejected @ ConstraintSetAnalysis::Unsatisfiable(_) => { + first_rejection.get_or_insert(rejected); } - }) - .reduce(|lhs, rhs| lhs.or(db, constraints, || rhs)); - let Some(combined) = combined else { - self.pending = ConstraintSet::from_bool(self.constraints, false); - if let Some(ConstraintSetInferenceError::InvalidTypeVar(error)) = first_error { - return Err(error); + analysis => accepted.push((when, analysis)), } - return Ok(()); + } + + let combined = accepted + .iter() + .map(|(when, _)| *when) + .reduce(|left, right| left.or(db, constraints, || right)); + let Some(combined) = combined else { + self.record_constraint_set(ConstraintSet::from_bool(self.constraints, false)); + return first_rejection + .and_then(|analysis| analysis.specialization_error(db, &env)) + .map_or(Ok(()), Err); }; - self.pending.intersect(db, self.constraints, combined); + + // One accepted alternative proves that their disjunction is satisfiable; solving + // the combined TDD again would eagerly repeat the expensive path enumeration. + self.record_constraint_set(combined); + for (_, analysis) in accepted { + self.project_for_legacy_fallback(&analysis); + } } } Ok(()) @@ -4142,4 +4254,115 @@ mod tests { assert!(t.is_inferable(db, inferable)); assert!(u.is_inferable(db, inferable)); } + + #[test] + fn recording_constraints_does_not_project_legacy_mappings() { + let db = setup_db(); + let db = &db; + let env = db.program_environment(); + let typevar = BoundTypeVarInstance::synthetic( + db, + &env, + Name::new_static("T"), + TypeVarVariance::Invariant, + ); + let context = GenericContext::from_typevar_instances(db, &env, [typevar]); + let constraints = ConstraintSetBuilder::new(); + let mut builder = + SpecializationBuilder::new(db, &env, &constraints, context.inferable_typevars(db)); + let int = KnownClass::Int.to_instance(db, &env); + let set = ConstraintSet::constrain_typevar(db, &env, &constraints, typevar, int, int); + + let analysis = builder.analyze_constraint_set(set); + assert!(builder.types.is_empty()); + assert!(builder.pending.is_always_satisfied(db, &env)); + + builder.record_constraint_set(set); + assert!(builder.types.is_empty()); + assert!(!builder.pending.is_always_satisfied(db, &env)); + + builder.project_for_legacy_fallback(&analysis); + assert!(builder.inferred_type_is_assignable_to(typevar.identity(db), int)); + } + + #[test] + fn satisfiable_constraint_analysis_discards_rejected_paths() { + let db = setup_db(); + let db = &db; + let env = db.program_environment(); + let int = KnownClass::Int.to_instance(db, &env); + let str = KnownClass::Str.to_instance(db, &env); + let typevar = BoundTypeVarInstance::synthetic( + db, + &env, + Name::new_static("T"), + TypeVarVariance::Invariant, + ) + .map_bound_or_constraints(db, |_| Some(TypeVarBoundOrConstraints::UpperBound(int))); + let context = GenericContext::from_typevar_instances(db, &env, [typevar]); + let constraints = ConstraintSetBuilder::new(); + let mut builder = + SpecializationBuilder::new(db, &env, &constraints, context.inferable_typevars(db)); + let lower_only = + ConstraintSet::constrain_typevar_lower_bound(db, &env, &constraints, typevar, str); + let rejected = ConstraintSet::constrain_typevar(db, &env, &constraints, typevar, str, str); + let accepted = ConstraintSet::constrain_typevar(db, &env, &constraints, typevar, int, int); + + for (set, variance) in [ + (lower_only, ConstraintFailureVariance::Contravariant), + (rejected, ConstraintFailureVariance::Invariant), + ] { + assert!(matches!( + builder.analyze_constraint_set(set), + ConstraintSetAnalysis::Unsatisfiable(failures) + if matches!(failures.as_slice(), [failure] if failure.variance == variance) + )); + } + + let analysis = builder.analyze_constraint_set(rejected.or(db, &constraints, || accepted)); + assert!(analysis.specialization_error(db, &env).is_none()); + + builder.project_for_legacy_fallback(&analysis); + assert!(builder.inferred_type_is_assignable_to(typevar.identity(db), int)); + assert!(!builder.inferred_type_is_assignable_to(typevar.identity(db), str)); + } + + #[test] + fn constraint_failure_diagnostics_preserve_variance() { + let db = setup_db(); + let db = &db; + let env = db.program_environment(); + let typevar = BoundTypeVarInstance::synthetic( + db, + &env, + Name::new_static("T"), + TypeVarVariance::Invariant, + ); + let int = KnownClass::Int.to_instance(db, &env); + let bool = KnownClass::Bool.to_instance(db, &env); + let analysis = |variance, first, second| { + ConstraintSetAnalysis::Unsatisfiable( + [first, second] + .into_iter() + .map(|argument| ConstraintFailure { + error: SpecializationError::MismatchedBound { + bound_typevar: typevar, + argument, + }, + variance, + }) + .collect(), + ) + }; + + let contravariant = analysis(ConstraintFailureVariance::Contravariant, int, bool) + .specialization_error(db, &env) + .map(|error| error.argument_type()); + assert_eq!(contravariant, Some(bool)); + + let invariant = analysis(ConstraintFailureVariance::Invariant, int, bool) + .specialization_error(db, &env) + .map(|error| error.argument_type()); + assert_eq!(invariant, Some(int)); + } } From 3242fc5abfb6124db82cb092539a1fc45ef0c852 Mon Sep 17 00:00:00 2001 From: Brent Westbrook <36778786+ntBre@users.noreply.github.com> Date: Tue, 18 Aug 2026 15:54:49 -0400 Subject: [PATCH 090/371] Offer display-only fixes and mark safe fixes preferred (#27807) Summary -- I thought this was the case before (https://github.com/astral-sh/ruff/issues/18686#issuecomment-2977459714), but apparently that only applied to `ruff-lsp`, which I was somehow still using at the time. This change exposes display-only fixes in the native server, like in `ruff-lsp` and the [playground]. We append `(suggestion)` to the fix title for display-only fixes in an effort to distinguish them from the more reliable safe and unsafe fixes. To further differentiate these from safe fixes, this PR also sets the `isPreferred` property to `true` for safe fixes. This means such fixes can be applied automatically with the "Auto Fix" command (distinct from the "Quick Fix" command, which always opens a menu), should sort first, and get a slightly different lightbulb icon, at least in VS Code. Before landing on this, I tried adding a [request for confirmation](https://code.visualstudio.com/api/references/vscode-api#WorkspaceEditEntryMetadata) to the fix, but the UI for this seems really annoying in VS Code. It opens this separate panel, and you have to click the check box and also click Apply: image That seemed to go a bit farther than we need, especially relative to the code change required and after I vetted our display-only fixes. Most of them are only a bit less safe than our unsafe fixes. We don't have any that seem obviously disastrous. Related to the scope of the code change, this functionality may also not be available in some clients, at least according to Codex. However, one cool thing about this approach is that we could pass down rule-specific information about why the fix needs confirmation. That would obviously increase the scope even more but was an interesting finding. These changes align pretty well with what rust-analyzer does. Its applicability levels are a bit different from ours, but it marks `MachineApplicable` fixes as preferred, `MaybeIncorrect` fixes as not preferred, and `HasPlaceholders` suggestions don't have quick fixes. The main difference is that our `DisplayOnly` fixes seem considerably safer than [`HasPlaceholders`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint_defs/enum.Applicability.html#variant.HasPlaceholders), so we can still set non-preferred fixes on them. Test Plan -- Updated existing snapshots and manual testing in VS Code. The quick fix list previously only offered "Disable for this line" but now also offers the display-only fix: image I also tested the `Ruff: Fix all auto-fixable problems` and `Fix all` commands. These still don't apply display-only fixes. Only inline quick fixes include display-only fixes. [playground]: https://play.ruff.rs/487aa90c-7809-4405-b9c5-ef9c5b8cf9c7 --- crates/ruff_server/src/lint.rs | 13 +++++++-- crates/ruff_server/src/server.rs | 3 +- .../src/server/api/requests/code_action.rs | 1 + crates/ruff_server/tests/e2e/code_action.rs | 1 + crates/ruff_server/tests/e2e/diagnostics.rs | 2 ++ ...notebook_without_ipynb_extension_open.snap | 29 +++++++++++++++++-- ...book__super_resolution_overview_final.snap | 29 +++++++++++++++++-- ...ebook__super_resolution_overview_open.snap | 29 +++++++++++++++++-- crates/ruff_server/tests/e2e/workspace.rs | 1 + 9 files changed, 98 insertions(+), 10 deletions(-) diff --git a/crates/ruff_server/src/lint.rs b/crates/ruff_server/src/lint.rs index 1b74b72222..308de886ee 100644 --- a/crates/ruff_server/src/lint.rs +++ b/crates/ruff_server/src/lint.rs @@ -47,6 +47,8 @@ pub(crate) struct AssociatedDiagnosticData { code: String, /// Possible edit to add a suppression comment which will disable this diagnostic. noqa_edit: Option, + /// Whether this fix corresponds to a preferred action that can be used by auto fix commands. + is_preferred: Option, } /// Describes a fix for `fixed_diagnostic` that may have quick fix @@ -64,6 +66,8 @@ pub(crate) struct DiagnosticFix { pub(crate) edits: Vec, /// Possible edit to add a suppression comment which will disable this diagnostic. pub(crate) noqa_edit: Option, + /// Whether this fix corresponds to a preferred action that can be used by auto fix commands. + pub(crate) is_preferred: Option, } /// A series of diagnostics across a single text document or an arbitrary number of notebook cells. @@ -306,6 +310,7 @@ pub(crate) fn fixes_for_diagnostics( title: associated_data.title, noqa_edit: associated_data.noqa_edit, edits: associated_data.edits, + is_preferred: associated_data.is_preferred, })) }) .filter_map(crate::Result::transpose) @@ -372,7 +377,6 @@ fn to_lsp_diagnostic( let name = diagnostic.name(); let fix = diagnostic.fix(); let suggestion = diagnostic.first_help_text(); - let fix = fix.and_then(|fix| fix.applies(Applicability::Unsafe).then_some(fix)); let (severity, code) = if let Some(code) = diagnostic.secondary_code() { let severity = severity(code); @@ -398,6 +402,10 @@ fn to_lsp_diagnostic( let data = (fix.is_some() || noqa_edit.is_some()) .then(|| { + let mut title = suggestion.unwrap_or(name).to_string(); + if fix.is_some_and(|fix| fix.applicability() == Applicability::DisplayOnly) { + title.push_str(" (suggestion)"); + } let edits = fix .into_iter() .flat_map(Fix::edits) @@ -411,10 +419,11 @@ fn to_lsp_diagnostic( new_text: noqa_edit.into_content().unwrap_or_default().into_string(), }); serde_json::to_value(AssociatedDiagnosticData { - title: suggestion.unwrap_or(name).to_string(), + title, noqa_edit, edits, code: code.clone(), + is_preferred: fix.map(|fix| fix.applicability().is_safe()), }) .ok() }) diff --git a/crates/ruff_server/src/server.rs b/crates/ruff_server/src/server.rs index c02b008a71..c046266d44 100644 --- a/crates/ruff_server/src/server.rs +++ b/crates/ruff_server/src/server.rs @@ -268,8 +268,7 @@ impl Server { #[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] pub(crate) enum SupportedCodeAction { /// Maps to the `quickfix` code action kind. Quick fix code actions are shown under - /// their respective diagnostics. Quick fixes are only created where the fix applicability is - /// at least [`ruff_diagnostics::Applicability::Unsafe`]. + /// their respective diagnostics, including display-only fixes that require manual review. QuickFix, /// Maps to the `source.fixAll` and `source.fixAll.ruff` code action kinds. /// This is a source action that applies all safe fixes to the currently open document. diff --git a/crates/ruff_server/src/server/api/requests/code_action.rs b/crates/ruff_server/src/server/api/requests/code_action.rs index fedf2239bd..9e15f6c0e5 100644 --- a/crates/ruff_server/src/server/api/requests/code_action.rs +++ b/crates/ruff_server/src/server/api/requests/code_action.rs @@ -152,6 +152,7 @@ fn quick_fix( data: Some( serde_json::to_value(document_uri).expect("document uri should serialize"), ), + is_preferred: fix.is_preferred, ..Default::default() })) }) diff --git a/crates/ruff_server/tests/e2e/code_action.rs b/crates/ruff_server/tests/e2e/code_action.rs index 3952c248c2..89ee2e1e84 100644 --- a/crates/ruff_server/tests/e2e/code_action.rs +++ b/crates/ruff_server/tests/e2e/code_action.rs @@ -135,6 +135,7 @@ extend-select = ["F401"] "tags": [] } ], + "isPreferred": true, "edit": { "changes": { "file:///ruff.toml": [ diff --git a/crates/ruff_server/tests/e2e/diagnostics.rs b/crates/ruff_server/tests/e2e/diagnostics.rs index be6cf48ee6..da2286bab9 100644 --- a/crates/ruff_server/tests/e2e/diagnostics.rs +++ b/crates/ruff_server/tests/e2e/diagnostics.rs @@ -57,6 +57,7 @@ fn uses_human_readable_names_in_preview() -> Result<()> { } } ], + "is_preferred": true, "noqa_edit": { "newText": " # ruff: ignore[unused-import]\n", "range": { @@ -138,6 +139,7 @@ extend-select = ["F401"] } } ], + "is_preferred": true, "noqa_edit": null, "title": "Replace rule code with `unused-import`" } diff --git a/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__notebook_without_ipynb_extension_open.snap b/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__notebook_without_ipynb_extension_open.snap index 60aa2d33ef..67c2f442ca 100644 --- a/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__notebook_without_ipynb_extension_open.snap +++ b/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__notebook_without_ipynb_extension_open.snap @@ -40,6 +40,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: I002\n", "range": { @@ -78,6 +79,7 @@ expression: diagnostics "data": { "code": "RUF900", "edits": [], + "is_preferred": null, "noqa_edit": { "newText": " # noqa: RUF900\n", "range": { @@ -130,6 +132,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: RUF901\n", "range": { @@ -182,6 +185,7 @@ expression: diagnostics } } ], + "is_preferred": false, "noqa_edit": { "newText": " # noqa: RUF902\n", "range": { @@ -219,7 +223,22 @@ expression: diagnostics "tags": [], "data": { "code": "RUF903", - "edits": [], + "edits": [ + { + "newText": "# fix from stable-test-rule-display-only-fix\n", + "range": { + "end": { + "character": 0, + "line": 0 + }, + "start": { + "character": 0, + "line": 0 + } + } + } + ], + "is_preferred": false, "noqa_edit": { "newText": " # noqa: RUF903\n", "range": { @@ -233,7 +252,7 @@ expression: diagnostics } } }, - "title": "stable-test-rule-display-only-fix" + "title": "stable-test-rule-display-only-fix (suggestion)" } }, { @@ -258,6 +277,7 @@ expression: diagnostics "data": { "code": "RUF950", "edits": [], + "is_preferred": null, "noqa_edit": { "newText": " # noqa: RUF950\n", "range": { @@ -313,6 +333,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: I001\n", "range": { @@ -367,6 +388,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -423,6 +445,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -475,6 +498,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -527,6 +551,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: F541\n", "range": { diff --git a/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__super_resolution_overview_final.snap b/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__super_resolution_overview_final.snap index 62e7b5f7ce..4ce8487c07 100644 --- a/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__super_resolution_overview_final.snap +++ b/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__super_resolution_overview_final.snap @@ -40,6 +40,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: I002\n", "range": { @@ -78,6 +79,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" "data": { "code": "RUF900", "edits": [], + "is_preferred": null, "noqa_edit": { "newText": " # noqa: RUF900\n", "range": { @@ -130,6 +132,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: RUF901\n", "range": { @@ -182,6 +185,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } ], + "is_preferred": false, "noqa_edit": { "newText": " # noqa: RUF902\n", "range": { @@ -219,7 +223,22 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" "tags": [], "data": { "code": "RUF903", - "edits": [], + "edits": [ + { + "newText": "# fix from stable-test-rule-display-only-fix\n", + "range": { + "end": { + "character": 0, + "line": 0 + }, + "start": { + "character": 0, + "line": 0 + } + } + } + ], + "is_preferred": false, "noqa_edit": { "newText": " # noqa: RUF903\n", "range": { @@ -233,7 +252,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } }, - "title": "stable-test-rule-display-only-fix" + "title": "stable-test-rule-display-only-fix (suggestion)" } }, { @@ -258,6 +277,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" "data": { "code": "RUF950", "edits": [], + "is_preferred": null, "noqa_edit": { "newText": " # noqa: RUF950\n", "range": { @@ -313,6 +333,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: I001\n", "range": { @@ -367,6 +388,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -423,6 +445,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -475,6 +498,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -527,6 +551,7 @@ expression: "final_diagnostics.expect(\"at least one notebook change\")" } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: F541\n", "range": { diff --git a/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__super_resolution_overview_open.snap b/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__super_resolution_overview_open.snap index 60aa2d33ef..67c2f442ca 100644 --- a/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__super_resolution_overview_open.snap +++ b/crates/ruff_server/tests/e2e/snapshots/e2e__notebook__super_resolution_overview_open.snap @@ -40,6 +40,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: I002\n", "range": { @@ -78,6 +79,7 @@ expression: diagnostics "data": { "code": "RUF900", "edits": [], + "is_preferred": null, "noqa_edit": { "newText": " # noqa: RUF900\n", "range": { @@ -130,6 +132,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: RUF901\n", "range": { @@ -182,6 +185,7 @@ expression: diagnostics } } ], + "is_preferred": false, "noqa_edit": { "newText": " # noqa: RUF902\n", "range": { @@ -219,7 +223,22 @@ expression: diagnostics "tags": [], "data": { "code": "RUF903", - "edits": [], + "edits": [ + { + "newText": "# fix from stable-test-rule-display-only-fix\n", + "range": { + "end": { + "character": 0, + "line": 0 + }, + "start": { + "character": 0, + "line": 0 + } + } + } + ], + "is_preferred": false, "noqa_edit": { "newText": " # noqa: RUF903\n", "range": { @@ -233,7 +252,7 @@ expression: diagnostics } } }, - "title": "stable-test-rule-display-only-fix" + "title": "stable-test-rule-display-only-fix (suggestion)" } }, { @@ -258,6 +277,7 @@ expression: diagnostics "data": { "code": "RUF950", "edits": [], + "is_preferred": null, "noqa_edit": { "newText": " # noqa: RUF950\n", "range": { @@ -313,6 +333,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: I001\n", "range": { @@ -367,6 +388,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -423,6 +445,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -475,6 +498,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: E703\n", "range": { @@ -527,6 +551,7 @@ expression: diagnostics } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: F541\n", "range": { diff --git a/crates/ruff_server/tests/e2e/workspace.rs b/crates/ruff_server/tests/e2e/workspace.rs index 242901fd6b..38405599bf 100644 --- a/crates/ruff_server/tests/e2e/workspace.rs +++ b/crates/ruff_server/tests/e2e/workspace.rs @@ -81,6 +81,7 @@ ignore = ["F401"] } } ], + "is_preferred": true, "noqa_edit": { "newText": " # noqa: F401\n", "range": { From 8e0dbf4bd982547be7ec6a69369107bfca8fe427 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Tue, 18 Aug 2026 21:41:48 +0100 Subject: [PATCH 091/371] [ty] Add an opt-in rule flagging function decorators that transform a non-dynamic type into a dynamic type (#27829) --- .github/ty-ecosystem.toml | 1 + crates/mdtest/src/lib.rs | 14 +- crates/ruff_mdtest/src/lib.rs | 1 + crates/ty/docs/rules.md | 342 +++++++---- .../dynamic-function-decorator-return.md | 93 +++ .../resources/mdtest/attributes.md | 8 +- .../mdtest/call/callables_as_descriptors.md | 1 + .../resources/mdtest/decorators.md | 544 +++++++++++++++++- .../mdtest/exception/control_flow.md | 6 +- .../mdtest/generics/legacy/functions.md | 1 + .../mdtest/generics/pep695/functions.md | 1 + .../ty_python_semantic/src/types/call/bind.rs | 4 + .../src/types/diagnostic.rs | 167 +++++- .../src/types/infer/builder.rs | 43 +- .../src/types/infer/builder/function.rs | 7 +- crates/ty_test/src/lib.rs | 47 ++ ty.schema.json | 10 + 17 files changed, 1151 insertions(+), 139 deletions(-) create mode 100644 crates/ty_python_semantic/resources/lint_docs/dynamic-function-decorator-return.md diff --git a/.github/ty-ecosystem.toml b/.github/ty-ecosystem.toml index 7d3f469aa7..27d933c18c 100644 --- a/.github/ty-ecosystem.toml +++ b/.github/ty-ecosystem.toml @@ -5,6 +5,7 @@ [rules] blanket-ignore-comment = "warn" division-by-zero = "warn" +dynamic-function-decorator-return = "warn" missing-type-argument = "warn" possibly-missing-attribute = "warn" possibly-missing-import = "warn" diff --git a/crates/mdtest/src/lib.rs b/crates/mdtest/src/lib.rs index d7d226bd36..bfc0ba0575 100644 --- a/crates/mdtest/src/lib.rs +++ b/crates/mdtest/src/lib.rs @@ -351,7 +351,6 @@ fn is_update_inline_snapshots_enabled() -> bool { fn apply_snapshot_filters(rendered: &str) -> std::borrow::Cow<'_, str> { static INLINE_SNAPSHOT_PATH_FILTER: std::sync::LazyLock = std::sync::LazyLock::new(|| regex::Regex::new(r#"\\(\w\w|\.|")"#).unwrap()); - INLINE_SNAPSHOT_PATH_FILTER.replace_all(rendered, "/$1") } @@ -361,6 +360,7 @@ pub fn validate_inline_snapshot( test_file: &TestFile<'_>, inline_diagnostics: &[Diagnostic], markdown_edits: &mut Vec, + snapshot_filter: impl Fn(&str) -> String, ) -> Result<(), matcher::FailuresByLine> { let update_snapshots = is_update_inline_snapshots_enabled(); let line_index = line_index(db, test_file.file); @@ -419,8 +419,8 @@ pub fn validate_inline_snapshot( continue; }; - let actual = apply_snapshot_filters(&render_diagnostics(db, tool_name, block_diagnostics)) - .into_owned(); + let rendered = render_diagnostics(db, tool_name, block_diagnostics); + let actual = snapshot_filter(&apply_snapshot_filters(&rendered)); let Some(snapshot_code_block) = code_block.inline_snapshot_block() else { if update_snapshots { @@ -742,6 +742,7 @@ pub fn snapshot_diagnostics( #[cfg(test)] pub(crate) mod tests { + use super::apply_snapshot_filters; use ruff_db::Db; use ruff_db::files::Files; use ruff_db::system::{DbWithTestSystem, System, TestSystem}; @@ -793,4 +794,11 @@ pub(crate) mod tests { #[salsa::db] impl salsa::Database for TestDb {} + + #[test] + fn preserves_site_packages_paths_in_inline_snapshots() { + let rendered = " ::: .venv/lib/python3.10/site-packages/dependency.py:1:5"; + + assert_eq!(apply_snapshot_filters(rendered), rendered); + } } diff --git a/crates/ruff_mdtest/src/lib.rs b/crates/ruff_mdtest/src/lib.rs index f9b570d7bc..aace07529a 100644 --- a/crates/ruff_mdtest/src/lib.rs +++ b/crates/ruff_mdtest/src/lib.rs @@ -172,6 +172,7 @@ fn run_test( test_file, &inline_diagnostics, &mut markdown_edits, + str::to_owned, ) }) { Ok(()) => None, diff --git a/crates/ty/docs/rules.md b/crates/ty/docs/rules.md index 34cc77f88f..fb1466a0c8 100644 --- a/crates/ty/docs/rules.md +++ b/crates/ty/docs/rules.md @@ -8,7 +8,7 @@ Default level: error · Added in 0.0.64 · Related issues · -View source +View source @@ -44,7 +44,7 @@ class Base(ABC): Default level: error · Added in 0.0.13 · Related issues · -View source +View source @@ -90,7 +90,7 @@ class Derived(Base): # error Default level: warn · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -154,7 +154,7 @@ class SubProto(BaseProto, Protocol): Default level: error · Added in 0.0.14 · Related issues · -View source +View source @@ -237,7 +237,7 @@ value = unknown # ty: ignore[unresolved-reference] Default level: error · Added in 0.0.16 · Related issues · -View source +View source @@ -292,7 +292,7 @@ Foo.method() # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -320,7 +320,7 @@ Calling a non-callable object will raise a `TypeError` at runtime. Default level: error · Added in 0.0.7 · Related issues · -View source +View source @@ -355,7 +355,7 @@ def f(x: object): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -389,7 +389,7 @@ a = 1 # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -424,7 +424,7 @@ class C(A, B): ... # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -460,7 +460,7 @@ class B(A): ... # error Default level: error · Added in 0.0.1-alpha.29 · Related issues · -View source +View source @@ -496,7 +496,7 @@ type B = A # error Default level: error · Added in 0.0.15 · Related issues · -View source +View source @@ -533,7 +533,7 @@ class Example: Default level: warn · Added in 0.0.1-alpha.16 · Related issues · -View source +View source @@ -572,7 +572,7 @@ old_func() # error: [deprecated] Default level: ignore · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -605,7 +605,7 @@ false positives it can produce. Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -636,7 +636,7 @@ class B(A, A): ... # error Default level: error · Added in 0.0.1-alpha.12 · Related issues · -View source +View source @@ -673,13 +673,121 @@ class A: # error d: bytes ``` +## `dynamic-function-decorator-return` + + +Default level: ignore · +Added in 0.0.73 · +Related issues · +View source + + + +**What it does** + + +Detects decorator applications that replace a function with `Any` or another [dynamic type]. + +**Why is this bad?** + + +A decorator can replace the function it receives with any object. Type checkers therefore use the +decorator's return type as the type of the decorated function. If the decorator returns `Any` or +`Unknown` (explicitly or implicitly), the original type is lost, along with the type checker's +ability to catch invalid calls and attribute accesses: + +```py +from collections.abc import Callable + + +def untyped_decorator(function: Callable[..., object]): + return function + + +# error: "Decorator returns `Unknown`" +@untyped_decorator +def stringify(value: int) -> str: + return str(value) + + +# No type error is reported, even though `stringify` expects an integer. +stringify("not an integer") +``` + +This rule identifies the point where a decorator erases useful type information, before that +imprecision spreads to every use of the decorated function. It can be especially useful in cases +where the decorator is defined in a third-party library. Whereas linter rules such as +[`ANN201`][ann201] and [`ANN202`][ann202] can complain about missing annotations in your +first-party code, they cannot identify instances where unsound types leak into your code due to +missing type annotations in third-party code installed into `site-packages`. + +**Examples** + + +`third_party_library.py`: + +```py +from collections.abc import Callable + + +def untyped_decorator(function: Callable[..., object]): + return function +``` + +`first_party.py`: + +```py +from third_party_library import untyped_decorator + + +# error: "Decorator returns `Unknown`" +@untyped_decorator +def greet(name: str) -> str: + return f"Hello, {name}!" +``` + +If making a PR to the third-party library to improve their annotations is not possible, fixes for +this diagnostic could include writing your own decorator or introducing a type-safe wrapper: + +```py +from collections.abc import Callable +from typing import TypeVar + +from third_party_library import untyped_decorator + + +FunctionT = TypeVar("FunctionT", bound=Callable[..., object]) + + +def typed_wrapper(f: FunctionT) -> FunctionT: + decorated = untyped_decorator(f) + assert decorated is f + return decorated + + +@typed_wrapper +def greet(name: str) -> str: + return f"Hello, {name}!" +``` + +**Default level** + + +This rule is disabled by default. It is intended for advanced users wanting additional soundness +checks from their type checker, not for users who have just started to use type checkers on their +Python code. + +[ann201]: https://docs.astral.sh/ruff/rules/missing-return-type-undocumented-public-function/ +[ann202]: https://docs.astral.sh/ruff/rules/missing-return-type-private-function/ +[dynamic type]: https://typing.python.org/en/latest/spec/glossary.html#term-dynamic-type + ## `empty-body` Default level: error · Added in 0.0.14 · Related issues · -View source +View source @@ -756,7 +864,7 @@ def foo() -> "intt\b": ... # error Default level: warn · Added in 0.0.50 · Related issues · -View source +View source @@ -796,7 +904,7 @@ def g(value: ~A) -> None: ... # error: [experimental-syntax] Default level: error · Added in 0.0.20 · Related issues · -View source +View source @@ -831,7 +939,7 @@ def my_function() -> int: Default level: error · Added in 0.0.15 · Related issues · -View source +View source @@ -947,7 +1055,7 @@ def test() -> "Literal[5]": Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -983,7 +1091,7 @@ class C(A, B): ... # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1013,7 +1121,7 @@ t[3] # error Default level: warn · Added in 0.0.1-alpha.33 · Related issues · -View source +View source @@ -1050,7 +1158,7 @@ class MyClass: ... Default level: error · Added in 0.0.1-alpha.12 · Related issues · -View source +View source @@ -1151,7 +1259,7 @@ an atypical memory layout. Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1183,7 +1291,7 @@ func("foo") # error: [invalid-argument-type] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1214,7 +1322,7 @@ a: int = "" # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1272,7 +1380,7 @@ C.instance_only_var = 56 # error Default level: error · Added in 0.0.33 · Related issues · -View source +View source @@ -1318,7 +1426,7 @@ class Sub(Base): Default level: error · Added in 0.0.1-alpha.19 · Related issues · -View source +View source @@ -1360,7 +1468,7 @@ asyncio.run(main()) Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1387,7 +1495,7 @@ class A(42): ... # error: [invalid-base] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1417,7 +1525,7 @@ with 1: # error Default level: error · Added in 0.0.12 · Related issues · -View source +View source @@ -1470,7 +1578,7 @@ See: Default level: error · Added in 0.0.13 · Related issues · -View source +View source @@ -1506,7 +1614,7 @@ class A: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1538,7 +1646,7 @@ a: str # error Default level: warn · Added in 0.0.20 · Related issues · -View source +View source @@ -1595,7 +1703,7 @@ class Pet(Enum): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1659,7 +1767,7 @@ This rule corresponds to Ruff's [`except-with-non-exception-classes` (`B030`)](h Default level: error · Added in 0.0.1-alpha.28 · Related issues · -View source +View source @@ -1712,7 +1820,7 @@ class D(A): Default level: error · Added in 0.0.1-alpha.35 · Related issues · -View source +View source @@ -1763,7 +1871,7 @@ class NonFrozenChild(FrozenBase): # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -1812,7 +1920,7 @@ class D(Generic[U, T]): ... # error Default level: error · Added in 0.0.12 · Related issues · -View source +View source @@ -1908,7 +2016,7 @@ a = 20 / 0 # type: ignore Default level: error · Added in 0.0.1-alpha.17 · Related issues · -View source +View source @@ -1956,7 +2064,7 @@ carol = Person(name="Carol", aeg=25) # typo! Default level: warn · Added in 0.0.15 · Related issues · -View source +View source @@ -2018,7 +2126,7 @@ def f(x, y, /): # Python 3.8+ syntax Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2058,7 +2166,7 @@ def f(t: TypeVar("U")): ... # ty: ignore[invalid-type-form] Default level: error · Added in 0.0.18 · Related issues · -View source +View source @@ -2108,7 +2216,7 @@ match object(): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2143,7 +2251,7 @@ class B(metaclass=42): ... # error Default level: error · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -2261,7 +2369,7 @@ Correct use of `@override` is enforced by ty's [`invalid-explicit-override`](#in Default level: error · Added in 0.0.72 · Related issues · -View source +View source @@ -2299,7 +2407,7 @@ from module import missing # error Default level: error · Added in 0.0.1-alpha.19 · Related issues · -View source +View source @@ -2366,7 +2474,7 @@ TypeError: typing.ClassVar[int] is not valid as type argument Default level: warn · Added in 0.0.31 · Related issues · -View source +View source @@ -2414,7 +2522,7 @@ admin[0] # "Alice" Default level: error · Added in 0.0.1-alpha.27 · Related issues · -View source +View source @@ -2452,7 +2560,7 @@ Baz = NewType("Baz", int | str) # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2509,7 +2617,7 @@ def foo(x: int) -> int: ... Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2538,7 +2646,7 @@ def f(a: int = ""): ... # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2574,7 +2682,7 @@ P2 = ParamSpec() # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2610,7 +2718,7 @@ TypeError: Protocols can only inherit from other protocols, got Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2681,7 +2789,7 @@ def g(): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2713,7 +2821,7 @@ def func() -> int: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -2824,7 +2932,7 @@ class C: ... Default level: error · Added in 0.0.10 · Related issues · -View source +View source @@ -2875,7 +2983,7 @@ class MyClass: Default level: error · Added in 0.0.1-alpha.6 · Related issues · -View source +View source @@ -2921,7 +3029,7 @@ InvalidAlias = TypeAliasType("InvalidAlias", list[T], type_params=(list[T],)) # Default level: error · Added in 0.0.1-alpha.29 · Related issues · -View source +View source @@ -2988,7 +3096,7 @@ Bar[int] # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3021,7 +3129,7 @@ TYPE_CHECKING = "" # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3057,7 +3165,7 @@ b: Annotated[int] # error Default level: error · Added in 0.0.1-alpha.11 · Related issues · -View source +View source @@ -3114,7 +3222,7 @@ class C: Default level: error · Added in 0.0.15 · Related issues · -View source +View source @@ -3158,7 +3266,7 @@ def g[U, T: U](): ... # error: [invalid-type-variable-bound] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3215,7 +3323,7 @@ V = TypeVar("V", list[int], int) # valid constrained Type Default level: error · Added in 0.0.16 · Related issues · -View source +View source @@ -3257,7 +3365,7 @@ U = TypeVar("U", int, str, default=bytes) # error: [invalid-type-variable-defau Default level: error · Added in 0.0.28 · Related issues · -View source +View source @@ -3293,7 +3401,7 @@ class Child(Base): Default level: error · Added in 0.0.14 · Related issues · -View source +View source @@ -3336,7 +3444,7 @@ def f(options: dict[str, object]): Default level: error · Added in 0.0.9 · Related issues · -View source +View source @@ -3371,7 +3479,7 @@ class Foo(TypedDict): Default level: error · Added in 0.0.25 · Related issues · -View source +View source @@ -3406,7 +3514,7 @@ def gen() -> Iterator[int]: Default level: error · Added in 0.0.14 · Related issues · -View source +View source @@ -3473,7 +3581,7 @@ def h(arg2: type): Default level: error · Added in 0.0.15 · Related issues · -View source +View source @@ -3523,7 +3631,7 @@ def g(arg: object): Default level: warn · Added in 0.0.30 · Related issues · -View source +View source @@ -3566,7 +3674,7 @@ Movie = TypedDict("Film", {"title": str}) # error: [mismatched-type-name] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3597,7 +3705,7 @@ func() # error Default level: ignore · Added in 0.0.41 · Related issues · -View source +View source @@ -3656,7 +3764,7 @@ class ExplicitChild(Parent): Default level: ignore · Added in 0.0.45 · Related issues · -View source +View source @@ -3695,7 +3803,7 @@ def handle(m: re.Match[str]) -> str: Default level: error · Added in 0.0.1-alpha.20 · Related issues · -View source +View source @@ -3734,7 +3842,7 @@ alice["age"] # KeyError Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3772,7 +3880,7 @@ func("string") # error: [no-matching-overload] Default level: error · Added in 0.0.30 · Related issues · -View source +View source @@ -3810,7 +3918,7 @@ class Sub(Super): ... # error: [non-callable-init-subclass] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3839,7 +3947,7 @@ for i in 34: # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3867,7 +3975,7 @@ Subscripting an object that does not support it will raise a `TypeError` at runt Default level: error · Added in 0.0.1-alpha.29 · Related issues · -View source +View source @@ -3904,7 +4012,7 @@ class B(A): Default level: error · Added in 0.0.16 · Related issues · -View source +View source @@ -3941,7 +4049,7 @@ class B(A): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -3972,7 +4080,7 @@ f(1, x=2) # error Default level: error · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -4003,7 +4111,7 @@ f(x=1) # error Default level: ignore · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -4042,7 +4150,7 @@ A.c # error Default level: warn · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -4081,7 +4189,7 @@ A()[0] # error Default level: ignore · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -4127,7 +4235,7 @@ from module import a # error Default level: warn · Added in 0.0.23 · Related issues · -View source +View source @@ -4159,7 +4267,7 @@ html.parser # error Default level: ignore · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4196,7 +4304,7 @@ print(x) # error Default level: warn · Added in 0.0.60 · Related issues · -View source +View source @@ -4271,7 +4379,7 @@ def test() -> "int": Default level: warn · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4306,7 +4414,7 @@ cast(int, f()) # error Default level: warn · Added in 0.0.18 · Related issues · -View source +View source @@ -4344,7 +4452,7 @@ class C: Default level: error · Added in 0.0.20 · Related issues · -View source +View source @@ -4388,7 +4496,7 @@ class Outer[T]: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4423,7 +4531,7 @@ static_assert(int(2.0 * 3.0) == 6) # error Default level: warn · Added in 0.0.39 · Related issues · -View source +View source @@ -4474,7 +4582,7 @@ Consider using [`functools.total_ordering`][total_ordering] instead, which does Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4508,7 +4616,7 @@ class B(A): ... # error Default level: error · Added in 0.0.1-alpha.30 · Related issues · -View source +View source @@ -4548,7 +4656,7 @@ class F(NamedTuple): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4578,7 +4686,7 @@ f("foo") # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4617,7 +4725,7 @@ def _(x: int): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4675,7 +4783,7 @@ class A: Default level: error · Added in 0.0.20 · Related issues · -View source +View source @@ -4719,7 +4827,7 @@ class C(Generic[T]): Default level: warn · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4748,7 +4856,7 @@ reveal_type(1) # revealed: Literal[1] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4779,7 +4887,7 @@ f(x=1, y=2) # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4812,7 +4920,7 @@ A().foo # error Default level: warn · Added in 0.0.1-alpha.15 · Related issues · -View source +View source @@ -4887,7 +4995,7 @@ def g(): Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4916,7 +5024,7 @@ import foo # error Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -4944,7 +5052,7 @@ print(x) # error Default level: ignore · Added in 0.0.70 · Related issues · -View source +View source @@ -5087,7 +5195,7 @@ Python code. Default level: ignore · Added in 0.0.70 · Related issues · -View source +View source @@ -5230,7 +5338,7 @@ generator boundaries. Default level: warn · Added in 0.0.1-alpha.7 · Related issues · -View source +View source @@ -5277,7 +5385,7 @@ class D(C): ... # error: [unsupported-base] Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -5326,7 +5434,7 @@ b1 < b2 < b1 # error Default level: ignore · Added in 0.0.12 · Related issues · -View source +View source @@ -5373,7 +5481,7 @@ def factory(base: type[Base]) -> type: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source @@ -5406,7 +5514,7 @@ A() + A() # error Default level: warn · Added in 0.0.21 · Related issues · -View source +View source @@ -5526,7 +5634,7 @@ to `false`. Default level: warn · Added in 0.0.1-alpha.22 · Related issues · -View source +View source @@ -5605,7 +5713,7 @@ def foo(x: int | str) -> int | str: Default level: error · Added in 0.0.1-alpha.1 · Related issues · -View source +View source diff --git a/crates/ty_python_semantic/resources/lint_docs/dynamic-function-decorator-return.md b/crates/ty_python_semantic/resources/lint_docs/dynamic-function-decorator-return.md new file mode 100644 index 0000000000..a67055bc35 --- /dev/null +++ b/crates/ty_python_semantic/resources/lint_docs/dynamic-function-decorator-return.md @@ -0,0 +1,93 @@ +## What it does + +Detects decorator applications that replace a function with `Any` or another [dynamic type]. + +## Why is this bad? + +A decorator can replace the function it receives with any object. Type checkers therefore use the +decorator's return type as the type of the decorated function. If the decorator returns `Any` or +`Unknown` (explicitly or implicitly), the original type is lost, along with the type checker's +ability to catch invalid calls and attribute accesses: + +```py +from collections.abc import Callable + + +def untyped_decorator(function: Callable[..., object]): + return function + + +# error: "Decorator returns `Unknown`" +@untyped_decorator +def stringify(value: int) -> str: + return str(value) + + +# No type error is reported, even though `stringify` expects an integer. +stringify("not an integer") +``` + +This rule identifies the point where a decorator erases useful type information, before that +imprecision spreads to every use of the decorated function. It can be especially useful in cases +where the decorator is defined in a third-party library. Whereas linter rules such as +[`ANN201`][ann201] and [`ANN202`][ann202] can complain about missing annotations in your +first-party code, they cannot identify instances where unsound types leak into your code due to +missing type annotations in third-party code installed into `site-packages`. + +## Examples + +`third_party_library.py`: + +```py +from collections.abc import Callable + + +def untyped_decorator(function: Callable[..., object]): + return function +``` + +`first_party.py`: + +```py +from third_party_library import untyped_decorator + + +# error: "Decorator returns `Unknown`" +@untyped_decorator +def greet(name: str) -> str: + return f"Hello, {name}!" +``` + +If making a PR to the third-party library to improve their annotations is not possible, fixes for +this diagnostic could include writing your own decorator or introducing a type-safe wrapper: + +```py +from collections.abc import Callable +from typing import TypeVar + +from third_party_library import untyped_decorator + + +FunctionT = TypeVar("FunctionT", bound=Callable[..., object]) + + +def typed_wrapper(f: FunctionT) -> FunctionT: + decorated = untyped_decorator(f) + assert decorated is f + return decorated + + +@typed_wrapper +def greet(name: str) -> str: + return f"Hello, {name}!" +``` + +## Default level + +This rule is disabled by default. It is intended for advanced users wanting additional soundness +checks from their type checker, not for users who have just started to use type checkers on their +Python code. + +[ann201]: https://docs.astral.sh/ruff/rules/missing-return-type-undocumented-public-function/ +[ann202]: https://docs.astral.sh/ruff/rules/missing-return-type-private-function/ +[dynamic type]: https://typing.python.org/en/latest/spec/glossary.html#term-dynamic-type diff --git a/crates/ty_python_semantic/resources/mdtest/attributes.md b/crates/ty_python_semantic/resources/mdtest/attributes.md index 4f9fb45be7..b64b7ae96d 100644 --- a/crates/ty_python_semantic/resources/mdtest/attributes.md +++ b/crates/ty_python_semantic/resources/mdtest/attributes.md @@ -971,7 +971,11 @@ reveal_type(D().x) # revealed: Unknown If `staticmethod` is something else, that should not influence the behavior: ```py -def staticmethod(f): +from typing import TypeVar + +T = TypeVar("T") + +def staticmethod(f: T) -> T: return f class C: @@ -4777,6 +4781,7 @@ declarations. from unknown_library import unknown_decorator class C: + # error: [dynamic-function-decorator-return] @unknown_decorator def f(self): self.x: int = 1 @@ -4789,6 +4794,7 @@ class D: def __init__(self): self.x: int = 1 + # error: [dynamic-function-decorator-return] @unknown_decorator def f(self): self.x = 2 diff --git a/crates/ty_python_semantic/resources/mdtest/call/callables_as_descriptors.md b/crates/ty_python_semantic/resources/mdtest/call/callables_as_descriptors.md index e702bad953..c4c61662b1 100644 --- a/crates/ty_python_semantic/resources/mdtest/call/callables_as_descriptors.md +++ b/crates/ty_python_semantic/resources/mdtest/call/callables_as_descriptors.md @@ -237,6 +237,7 @@ except* Exception: unknown_decorator: Any +# error: [dynamic-function-decorator-return] @unknown_decorator # error: [unresolved-reference] def decorated(argument: lambda: decorated, /): # error: [invalid-type-form] pass diff --git a/crates/ty_python_semantic/resources/mdtest/decorators.md b/crates/ty_python_semantic/resources/mdtest/decorators.md index dcfed4f5a2..eadbe05a16 100644 --- a/crates/ty_python_semantic/resources/mdtest/decorators.md +++ b/crates/ty_python_semantic/resources/mdtest/decorators.md @@ -62,8 +62,12 @@ Decorator expressions can also introduce bindings that remain visible after the definition: ```py -def decorator_factory(flag: bool): - def decorator(func): +from typing import TypeVar + +T = TypeVar("T") + +def decorator_factory(flag: bool) -> Callable[[T], T]: + def decorator(func: T) -> T: return func return decorator @@ -206,6 +210,8 @@ reveal_type(Box[int]().values) # revealed: list[int] | None ## Lambdas as decorators ```py +# TODO: infer the `lambda` as a generic function and avoid the false-positive diagnostic here: +# error: [dynamic-function-decorator-return] @lambda f: f def g(x: int) -> str: return "a" @@ -220,6 +226,7 @@ reveal_type(g) # revealed: Unknown ```py # error: [unresolved-reference] "Name `unknown_decorator` used when not defined" +# error: [dynamic-function-decorator-return] @unknown_decorator def f(x): ... @@ -230,6 +237,7 @@ reveal_type(f) # revealed: Unknown ```py # error: [unsupported-operator] +# error: [dynamic-function-decorator-return] @(1 + "a") def f(x): ... @@ -768,3 +776,535 @@ class RegisteredIdentity: reveal_type(RegisteredIdentity.resource.fetch()) # revealed: str ``` + +## Dynamic function decorator returns + +### Basics + +A decorator that returns `Any` erases the original function's signature. Unannotated decorators have +the same effect because their `Unknown` return type is equivalent to `Any`. Our opt-in diagnostic +`dynamic-function-decorator-return` identifies and flags these cases, which will be undesirable for +users who want strict typing enforced on their codebases: + +```py +from typing import Any, Callable + +def returns_any(function: Callable[..., object]) -> Any: + return function + +# snapshot: dynamic-function-decorator-return +@returns_any +def fully_typed(value: int) -> str: + return str(value) + +reveal_type(fully_typed) # revealed: Any +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Any` + --> src/mdtest_snippet.py:7:1 + | +7 | @returns_any + | ^^^^^^^^^^^^ +8 | def fully_typed(value: int) -> str: + | ----------- Signature of `fully_typed` will be obscured by the decorator + | + ::: src/mdtest_snippet.py:3:5 + | +3 | def returns_any(function: Callable[..., object]) -> Any: + | --------------------------------------------------- `returns_any` defined here +``` + +### Dynamic decorators implemented by callable instances + +A callable-instance decorator points to its `__call__` method and suggests adding a return +annotation when that method is unannotated. + +```py +class CallableDecorator: + def __call__(self, function: object): + return function + +# snapshot: dynamic-function-decorator-return +@CallableDecorator() +def decorated(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Unknown` + --> src/mdtest_snippet.py:6:1 + | +6 | @CallableDecorator() + | ^^^^^^^^^^^^^^^^^^^^ +7 | def decorated(value: int) -> str: + | --------- Signature of `decorated` will be obscured by the decorator + | + ::: src/mdtest_snippet.py:2:9 + | +2 | def __call__(self, function: object): + | -------------------------------- `CallableDecorator.__call__` defined here +help: Add a return type annotation to `CallableDecorator.__call__` +``` + +### Dynamic decorators on overloaded function implementations + +A dynamic decorator on an overload implementation does not obscure the externally visible overload +signatures, so it does not trigger `dynamic-function-decorator-return`: + +```py +from collections.abc import Callable +from typing import Any, overload + +def dynamic(function: Callable[..., object]) -> Any: + return function + +@overload +def decorated(value: int) -> int: ... +@overload +def decorated(value: str) -> str: ... +@dynamic +def decorated(value: int | str) -> int | str: + return value + +reveal_type(decorated) # revealed: Overload[(value: int) -> int, (value: str) -> str] +reveal_type(decorated(1)) # revealed: int +reveal_type(decorated("hello")) # revealed: str +``` + +### Subdiagnostics suggest adding annotations, where appropriate + +Decorators imported from another first-party module point to their definition. If the diagnostic was +triggered due to a missing return-type annotation, we suggest adding one: + +`decorator.py`: + +```py +def dynamic(value: object): + return value +``` + +`main.py`: + +```py +from decorator import dynamic + +# snapshot: dynamic-function-decorator-return +@dynamic +def decorated(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Unknown` + --> src/main.py:4:1 + | +4 | @dynamic + | ^^^^^^^^ +5 | def decorated(value: int) -> str: + | --------- Signature of `decorated` will be obscured by the decorator + | + ::: src/decorator.py:1:5 + | +1 | def dynamic(value: object): + | ---------------------- `dynamic` defined here +help: Add a return type annotation to `dynamic` +``` + +But we refrain from suggesting the user add a return annotation to the implementation of an +overloaded decorator function: the return annotation of the implementation is irrelevant to the +diagnostic in the following example: + +```py +from typing import overload, Callable, Any + +@overload +def overloaded_dynamic(function: Callable[..., object]) -> Any: ... +@overload +def overloaded_dynamic(function: None) -> None: ... +def overloaded_dynamic(function): + return function + +# snapshot: dynamic-function-decorator-return +@overloaded_dynamic +def decorated2(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Any` + --> src/mdtest_snippet.py:11:1 + | +11 | @overloaded_dynamic + | ^^^^^^^^^^^^^^^^^^^ +12 | def decorated2(value: int) -> str: + | ---------- Signature of `decorated2` will be obscured by the decorator + | + ::: src/mdtest_snippet.py:4:5 + | + 4 | def overloaded_dynamic(function: Callable[..., object]) -> Any: ... + | ---------------------------------------------------------- Matching overload defined here +``` + +### Dynamic decorators use the matched overload definition(s) + +The definition annotation points to the overload selected by the implicit decorator call, even when +that overload is not the first declaration. + +```py +from typing import overload + +@overload +def dynamic(function, extra): ... +@overload +def dynamic(function): ... +def dynamic(function, extra=None): + return function + +# snapshot: dynamic-function-decorator-return +@dynamic +def decorated(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Unknown` + --> src/mdtest_snippet.py:11:1 + | +11 | @dynamic + | ^^^^^^^^ +12 | def decorated(value: int) -> str: + | --------- Signature of `decorated` will be obscured by the decorator + | + ::: src/mdtest_snippet.py:6:5 + | + 6 | def dynamic(function): ... + | ----------------- Matching overload defined here +help: Add a return type annotation to `dynamic` +``` + +When multiple overloads match, the definition annotation spans every overload: + +```py +from collections.abc import Callable +from typing import Any, overload + +@overload +def dynamic(function: Callable[[int], object]) -> Any: ... +@overload +def dynamic(function: None) -> None: ... +@overload +def dynamic(function: Callable[[str], object]): ... +def dynamic(function): + return function + +# snapshot: dynamic-function-decorator-return +@dynamic +def decorated(value: Any) -> object: + return value +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Any` + --> src/mdtest_snippet.py:27:1 + | +27 | @dynamic + | ^^^^^^^^ +28 | def decorated(value: Any) -> object: + | --------- Signature of `decorated` will be obscured by the decorator + | + ::: src/mdtest_snippet.py:17:1 + | +17 | / @overload +18 | | def dynamic(function: Callable[[int], object]) -> Any: ... +19 | | @overload +20 | | def dynamic(function: None) -> None: ... +21 | | @overload +22 | | def dynamic(function: Callable[[str], object]): ... + | |______________________________________________- Overloads of `dynamic` defined here +help: Ensure all `dynamic` overloads have a return annotation +``` + +### Fully annotated decorators with multiple matching overloads + +Overload ambiguity can produce `Unknown` even when every overload already has a return annotation. +In that case, we do not suggest adding annotations that already exist. + +```py +from collections.abc import Callable +from typing import Any, overload + +@overload +def dynamic(function: Callable[[int], object]) -> int: ... +@overload +def dynamic(function: None) -> None: ... +@overload +def dynamic(function: Callable[[str], object]) -> str: ... +def dynamic(function): + return function + +# snapshot: dynamic-function-decorator-return +@dynamic +def decorated(value: Any) -> object: + return value +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Unknown` + --> src/mdtest_snippet.py:14:1 + | +14 | @dynamic + | ^^^^^^^^ +15 | def decorated(value: Any) -> object: + | --------- Signature of `decorated` will be obscured by the decorator + | + ::: src/mdtest_snippet.py:4:1 + | + 4 | / @overload + 5 | | def dynamic(function: Callable[[int], object]) -> int: ... + 6 | | @overload + 7 | | def dynamic(function: None) -> None: ... + 8 | | @overload + 9 | | def dynamic(function: Callable[[str], object]) -> str: ... + | |_____________________________________________________- Overloads of `dynamic` defined here +``` + +### Dynamic decorators defined in third-party packages + +A decorator from a dependency still points to its definition, but ty does not suggest editing code +outside the current project. + +```toml +[environment] +python = "/.venv" +``` + +`/.venv//dependency.py`: + +```py +def dynamic(value: object): + return value +``` + +`main.py`: + +```py +from dependency import dynamic + +# snapshot: dynamic-function-decorator-return +@dynamic +def decorated(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Unknown` + --> src/main.py:4:1 + | +4 | @dynamic + | ^^^^^^^^ +5 | def decorated(value: int) -> str: + | --------- Signature of `decorated` will be obscured by the decorator + | + ::: .venv//dependency.py:1:5 + | +1 | def dynamic(value: object): + | ---------------------- `dynamic` defined here +``` + +### Edge case: dynamic decorators defined in non-module scripts + +Decorators defined in the checked file receive a suggestion to add a return-type annotation even +when the filename is not a valid Python module name. (`typed-script.py` cannot be resolved to a +valid Python module by our module resolver, so cannot be recognised as having a "first-party search +path", but we nonetheless recognise it as a first-party file and offer the suggestion.) + +`typed-script.py`: + +```py +def dynamic(function: object): + return function + +# snapshot: dynamic-function-decorator-return +@dynamic +def decorated(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Unknown` + --> src/typed-script.py:5:1 + | +5 | @dynamic + | ^^^^^^^^ +6 | def decorated(value: int) -> str: + | --------- Signature of `decorated` will be obscured by the decorator + | + ::: src/typed-script.py:1:5 + | +1 | def dynamic(function: object): + | ------------------------- `dynamic` defined here +help: Add a return type annotation to `dynamic` +``` + +### Edge case: the decorator has a union type + +This comes up very rarely, so for simplicity's sake we just don't have any secondary annotations +here: + +```py +from collections.abc import Callable +from typing import Any + +def annotated_dynamic(function: Callable[..., object]) -> Any: + return function + +def unannotated_dynamic(function: Callable[..., object]): + return function + +def condition() -> bool: + return True + +decorator = annotated_dynamic if condition() else unannotated_dynamic + +# snapshot: dynamic-function-decorator-return +@decorator +def decorated(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Any` + --> src/mdtest_snippet.py:16:1 + | +16 | @decorator + | ^^^^^^^^^^ +17 | def decorated(value: int) -> str: + | --------- Signature of `decorated` will be obscured by the decorator +``` + +### Edge case: the decorator is a union of an overloaded function and `Callable` + +When one union member is an overloaded function and another is a `Callable`, the latter's bindings +must not be attributed to the overloaded function. + +```py +from typing import Any, Callable, overload + +@overload +def overloaded_dynamic(value: None): ... +@overload +def overloaded_dynamic(value: Callable[[int], str]) -> Any: ... +def overloaded_dynamic(value: object) -> Any: + return value + +def apply_decorator(flag: bool, other: Callable[[Callable[..., object]], Any]) -> None: + decorator = overloaded_dynamic if flag else other + + # snapshot: dynamic-function-decorator-return + @decorator + def decorated(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Any` + --> src/mdtest_snippet.py:14:5 + | +14 | @decorator + | ^^^^^^^^^^ +15 | def decorated(value: int) -> str: + | --------- Signature of `decorated` will be obscured by the decorator +``` + +### Multiple dynamic decorators + +Only the first decorator that replaces a precise type with a dynamic type is reported. Outer +decorators receive an already-dynamic type, so they do not lose any additional information. + +```py +from typing import Any + +def dynamic(value: Any) -> Any: + return value + +# no error for this outer decorator (the received type was already `Any`) +@dynamic +# error: [dynamic-function-decorator-return] +@dynamic +def decorated_function(value: int) -> str: + return str(value) +``` + +### Dynamic decorators applied to replacement values + +When an inner decorator replaces a function with a non-callable value, an outer dynamic decorator +obscures that replacement value's type rather than the original function signature. + +```py +from collections.abc import Callable +from typing import Any + +def replace_with_int(function: Callable[..., object]) -> int: + return 1 + +def dynamic(value: int) -> Any: + return value + +# snapshot: dynamic-function-decorator-return +@dynamic +@replace_with_int +def decorated(value: int) -> str: + return str(value) +``` + +```snapshot +info[dynamic-function-decorator-return]: Decorator returns `Any` + --> src/mdtest_snippet.py:11:1 + | +11 | @dynamic + | ^^^^^^^^ +12 | @replace_with_int +13 | def decorated(value: int) -> str: + | --------- Previous type of `decorated` will be obscured by the decorator + | + ::: src/mdtest_snippet.py:7:5 + | + 7 | def dynamic(value: int) -> Any: + | -------------------------- `dynamic` defined here +``` + +### Decorator return types equivalent to `Any` + +Aliases of `Any` erase the decorated type in the same way as a direct `Any` annotation. + +```py +from typing import Any, TypeAlias + +DynamicAlias: TypeAlias = Any + +def returns_alias(value: object) -> DynamicAlias: + return value + +# error: [dynamic-function-decorator-return] +@returns_alias +def decorated_function(value: int) -> str: + return str(value) +``` + +### Partially dynamic decorator returns + +A decorator does not erase all information when its return type merely contains `Any`. Such a type +is not equivalent to `Any`, so the decorator is not reported. + +```py +from collections.abc import Callable +from typing import Any + +def returns_callable(function: Callable[..., object]) -> Callable[..., Any]: + return function + +@returns_callable +def decorated_function(value: int) -> str: + return str(value) +``` diff --git a/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md b/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md index ae36f031ba..573b07c444 100644 --- a/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md +++ b/crates/ty_python_semantic/resources/mdtest/exception/control_flow.md @@ -931,7 +931,9 @@ def class_decorator_raises(decorator) -> None: A function decorator is applied after its parameter defaults have been evaluated: ```py -def function_decorator_raises(decorator) -> None: +from typing import Any, Callable + +def function_decorator_raises(decorator: Callable[[Any], None]) -> None: state = 0 try: @decorator @@ -945,7 +947,7 @@ def function_decorator_raises(decorator) -> None: Decorator application can also raise when the function has no parameter defaults: ```py -def function_decorator_without_defaults(decorator) -> None: +def function_decorator_without_defaults(decorator: Callable[[Any], None]) -> None: caught = False try: @decorator diff --git a/crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md b/crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md index 9008b40c94..e889e8ac1c 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/legacy/functions.md @@ -985,6 +985,7 @@ def opaque_decorator(f: Any) -> Any: def transparent_decorator(f: F) -> F: return f +# error: [dynamic-function-decorator-return] @opaque_decorator def decorated(t: T) -> None: # error: [redundant-cast] diff --git a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md index 924622934c..9ceacf1138 100644 --- a/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md +++ b/crates/ty_python_semantic/resources/mdtest/generics/pep695/functions.md @@ -1274,6 +1274,7 @@ def opaque_decorator(f: Any) -> Any: def transparent_decorator[F: Callable[..., Any]](f: F) -> F: return f +# error: [dynamic-function-decorator-return] @opaque_decorator def decorated[T](t: T) -> None: # error: [redundant-cast] diff --git a/crates/ty_python_semantic/src/types/call/bind.rs b/crates/ty_python_semantic/src/types/call/bind.rs index b50442a49e..6d47ac208d 100644 --- a/crates/ty_python_semantic/src/types/call/bind.rs +++ b/crates/ty_python_semantic/src/types/call/bind.rs @@ -8332,6 +8332,10 @@ impl<'db> CallableDescription<'db> { Self::new_with_settings(db, callable_type, None) } + pub(crate) fn name(&self) -> &str { + &self.name + } + fn new_with_settings( db: &'db dyn Db, callable_type: Type<'db>, diff --git a/crates/ty_python_semantic/src/types/diagnostic.rs b/crates/ty_python_semantic/src/types/diagnostic.rs index 04029d9098..70b61f492c 100644 --- a/crates/ty_python_semantic/src/types/diagnostic.rs +++ b/crates/ty_python_semantic/src/types/diagnostic.rs @@ -9,7 +9,8 @@ use crate::diagnostic::{did_you_mean, format_enumeration}; use crate::lint::{Level, LintRegistryBuilder, LintStatus}; use crate::place::{DefinedPlace, Place, place_from_bindings}; use crate::suppression::FileSuppressionId; -use crate::types::call::{CallDiagnosticOverride, CallError}; +use crate::types::call::bind::CallableDescription; +use crate::types::call::{Bindings, CallDiagnosticOverride, CallError}; use crate::types::class::{ CodeGeneratorKind, DisjointBase, DisjointBaseKind, ExpandedClassBaseEntry, MethodDecorator, }; @@ -47,7 +48,7 @@ use ruff_source_file::LineRanges; use ruff_text_size::{Ranged, TextRange}; use rustc_hash::{FxHashMap, FxHashSet}; use std::fmt::{self, Formatter}; -use ty_module_resolver::{KnownModule, Module, ModuleName, file_to_module}; +use ty_module_resolver::{KnownModule, Module, ModuleName, SearchPath, file_to_module}; use ty_python_core::definition::{Definition, DefinitionKind}; use ty_python_core::place::{PlaceTable, ScopedPlaceId}; use ty_python_core::{ProgramFile, global_scope, place_table, use_def_map}; @@ -69,6 +70,7 @@ pub(crate) fn register_lints(registry: &mut LintRegistryBuilder) { registry.register_lint(&CYCLIC_TYPE_ALIAS_DEFINITION); registry.register_lint(&DEPRECATED); registry.register_lint(&DIVISION_BY_ZERO); + registry.register_lint(&DYNAMIC_FUNCTION_DECORATOR_RETURN); registry.register_lint(&DUPLICATE_BASE); registry.register_lint(&DUPLICATE_KW_ONLY); registry.register_lint(&DATACLASS_FIELD_ORDER); @@ -429,6 +431,15 @@ declare_lint! { } } +declare_lint! { + #[doc = include_str!("../../resources/lint_docs/dynamic-function-decorator-return.md")] + pub(crate) static DYNAMIC_FUNCTION_DECORATOR_RETURN = { + summary: "detects decorators that replace a function with a dynamic type such as `Any`", + status: LintStatus::stable("0.0.73"), + default_level: Level::Ignore, + } +} + declare_lint! { #[expect(clippy::doc_link_with_quotes)] #[doc = include_str!("../../resources/lint_docs/unsound-return-statement.md")] @@ -2048,6 +2059,158 @@ pub(super) fn report_bad_dunder_delattr_call( } } +pub(super) fn report_dynamic_function_decorator_return<'db>( + context: &InferContext<'db, '_>, + decorator: &ast::Decorator, + decorated_ty: Type<'db>, + decorator_bindings: &Bindings<'db>, + decorated_function: &ast::StmtFunctionDef, + return_ty: Type<'db>, +) { + let Some(builder) = context.report_lint(&DYNAMIC_FUNCTION_DECORATOR_RETURN, decorator) else { + return; + }; + + let db = context.db(); + let env = context.program_environment(); + let returned = return_ty.display(db, env); + + let mut diagnostic = builder.into_diagnostic(format_args!("Decorator returns `{returned}`")); + + let mut secondary_annotation = context.secondary(&decorated_function.name); + + // Special-casing of function-literal types is necessary to workaround + secondary_annotation = if decorated_ty.is_function_literal() + || decorated_ty.try_upcast_to_callable(db, env).is_some() + { + secondary_annotation.message(format_args!( + "Signature of `{}` will be obscured by the decorator", + decorated_function.name.id + )) + } else { + secondary_annotation.message(format_args!( + "Previous type of `{}` will be obscured by the decorator", + decorated_function.name.id + )) + }; + + diagnostic.annotate(secondary_annotation); + + // Union and intersection bindings can refer to different callables, so there is no single + // decorator definition or set of overloads that can be safely highlighted. + let Some(decorator_binding) = decorator_bindings.single_element() else { + return; + }; + + let decorator_function = match decorator_binding.signature_type { + Type::FunctionLiteral(function) => function, + Type::BoundMethod(method) => method.function(db), + _ => return, + }; + + let decorator_definition = decorator_function.definition(db); + let (overloads, _) = decorator_function.overloads_and_implementation(db); + + let mut matching_overloads = + decorator_binding + .matching_overloads() + .filter_map(|(overload_index, binding)| { + let overload_index = binding + .signature + .source_overload_index() + .unwrap_or(overload_index); + overloads.get(overload_index).copied() + }); + + let matched_overload = matching_overloads.next(); + let next_matching_overload = matching_overloads.next(); + let has_multiple_matching_overloads = next_matching_overload.is_some(); + + let definition_span = match (overloads, has_multiple_matching_overloads) { + ([first, .., last], true) => { + let first_span = first.spans(db).decorators_and_header; + let last_span = last.spans(db).decorators_and_header; + match (first_span.range(), last_span.range()) { + (Some(first_range), Some(last_range)) => { + first_span.with_range(first_range.cover(last_range)) + } + _ => decorator_function.spans(db).signature, + } + } + _ => matched_overload + .map(|overload| overload.spans(db).signature) + .unwrap_or_else(|| decorator_function.spans(db).signature), + }; + + let definition_annotation = Annotation::secondary(definition_span); + + let missing_return_annotations = if let Some(matched_overload) = matched_overload { + !matched_overload.has_explicit_return_annotation(db) + || next_matching_overload + .into_iter() + .chain(matching_overloads) + .any(|overload| !overload.has_explicit_return_annotation(db)) + } else { + !decorator_function.has_explicit_return_annotation(db) + }; + + let should_add_hint = missing_return_annotations + && (decorator_definition.file(db) == context.file() + || file_to_module(db, decorator_definition.program_file(db).resolver_file(db)) + .and_then(|module| module.search_path(db)) + .is_some_and(SearchPath::is_first_party)); + + match decorator_definition.name(db) { + Some(name) => { + let decorator_description = + CallableDescription::new(db, Type::FunctionLiteral(decorator_function)); + let name = decorator_description + .as_ref() + .map(CallableDescription::name) + .unwrap_or_else(|| name.as_str()); + + if has_multiple_matching_overloads { + diagnostic.annotate( + definition_annotation + .message(format_args!("Overloads of `{name}` defined here")), + ); + } else if matched_overload.is_some() { + diagnostic + .annotate(definition_annotation.message("Matching overload defined here")); + } else { + diagnostic + .annotate(definition_annotation.message(format_args!("`{name}` defined here"))); + } + if should_add_hint { + if has_multiple_matching_overloads { + diagnostic.help(format_args!( + "Ensure all `{name}` overloads have a return annotation" + )); + } else { + diagnostic.help(format_args!("Add a return type annotation to `{name}`")); + } + } + } + None => { + diagnostic.annotate(definition_annotation.message( + if has_multiple_matching_overloads { + "Decorator overloads defined here" + } else { + "Decorator defined here" + }, + )); + + if should_add_hint { + diagnostic.help(if has_multiple_matching_overloads { + "Ensure all overloads have a return annotation" + } else { + "Add a return type annotation to the decorator" + }); + } + } + } +} + pub(super) fn report_invalid_return_type( context: &InferContext, object_range: impl Ranged, diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index c57dc662a1..032ff92990 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -61,17 +61,17 @@ use crate::types::context::InferContext; use crate::types::dedicated::pydantic; use crate::types::diagnostic::{ self, CALL_NON_CALLABLE, CONFLICTING_DECLARATIONS, CYCLIC_TYPE_ALIAS_DEFINITION, - GeneratorMismatchKind, INEFFECTIVE_FINAL, INVALID_ARGUMENT_TYPE, INVALID_ASSIGNMENT, - INVALID_DECLARATION, INVALID_ENUM_MEMBER_ANNOTATION, INVALID_LEGACY_TYPE_VARIABLE, - INVALID_NEWTYPE, INVALID_PARAMSPEC, INVALID_TYPE_ALIAS_TYPE, INVALID_TYPE_FORM, - INVALID_TYPE_VARIABLE_BOUND, INVALID_TYPE_VARIABLE_CONSTRAINTS, INVALID_TYPE_VARIABLE_DEFAULT, - POSSIBLY_MISSING_IMPLICIT_CALL, POSSIBLY_MISSING_SUBMODULE, TypeCheckDiagnostics, - UNDEFINED_REVEAL, UNRESOLVED_ATTRIBUTE, UNRESOLVED_GLOBAL, UNRESOLVED_REFERENCE, UNSOUND_YIELD, - UNSUPPORTED_OPERATOR, UNUSED_AWAITABLE, YieldKind, + DYNAMIC_FUNCTION_DECORATOR_RETURN, GeneratorMismatchKind, INEFFECTIVE_FINAL, + INVALID_ARGUMENT_TYPE, INVALID_ASSIGNMENT, INVALID_DECLARATION, INVALID_ENUM_MEMBER_ANNOTATION, + INVALID_LEGACY_TYPE_VARIABLE, INVALID_NEWTYPE, INVALID_PARAMSPEC, INVALID_TYPE_ALIAS_TYPE, + INVALID_TYPE_FORM, INVALID_TYPE_VARIABLE_BOUND, INVALID_TYPE_VARIABLE_CONSTRAINTS, + INVALID_TYPE_VARIABLE_DEFAULT, POSSIBLY_MISSING_IMPLICIT_CALL, POSSIBLY_MISSING_SUBMODULE, + TypeCheckDiagnostics, UNDEFINED_REVEAL, UNRESOLVED_ATTRIBUTE, UNRESOLVED_GLOBAL, + UNRESOLVED_REFERENCE, UNSOUND_YIELD, UNSUPPORTED_OPERATOR, UNUSED_AWAITABLE, YieldKind, hint_if_stdlib_attribute_exists_on_other_versions, report_attempted_protocol_instantiation, report_bad_dunder_delattr_call, report_bad_dunder_delete_call, report_call_to_abstract_method, - report_cannot_pop_required_field_on_typed_dict, report_invalid_assignment, - report_invalid_class_match_pattern, report_invalid_exception_caught, + report_cannot_pop_required_field_on_typed_dict, report_dynamic_function_decorator_return, + report_invalid_assignment, report_invalid_class_match_pattern, report_invalid_exception_caught, report_invalid_exception_cause, report_invalid_exception_raised, report_invalid_exception_tuple_caught, report_invalid_generator_yield_type, report_invalid_key_on_typed_dict, report_invalid_match_args_type, @@ -5271,6 +5271,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { decorator_ty: Type<'db>, decorated_ty: Type<'db>, decorator_node: &ast::Decorator, + decorated_function: Option<&ast::StmtFunctionDef>, ) -> Type<'db> { fn propagate_callable_kind<'d>( db: &'d dyn Db, @@ -5377,11 +5378,31 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { // classmethod-like or staticmethod-like). See "Decorating a method with // a `Callable`-typed decorator" in `callables_as_descriptors.md` for the // extended explanation. - propagatable_kind + let inferred_ty = propagatable_kind .and_then(|(kind, provenance)| { propagate_callable_kind(db, env, return_ty, kind, provenance) }) - .unwrap_or(return_ty) + .unwrap_or(return_ty); + + if let Some(decorated_function) = decorated_function + && let Some(decorator_bindings) = decorator_bindings.as_ref() + && self + .context + .is_lint_enabled(&DYNAMIC_FUNCTION_DECORATOR_RETURN) + && inferred_ty.is_equivalent_to(db, env, Type::any()) + && !decorated_ty.is_equivalent_to(db, env, Type::any()) + { + report_dynamic_function_decorator_return( + &self.context, + decorator_node, + decorated_ty, + decorator_bindings, + decorated_function, + inferred_ty, + ); + } + + inferred_ty } #[expect(clippy::too_many_arguments)] diff --git a/crates/ty_python_semantic/src/types/infer/builder/function.rs b/crates/ty_python_semantic/src/types/infer/builder/function.rs index 98b0d99608..bf2779ae5d 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/function.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/function.rs @@ -594,7 +594,12 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { { Type::FunctionLiteral(function.with_deprecated(db, *deprecated)) } else { - self.apply_decorator(*decorator_ty, inferred_ty, decorator_node) + self.apply_decorator( + *decorator_ty, + inferred_ty, + decorator_node, + (!is_decorated_overload_implementation).then_some(function), + ) }; } diff --git a/crates/ty_test/src/lib.rs b/crates/ty_test/src/lib.rs index 7e9c793191..37cd8bf28a 100644 --- a/crates/ty_test/src/lib.rs +++ b/crates/ty_test/src/lib.rs @@ -351,6 +351,7 @@ fn run_test( test_file, &inline_diagnostics, &mut markdown_edits, + |rendered| normalize_site_packages_paths(rendered, python_version), ) }) { Ok(()) => None, @@ -565,6 +566,28 @@ impl std::fmt::Display for ModuleInconsistency<'_> { } } +// Site-packages placeholders are specific to ty's fixtures. Keeping their normalization outside +// the shared mdtest crate avoids rewriting Ruff snapshots or paths in displayed source and messages. +fn normalize_site_packages_paths(rendered: &str, python_version: PythonVersion) -> String { + let unix_site_packages_path = format!("/lib/python{python_version}/site-packages/"); + let mut normalized = String::with_capacity(rendered.len()); + + for line in rendered.split_inclusive('\n') { + let trimmed = line.trim_start(); + + if trimmed.starts_with("--> ") || trimmed.starts_with("::: ") { + let line = line + .replace(&unix_site_packages_path, "//") + .replace("/Lib/site-packages/", "//"); + normalized.push_str(&line); + } else { + normalized.push_str(line); + } + } + + normalized +} + fn expand_site_packages_placeholder( path: &SystemPath, python_version: PythonVersion, @@ -608,8 +631,32 @@ fn parse<'s>( #[cfg(test)] mod tests { + use ruff_python_ast::PythonVersion; use ruff_python_trivia::textwrap::dedent; + #[test] + fn normalizes_site_packages_paths_only_in_diagnostic_locations() { + let rendered = "warning[example]: Invalid value\n\ + --> .venv/lib/python3.10/site-packages/dependency.py:1:5\n\ + |\n\ + 1 | path = \".venv/lib/python3.10/site-packages/dependency.py\"\n\ + |\n\ + ::: .venv/Lib/site-packages/other.py:2:1\n\ + help: Inspect .venv/lib/python3.10/site-packages/dependency.py"; + let expected = "warning[example]: Invalid value\n\ + --> .venv//dependency.py:1:5\n\ + |\n\ + 1 | path = \".venv/lib/python3.10/site-packages/dependency.py\"\n\ + |\n\ + ::: .venv//other.py:2:1\n\ + help: Inspect .venv/lib/python3.10/site-packages/dependency.py"; + + assert_eq!( + super::normalize_site_packages_paths(rendered, PythonVersion::PY310), + expected, + ); + } + #[test] fn multiple_sections_with_dependencies_not_allowed() { let source = dedent( diff --git a/ty.schema.json b/ty.schema.json index 3f95750842..041ffbb4ff 100644 --- a/ty.schema.json +++ b/ty.schema.json @@ -514,6 +514,16 @@ } ] }, + "dynamic-function-decorator-return": { + "title": "detects decorators that replace a function with a dynamic type such as `Any`", + "description": "## What it does\n\nDetects decorator applications that replace a function with `Any` or another [dynamic type].\n\n## Why is this bad?\n\nA decorator can replace the function it receives with any object. Type checkers therefore use the\ndecorator's return type as the type of the decorated function. If the decorator returns `Any` or\n`Unknown` (explicitly or implicitly), the original type is lost, along with the type checker's\nability to catch invalid calls and attribute accesses:\n\n```py\nfrom collections.abc import Callable\n\n\ndef untyped_decorator(function: Callable[..., object]):\n return function\n\n\n# error: \"Decorator returns `Unknown`\"\n@untyped_decorator\ndef stringify(value: int) -> str:\n return str(value)\n\n\n# No type error is reported, even though `stringify` expects an integer.\nstringify(\"not an integer\")\n```\n\nThis rule identifies the point where a decorator erases useful type information, before that\nimprecision spreads to every use of the decorated function. It can be especially useful in cases\nwhere the decorator is defined in a third-party library. Whereas linter rules such as\n[`ANN201`][ann201] and [`ANN202`][ann202] can complain about missing annotations in your\nfirst-party code, they cannot identify instances where unsound types leak into your code due to\nmissing type annotations in third-party code installed into `site-packages`.\n\n## Examples\n\n`third_party_library.py`:\n\n```py\nfrom collections.abc import Callable\n\n\ndef untyped_decorator(function: Callable[..., object]):\n return function\n```\n\n`first_party.py`:\n\n```py\nfrom third_party_library import untyped_decorator\n\n\n# error: \"Decorator returns `Unknown`\"\n@untyped_decorator\ndef greet(name: str) -> str:\n return f\"Hello, {name}!\"\n```\n\nIf making a PR to the third-party library to improve their annotations is not possible, fixes for\nthis diagnostic could include writing your own decorator or introducing a type-safe wrapper:\n\n```py\nfrom collections.abc import Callable\nfrom typing import TypeVar\n\nfrom third_party_library import untyped_decorator\n\n\nFunctionT = TypeVar(\"FunctionT\", bound=Callable[..., object])\n\n\ndef typed_wrapper(f: FunctionT) -> FunctionT:\n decorated = untyped_decorator(f)\n assert decorated is f\n return decorated\n\n\n@typed_wrapper\ndef greet(name: str) -> str:\n return f\"Hello, {name}!\"\n```\n\n## Default level\n\nThis rule is disabled by default. It is intended for advanced users wanting additional soundness\nchecks from their type checker, not for users who have just started to use type checkers on their\nPython code.\n\n[ann201]: https://docs.astral.sh/ruff/rules/missing-return-type-undocumented-public-function/\n[ann202]: https://docs.astral.sh/ruff/rules/missing-return-type-private-function/\n[dynamic type]: https://typing.python.org/en/latest/spec/glossary.html#term-dynamic-type", + "default": "ignore", + "oneOf": [ + { + "$ref": "#/definitions/Level" + } + ] + }, "empty-body": { "title": "detects functions with empty bodies that have a non-`None` return type annotation", "description": "## What it does\n\nDetects functions with empty bodies that have a non-`None` return type annotation.\n\nThe errors reported by this rule have the same motivation as the `invalid-return-type`\nrule. The diagnostic exists as a separate error code to allow users to disable this\nrule while prototyping code. While we strongly recommend enabling this rule if\npossible, users migrating from other type checkers may also find it useful to\ntemporarily disable this rule on some or all of their codebase if they find it\nresults in a large number of diagnostics.\n\n## Why is this bad?\n\nA function with an empty body (containing only `...`, `pass`, or a docstring) will\nimplicitly return `None` at runtime. Returning `None` when the return type is non-`None`\nis unsound, and will lead to ty inferring incorrect types elsewhere.\n\nFunctions with empty bodies are permitted in certain contexts where they serve as\ndeclarations rather than implementations:\n\n- Functions in stub files (`.pyi`)\n- Methods in Protocol classes\n- Abstract methods decorated with `@abstractmethod`\n- Overload declarations decorated with `@overload`\n- Functions in `if TYPE_CHECKING` blocks\n\n## Examples\n\n```python\ndef foo() -> int: ... # error: [empty-body]\n\n\ndef bar() -> str: # error: [empty-body]\n \"\"\"A function that does nothing.\"\"\"\n pass\n```", From 554f5b38bf61eb883237e5f3aa9fcb0038673bdc Mon Sep 17 00:00:00 2001 From: zaniebot Date: Tue, 18 Aug 2026 15:44:19 -0500 Subject: [PATCH 092/371] Pin Node.js for WebAssembly testing and publishing (#27855) The WebAssembly test and npm publication jobs select `node-version: 24`, so the Node.js and bundled `npm` versions can change between runs without a workflow change. Pin both selectors to `24.19.0`, keeping the existing Node major and publication permissions. The existing Renovate GitHub Actions manager continues to track these inputs. Related: astral-sh/ruff#27838, astral-sh/ruff#27839, and astral-sh/ruff#27844 pin the playground runtimes; astral-sh/ruff#27842 covers the `prek`-managed runtime. astral-sh/packse#326 makes the corresponding exact Node.js 24 pin for formatter CI. Co-authored-by: zaniebot <242828183+zaniebot@users.noreply.github.com> --- .github/workflows/ci.yaml | 2 +- .github/workflows/publish-wasm.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 3b2e823123..c09b4c343e 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -499,7 +499,7 @@ jobs: run: rustup target add wasm32-unknown-unknown - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: - node-version: 24 + node-version: 24.19.0 cache: "npm" cache-dependency-path: playground/package-lock.json - uses: jetli/wasm-pack-action@0d096b08b4e5a7de8c28de67e11e945404e9eefa # v0.4.0 diff --git a/.github/workflows/publish-wasm.yml b/.github/workflows/publish-wasm.yml index 3088ae5850..fc292ebaea 100644 --- a/.github/workflows/publish-wasm.yml +++ b/.github/workflows/publish-wasm.yml @@ -29,7 +29,7 @@ jobs: path: pkg - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 with: - node-version: 24 + node-version: 24.19.0 registry-url: "https://registry.npmjs.org" - name: "Publish (dry-run)" if: ${{ inputs.plan == '' || fromJson(inputs.plan).announcement_tag_is_implicit }} From 7ed8d1fbdd29e4bc9a62112cca2427e8b0767e82 Mon Sep 17 00:00:00 2001 From: William Woodruff Date: Tue, 18 Aug 2026 16:56:20 -0400 Subject: [PATCH 093/371] Manage docs build with a dep group (#27843) Co-authored-by: Alex Waygood Signed-off-by: William Woodruff --- .github/workflows/ci.yaml | 13 +- .github/workflows/publish-docs.yml | 13 +- CONTRIBUTING.md | 10 +- crates/ruff_python_formatter/CONTRIBUTING.md | 12 - docs/requirements.txt | 9 - pyproject.toml | 10 + scripts/Dockerfile.ecosystem | 34 - scripts/_utils.py | 26 - scripts/add_plugin.py | 26 +- scripts/add_plugin.py.lock | 7 + scripts/add_rule.py | 32 +- scripts/add_rule.py.lock | 7 + scripts/check_docs_formatted.py | 16 +- scripts/check_docs_formatted.py.lock | 35 + scripts/check_ecosystem.py | 550 -------------- scripts/conformance.py | 8 + scripts/conformance.py.lock | 7 + scripts/ecosystem_all_check.py | 10 +- scripts/ecosystem_all_check.py.lock | 31 + scripts/ecosystem_all_check.sh | 2 +- scripts/generate_builtin_modules.py | 8 + scripts/generate_builtin_modules.py.lock | 7 + scripts/generate_known_standard_library.py | 8 + .../generate_known_standard_library.py.lock | 19 + scripts/generate_mkdocs.py | 12 + scripts/{uv.lock => generate_mkdocs.py.lock} | 174 ++--- scripts/memory_report.py | 8 + scripts/memory_report.py.lock | 7 + scripts/publish-crates.py | 8 + scripts/publish-crates.py.lock | 7 + scripts/pyproject.toml | 26 - scripts/transform_readme.py | 8 + scripts/transform_readme.py.lock | 7 + scripts/ty.toml | 4 + scripts/update_ambiguous_characters.py | 8 + scripts/update_ambiguous_characters.py.lock | 7 + scripts/update_schemastore.py | 10 +- scripts/update_schemastore.py.lock | 7 + uv.lock | 680 ++++++++++++++++++ 39 files changed, 1056 insertions(+), 817 deletions(-) delete mode 100644 docs/requirements.txt delete mode 100644 scripts/Dockerfile.ecosystem delete mode 100644 scripts/_utils.py create mode 100644 scripts/add_plugin.py.lock create mode 100644 scripts/add_rule.py.lock create mode 100644 scripts/check_docs_formatted.py.lock delete mode 100755 scripts/check_ecosystem.py create mode 100644 scripts/conformance.py.lock create mode 100644 scripts/ecosystem_all_check.py.lock create mode 100644 scripts/generate_builtin_modules.py.lock create mode 100644 scripts/generate_known_standard_library.py.lock rename scripts/{uv.lock => generate_mkdocs.py.lock} (53%) create mode 100644 scripts/memory_report.py.lock create mode 100644 scripts/publish-crates.py.lock delete mode 100644 scripts/pyproject.toml create mode 100644 scripts/transform_readme.py.lock create mode 100644 scripts/ty.toml create mode 100644 scripts/update_ambiguous_characters.py.lock create mode 100644 scripts/update_schemastore.py.lock diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index c09b4c343e..474af00487 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -388,8 +388,6 @@ jobs: run: cargo test --doc --all-features - name: Dogfood ty on py-fuzzer run: uv run --project=./python/py-fuzzer cargo run -p ty check --project=./python/py-fuzzer - - name: Dogfood ty on the scripts directory - run: uv run --project=./scripts cargo run -p ty check --project=./scripts - name: Dogfood ty on ty_benchmark run: uv run --project=./scripts/ty_benchmark cargo run -p ty check --project=./scripts/ty_benchmark # Check for broken links in the documentation. @@ -966,18 +964,15 @@ jobs: uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 with: python-version: 3.13 - activate-environment: true version: "0.12.3" - - name: "Install dependencies" - run: uv pip install -r docs/requirements.txt - name: "Update README File" - run: python scripts/transform_readme.py --target mkdocs + run: uv run scripts/transform_readme.py --target mkdocs - name: "Generate docs" - run: python scripts/generate_mkdocs.py + run: uv run scripts/generate_mkdocs.py - name: "Check docs formatting" - run: python scripts/check_docs_formatted.py + run: uv run scripts/check_docs_formatted.py - name: "Build docs" - run: mkdocs build --strict -f mkdocs.yml + run: uv run --only-group=docs mkdocs build --strict -f mkdocs.yml check-formatter-instability-and-black-similarity: name: "formatter instabilities and black similarity" diff --git a/.github/workflows/publish-docs.yml b/.github/workflows/publish-docs.yml index b95f7d0b6e..a4369d836f 100644 --- a/.github/workflows/publish-docs.yml +++ b/.github/workflows/publish-docs.yml @@ -35,9 +35,9 @@ jobs: ref: ${{ inputs.ref }} persist-credentials: true - - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + - uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0 with: - python-version: 3.12 + version: "0.12.3" - name: "Set docs version" env: @@ -71,16 +71,13 @@ jobs: - uses: Swatinem/rust-cache@6323deb102c322ba6fcbdcafc7e3dddab59af2b6 # v2.9.2 - - name: "Install dependencies" - run: pip install -r docs/requirements.txt - - name: "Copy README File" run: | - python scripts/transform_readme.py --target mkdocs - python scripts/generate_mkdocs.py + uv run scripts/transform_readme.py --target mkdocs + uv run scripts/generate_mkdocs.py - name: "Build docs" - run: mkdocs build --strict -f mkdocs.yml + run: uv run --only-group=docs mkdocs build --strict -f mkdocs.yml - name: "Clone docs repo" run: git clone https://${{ secrets.ASTRAL_DOCS_PAT }}@github.com/astral-sh/docs.git astral-docs diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3ebd8d449b..24834dfdfc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -184,14 +184,14 @@ crates.io as part of Ruff's releases: For a publishable crate, generate its README and verify that the workspace can still be packaged: ```shell -uv run --script scripts/generate-crate-readmes.py +uv run scripts/generate-crate-readmes.py cargo publish --workspace --dry-run ``` Before merging a publishable crate, ask a crates.io owner to bootstrap it by running: ```shell -CARGO_REGISTRY_TOKEN= uv run --no-config --script scripts/setup-crates-io-publish.py +CARGO_REGISTRY_TOKEN= uv run --no-config scripts/setup-crates-io-publish.py ``` The bootstrap script reserves the crate name, configures the release workflow as its trusted @@ -468,13 +468,13 @@ To preview any changes to the documentation locally: 1. Generate the MkDocs site with: ```shell - uv run --no-project --isolated --with-requirements docs/requirements.txt scripts/generate_mkdocs.py + uv run scripts/generate_mkdocs.py ``` 1. Run the development server with: ```shell - uvx --with-requirements docs/requirements.txt -- mkdocs serve -f mkdocs.yml + uv run --only-group=docs mkdocs serve -f mkdocs.yml ``` The documentation should then be available locally at @@ -560,7 +560,7 @@ Commit each step of this process separately for easier review. 1. One can determine if an update is needed when `git diff old-version-tag new-version-tag -- ruff.schema.json` returns a non-empty diff. - 1. Run `uv run --only-dev --no-sync scripts/update_schemastore.py --proto ` + 1. Run `uv run scripts/update_schemastore.py --proto ` 1. Once run successfully, you should follow the link in the output to create a PR. 1. Update the [`ruff-vscode`](https://github.com/astral-sh/ruff-vscode) repository by following diff --git a/crates/ruff_python_formatter/CONTRIBUTING.md b/crates/ruff_python_formatter/CONTRIBUTING.md index b30b628c03..111ec6c078 100644 --- a/crates/ruff_python_formatter/CONTRIBUTING.md +++ b/crates/ruff_python_formatter/CONTRIBUTING.md @@ -121,18 +121,6 @@ Available options: - `--stats-file`: Use together with `--multi-project`, this writes the similarity index as unicode table to the given file. -**Large ecosystem checks** It is also possible to check a large number of repositories. This dataset -is large (~60GB), so we only do this occasionally: - -```shell -# Get the list of projects -curl https://raw.githubusercontent.com/akx/ruff-usage-aggregate/master/data/known-github-tomls-clean.jsonl > github_search.jsonl -# Repurpose this script to download the repositories for us -python scripts/check_ecosystem.py --checkouts target/checkouts --projects github_search.jsonl -v $(which true) $(which true) -# Check each project for formatter stability -cargo run --bin ruff_dev -- format-dev --stability-check --error-file target/formatter-ecosystem-errors.txt --multi-project target/checkouts -``` - ## Helper structs To abstract formatting something into a helper, create a new struct with the data you want to diff --git a/docs/requirements.txt b/docs/requirements.txt deleted file mode 100644 index c26757440a..0000000000 --- a/docs/requirements.txt +++ /dev/null @@ -1,9 +0,0 @@ -PyYAML==6.0.3 -ruff==0.16.2 -mkdocs==1.6.1 -mkdocs-material==9.7.7 -mkdocs-redirects==1.2.3 -mdformat==1.0.0 -mdformat-mkdocs==5.3.0 -mkdocs-github-admonitions-plugin @ git+https://github.com/PGijsbers/admonitions.git#7343d2f4a92e4d1491094530ef3d0d02d93afbb7 -mkdocs-llmstxt==0.2.0 diff --git a/pyproject.toml b/pyproject.toml index 3da925242f..dc74635004 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -62,6 +62,14 @@ dev = [ release = [ "rooster==0.1.1", ] +docs = [ + "mkdocs>=1.6.1", + "mkdocs-github-admonitions-plugin>=0.1.1", + "mkdocs-llmstxt>=0.2.0", + "mkdocs-material>=9.7.7", + "mkdocs-redirects>=1.2.3", + "pyyaml>=6.0.3", +] [tool.uv] exclude-newer = "P7D" @@ -69,6 +77,7 @@ exclude-newer = "P7D" [tool.uv.dependency-groups] dev = { requires-python = ">=3.12" } release = { requires-python = ">=3.12" } +docs = { requires-python = ">=3.12" } [tool.ruff] target-version = "py38" @@ -114,6 +123,7 @@ combine-as-imports = true [tool.ruff.per-file-target-version] "crates/ty_python_semantic/mdtest.py" = "py310" "crates/ty_vendored/ty_extensions/*.pyi" = "py312" +"scripts/*.py" = "py312" [tool.black] force-exclude = ''' diff --git a/scripts/Dockerfile.ecosystem b/scripts/Dockerfile.ecosystem deleted file mode 100644 index d671b20c1e..0000000000 --- a/scripts/Dockerfile.ecosystem +++ /dev/null @@ -1,34 +0,0 @@ -# [crater](https://github.com/rust-lang/crater)-inspired check that tests against a large number of -# projects, mainly from https://github.com/akx/ruff-usage-aggregate. -# -# We run this in a Docker container as Ruff isn't designed for untrusted inputs. -# -# Either download https://github.com/akx/ruff-usage-aggregate/blob/master/data/known-github-tomls.jsonl as -# `github_search.jsonl` or follow the instructions in the README to scrape your own dataset. -# -# Setup: -# ``` -# apt-get install musl-tools # or corresponding command to install musl on your platform, e.g. `yay musl` -# rustup target add x86_64-unknown-linux-musl -# ``` -# From the project root: -# ``` -# cargo build --target x86_64-unknown-linux-musl -# docker buildx build -f scripts/Dockerfile.ecosystem -t ruff-ecosystem-checker --load . -# docker run --rm -v ./target/x86_64-unknown-linux-musl/debug/ruff:/app/ruff-new -v ./ruff-old:/app/ruff-old ruff-ecosystem-checker -# ``` -# You can customize this, e.g. cache the git checkouts, a custom json file and a glibc build: -# ``` -# docker run -v ./target/debug/ruff:/app/ruff-new -v ./ruff-old:/app/ruff-old -v ./target/checkouts:/app/checkouts \ -# -v ./github_search.jsonl:/app/github_search.jsonl --rm ruff-ecosystem-checker \ -# python check_ecosystem.py --verbose ruff-new ruff-old --projects github_search.jsonl --checkouts checkouts \ -# > target/ecosystem-ci.txt -# ``` - -FROM python:3.11 -RUN mkdir /app -WORKDIR /app -ADD scripts/check_ecosystem.py check_ecosystem.py -ADD github_search.jsonl github_search.jsonl - -CMD ["python", "check_ecosystem.py", "--verbose", "--projects", "github_search.jsonl", "ruff-new", "ruff-old"] diff --git a/scripts/_utils.py b/scripts/_utils.py deleted file mode 100644 index 6807c74a91..0000000000 --- a/scripts/_utils.py +++ /dev/null @@ -1,26 +0,0 @@ -from __future__ import annotations - -import re -from pathlib import Path - -ROOT_DIR = Path(__file__).resolve().parent.parent - - -def dir_name(linter_name: str) -> str: - return linter_name.replace("-", "_") - - -def pascal_case(linter_name: str) -> str: - """Convert from snake-case to PascalCase.""" - return "".join(word.title() for word in linter_name.split("-")) - - -def snake_case(name: str) -> str: - """Convert from PascalCase to snake_case.""" - return "".join( - f"_{word.lower()}" if word.isupper() else word for word in name - ).lstrip("_") - - -def get_indent(line: str) -> str: - return re.match(r"^\s*", line).group() # type: ignore[union-attr, ty:unresolved-attribute] diff --git a/scripts/add_plugin.py b/scripts/add_plugin.py index d50bcb9f46..e64c9f5a4c 100755 --- a/scripts/add_plugin.py +++ b/scripts/add_plugin.py @@ -1,4 +1,13 @@ #!/usr/bin/env python3 +# +# /// script +# requires-python = ">=3.12" +# dependencies = [] +# +# [tool.uv] +# exclude-newer = "P7D" +# /// + """Generate boilerplate for a new Flake8 plugin. Example usage: @@ -12,8 +21,23 @@ from __future__ import annotations import argparse +import re +from pathlib import Path + +ROOT_DIR = Path(__file__).resolve().parent.parent + + +def dir_name(linter_name: str) -> str: + return linter_name.replace("-", "_") + + +def pascal_case(linter_name: str) -> str: + """Convert from snake-case to PascalCase.""" + return "".join(word.title() for word in linter_name.split("-")) + -from _utils import ROOT_DIR, dir_name, get_indent, pascal_case +def get_indent(line: str) -> str: + return re.match(r"^\s*", line).group() # type: ignore[union-attr, ty:unresolved-attribute] def main(*, plugin: str, url: str, prefix_code: str) -> None: diff --git a/scripts/add_plugin.py.lock b/scripts/add_plugin.py.lock new file mode 100644 index 0000000000..9a5fd10568 --- /dev/null +++ b/scripts/add_plugin.py.lock @@ -0,0 +1,7 @@ +version = 1 +revision = 3 +requires-python = ">=3.12" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" diff --git a/scripts/add_rule.py b/scripts/add_rule.py index 74dbea01ad..37a396c07c 100755 --- a/scripts/add_rule.py +++ b/scripts/add_rule.py @@ -1,4 +1,13 @@ #!/usr/bin/env python3 +# +# /// script +# requires-python = ">=3.12" +# dependencies = [] +# +# [tool.uv] +# exclude-newer = "P7D" +# /// + """Generate boilerplate for a new rule. Example usage: @@ -13,10 +22,31 @@ from __future__ import annotations import argparse +import re import subprocess from pathlib import Path -from _utils import ROOT_DIR, dir_name, get_indent, pascal_case, snake_case +ROOT_DIR = Path(__file__).resolve().parent.parent + + +def dir_name(linter_name: str) -> str: + return linter_name.replace("-", "_") + + +def pascal_case(linter_name: str) -> str: + """Convert from snake-case to PascalCase.""" + return "".join(word.title() for word in linter_name.split("-")) + + +def snake_case(name: str) -> str: + """Convert from PascalCase to snake_case.""" + return "".join( + f"_{word.lower()}" if word.isupper() else word for word in name + ).lstrip("_") + + +def get_indent(line: str) -> str: + return re.match(r"^\s*", line).group() # type: ignore[union-attr, ty:unresolved-attribute] def main(*, name: str, prefix: str, code: str, linter: str) -> None: diff --git a/scripts/add_rule.py.lock b/scripts/add_rule.py.lock new file mode 100644 index 0000000000..9a5fd10568 --- /dev/null +++ b/scripts/add_rule.py.lock @@ -0,0 +1,7 @@ +version = 1 +revision = 3 +requires-python = ">=3.12" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" diff --git a/scripts/check_docs_formatted.py b/scripts/check_docs_formatted.py index e975847491..50c5448d32 100755 --- a/scripts/check_docs_formatted.py +++ b/scripts/check_docs_formatted.py @@ -1,4 +1,13 @@ #!/usr/bin/env python3 +# +# /// script +# requires-python = ">=3.12" +# dependencies = ["ruff"] +# +# [tool.uv] +# exclude-newer = "P7D" +# /// + """Check code snippets in docs are formatted by Ruff.""" from __future__ import annotations @@ -286,15 +295,8 @@ def main(argv: Sequence[str] | None = None) -> int: description="Check code snippets in docs are formatted by Ruff.", ) parser.add_argument("--skip-errors", action="store_true") - parser.add_argument("--generate-docs", action="store_true") args = parser.parse_args(argv) - if args.generate_docs: - # Generate docs - from generate_mkdocs import main as generate_docs - - generate_docs() - # Get static docs static_docs = [Path("docs") / f for f in os.listdir("docs") if f.endswith(".md")] diff --git a/scripts/check_docs_formatted.py.lock b/scripts/check_docs_formatted.py.lock new file mode 100644 index 0000000000..cffb5efc41 --- /dev/null +++ b/scripts/check_docs_formatted.py.lock @@ -0,0 +1,35 @@ +version = 1 +revision = 3 +requires-python = ">=3.12" + +[options] +exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values. +exclude-newer-span = "P7D" + +[manifest] +requirements = [{ name = "ruff" }] + +[[package]] +name = "ruff" +version = "0.16.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/73/e1/4508a569211b35599016e84ba65c1a992b7a4004b4b6c4bea02a851cba1b/ruff-0.16.2.tar.gz", hash = "sha256:c3d7828d12e8927a6fc65fe38e2c2541b9e762d360a1786d752cb1b8883b3c9c", size = 4885811, upload-time = "2026-08-07T13:31:01.432Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/14/57/db19951540f98859c956b50bdb4d31089b4d91e9f15e2968e7d5193806d5/ruff-0.16.2-py3-none-linux_armv6l.whl", hash = "sha256:3c8de4cf2181f01d57946d87d777aa52916976fc09942aed89938fab5e013318", size = 10847925, upload-time = "2026-08-07T13:30:14.468Z" }, + { url = "https://files.pythonhosted.org/packages/13/5a/995fe85a8470d3e391ac0f7fa8054bb454eaf33ee138196d6172ed1079c0/ruff-0.16.2-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:9a48cc05c6fbc811ca81b5d7ba95375affea6582d1b8024e455e41afbbf55344", size = 11072662, upload-time = "2026-08-07T13:30:18.143Z" }, + { url = "https://files.pythonhosted.org/packages/32/53/370d767c61c71a971a4ace36703a7ecd8c393956349a7325d7fab2b56827/ruff-0.16.2-py3-none-macosx_11_0_arm64.whl", hash = "sha256:a2c0d14fcbb26c91f0f867a6dc9bd71bbc30b1b6151829c884f23faeab2e5700", size = 10566771, upload-time = "2026-08-07T13:30:20.899Z" }, + { url = "https://files.pythonhosted.org/packages/85/d6/9d96948caf5a632be62d62202d5ec914d6856f204fd79eb036e5915e79ea/ruff-0.16.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:335c621622c4650330be50842561c6586ac6971bb8ab5407fe34dcc9efb16bbe", size = 10975825, upload-time = "2026-08-07T13:30:23.517Z" }, + { url = "https://files.pythonhosted.org/packages/3b/92/ea87129b3414acb0b5770563779c51804d37ac67675c7ba35447ddb14773/ruff-0.16.2-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:20e66910f2c37cc753f9ef6580c914a621b80c4fa3549d3e3521e29d0f5bfc3f", size = 10649437, upload-time = "2026-08-07T13:30:26.097Z" }, + { url = "https://files.pythonhosted.org/packages/ac/43/f8f291dcd4af5bb7872b74fdfa41a7cd7c856ca1d4069670971cf1b9f5cb/ruff-0.16.2-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c7e36fbfba65510548156902bcf1350a979a958ce0347ce0f90d73894036b39f", size = 11446761, upload-time = "2026-08-07T13:30:28.752Z" }, + { url = "https://files.pythonhosted.org/packages/71/4a/ef991fb2fcf516ab71f0808adcdd8da5e18c8cde447f4ceaf5f47a5132a5/ruff-0.16.2-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f0eab35f80df8f134aae5d1630e751901321d317cc8e50dc39e36fa3ed34cd12", size = 12336364, upload-time = "2026-08-07T13:30:31.468Z" }, + { url = "https://files.pythonhosted.org/packages/f3/24/f615e74f307e6ca0e56a482872477b856c70d530aa356abfb6dfe5ca8a80/ruff-0.16.2-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:40ea8c0594feb894e89c8c61ab9c103d38b0ea72dfde6c594107147ca31b1140", size = 11630720, upload-time = "2026-08-07T13:30:34.426Z" }, + { url = "https://files.pythonhosted.org/packages/c5/d3/8ef50149e8412a77f7ab409efdef0e2b23803707a3863da4fc64cb23d459/ruff-0.16.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ab3d62dde0b19facdd632008cc4827fc28ada7736c6bd35ab6f1050f0bfed53f", size = 11466130, upload-time = "2026-08-07T13:30:36.958Z" }, + { url = "https://files.pythonhosted.org/packages/dd/a7/a19334985c4dea8c381981fa252cd854c7ee52dc4b1686dc16f4a911c702/ruff-0.16.2-py3-none-manylinux_2_31_riscv64.whl", hash = "sha256:e43e1f5b8388da9eca1b9e88328d47a5cec794633ccf6f7484ac2dd15eee92c0", size = 11523634, upload-time = "2026-08-07T13:30:39.822Z" }, + { url = "https://files.pythonhosted.org/packages/6e/6c/96d192b0e742412ceda08c0a50f9669b253dde9fd6a60ea1a10c9fa79a63/ruff-0.16.2-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:c24788a980581e1d7ea3a0cbe4344c4fbeb0a6a9b1f4713aa46bb104f8294690", size = 10949807, upload-time = "2026-08-07T13:30:42.745Z" }, + { url = "https://files.pythonhosted.org/packages/fa/51/e26599ceca11e79ee255c7df515995561edf87e9ca1893284e44d98f5a86/ruff-0.16.2-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:81806b08329130005dd4a8a8394a0c9da8c6f4cafb16ba438d2a2ee6a18bedf1", size = 10646891, upload-time = "2026-08-07T13:30:45.522Z" }, + { url = "https://files.pythonhosted.org/packages/68/01/800c4b1f97bc8d7c6029e06b1f20473a3cf1e13c4933d8f3342add83fc55/ruff-0.16.2-py3-none-musllinux_1_2_i686.whl", hash = "sha256:4ce4e02bad779bef557f541a1b31f20d6abeae1cc05ed1b1ac019d4ffd1044c8", size = 11162063, upload-time = "2026-08-07T13:30:48.131Z" }, + { url = "https://files.pythonhosted.org/packages/e4/d0/1477ea50fc5a0d4b0b71d1d63d50770bdd794d90b43e37a7618e63ec9894/ruff-0.16.2-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:e0422abdf70070255fc4073ce9dfc814cc03db577013761ddd09bc1e4a9a4fbd", size = 11556038, upload-time = "2026-08-07T13:30:50.686Z" }, + { url = "https://files.pythonhosted.org/packages/b8/76/a7776f32048d991e16d4fa8ff91790b877342d3596cc3ed04acdbf1aaedc/ruff-0.16.2-py3-none-win32.whl", hash = "sha256:bf3a63d78fb39f4bf5ac8ae52051c5520505301abe19ba4e204c453b3f09bb0b", size = 10872850, upload-time = "2026-08-07T13:30:53.471Z" }, + { url = "https://files.pythonhosted.org/packages/00/0d/929c800d920e61397d82a01b60bffc68da3052c17d31de59efaad2e4ed75/ruff-0.16.2-py3-none-win_amd64.whl", hash = "sha256:bcabe2f6d0fc7819f1431793005af4e4de7371927d037345bf941252b195b9fa", size = 12023338, upload-time = "2026-08-07T13:30:56.193Z" }, + { url = "https://files.pythonhosted.org/packages/5b/6c/93e26c22c5f78ff87363e07da49c84955affbeb1098bd1936bf3b3f293bf/ruff-0.16.2-py3-none-win_arm64.whl", hash = "sha256:d614e95cedf38a2053fd351c55b103ba30d017d61688fdbfd40ee0412852a99f", size = 11374065, upload-time = "2026-08-07T13:30:58.775Z" }, +] diff --git a/scripts/check_ecosystem.py b/scripts/check_ecosystem.py deleted file mode 100755 index 739d60430a..0000000000 --- a/scripts/check_ecosystem.py +++ /dev/null @@ -1,550 +0,0 @@ -#!/usr/bin/env python3 -""" -**DEPRECATED** This script is being replaced by the ruff-ecosystem package. - - -Check two versions of ruff against a corpus of open-source code. - -Example usage: - - scripts/check_ecosystem.py -""" - -from __future__ import annotations - -import argparse -import asyncio -import difflib -import heapq -import json -import logging -import re -import tempfile -import time -from asyncio.subprocess import PIPE, create_subprocess_exec -from collections.abc import Awaitable -from contextlib import asynccontextmanager, nullcontext -from pathlib import Path -from signal import SIGINT, SIGTERM -from typing import TYPE_CHECKING, NamedTuple, Self, TypeVar - -if TYPE_CHECKING: - from collections.abc import AsyncIterator, Iterator, Sequence - -logger = logging.getLogger(__name__) - - -class Repository(NamedTuple): - """A GitHub repository at a specific ref.""" - - org: str - repo: str - ref: str | None - select: str = "" - ignore: str = "" - exclude: str = "" - # Generating fixes is slow and verbose - show_fixes: bool = False - - @asynccontextmanager - async def clone(self: Self, checkout_dir: Path) -> AsyncIterator[str]: - """Shallow clone this repository to a temporary directory.""" - if checkout_dir.exists(): - logger.debug(f"Reusing {self.org}:{self.repo}") - yield await self._get_commit(checkout_dir) - return - - logger.debug(f"Cloning {self.org}:{self.repo}") - git_clone_command = [ - "git", - "clone", - "--config", - "advice.detachedHead=false", - "--quiet", - "--depth", - "1", - "--no-tags", - ] - if self.ref: - git_clone_command.extend(["--branch", self.ref]) - - git_clone_command.extend( - [ - f"https://github.com/{self.org}/{self.repo}", - str(checkout_dir), - ], - ) - - git_clone_process = await create_subprocess_exec( - *git_clone_command, - env={"GIT_TERMINAL_PROMPT": "0"}, - ) - - status_code = await git_clone_process.wait() - - logger.debug( - f"Finished cloning {self.org}/{self.repo} with status {status_code}", - ) - yield await self._get_commit(checkout_dir) - - def url_for(self: Self, commit_sha: str, path: str, lnum: int | None = None) -> str: - """ - Return the GitHub URL for the given commit, path, and line number, if given. - """ - # Default to main branch - url = f"https://github.com/{self.org}/{self.repo}/blob/{commit_sha}/{path}" - if lnum: - url += f"#L{lnum}" - return url - - async def _get_commit(self: Self, checkout_dir: Path) -> str: - """Return the commit sha for the repository in the checkout directory.""" - git_sha_process = await create_subprocess_exec( - *["git", "rev-parse", "HEAD"], - cwd=checkout_dir, - stdout=PIPE, - ) - git_sha_stdout, _ = await git_sha_process.communicate() - assert await git_sha_process.wait() == 0, ( - f"Failed to retrieve commit sha at {checkout_dir}" - ) - return git_sha_stdout.decode().strip() - - -# Repositories to check -# We check most repositories with the default ruleset instead of all rules to avoid -# noisy reports when new rules are added; see https://github.com/astral-sh/ruff/pull/3590 -REPOSITORIES: list[Repository] = [ - Repository("DisnakeDev", "disnake", "master"), - Repository("PostHog", "HouseWatch", "main"), - Repository("RasaHQ", "rasa", "main"), - Repository("Snowflake-Labs", "snowcli", "main"), - Repository("aiven", "aiven-client", "main"), - Repository("alteryx", "featuretools", "main"), - Repository("apache", "airflow", "main", select="ALL"), - Repository("apache", "superset", "master", select="ALL"), - Repository("aws", "aws-sam-cli", "develop"), - Repository("binary-husky", "gpt_academic", "master"), - Repository("bloomberg", "pytest-memray", "main"), - Repository("bokeh", "bokeh", "branch-3.10", select="ALL"), - # Disabled due to use of explicit `select` with `E999`, which has been removed. - # See: https://github.com/astral-sh/ruff/pull/12129 - # Repository("demisto", "content", "master"), - Repository("docker", "docker-py", "main"), - Repository("facebookresearch", "chameleon", "main"), - Repository("freedomofpress", "securedrop", "develop"), - Repository("fronzbot", "blinkpy", "dev"), - Repository("ibis-project", "ibis", "master"), - Repository("ing-bank", "probatus", "main"), - Repository("jrnl-org", "jrnl", "main"), - Repository("langchain-ai", "langchain", "main"), - Repository("latchbio", "latch", "main"), - Repository("lnbits", "lnbits", "main"), - Repository("milvus-io", "pymilvus", "master"), - Repository("mlflow", "mlflow", "master"), - Repository("model-bakers", "model_bakery", "main"), - Repository("pandas-dev", "pandas", "main"), - Repository("prefecthq", "prefect", "main"), - Repository("pypa", "build", "main"), - Repository("pypa", "cibuildwheel", "main"), - Repository("pypa", "pip", "main"), - Repository("pypa", "setuptools", "main"), - Repository("python", "mypy", "master"), - Repository("python", "typeshed", "main", select="PYI"), - Repository("python-poetry", "poetry", "master"), - Repository("qdrant", "qdrant-client", "master"), - Repository("reflex-dev", "reflex", "main"), - Repository("rotki", "rotki", "develop"), - Repository("scikit-build", "scikit-build", "main"), - Repository("scikit-build", "scikit-build-core", "main"), - Repository("sphinx-doc", "sphinx", "master"), - Repository("spruceid", "siwe-py", "main"), - Repository("tiangolo", "fastapi", "master"), - Repository("yandex", "ch-backup", "main"), - Repository("zulip", "zulip", "main", select="ALL"), -] - -SUMMARY_LINE_RE = re.compile(r"^(Found \d+ error.*)|(.*potentially fixable with.*)$") - - -class RuffError(Exception): - """An error reported by ruff.""" - - -async def check( - *, - ruff: Path, - path: Path, - name: str, - select: str = "", - ignore: str = "", - exclude: str = "", - show_fixes: bool = False, -) -> Sequence[str]: - """Run the given ruff binary against the specified path.""" - logger.debug(f"Checking {name} with {ruff}") - ruff_args = ["check", "--no-cache", "--exit-zero"] - if select: - ruff_args.extend(["--select", select]) - if ignore: - ruff_args.extend(["--ignore", ignore]) - if exclude: - ruff_args.extend(["--exclude", exclude]) - if show_fixes: - ruff_args.extend(["--show-fixes"]) - - start = time.time() - proc = await create_subprocess_exec( - ruff.absolute(), - *ruff_args, - ".", - stdout=PIPE, - stderr=PIPE, - cwd=path, - ) - result, err = await proc.communicate() - end = time.time() - - logger.debug(f"Finished checking {name} with {ruff} in {end - start:.2f}") - - if proc.returncode != 0: - raise RuffError(err.decode("utf8")) - - lines = [ - line - for line in result.decode("utf8").splitlines() - if not SUMMARY_LINE_RE.match(line) - ] - - return sorted(lines) - - -class Diff(NamedTuple): - """A diff between two runs of ruff.""" - - removed: set[str] - added: set[str] - source_sha: str - - def __bool__(self: Self) -> bool: - """Return true if this diff is non-empty.""" - return bool(self.removed or self.added) - - def __iter__(self: Self) -> Iterator[str]: - """Iterate through the changed lines in diff format.""" - for line in heapq.merge(sorted(self.removed), sorted(self.added)): - if line in self.removed: - yield f"- {line}" - else: - yield f"+ {line}" - - -async def compare( - ruff1: Path, - ruff2: Path, - repo: Repository, - checkouts: Path | None = None, -) -> Diff: - """Check a specific repository against two versions of ruff.""" - removed, added = set(), set() - - # By the default, the git clone are transient, but if the user provides a - # directory for permanent storage we keep it there - if checkouts: - location_context = nullcontext(checkouts) - else: - location_context = tempfile.TemporaryDirectory() - - with location_context as checkout_parent: - assert ":" not in repo.org - assert ":" not in repo.repo - checkout_dir = Path(checkout_parent).joinpath(f"{repo.org}:{repo.repo}") - async with repo.clone(checkout_dir) as checkout_sha: - try: - async with asyncio.TaskGroup() as tg: - check1 = tg.create_task( - check( - ruff=ruff1, - path=checkout_dir, - name=f"{repo.org}/{repo.repo}", - select=repo.select, - ignore=repo.ignore, - exclude=repo.exclude, - show_fixes=repo.show_fixes, - ), - ) - check2 = tg.create_task( - check( - ruff=ruff2, - path=checkout_dir, - name=f"{repo.org}/{repo.repo}", - select=repo.select, - ignore=repo.ignore, - exclude=repo.exclude, - show_fixes=repo.show_fixes, - ), - ) - except ExceptionGroup as e: - raise e.exceptions[0] from e - - for line in difflib.ndiff(check1.result(), check2.result()): - if line.startswith("- "): - removed.add(line[2:]) - elif line.startswith("+ "): - added.add(line[2:]) - - return Diff(removed, added, checkout_sha) - - -def read_projects_jsonl(projects_jsonl: Path) -> dict[tuple[str, str], Repository]: - """Read either of the two formats of https://github.com/akx/ruff-usage-aggregate.""" - repositories = {} - for line in projects_jsonl.read_text().splitlines(): - data = json.loads(line) - # Check the input format. - if "items" in data: - for item in data["items"]: - # Pick only the easier case for now. - if item["path"] != "pyproject.toml": - continue - repository = item["repository"] - assert re.fullmatch(r"[a-zA-Z0-9_.-]+", repository["name"]), repository[ - "name" - ] - # GitHub doesn't give us any branch or pure rev info. This would give - # us the revision, but there's no way with git to just do - # `git clone --depth 1` with a specific ref. - # `ref = item["url"].split("?ref=")[1]` would be exact - repositories[(repository["owner"], repository["repo"])] = Repository( - repository["owner"]["login"], - repository["name"], - None, - select=repository.get("select"), - ignore=repository.get("ignore"), - exclude=repository.get("exclude"), - ) - else: - assert "owner" in data, "Unknown ruff-usage-aggregate format" - # Pick only the easier case for now. - if data["path"] != "pyproject.toml": - continue - repositories[(data["owner"], data["repo"])] = Repository( - data["owner"], - data["repo"], - data.get("ref"), - select=data.get("select"), - ignore=data.get("ignore"), - exclude=data.get("exclude"), - ) - return repositories - - -DIFF_LINE_RE = re.compile( - r"^(?P
[+-]) (?P(?P[^:]+):(?P\d+):\d+:) (?P.*)$",
-)
-
-T = TypeVar("T")
-
-
-async def main(
-    *,
-    ruff1: Path,
-    ruff2: Path,
-    projects_jsonl: Path | None,
-    checkouts: Path | None = None,
-) -> None:
-    """Check two versions of ruff against a corpus of open-source code."""
-    if projects_jsonl:
-        repositories = read_projects_jsonl(projects_jsonl)
-    else:
-        repositories = {(repo.org, repo.repo): repo for repo in REPOSITORIES}
-
-    logger.debug(f"Checking {len(repositories)} projects")
-
-    # https://stackoverflow.com/a/61478547/3549270
-    # Otherwise doing 3k repositories can take >8GB RAM
-    semaphore = asyncio.Semaphore(50)
-
-    async def limited_parallelism(coroutine: Awaitable[T]) -> T:
-        async with semaphore:
-            return await coroutine
-
-    results = await asyncio.gather(
-        *[
-            limited_parallelism(compare(ruff1, ruff2, repo, checkouts))
-            for repo in repositories.values()
-        ],
-        return_exceptions=True,
-    )
-
-    diffs = dict(zip(repositories, results, strict=True))
-
-    total_removed = total_added = 0
-    errors = 0
-
-    for diff in diffs.values():
-        if isinstance(diff, BaseException):
-            errors += 1
-        else:
-            total_removed += len(diff.removed)
-            total_added += len(diff.added)
-
-    if total_removed == 0 and total_added == 0 and errors == 0:
-        print("\u2705 ecosystem check detected no changes.")
-    else:
-        rule_changes: dict[str, tuple[int, int]] = {}
-        changes = f"(+{total_added}, -{total_removed}, {errors} error(s))"
-
-        print(f"\u2139\ufe0f ecosystem check **detected changes**. {changes}")
-        print()
-
-        for (org, repo), diff in diffs.items():
-            if isinstance(diff, BaseException):
-                changes = "error"
-                print(f"
{repo} ({changes})")
-                repo = repositories[(org, repo)]
-                print(
-                    f"https://github.com/{repo.org}/{repo.repo} ref {repo.ref} "
-                    f"select {repo.select} ignore {repo.ignore} exclude {repo.exclude}",
-                )
-                print("
")
-                print()
-
-                print("```")
-                print(str(diff))
-                print("```")
-
-                print()
-                print("
")
-                print("
")
-            elif diff:
-                changes = f"+{len(diff.added)}, -{len(diff.removed)}"
-                print(f"
{repo} ({changes})")
-                print("
")
-                print()
-
-                repo = repositories[(org, repo)]
-                diff_lines = list(diff)
-
-                print("
")
-                for line in diff_lines:
-                    match = DIFF_LINE_RE.match(line)
-                    if match is None:
-                        print(line)
-                        continue
-
-                    pre, inner, path, lnum, post = match.groups()
-                    url = repo.url_for(diff.source_sha, path, int(lnum))
-                    print(f"{pre} {inner} {post}")
-                print("
")
-
-                print()
-                print("
")
-                print("
")
-
-                # Count rule changes
-                for line in diff_lines:
-                    # Find rule change for current line or construction
-                    # + /:::  
-                    matches = re.search(r": ([A-Z]{1,4}[0-9]{3,4})", line)
-
-                    if matches is None:
-                        # Handle case where there are no regex matches e.g.
-                        # +                 "?application=AIRFLOW&authenticator=TEST_AUTH&role=TEST_ROLE&warehouse=TEST_WAREHOUSE"
-                        # Which was found in local testing
-                        continue
-
-                    rule_code = matches.group(1)
-
-                    # Get current additions and removals for this rule
-                    current_changes = rule_changes.get(rule_code, (0, 0))
-
-                    # Check if addition or removal depending on the first character
-                    if line[0] == "+":
-                        current_changes = (current_changes[0] + 1, current_changes[1])
-                    elif line[0] == "-":
-                        current_changes = (current_changes[0], current_changes[1] + 1)
-
-                    rule_changes[rule_code] = current_changes
-
-            else:
-                continue
-
-        if len(rule_changes.keys()) > 0:
-            print(f"Rules changed: {len(rule_changes.keys())}")
-            print()
-            print("| Rule | Changes | Additions | Removals |")
-            print("| ---- | ------- | --------- | -------- |")
-            for rule, (additions, removals) in sorted(
-                rule_changes.items(),
-                key=lambda x: x[1][0] + x[1][1],
-                reverse=True,
-            ):
-                print(f"| {rule} | {additions + removals} | {additions} | {removals} |")
-
-    logger.debug(f"Finished {len(repositories)} repositories")
-
-
-if __name__ == "__main__":
-    parser = argparse.ArgumentParser(
-        description="Check two versions of ruff against a corpus of open-source code.",
-        epilog="scripts/check_ecosystem.py  ",
-    )
-
-    parser.add_argument(
-        "--projects",
-        type=Path,
-        help=(
-            "Optional JSON files to use over the default repositories. "
-            "Supports both github_search_*.jsonl and known-github-tomls.jsonl."
-        ),
-    )
-    parser.add_argument(
-        "--checkouts",
-        type=Path,
-        help=(
-            "Location for the git checkouts, in case you want to save them"
-            " (defaults to temporary directory)"
-        ),
-    )
-    parser.add_argument(
-        "-v",
-        "--verbose",
-        action="store_true",
-        help="Activate debug logging",
-    )
-    parser.add_argument(
-        "ruff1",
-        type=Path,
-    )
-    parser.add_argument(
-        "ruff2",
-        type=Path,
-    )
-
-    args = parser.parse_args()
-
-    if args.verbose:
-        logging.basicConfig(level=logging.DEBUG)
-    else:
-        logging.basicConfig(level=logging.INFO)
-
-    loop = asyncio.new_event_loop()
-    asyncio.set_event_loop(loop)
-    if args.checkouts:
-        args.checkouts.mkdir(exist_ok=True, parents=True)
-    main_task = asyncio.ensure_future(
-        main(
-            ruff1=args.ruff1,
-            ruff2=args.ruff2,
-            projects_jsonl=args.projects,
-            checkouts=args.checkouts,
-        ),
-    )
-    # https://stackoverflow.com/a/58840987/3549270
-    for signal in [SIGINT, SIGTERM]:
-        loop.add_signal_handler(signal, main_task.cancel)
-    try:
-        loop.run_until_complete(main_task)
-    finally:
-        loop.close()
diff --git a/scripts/conformance.py b/scripts/conformance.py
index 011217a18b..7d34758090 100644
--- a/scripts/conformance.py
+++ b/scripts/conformance.py
@@ -1,3 +1,11 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = []
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 """
 Run typing conformance tests and compare results between two ty versions.
 
diff --git a/scripts/conformance.py.lock b/scripts/conformance.py.lock
new file mode 100644
index 0000000000..9a5fd10568
--- /dev/null
+++ b/scripts/conformance.py.lock
@@ -0,0 +1,7 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
diff --git a/scripts/ecosystem_all_check.py b/scripts/ecosystem_all_check.py
index 7d3c2e5cfe..cbcbd4ce17 100644
--- a/scripts/ecosystem_all_check.py
+++ b/scripts/ecosystem_all_check.py
@@ -1,7 +1,15 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = ["tqdm"]
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 """This is @konstin's scripts for checking an entire checkout of ~2.1k packages for
 panics, fix errors and similar problems.
 
-It's a less elaborate, more hacky version of check_ecosystem.py
+It's a less elaborate, more hacky ecosystem checker.
 """
 
 from __future__ import annotations
diff --git a/scripts/ecosystem_all_check.py.lock b/scripts/ecosystem_all_check.py.lock
new file mode 100644
index 0000000000..0295575253
--- /dev/null
+++ b/scripts/ecosystem_all_check.py.lock
@@ -0,0 +1,31 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
+
+[manifest]
+requirements = [{ name = "tqdm" }]
+
+[[package]]
+name = "colorama"
+version = "0.4.6"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
+]
+
+[[package]]
+name = "tqdm"
+version = "4.70.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "colorama", marker = "sys_platform == 'win32'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/21/3b/6c24bec5be5e743ffd99576daa5cc077722fc7d5bbc00bd133fa0c698dc6/tqdm-4.70.0.tar.gz", hash = "sha256:55b0b0dbd97462d06ebee91e4dac24ed4d4702be82b24f07e6c1d27e08cea220", size = 795438, upload-time = "2026-07-27T11:33:15.271Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f9/1c/01bfd571a64e7f270e6bab5e33777debe0edc56759233ce84f27dec92d14/tqdm-4.70.0-py3-none-any.whl", hash = "sha256:7f585706bfddbdebf89daac705b2dfcc16890130727d3197ca62c732b4310953", size = 80184, upload-time = "2026-07-27T11:33:13.167Z" },
+]
diff --git a/scripts/ecosystem_all_check.sh b/scripts/ecosystem_all_check.sh
index 2018108280..da6dd3dea4 100755
--- a/scripts/ecosystem_all_check.sh
+++ b/scripts/ecosystem_all_check.sh
@@ -7,7 +7,7 @@
 #
 # Usage:
 # ```shell
-# # You can also use any other check_ecosystem.py input file
+# # You can also use any compatible JSONL input file
 # curl https://raw.githubusercontent.com/akx/ruff-usage-aggregate/master/data/known-github-tomls-clean.jsonl > github_search.jsonl
 # cargo build --release --target x86_64-unknown-linux-musl --bin ruff
 # scripts/ecosystem_all_check.sh check --select RUF200
diff --git a/scripts/generate_builtin_modules.py b/scripts/generate_builtin_modules.py
index 568bb8604c..a2732cfe94 100644
--- a/scripts/generate_builtin_modules.py
+++ b/scripts/generate_builtin_modules.py
@@ -1,3 +1,11 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = []
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 """Script to generate `crates/ruff_python_stdlib/src/sys/builtin_modules.rs`.
 
 This script requires `uvx` to be available on PATH.
diff --git a/scripts/generate_builtin_modules.py.lock b/scripts/generate_builtin_modules.py.lock
new file mode 100644
index 0000000000..9a5fd10568
--- /dev/null
+++ b/scripts/generate_builtin_modules.py.lock
@@ -0,0 +1,7 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
diff --git a/scripts/generate_known_standard_library.py b/scripts/generate_known_standard_library.py
index 00cda45efc..404a24972c 100644
--- a/scripts/generate_known_standard_library.py
+++ b/scripts/generate_known_standard_library.py
@@ -1,3 +1,11 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = ["stdlibs"]
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 from __future__ import annotations
 
 from pathlib import Path
diff --git a/scripts/generate_known_standard_library.py.lock b/scripts/generate_known_standard_library.py.lock
new file mode 100644
index 0000000000..18e05bb895
--- /dev/null
+++ b/scripts/generate_known_standard_library.py.lock
@@ -0,0 +1,19 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
+
+[manifest]
+requirements = [{ name = "stdlibs" }]
+
+[[package]]
+name = "stdlibs"
+version = "2026.2.26"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/d5/cd/2710eaacaefc8be2f520b55c313498a50a295a8378e932c70d4ea34250aa/stdlibs-2026.2.26.tar.gz", hash = "sha256:10f911bdd8d3e45b452cc187b3527e6f9d288c8a943c5f973da94c71b2757d5b", size = 20203, upload-time = "2026-02-26T23:30:04.775Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/90/ec/b6a5a568d584659e037c8f53fc25acc79950ac32796b8861b2015446b7b2/stdlibs-2026.2.26-py3-none-any.whl", hash = "sha256:3257486216eac5ac627a3a4c5665802aca72fe7fc9e4ab1f232b1fb47bfd3db6", size = 59288, upload-time = "2026-02-26T23:30:03.597Z" },
+]
diff --git a/scripts/generate_mkdocs.py b/scripts/generate_mkdocs.py
index 943ddc517e..63e3565795 100644
--- a/scripts/generate_mkdocs.py
+++ b/scripts/generate_mkdocs.py
@@ -1,3 +1,15 @@
+# /// script
+# requires-python = ">=3.13"
+# dependencies = [
+#     "mdformat>=1.0.0",
+#     "mdformat-mkdocs>=5.3.0",
+#     "pyyaml>=6.0.3",
+# ]
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 """Generate an MkDocs-compatible `docs` and `mkdocs.yml` from the README.md."""
 
 from __future__ import annotations
diff --git a/scripts/uv.lock b/scripts/generate_mkdocs.py.lock
similarity index 53%
rename from scripts/uv.lock
rename to scripts/generate_mkdocs.py.lock
index 6461f0ce9b..6eb1f9029b 100644
--- a/scripts/uv.lock
+++ b/scripts/generate_mkdocs.py.lock
@@ -1,110 +1,82 @@
 version = 1
 revision = 3
-requires-python = ">=3.12"
+requires-python = ">=3.13"
 
 [options]
 exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
 exclude-newer-span = "P7D"
 
-[[package]]
-name = "anyio"
-version = "4.14.0"
-source = { registry = "https://pypi.org/simple" }
-dependencies = [
-    { name = "idna" },
-    { name = "typing-extensions", marker = "python_full_version < '3.13'" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/1c/b5/001890774a9552aff22502b8da382593109ce0c95314abaebbb116567545/anyio-4.14.0.tar.gz", hash = "sha256:b47c1f9ccf73e67021df785332508f99379c68fa7d0684e8e3492cb1d4b23f89", size = 253586, upload-time = "2026-06-15T22:00:49.021Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/ba/16/9826f089383c593cdfc4a6e5aca94d9e91ae1692c57af82c3b2aa5e810f7/anyio-4.14.0-py3-none-any.whl", hash = "sha256:dd9b7a2a9799ed6552fde617b2c5df02b7fdd7d88392fc48101e51bae46164d9", size = 123506, upload-time = "2026-06-15T22:00:47.595Z" },
+[manifest]
+requirements = [
+    { name = "mdformat", specifier = ">=1.0.0" },
+    { name = "mdformat-mkdocs", specifier = ">=5.3.0" },
+    { name = "pyyaml", specifier = ">=6.0.3" },
 ]
 
 [[package]]
-name = "certifi"
-version = "2026.6.17"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/c9/c7/424b75da314c1045981bd9777432fad05a9e0c69daa4ed7e308bbaffe405/certifi-2026.6.17.tar.gz", hash = "sha256:024c88eeec92ca068db80f02b8b07c9cef7b9fe261d1d535abfd5abd6f6af432", size = 134594, upload-time = "2026-06-17T10:31:07.894Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/ef/2f/c5464532e965badff2f4c4c1a3a83f5697f0d7c407ed0cda44aaa99bb451/certifi-2026.6.17-py3-none-any.whl", hash = "sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db", size = 133289, upload-time = "2026-06-17T10:31:06.348Z" },
-]
-
-[[package]]
-name = "colorama"
-version = "0.4.6"
+name = "markdown-it-py"
+version = "4.2.0"
 source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
+dependencies = [
+    { name = "mdurl" },
 ]
-
-[[package]]
-name = "h11"
-version = "0.16.0"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/01/ee/02a2c011bdab74c6fb3c75474d40b3052059d95df7e73351460c8588d963/h11-0.16.0.tar.gz", hash = "sha256:4e35b956cf45792e4caa5885e69fba00bdbc6ffafbfa020300e549b208ee5ff1", size = 101250, upload-time = "2025-04-24T03:35:25.427Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/04/4b/29cac41a4d98d144bf5f6d33995617b185d14b22401f75ca86f384e87ff1/h11-0.16.0-py3-none-any.whl", hash = "sha256:63cf8bbe7522de3bf65932fda1d9c2772064ffb3dae62d55932da54b31cb6c86", size = 37515, upload-time = "2025-04-24T03:35:24.344Z" },
+    { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" },
 ]
 
 [[package]]
-name = "httpcore"
-version = "1.0.9"
+name = "mdformat"
+version = "1.0.0"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "certifi" },
-    { name = "h11" },
+    { name = "markdown-it-py" },
 ]
-sdist = { url = "https://files.pythonhosted.org/packages/06/94/82699a10bca87a5556c9c59b5963f2d039dbd239f25bc2a63907a05a14cb/httpcore-1.0.9.tar.gz", hash = "sha256:6e34463af53fd2ab5d807f399a9b45ea31c3dfa2276f15a2c3f00afff6e176e8", size = 85484, upload-time = "2025-04-24T22:06:22.219Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/3f/05/32b5e14b192b0a8a309f32232c580aefedd9d06017cb8fe8fce34bec654c/mdformat-1.0.0.tar.gz", hash = "sha256:4954045fcae797c29f86d4ad879e43bb151fa55dbaf74ac6eaeacf1d45bb3928", size = 56953, upload-time = "2025-10-16T12:05:03.695Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/7e/f5/f66802a942d491edb555dd61e3a9961140fd64c90bce1eafd741609d334d/httpcore-1.0.9-py3-none-any.whl", hash = "sha256:2d400746a40668fc9dec9810239072b40b4484b640a8c38fd654a024c7a1bf55", size = 78784, upload-time = "2025-04-24T22:06:20.566Z" },
+    { url = "https://files.pythonhosted.org/packages/54/9a/8fe71b95985ca7a4001effbcc58e5a07a1f2a2884203f74dcf48a3b08315/mdformat-1.0.0-py3-none-any.whl", hash = "sha256:bca015d65a1d063a02e885a91daee303057bc7829c2cd37b2075a50dbb65944b", size = 53288, upload-time = "2025-10-16T12:05:02.607Z" },
 ]
 
 [[package]]
-name = "httpx"
-version = "0.28.1"
+name = "mdformat-gfm"
+version = "1.0.0"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "anyio" },
-    { name = "certifi" },
-    { name = "httpcore" },
-    { name = "idna" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/b1/df/48c586a5fe32a0f01324ee087459e112ebb7224f646c0b5023f5e79e9956/httpx-0.28.1.tar.gz", hash = "sha256:75e98c5f16b0f35b567856f597f06ff2270a374470a5c2392242528e3e3e42fc", size = 141406, upload-time = "2024-12-06T15:37:23.222Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/2a/39/e50c7c3a983047577ee07d2a9e53faf5a69493943ec3f6a384bdc792deb2/httpx-0.28.1-py3-none-any.whl", hash = "sha256:d909fcccc110f8c7faf814ca82a9a4d816bc5a6dbfea25d6591d6985b8ba59ad", size = 73517, upload-time = "2024-12-06T15:37:21.509Z" },
+    { name = "markdown-it-py" },
+    { name = "mdformat" },
+    { name = "mdit-py-plugins" },
+    { name = "wcwidth" },
 ]
-
-[[package]]
-name = "idna"
-version = "3.18"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848", size = 196711, upload-time = "2026-06-02T14:34:07.794Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/56/6f/a626ebb142a290474401b67e2d61e73ce096bf7798ee22dfe6270f924b3f/mdformat_gfm-1.0.0.tar.gz", hash = "sha256:d1d49a409a6acb774ce7635c72d69178df7dce1dc8cdd10e19f78e8e57b72623", size = 10112, upload-time = "2025-10-16T09:12:22.402Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
+    { url = "https://files.pythonhosted.org/packages/e6/18/6bc2189b744dd383cad03764f41f30352b1278d2205096f77a29c0b327ad/mdformat_gfm-1.0.0-py3-none-any.whl", hash = "sha256:7305a50efd2a140d7c83505b58e3ac5df2b09e293f9bbe72f6c7bee8c678b005", size = 10970, upload-time = "2025-10-16T09:12:21.276Z" },
 ]
 
 [[package]]
-name = "markdown-it-py"
-version = "4.2.0"
+name = "mdformat-mkdocs"
+version = "5.3.0"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
-    { name = "mdurl" },
+    { name = "mdformat" },
+    { name = "mdformat-gfm" },
+    { name = "mdit-py-plugins" },
+    { name = "more-itertools" },
 ]
-sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/2e/8c/ca9c13017fcb224e9a0c17c214279eb7273318d1890cd0adc80a3c30e443/mdformat_mkdocs-5.3.0.tar.gz", hash = "sha256:9ae35940cfc1d350c41dda717963c90c669937fbbe3be32412a2b975e4bf891d", size = 33319, upload-time = "2026-08-02T18:38:07.954Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" },
+    { url = "https://files.pythonhosted.org/packages/c0/71/1e1a81c7ed1629ac41f5b3d7dd7bc93c91e106398a88f0e2949c39e04b8e/mdformat_mkdocs-5.3.0-py3-none-any.whl", hash = "sha256:46938724df5892f517130a42d5652276f9a40e80ad0bae4a2af6bde71b53f861", size = 43928, upload-time = "2026-08-02T18:38:06.347Z" },
 ]
 
 [[package]]
-name = "mdformat"
-version = "1.0.0"
+name = "mdit-py-plugins"
+version = "0.6.1"
 source = { registry = "https://pypi.org/simple" }
 dependencies = [
     { name = "markdown-it-py" },
 ]
-sdist = { url = "https://files.pythonhosted.org/packages/3f/05/32b5e14b192b0a8a309f32232c580aefedd9d06017cb8fe8fce34bec654c/mdformat-1.0.0.tar.gz", hash = "sha256:4954045fcae797c29f86d4ad879e43bb151fa55dbaf74ac6eaeacf1d45bb3928", size = 56953, upload-time = "2025-10-16T12:05:03.695Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/54/9a/8fe71b95985ca7a4001effbcc58e5a07a1f2a2884203f74dcf48a3b08315/mdformat-1.0.0-py3-none-any.whl", hash = "sha256:bca015d65a1d063a02e885a91daee303057bc7829c2cd37b2075a50dbb65944b", size = 53288, upload-time = "2025-10-16T12:05:02.607Z" },
+    { url = "https://files.pythonhosted.org/packages/a5/69/6da5581c6a7fede7dc261bf4e67d6adca4196f176b43288b55b3db395b6e/mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d", size = 66663, upload-time = "2026-05-13T09:03:37.76Z" },
 ]
 
 [[package]]
@@ -117,9 +89,13 @@ wheels = [
 ]
 
 [[package]]
-name = "mypy-primer"
-version = "0.1.0"
-source = { git = "https://github.com/hauntsaninja/mypy_primer#23bbdd55fea37ca2489043d1327dbe35c4fc7083" }
+name = "more-itertools"
+version = "11.1.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/de/1d/f4da6f02cdffe04d6362210b807146a26044c88d839208aec273bb0d9184/more_itertools-11.1.0.tar.gz", hash = "sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d", size = 145772, upload-time = "2026-05-22T14:14:29.909Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/e8/3d/1087453384dbde46a8c7f9356eead2c58be8a7bf156bca40243377c85715/more_itertools-11.1.0-py3-none-any.whl", hash = "sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192", size = 72226, upload-time = "2026-05-22T14:14:28.824Z" },
+]
 
 [[package]]
 name = "pyyaml"
@@ -127,16 +103,6 @@ version = "6.0.3"
 source = { registry = "https://pypi.org/simple" }
 sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" },
-    { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" },
-    { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" },
-    { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" },
-    { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" },
-    { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" },
-    { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" },
-    { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" },
-    { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" },
-    { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" },
     { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" },
     { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" },
     { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" },
@@ -168,54 +134,10 @@ wheels = [
 ]
 
 [[package]]
-name = "scripts"
-version = "0.0.1"
-source = { virtual = "." }
-dependencies = [
-    { name = "httpx" },
-    { name = "mdformat" },
-    { name = "mypy-primer" },
-    { name = "pyyaml" },
-    { name = "stdlibs" },
-    { name = "tqdm" },
-]
-
-[package.metadata]
-requires-dist = [
-    { name = "httpx" },
-    { name = "mdformat" },
-    { name = "mypy-primer", git = "https://github.com/hauntsaninja/mypy_primer" },
-    { name = "pyyaml" },
-    { name = "stdlibs" },
-    { name = "tqdm" },
-]
-
-[[package]]
-name = "stdlibs"
-version = "2026.2.26"
-source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/d5/cd/2710eaacaefc8be2f520b55c313498a50a295a8378e932c70d4ea34250aa/stdlibs-2026.2.26.tar.gz", hash = "sha256:10f911bdd8d3e45b452cc187b3527e6f9d288c8a943c5f973da94c71b2757d5b", size = 20203, upload-time = "2026-02-26T23:30:04.775Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/90/ec/b6a5a568d584659e037c8f53fc25acc79950ac32796b8861b2015446b7b2/stdlibs-2026.2.26-py3-none-any.whl", hash = "sha256:3257486216eac5ac627a3a4c5665802aca72fe7fc9e4ab1f232b1fb47bfd3db6", size = 59288, upload-time = "2026-02-26T23:30:03.597Z" },
-]
-
-[[package]]
-name = "tqdm"
-version = "4.67.3"
-source = { registry = "https://pypi.org/simple" }
-dependencies = [
-    { name = "colorama", marker = "sys_platform == 'win32'" },
-]
-sdist = { url = "https://files.pythonhosted.org/packages/09/a9/6ba95a270c6f1fbcd8dac228323f2777d886cb206987444e4bce66338dd4/tqdm-4.67.3.tar.gz", hash = "sha256:7d825f03f89244ef73f1d4ce193cb1774a8179fd96f31d7e1dcde62092b960bb", size = 169598, upload-time = "2026-02-03T17:35:53.048Z" }
-wheels = [
-    { url = "https://files.pythonhosted.org/packages/16/e1/3079a9ff9b8e11b846c6ac5c8b5bfb7ff225eee721825310c91b3b50304f/tqdm-4.67.3-py3-none-any.whl", hash = "sha256:ee1e4c0e59148062281c49d80b25b67771a127c85fc9676d3be5f243206826bf", size = 78374, upload-time = "2026-02-03T17:35:50.982Z" },
-]
-
-[[package]]
-name = "typing-extensions"
-version = "4.15.0"
+name = "wcwidth"
+version = "0.8.2"
 source = { registry = "https://pypi.org/simple" }
-sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
+sdist = { url = "https://files.pythonhosted.org/packages/34/74/c6428f875774288bec1396f5bfcbc2d925700a4dad61727fd5f2b12f249d/wcwidth-0.8.2.tar.gz", hash = "sha256:91fbef97204b96a3d4d421609b80340b760cf33e26da123ff243d76b1fda8dda", size = 1466253, upload-time = "2026-06-29T18:11:11.601Z" }
 wheels = [
-    { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
+    { url = "https://files.pythonhosted.org/packages/96/42/3e5985a0a7e57de470b320c6d6a1a67c844f6737a587f3d44dd13d1819e7/wcwidth-0.8.2-py3-none-any.whl", hash = "sha256:d63947694a0539a1d51e01eda7caf800c291020e6cdd7e28ad7b14dd33ad4f85", size = 323166, upload-time = "2026-06-29T18:11:09.888Z" },
 ]
diff --git a/scripts/memory_report.py b/scripts/memory_report.py
index b4c4fd3224..659a1f1aca 100644
--- a/scripts/memory_report.py
+++ b/scripts/memory_report.py
@@ -1,3 +1,11 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = []
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 """
 Compare memory usage reports between two ty versions and generate a PR comment.
 
diff --git a/scripts/memory_report.py.lock b/scripts/memory_report.py.lock
new file mode 100644
index 0000000000..9a5fd10568
--- /dev/null
+++ b/scripts/memory_report.py.lock
@@ -0,0 +1,7 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
diff --git a/scripts/publish-crates.py b/scripts/publish-crates.py
index a1a9ff3f59..2dbe5175c9 100644
--- a/scripts/publish-crates.py
+++ b/scripts/publish-crates.py
@@ -1,3 +1,11 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = []
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 # Publish workspace crates to crates.io idempotently.
 #
 # `cargo publish --workspace` fails if any selected crate version already exists on crates.io. That
diff --git a/scripts/publish-crates.py.lock b/scripts/publish-crates.py.lock
new file mode 100644
index 0000000000..9a5fd10568
--- /dev/null
+++ b/scripts/publish-crates.py.lock
@@ -0,0 +1,7 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
diff --git a/scripts/pyproject.toml b/scripts/pyproject.toml
deleted file mode 100644
index 6993c23543..0000000000
--- a/scripts/pyproject.toml
+++ /dev/null
@@ -1,26 +0,0 @@
-[project]
-name = "scripts"
-version = "0.0.1"
-dependencies = ["stdlibs", "tqdm", "mdformat", "pyyaml", "mypy-primer", "httpx"]
-requires-python = ">=3.12"
-
-[tool.black]
-line-length = 88
-
-[tool.ruff]
-extend = "../pyproject.toml"
-
-[tool.ty.src]
-# `ty_benchmark` is a standalone project with its own pyproject.toml files, search paths, etc.
-exclude = ["./ty_benchmark"]
-
-[tool.uv]
-exclude-newer = "P7D"
-
-[tool.uv.sources]
-mypy-primer = { git = "https://github.com/hauntsaninja/mypy_primer" }
-
-[tool.ty.rules]
-possibly-unresolved-reference = "error"
-division-by-zero = "error"
-unused-ignore-comment = "error"
diff --git a/scripts/transform_readme.py b/scripts/transform_readme.py
index 63f5c00267..5ba8958f72 100644
--- a/scripts/transform_readme.py
+++ b/scripts/transform_readme.py
@@ -1,3 +1,11 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = []
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 """Transform the README.md to support a specific deployment target.
 
 By default, we assume that our README.md will be rendered on GitHub. However, different
diff --git a/scripts/transform_readme.py.lock b/scripts/transform_readme.py.lock
new file mode 100644
index 0000000000..9a5fd10568
--- /dev/null
+++ b/scripts/transform_readme.py.lock
@@ -0,0 +1,7 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
diff --git a/scripts/ty.toml b/scripts/ty.toml
new file mode 100644
index 0000000000..9216a26b46
--- /dev/null
+++ b/scripts/ty.toml
@@ -0,0 +1,4 @@
+[rules]
+division-by-zero = "error"
+possibly-unresolved-reference = "error"
+unused-ignore-comment = "error"
diff --git a/scripts/update_ambiguous_characters.py b/scripts/update_ambiguous_characters.py
index bf1dca7b8f..5a5991ac2c 100644
--- a/scripts/update_ambiguous_characters.py
+++ b/scripts/update_ambiguous_characters.py
@@ -1,3 +1,11 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = []
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 """Generate the confusables.rs file from the VS Code ambiguous.json file."""
 
 from __future__ import annotations
diff --git a/scripts/update_ambiguous_characters.py.lock b/scripts/update_ambiguous_characters.py.lock
new file mode 100644
index 0000000000..9a5fd10568
--- /dev/null
+++ b/scripts/update_ambiguous_characters.py.lock
@@ -0,0 +1,7 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
diff --git a/scripts/update_schemastore.py b/scripts/update_schemastore.py
index e19f25f515..d846d39ccd 100644
--- a/scripts/update_schemastore.py
+++ b/scripts/update_schemastore.py
@@ -1,3 +1,11 @@
+# /// script
+# requires-python = ">=3.12"
+# dependencies = []
+#
+# [tool.uv]
+# exclude-newer = "P7D"
+# ///
+
 """Update ruff.json in schemastore.
 
 This script will clone `astral-sh/schemastore`, update the schema and push the changes
@@ -6,7 +14,7 @@
 
 Usage:
 
-    uv run --only-dev scripts/update_schemastore.py
+    uv run --script scripts/update_schemastore.py
 """
 
 from __future__ import annotations
diff --git a/scripts/update_schemastore.py.lock b/scripts/update_schemastore.py.lock
new file mode 100644
index 0000000000..9a5fd10568
--- /dev/null
+++ b/scripts/update_schemastore.py.lock
@@ -0,0 +1,7 @@
+version = 1
+revision = 3
+requires-python = ">=3.12"
+
+[options]
+exclude-newer = "0001-01-01T00:00:00Z" # This has no effect and is included for backwards compatibility when using relative exclude-newer values.
+exclude-newer-span = "P7D"
diff --git a/uv.lock b/uv.lock
index a9b3550aa3..0a71404ae7 100644
--- a/uv.lock
+++ b/uv.lock
@@ -54,6 +54,41 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/0b/31/349eae2bc9d9331dd8951684cf94528d91efaa71129dc30822ac111dfc66/anysqlite-0.0.5-py3-none-any.whl", hash = "sha256:cb345dc4f76f6b37f768d7a0b3e9cf5c700dfcb7a6356af8ab46a11f666edbe7", size = 3907, upload-time = "2023-10-02T13:49:26.943Z" },
 ]
 
+[[package]]
+name = "babel"
+version = "2.18.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/7d/b2/51899539b6ceeeb420d40ed3cd4b7a40519404f9baf3d4ac99dc413a834b/babel-2.18.0.tar.gz", hash = "sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d", size = 9959554, upload-time = "2026-02-01T12:30:56.078Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/77/f5/21d2de20e8b8b0408f0681956ca2c69f1320a3848ac50e6e7f39c6159675/babel-2.18.0-py3-none-any.whl", hash = "sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35", size = 10196845, upload-time = "2026-02-01T12:30:53.445Z" },
+]
+
+[[package]]
+name = "backrefs"
+version = "8.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ec/56/4744bcd0c82184e80c52b0ac4076c261a8ffa1f1b343ff2f6e89ce0e1cef/backrefs-8.0.tar.gz", hash = "sha256:b556cd7d36c3a3a2f256b89590b176b8eddfb73bcfaee3a3ddd84ea66d21ce50", size = 7013081, upload-time = "2026-07-26T19:54:24.638Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/e3/fd/9bf53b6a6f6f519ffaac765df2f2a25e5c2fc6d32cfd2b2747099e72c911/backrefs-8.0-py310-none-any.whl", hash = "sha256:4a627b817fd2dce43b79ab48da63613340509381cd8ce0897078a0bce79a2ab8", size = 380377, upload-time = "2026-07-26T19:54:17.457Z" },
+    { url = "https://files.pythonhosted.org/packages/e1/29/4bd7ae72a2634da00379c2b3bcc5439e7c94620235c6afea8af15229a973/backrefs-8.0-py311-none-any.whl", hash = "sha256:f0c35cf0102ba6b6070c12a492be3c1c1d3f5839529784b9a9565d6d04569a01", size = 392169, upload-time = "2026-07-26T19:54:18.782Z" },
+    { url = "https://files.pythonhosted.org/packages/29/13/232505664e8e2a0c7a2eb0c505cfade9d715538f89a5d62bc4c272968f62/backrefs-8.0-py312-none-any.whl", hash = "sha256:87f0fae8c5f207fe9f4b2887efc71d42f4900ac78faa1af08d675ef303692dc5", size = 398084, upload-time = "2026-07-26T19:54:19.954Z" },
+    { url = "https://files.pythonhosted.org/packages/8a/69/47a3dc20abc4fa5486655fde681bd55e63211b46c886d8c02223d6468431/backrefs-8.0-py313-none-any.whl", hash = "sha256:601ce68ca12385dbda06ce264406b4c4210cf5b79fd0fd627592365c92f29a88", size = 400040, upload-time = "2026-07-26T19:54:21.194Z" },
+    { url = "https://files.pythonhosted.org/packages/1c/cf/e5f9b68a5b0e939a2fb933a66c20180d0c9241bf8927f7a47fa48c1675e9/backrefs-8.0-py314-none-any.whl", hash = "sha256:9ec96efa080938be92323e8e730e57718c9c88eb15ad70bbef4e1766df591408", size = 411903, upload-time = "2026-07-26T19:54:23.221Z" },
+]
+
+[[package]]
+name = "beautifulsoup4"
+version = "4.15.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "soupsieve" },
+    { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/43/65/318323f98dbee45d42dff61d8f047181bc6f2268a9068cfad035a46be5af/beautifulsoup4-4.15.0.tar.gz", hash = "sha256:288e3ca7d54b06f2ac191970bc275c1939cb46d450b255bf6718b04aa37ab4f7", size = 632571, upload-time = "2026-06-07T16:44:20.453Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/88/c6/92fcd42f1ba33e1184263f25bfabf3d27c383410470f169e4b8163bf9c17/beautifulsoup4-4.15.0-py3-none-any.whl", hash = "sha256:d6f88de62e1d4e38ecb1077eb9724cd0eff29d2a08ca16a401e9b9e93f117cf9", size = 109924, upload-time = "2026-06-07T16:44:21.566Z" },
+]
+
 [[package]]
 name = "certifi"
 version = "2026.6.17"
@@ -157,6 +192,118 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/54/8f/a1e836f82d8e32a97e6b29cc8f641779181ac7363734f12df27db803ebda/cffi-2.0.0-cp39-cp39-win_amd64.whl", hash = "sha256:b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9", size = 182794, upload-time = "2025-09-08T23:24:02.943Z" },
 ]
 
+[[package]]
+name = "charset-normalizer"
+version = "3.4.9"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/bd/2a/23f34ec9d04624958e137efdc394888716353190e75f25dd22c7a2c7a8aa/charset_normalizer-3.4.9.tar.gz", hash = "sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b", size = 152439, upload-time = "2026-07-07T14:34:58.454Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/ad/81/8e983840c6e5b93b33c2ba81aa3d52c2e42f0e9a690ce7607a2e61da4a5c/charset_normalizer-3.4.9-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a", size = 322240, upload-time = "2026-07-07T14:32:36.236Z" },
+    { url = "https://files.pythonhosted.org/packages/de/d1/b4319dc3229d8272fba305e206fc0a148e2de8d4087917ce62ae6382f359/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616", size = 216475, upload-time = "2026-07-07T14:32:38.142Z" },
+    { url = "https://files.pythonhosted.org/packages/80/33/6c99c1b3e6b8bf730e1bc809b9a2608f224145069114c479a2e9e1494346/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209", size = 238670, upload-time = "2026-07-07T14:32:39.658Z" },
+    { url = "https://files.pythonhosted.org/packages/7f/f4/ffbb83546e1f198ecc70ecd372b65cf2b50f9068b380abd67640f17a8e18/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99", size = 233476, upload-time = "2026-07-07T14:32:41.155Z" },
+    { url = "https://files.pythonhosted.org/packages/e8/5f/b98b8da398637b551e427e7be922bdec19177dc54d6811dcdaa503f23aac/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8", size = 223817, upload-time = "2026-07-07T14:32:42.592Z" },
+    { url = "https://files.pythonhosted.org/packages/36/31/a276bb2e66243072a3fd06fdcab9cbb61a305b02143d70d2bda21d888fa8/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_armv7l.whl", hash = "sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b", size = 207974, upload-time = "2026-07-07T14:32:44.258Z" },
+    { url = "https://files.pythonhosted.org/packages/5e/be/7ee4453d7e88dfbc4104ccd34900b9f2c7c17dac22881865fe0e82424a25/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2", size = 221655, upload-time = "2026-07-07T14:32:45.64Z" },
+    { url = "https://files.pythonhosted.org/packages/1d/85/181c652953eb5276d198f375b1dd641047392050098100a3a02d6534f657/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9", size = 219229, upload-time = "2026-07-07T14:32:47.376Z" },
+    { url = "https://files.pythonhosted.org/packages/0c/e7/aaf6da33fc9f4691cda8f7efbc9f69179d3d39ec8a4799baf273ee1d8db0/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15", size = 209704, upload-time = "2026-07-07T14:32:48.855Z" },
+    { url = "https://files.pythonhosted.org/packages/63/01/f2fb3bd3a73be48b173ee0c6aa8d2497af97d5663a8c4c4b491de4c62f7a/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d", size = 226243, upload-time = "2026-07-07T14:32:50.239Z" },
+    { url = "https://files.pythonhosted.org/packages/c4/02/c57a22739fe05246b0b5783b3bfb6afaac4eebb46f3ececdfb2f048f780e/charset_normalizer-3.4.9-cp310-cp310-win32.whl", hash = "sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381", size = 150935, upload-time = "2026-07-07T14:32:51.676Z" },
+    { url = "https://files.pythonhosted.org/packages/37/8d/ca39a7559a4797505530d084fd3a49a2c959efbbbff146302fb7be4e3b35/charset_normalizer-3.4.9-cp310-cp310-win_amd64.whl", hash = "sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee", size = 162314, upload-time = "2026-07-07T14:32:53.193Z" },
+    { url = "https://files.pythonhosted.org/packages/01/da/a44bd7a13d426e69e4894557106cd58669097bfad4a8681123b618fbfc5d/charset_normalizer-3.4.9-cp310-cp310-win_arm64.whl", hash = "sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419", size = 153075, upload-time = "2026-07-07T14:32:54.554Z" },
+    { url = "https://files.pythonhosted.org/packages/0b/e3/85ec501f206fb049259288c1f3506e53876937fb00edb47009348e66756b/charset_normalizer-3.4.9-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5", size = 317075, upload-time = "2026-07-07T14:32:56.021Z" },
+    { url = "https://files.pythonhosted.org/packages/c3/69/2a5385192e67175f7d8bd5ce4f57c24bc956439adeae5c13a99aa28a53d1/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2", size = 213837, upload-time = "2026-07-07T14:32:57.78Z" },
+    { url = "https://files.pythonhosted.org/packages/b3/46/03ddc7da576d814fe0a36dd1f0fd3258e95404b4b2e3c026b7923d7e133f/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a", size = 235503, upload-time = "2026-07-07T14:32:59.205Z" },
+    { url = "https://files.pythonhosted.org/packages/4e/6e/de0229a7ef40f6f9d28a837eebf4ec47bdca5dab4e900c84f22919af636a/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29", size = 229944, upload-time = "2026-07-07T14:33:00.803Z" },
+    { url = "https://files.pythonhosted.org/packages/a5/34/49b9060e8418b14fb5cba9cf6bfb383111e2538a03a1fb18e66a95aeb3d5/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c", size = 221276, upload-time = "2026-07-07T14:33:02.199Z" },
+    { url = "https://files.pythonhosted.org/packages/44/95/80282cce0fae9c3061203d723ee87da996aed79679e65d8935050ee7ca1f/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b", size = 205260, upload-time = "2026-07-07T14:33:03.698Z" },
+    { url = "https://files.pythonhosted.org/packages/0c/74/2f62c8821b969ea3bd67cc2e6976834f48ca5d12664d2559ebcd9bcfbed7/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db", size = 217786, upload-time = "2026-07-07T14:33:05.12Z" },
+    { url = "https://files.pythonhosted.org/packages/d9/8d/feabb82cb49fcad14515b1d7d1ca4787b0da7fc723a212bf89bc9e0fac52/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993", size = 216798, upload-time = "2026-07-07T14:33:06.629Z" },
+    { url = "https://files.pythonhosted.org/packages/a5/ff/c946d63bc3786d5b84d960b0f7ab7e25b828486a946b5aa997625bcaf6a6/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da", size = 206429, upload-time = "2026-07-07T14:33:08.006Z" },
+    { url = "https://files.pythonhosted.org/packages/af/ba/5e5007c370702f85d2ef75791fac7943ed41e080364a673b20142e430e3e/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3", size = 223066, upload-time = "2026-07-07T14:33:09.783Z" },
+    { url = "https://files.pythonhosted.org/packages/83/d5/9096aa3cf532dfad237861544eb47a0f20d5adbf1039760fed8eaae935d9/charset_normalizer-3.4.9-cp311-cp311-win32.whl", hash = "sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d", size = 150456, upload-time = "2026-07-07T14:33:11.217Z" },
+    { url = "https://files.pythonhosted.org/packages/ed/a1/e29995109e455dc8eff8d0fac6ae509be39561318a7cfeac5d33ad029213/charset_normalizer-3.4.9-cp311-cp311-win_amd64.whl", hash = "sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1", size = 161410, upload-time = "2026-07-07T14:33:12.743Z" },
+    { url = "https://files.pythonhosted.org/packages/4f/8d/1569f4d0032d6ba2a4fe4591c35bf87868c600c41a71eb5c2e1ffa8464c2/charset_normalizer-3.4.9-cp311-cp311-win_arm64.whl", hash = "sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec", size = 152649, upload-time = "2026-07-07T14:33:14.173Z" },
+    { url = "https://files.pythonhosted.org/packages/70/4a/ecbd131485c07fcdfad54e28946d513e3da22ef3b4bd854dcafae54ec739/charset_normalizer-3.4.9-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0", size = 319300, upload-time = "2026-07-07T14:33:15.666Z" },
+    { url = "https://files.pythonhosted.org/packages/ec/96/5d9364e3342d69f3a045e1777bc47c85c383e6e9466d561b33fdb419d1f9/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9", size = 215802, upload-time = "2026-07-07T14:33:17.031Z" },
+    { url = "https://files.pythonhosted.org/packages/4b/4c/5361f9aa7f2cb58d94f2ab831b3d493f69efb1d239654b4744e3c09527cb/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44", size = 237171, upload-time = "2026-07-07T14:33:18.576Z" },
+    { url = "https://files.pythonhosted.org/packages/50/78/ce342ca4ff30b2eb49fe6d9578df85974f90c67d294113e94efdd9664cbd/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9", size = 233075, upload-time = "2026-07-07T14:33:20.084Z" },
+    { url = "https://files.pythonhosted.org/packages/01/c4/4fa4c8b3097a11f3c5f09a35b72ed6855fb1d332469504962ab7bafcc702/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd", size = 224256, upload-time = "2026-07-07T14:33:21.747Z" },
+    { url = "https://files.pythonhosted.org/packages/87/3a/ad914516df7e358a81aae018caa5e0470ba827fa6d763b1d2e87d920a5f6/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84", size = 208784, upload-time = "2026-07-07T14:33:23.313Z" },
+    { url = "https://files.pythonhosted.org/packages/d7/74/3c12f9755717dfe5c5c87da63f35d765fa0c00382ec26bf23f7fae34f2ba/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b", size = 219928, upload-time = "2026-07-07T14:33:24.814Z" },
+    { url = "https://files.pythonhosted.org/packages/33/9a/895095b83e7907abd6d3d99aad3a38ad0d9686cc186cb0c94c24320fe63e/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde", size = 218489, upload-time = "2026-07-07T14:33:26.42Z" },
+    { url = "https://files.pythonhosted.org/packages/a1/34/ef5c05f412f42520d7709b7d3784d19640839eb7366ded1755511585429f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39", size = 210267, upload-time = "2026-07-07T14:33:27.952Z" },
+    { url = "https://files.pythonhosted.org/packages/83/dc/9b29fa4412b318bf3bfea985c35d67eb55e04b59a7c3f2237168b0e0be6f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62", size = 226030, upload-time = "2026-07-07T14:33:29.397Z" },
+    { url = "https://files.pythonhosted.org/packages/0e/42/6dbc00b8cd16011691203e33570fa42ed5746599a2e878112d16eab403a3/charset_normalizer-3.4.9-cp312-cp312-win32.whl", hash = "sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642", size = 151185, upload-time = "2026-07-07T14:33:30.781Z" },
+    { url = "https://files.pythonhosted.org/packages/80/cc/f920afd1a23c58ccd53c1d36085a71893a4737ff5e66e0371efab6809850/charset_normalizer-3.4.9-cp312-cp312-win_amd64.whl", hash = "sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0", size = 162557, upload-time = "2026-07-07T14:33:32.176Z" },
+    { url = "https://files.pythonhosted.org/packages/f0/e6/0386d43a261ff4e4b30c5857af7df877254b46bec7b9d1b74b6bf969a90b/charset_normalizer-3.4.9-cp312-cp312-win_arm64.whl", hash = "sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2", size = 152665, upload-time = "2026-07-07T14:33:33.711Z" },
+    { url = "https://files.pythonhosted.org/packages/b2/06/97ec2aeae780b31d742b6352218b43841a6871e2564578ca522dce4a45c3/charset_normalizer-3.4.9-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614", size = 317688, upload-time = "2026-07-07T14:33:35.408Z" },
+    { url = "https://files.pythonhosted.org/packages/d0/39/8ff066c672434225f8d25f8b739f992af250944392173dcc88362681c9bf/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698", size = 214982, upload-time = "2026-07-07T14:33:36.996Z" },
+    { url = "https://files.pythonhosted.org/packages/92/8f/3a47a3667c83c2df9483d91644c6c107de3bf8874aa1793da9d3012eb986/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b", size = 236460, upload-time = "2026-07-07T14:33:38.536Z" },
+    { url = "https://files.pythonhosted.org/packages/f1/60/b22cdbee7e4013dab8b0d7647fc6181120fbbbc8f7025c226d15bd5a47fc/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9", size = 232003, upload-time = "2026-07-07T14:33:40.059Z" },
+    { url = "https://files.pythonhosted.org/packages/ea/f8/72eb13dcabe7257035cea8aefd922caad2f110d252bf9f67c4c2ca763aee/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33", size = 223149, upload-time = "2026-07-07T14:33:41.631Z" },
+    { url = "https://files.pythonhosted.org/packages/b0/3e/faee8f9de92b14ee1198e9163252bb15efee7301b31256a3b6d9ebfdd0dd/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63", size = 207901, upload-time = "2026-07-07T14:33:43.209Z" },
+    { url = "https://files.pythonhosted.org/packages/3a/25/45f30093ae27dd7b92a793b61882a38685f993700113ca36e0c9c14965e1/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0", size = 219176, upload-time = "2026-07-07T14:33:44.725Z" },
+    { url = "https://files.pythonhosted.org/packages/48/18/c8f397329c35e32f6a837e488986f4ae03bd2abebc453b48714991630c2f/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe", size = 217356, upload-time = "2026-07-07T14:33:46.192Z" },
+    { url = "https://files.pythonhosted.org/packages/86/7e/5ce0bba863470fd1902d5e5843968951bddf38abe4742fc97116ef4598b3/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35", size = 209614, upload-time = "2026-07-07T14:33:47.705Z" },
+    { url = "https://files.pythonhosted.org/packages/6c/ef/2473d3c4d869155be4af1191111d59c4d5c4e0173026f7e85b176e23bf65/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8", size = 224991, upload-time = "2026-07-07T14:33:49.238Z" },
+    { url = "https://files.pythonhosted.org/packages/d0/a3/53ddae3db108a088156aa8ddfafd411ebbc1340f48c5573f697b27f69a39/charset_normalizer-3.4.9-cp313-cp313-win32.whl", hash = "sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9", size = 150622, upload-time = "2026-07-07T14:33:50.711Z" },
+    { url = "https://files.pythonhosted.org/packages/e8/ef/6953a77c7cf2c2ff9998e6f575ab3e380119f100223381565a4f94c1f836/charset_normalizer-3.4.9-cp313-cp313-win_amd64.whl", hash = "sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115", size = 161947, upload-time = "2026-07-07T14:33:52.197Z" },
+    { url = "https://files.pythonhosted.org/packages/6e/fb/d560d1d1555debbfe7849d9cac6145c1b537709d79576bf22557ed803b82/charset_normalizer-3.4.9-cp313-cp313-win_arm64.whl", hash = "sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012", size = 152594, upload-time = "2026-07-07T14:33:53.486Z" },
+    { url = "https://files.pythonhosted.org/packages/7e/8d/496817fa0944239ecae662dd57ea765cfeaec6a735f9f025d4b7b72e7143/charset_normalizer-3.4.9-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380", size = 317253, upload-time = "2026-07-07T14:33:54.994Z" },
+    { url = "https://files.pythonhosted.org/packages/2b/f9/ef4a69ea338ad3c0deceea0f5f7d2380ae8b52132b06d652cb0d2cd86706/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9", size = 215898, upload-time = "2026-07-07T14:33:56.334Z" },
+    { url = "https://files.pythonhosted.org/packages/8c/e7/5ddfd76fc061eb52de219658a4aa431cbacadf0a0219c8854f00da50d289/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4", size = 236718, upload-time = "2026-07-07T14:33:57.9Z" },
+    { url = "https://files.pythonhosted.org/packages/49/ba/768fa3f36048d81c477a0ce61f813bc1454d80917ccfe550abd9f44f5e24/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a", size = 232519, upload-time = "2026-07-07T14:33:59.811Z" },
+    { url = "https://files.pythonhosted.org/packages/f4/c4/b3e049d2aa3766180c78507110543d9d50894cc97f57de543f1be521dcdc/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046", size = 223143, upload-time = "2026-07-07T14:34:01.517Z" },
+    { url = "https://files.pythonhosted.org/packages/19/79/55c32d06d76ae4feafe053f061f3e3ab70bcf19f4007797ce8c3efda7830/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81", size = 206742, upload-time = "2026-07-07T14:34:03.04Z" },
+    { url = "https://files.pythonhosted.org/packages/10/e0/47c079dd82d217c807479cd59ffd30af56307ea31c108b75758970459ad3/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917", size = 219191, upload-time = "2026-07-07T14:34:04.657Z" },
+    { url = "https://files.pythonhosted.org/packages/42/ab/b9bc2e77d6b44a7e46ef62ec5cac1c9a6ba7b9135a5d560f002696ec9995/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41", size = 218328, upload-time = "2026-07-07T14:34:06.115Z" },
+    { url = "https://files.pythonhosted.org/packages/f1/78/c9c71d599f5aa2d42bcdd35cbbd46d7f535351a57e40ff7d8e5a7e219401/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1", size = 207406, upload-time = "2026-07-07T14:34:07.554Z" },
+    { url = "https://files.pythonhosted.org/packages/f6/39/c914445c321a845097ce4f6ac7de9a18228a77b766272125a1ce00d851eb/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf", size = 225157, upload-time = "2026-07-07T14:34:09.061Z" },
+    { url = "https://files.pythonhosted.org/packages/9b/f2/c0d4b8508565a36bc5c624e88ed297f5b0b1095011034d7f5b83a69908b5/charset_normalizer-3.4.9-cp314-cp314-win32.whl", hash = "sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48", size = 151095, upload-time = "2026-07-07T14:34:10.901Z" },
+    { url = "https://files.pythonhosted.org/packages/49/fd/a1d26144398c67486422a72bf5812cda22cb4ccfcd95a290fb41ceb4b8e2/charset_normalizer-3.4.9-cp314-cp314-win_amd64.whl", hash = "sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b", size = 162796, upload-time = "2026-07-07T14:34:12.47Z" },
+    { url = "https://files.pythonhosted.org/packages/20/95/d75e82f8ce9fd323ebf059c16c9aadefb22a1ecde13b7840b35835e4886c/charset_normalizer-3.4.9-cp314-cp314-win_arm64.whl", hash = "sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519", size = 153334, upload-time = "2026-07-07T14:34:14.044Z" },
+    { url = "https://files.pythonhosted.org/packages/00/5e/17398df3a139985ba9d11ed072531986f408c8fca952835ef1ab1820c02b/charset_normalizer-3.4.9-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198", size = 338848, upload-time = "2026-07-07T14:34:15.688Z" },
+    { url = "https://files.pythonhosted.org/packages/cd/91/7253a32e86b7e1d1239b1b36ba6dd0f021a21107ab33054b53119cc083b9/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32", size = 223022, upload-time = "2026-07-07T14:34:17.248Z" },
+    { url = "https://files.pythonhosted.org/packages/cb/32/2e64bd2be10e89c61e57ebe6a93fd98ae88eb7ebe414b5121f22c96c69eb/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632", size = 241590, upload-time = "2026-07-07T14:34:18.813Z" },
+    { url = "https://files.pythonhosted.org/packages/3d/ef/d96ec496cfea0c21db43b0ad03891308b02388d054cc902cf0e5a1ad6a88/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf", size = 239584, upload-time = "2026-07-07T14:34:20.52Z" },
+    { url = "https://files.pythonhosted.org/packages/d4/ce/9af95f7876194bd7a14e3dfe4a4de2e0bff02666a3910d72beafd06cc297/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990", size = 230224, upload-time = "2026-07-07T14:34:22.189Z" },
+    { url = "https://files.pythonhosted.org/packages/52/94/af74dde74a3996bd959c350709bfe50e297823d70a8c1cbd54b838880863/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d", size = 212667, upload-time = "2026-07-07T14:34:23.857Z" },
+    { url = "https://files.pythonhosted.org/packages/ee/f0/f1c4fe746c395922961b5916ed1d7d6e7d4c84851d19ed43cc89980ec953/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e", size = 227179, upload-time = "2026-07-07T14:34:25.586Z" },
+    { url = "https://files.pythonhosted.org/packages/e4/56/6c745619ac397e8871e2bcd3cea1eec86b877488f33888b3aef5c3ed506e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c", size = 225372, upload-time = "2026-07-07T14:34:27.212Z" },
+    { url = "https://files.pythonhosted.org/packages/78/ad/98aae8630ac71f16711968e38a5acfecce41b778bf2f0312851020f565a8/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2", size = 215222, upload-time = "2026-07-07T14:34:28.774Z" },
+    { url = "https://files.pythonhosted.org/packages/f7/40/9593d54209765207a7f11073c06494c1721e4ca4a0a426c597679bf7f91e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534", size = 231958, upload-time = "2026-07-07T14:34:30.345Z" },
+    { url = "https://files.pythonhosted.org/packages/b1/27/693ee5e8a18191eb38647360c51cd505013e2bd3b366aa43fd5344c21e3c/charset_normalizer-3.4.9-cp314-cp314t-win32.whl", hash = "sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226", size = 155580, upload-time = "2026-07-07T14:34:31.884Z" },
+    { url = "https://files.pythonhosted.org/packages/80/3f/bd97d3d9c613013d07cb7733d299385b41df37f0471310f5a73dc359f0b8/charset_normalizer-3.4.9-cp314-cp314t-win_amd64.whl", hash = "sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177", size = 167620, upload-time = "2026-07-07T14:34:33.438Z" },
+    { url = "https://files.pythonhosted.org/packages/3d/c6/eee9dca4439b1061f76373f06ea855678cc4a64c1c3c90b50e479edbb8eb/charset_normalizer-3.4.9-cp314-cp314t-win_arm64.whl", hash = "sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501", size = 158037, upload-time = "2026-07-07T14:34:35.018Z" },
+    { url = "https://files.pythonhosted.org/packages/a6/ec/81e22253f4b7091eca6515bb3da5e45d05a663f7f567bb745695dc60f892/charset_normalizer-3.4.9-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a", size = 306122, upload-time = "2026-07-07T14:34:36.607Z" },
+    { url = "https://files.pythonhosted.org/packages/c8/53/a8c042eb9eee4716f4d42a0f5a571eb32a09ec429be9fb0b8b9d765393ba/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4", size = 206284, upload-time = "2026-07-07T14:34:38.166Z" },
+    { url = "https://files.pythonhosted.org/packages/14/cb/1db8b96547ee3186cd2dd7f2e59dd560a9b80748f3604171f3c153d62811/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94", size = 226837, upload-time = "2026-07-07T14:34:39.77Z" },
+    { url = "https://files.pythonhosted.org/packages/6a/05/c94d5cd23396289c54c93b02e0273b4dd8921641d9968c4828caf9bbaad9/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5", size = 222199, upload-time = "2026-07-07T14:34:41.391Z" },
+    { url = "https://files.pythonhosted.org/packages/6d/46/79847edd07244a4a2d443c6655a7b6ee94203c21539414b059f32713c357/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84", size = 214344, upload-time = "2026-07-07T14:34:42.986Z" },
+    { url = "https://files.pythonhosted.org/packages/ec/b4/ef5a49b2e77c00deb43bb3256592b115ba9e4346016e82c516b8d215bf68/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_armv7l.whl", hash = "sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4", size = 199988, upload-time = "2026-07-07T14:34:44.685Z" },
+    { url = "https://files.pythonhosted.org/packages/3d/ca/ad1d7c7d3077dab873f539d3e1d083c0845a762cb0bafdfbe3ef93add598/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f", size = 211908, upload-time = "2026-07-07T14:34:46.227Z" },
+    { url = "https://files.pythonhosted.org/packages/ed/61/710738687f90d01c06a04ed52d6ca1e62dd9b1d8cc2567098167c4691034/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833", size = 209320, upload-time = "2026-07-07T14:34:47.753Z" },
+    { url = "https://files.pythonhosted.org/packages/5f/c0/6eec7bdabe6cbbcc274ec04596f6d93865751a0541d33d60d1ce179bd372/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_armv7l.whl", hash = "sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba", size = 200980, upload-time = "2026-07-07T14:34:49.362Z" },
+    { url = "https://files.pythonhosted.org/packages/eb/78/59344ff9a4a7b5f6530bf7bec2c980047cc42c3a616596cdbd8cb5c1a1af/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29", size = 216545, upload-time = "2026-07-07T14:34:50.98Z" },
+    { url = "https://files.pythonhosted.org/packages/17/6d/bff78a4bacc4891bc63ec5bdc6776d8c85e47fab93d0d5f6223068fad0a4/charset_normalizer-3.4.9-cp39-cp39-win32.whl", hash = "sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9", size = 146256, upload-time = "2026-07-07T14:34:52.509Z" },
+    { url = "https://files.pythonhosted.org/packages/a2/55/86048bde1c9d0352940bd7b87d825091a52aef67d01cde6c6f7342c5b552/charset_normalizer-3.4.9-cp39-cp39-win_amd64.whl", hash = "sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b", size = 156413, upload-time = "2026-07-07T14:34:54.117Z" },
+    { url = "https://files.pythonhosted.org/packages/28/e9/9fb6099b868c82a40698a748ae0fbd4f31ccc13844c176a07158ba2abbfd/charset_normalizer-3.4.9-cp39-cp39-win_arm64.whl", hash = "sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe", size = 147887, upload-time = "2026-07-07T14:34:55.51Z" },
+    { url = "https://files.pythonhosted.org/packages/98/2b/f97f1c193fb855c345d678f5077d6926034db0722df74c8f057020e05a25/charset_normalizer-3.4.9-py3-none-any.whl", hash = "sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5", size = 64538, upload-time = "2026-07-07T14:34:56.993Z" },
+]
+
+[[package]]
+name = "click"
+version = "8.4.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "colorama", marker = "sys_platform == 'win32'" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/76/d4/81420972a676e8ffea40450d8c8c92943e7218a78fe9b64359836cc9876b/click-8.4.2.tar.gz", hash = "sha256:9a6cea6e60b17ebe0a44c5cc636d94f09bd66142c1cd7d8b4cd731c4917a15f6", size = 338000, upload-time = "2026-06-24T17:45:15.148Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/fb/e2/79c688af8b210d232694e31e59da9f6ec747bae31c3f5946e4e9b98860d5/click-8.4.2-py3-none-any.whl", hash = "sha256:e6f9f66136c816745b9d65817da91d61d957fb16e02e4dcd0552553c5a197b76", size = 119243, upload-time = "2026-06-24T17:45:13.73Z" },
+]
+
 [[package]]
 name = "colorama"
 version = "0.4.6"
@@ -166,6 +313,18 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
 ]
 
+[[package]]
+name = "ghp-import"
+version = "2.1.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "python-dateutil" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/d9/29/d40217cbe2f6b1359e00c6c307bb3fc876ba74068cbab3dde77f03ca0dc4/ghp-import-2.1.0.tar.gz", hash = "sha256:9c535c4c61193c2df8871222567d7fd7e5014d835f97dc7b7439069e2413d343", size = 10943, upload-time = "2022-05-02T15:47:16.11Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f7/ec/67fbef5d497f86283db54c22eec6f6140243aae73265799baaaa19cd17fb/ghp_import-2.1.0-py3-none-any.whl", hash = "sha256:8337dd7b50877f163d4c0289bc1f1c7f127550241988d568c1db512c4324a619", size = 11034, upload-time = "2022-05-02T15:47:14.552Z" },
+]
+
 [[package]]
 name = "h11"
 version = "0.16.0"
@@ -228,6 +387,27 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" },
 ]
 
+[[package]]
+name = "jinja2"
+version = "3.1.6"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "markupsafe" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" },
+]
+
+[[package]]
+name = "markdown"
+version = "3.10.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/29/6f/da4c6aea59b3001f2e8c0ec7497475aadaf3b021c10cab5b2858f0f32b26/markdown-3.10.3.tar.gz", hash = "sha256:3589362618f743188b4d955b874402bc814f4f83f544dc207719f4baa7d9c45f", size = 372596, upload-time = "2026-07-30T19:05:29.005Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/64/69/4a5af2bc115a9a33fefe51709749de8262be3f9ba063d1753a837cdbc49c/markdown-3.10.3-py3-none-any.whl", hash = "sha256:fa6c92a00a4a3c98b22728c64a935ae1928250ae65058a6ded814d2cc29a4cea", size = 110757, upload-time = "2026-07-30T19:05:27.883Z" },
+]
+
 [[package]]
 name = "markdown-it-py"
 version = "4.2.0"
@@ -240,6 +420,19 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" },
 ]
 
+[[package]]
+name = "markdownify"
+version = "1.2.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "beautifulsoup4" },
+    { name = "six" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/92/ab/d1297139c0e2ceb151ae564c8c4f57ac0155d8f1f8b4cbd5d6523c82ea36/markdownify-1.2.3.tar.gz", hash = "sha256:1a176f05522c8a2cb1dd3ab9d307dcdadbed5c26ae717855bfc42b3b6d38d937", size = 18852, upload-time = "2026-06-30T20:27:39.06Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/04/10/fa543d484e8b1199243fe20eedd02cc5af050edebce98a7293a5773df592/markdownify-1.2.3-py3-none-any.whl", hash = "sha256:a189a0bedfd14009030fde5f85bb6f77c56897cb839b5c25315dd7d4e3e290ba", size = 15732, upload-time = "2026-06-30T20:27:38.094Z" },
+]
+
 [[package]]
 name = "marko"
 version = "2.2.3"
@@ -249,6 +442,114 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/97/50/0a8fab45fa374820c27cc4c3178c4914c60902ba9d6404a692a979e20dbc/marko-2.2.3-py3-none-any.whl", hash = "sha256:8e1d7a0387281e59dfbc52a381b58c570156970e36b2bbe047f8a3a2f368cacc", size = 42951, upload-time = "2026-05-28T02:07:38.373Z" },
 ]
 
+[[package]]
+name = "markupsafe"
+version = "3.0.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/e8/4b/3541d44f3937ba468b75da9eebcae497dcf67adb65caa16760b0a6807ebb/markupsafe-3.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559", size = 11631, upload-time = "2025-09-27T18:36:05.558Z" },
+    { url = "https://files.pythonhosted.org/packages/98/1b/fbd8eed11021cabd9226c37342fa6ca4e8a98d8188a8d9b66740494960e4/markupsafe-3.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419", size = 12057, upload-time = "2025-09-27T18:36:07.165Z" },
+    { url = "https://files.pythonhosted.org/packages/40/01/e560d658dc0bb8ab762670ece35281dec7b6c1b33f5fbc09ebb57a185519/markupsafe-3.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695", size = 22050, upload-time = "2025-09-27T18:36:08.005Z" },
+    { url = "https://files.pythonhosted.org/packages/af/cd/ce6e848bbf2c32314c9b237839119c5a564a59725b53157c856e90937b7a/markupsafe-3.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591", size = 20681, upload-time = "2025-09-27T18:36:08.881Z" },
+    { url = "https://files.pythonhosted.org/packages/c9/2a/b5c12c809f1c3045c4d580b035a743d12fcde53cf685dbc44660826308da/markupsafe-3.0.3-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c", size = 20705, upload-time = "2025-09-27T18:36:10.131Z" },
+    { url = "https://files.pythonhosted.org/packages/cf/e3/9427a68c82728d0a88c50f890d0fc072a1484de2f3ac1ad0bfc1a7214fd5/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f", size = 21524, upload-time = "2025-09-27T18:36:11.324Z" },
+    { url = "https://files.pythonhosted.org/packages/bc/36/23578f29e9e582a4d0278e009b38081dbe363c5e7165113fad546918a232/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6", size = 20282, upload-time = "2025-09-27T18:36:12.573Z" },
+    { url = "https://files.pythonhosted.org/packages/56/21/dca11354e756ebd03e036bd8ad58d6d7168c80ce1fe5e75218e4945cbab7/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1", size = 20745, upload-time = "2025-09-27T18:36:13.504Z" },
+    { url = "https://files.pythonhosted.org/packages/87/99/faba9369a7ad6e4d10b6a5fbf71fa2a188fe4a593b15f0963b73859a1bbd/markupsafe-3.0.3-cp310-cp310-win32.whl", hash = "sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa", size = 14571, upload-time = "2025-09-27T18:36:14.779Z" },
+    { url = "https://files.pythonhosted.org/packages/d6/25/55dc3ab959917602c96985cb1253efaa4ff42f71194bddeb61eb7278b8be/markupsafe-3.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8", size = 15056, upload-time = "2025-09-27T18:36:16.125Z" },
+    { url = "https://files.pythonhosted.org/packages/d0/9e/0a02226640c255d1da0b8d12e24ac2aa6734da68bff14c05dd53b94a0fc3/markupsafe-3.0.3-cp310-cp310-win_arm64.whl", hash = "sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1", size = 13932, upload-time = "2025-09-27T18:36:17.311Z" },
+    { url = "https://files.pythonhosted.org/packages/08/db/fefacb2136439fc8dd20e797950e749aa1f4997ed584c62cfb8ef7c2be0e/markupsafe-3.0.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad", size = 11631, upload-time = "2025-09-27T18:36:18.185Z" },
+    { url = "https://files.pythonhosted.org/packages/e1/2e/5898933336b61975ce9dc04decbc0a7f2fee78c30353c5efba7f2d6ff27a/markupsafe-3.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a", size = 12058, upload-time = "2025-09-27T18:36:19.444Z" },
+    { url = "https://files.pythonhosted.org/packages/1d/09/adf2df3699d87d1d8184038df46a9c80d78c0148492323f4693df54e17bb/markupsafe-3.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50", size = 24287, upload-time = "2025-09-27T18:36:20.768Z" },
+    { url = "https://files.pythonhosted.org/packages/30/ac/0273f6fcb5f42e314c6d8cd99effae6a5354604d461b8d392b5ec9530a54/markupsafe-3.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf", size = 22940, upload-time = "2025-09-27T18:36:22.249Z" },
+    { url = "https://files.pythonhosted.org/packages/19/ae/31c1be199ef767124c042c6c3e904da327a2f7f0cd63a0337e1eca2967a8/markupsafe-3.0.3-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f", size = 21887, upload-time = "2025-09-27T18:36:23.535Z" },
+    { url = "https://files.pythonhosted.org/packages/b2/76/7edcab99d5349a4532a459e1fe64f0b0467a3365056ae550d3bcf3f79e1e/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a", size = 23692, upload-time = "2025-09-27T18:36:24.823Z" },
+    { url = "https://files.pythonhosted.org/packages/a4/28/6e74cdd26d7514849143d69f0bf2399f929c37dc2b31e6829fd2045b2765/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115", size = 21471, upload-time = "2025-09-27T18:36:25.95Z" },
+    { url = "https://files.pythonhosted.org/packages/62/7e/a145f36a5c2945673e590850a6f8014318d5577ed7e5920a4b3448e0865d/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a", size = 22923, upload-time = "2025-09-27T18:36:27.109Z" },
+    { url = "https://files.pythonhosted.org/packages/0f/62/d9c46a7f5c9adbeeeda52f5b8d802e1094e9717705a645efc71b0913a0a8/markupsafe-3.0.3-cp311-cp311-win32.whl", hash = "sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19", size = 14572, upload-time = "2025-09-27T18:36:28.045Z" },
+    { url = "https://files.pythonhosted.org/packages/83/8a/4414c03d3f891739326e1783338e48fb49781cc915b2e0ee052aa490d586/markupsafe-3.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01", size = 15077, upload-time = "2025-09-27T18:36:29.025Z" },
+    { url = "https://files.pythonhosted.org/packages/35/73/893072b42e6862f319b5207adc9ae06070f095b358655f077f69a35601f0/markupsafe-3.0.3-cp311-cp311-win_arm64.whl", hash = "sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c", size = 13876, upload-time = "2025-09-27T18:36:29.954Z" },
+    { url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615, upload-time = "2025-09-27T18:36:30.854Z" },
+    { url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020, upload-time = "2025-09-27T18:36:31.971Z" },
+    { url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332, upload-time = "2025-09-27T18:36:32.813Z" },
+    { url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947, upload-time = "2025-09-27T18:36:33.86Z" },
+    { url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962, upload-time = "2025-09-27T18:36:35.099Z" },
+    { url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760, upload-time = "2025-09-27T18:36:36.001Z" },
+    { url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529, upload-time = "2025-09-27T18:36:36.906Z" },
+    { url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015, upload-time = "2025-09-27T18:36:37.868Z" },
+    { url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540, upload-time = "2025-09-27T18:36:38.761Z" },
+    { url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105, upload-time = "2025-09-27T18:36:39.701Z" },
+    { url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906, upload-time = "2025-09-27T18:36:40.689Z" },
+    { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" },
+    { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" },
+    { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" },
+    { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" },
+    { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" },
+    { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" },
+    { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" },
+    { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" },
+    { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" },
+    { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" },
+    { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" },
+    { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" },
+    { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" },
+    { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" },
+    { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" },
+    { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" },
+    { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" },
+    { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" },
+    { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" },
+    { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" },
+    { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" },
+    { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" },
+    { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" },
+    { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" },
+    { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" },
+    { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" },
+    { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" },
+    { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" },
+    { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" },
+    { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" },
+    { url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" },
+    { url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" },
+    { url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" },
+    { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" },
+    { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" },
+    { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" },
+    { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" },
+    { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" },
+    { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" },
+    { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" },
+    { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" },
+    { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" },
+    { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" },
+    { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" },
+    { url = "https://files.pythonhosted.org/packages/56/23/0d8c13a44bde9154821586520840643467aee574d8ce79a17da539ee7fed/markupsafe-3.0.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26", size = 11623, upload-time = "2025-09-27T18:37:29.296Z" },
+    { url = "https://files.pythonhosted.org/packages/fd/23/07a2cb9a8045d5f3f0890a8c3bc0859d7a47bfd9a560b563899bec7b72ed/markupsafe-3.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc", size = 12049, upload-time = "2025-09-27T18:37:30.234Z" },
+    { url = "https://files.pythonhosted.org/packages/bc/e4/6be85eb81503f8e11b61c0b6369b6e077dcf0a74adbd9ebf6b349937b4e9/markupsafe-3.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c", size = 21923, upload-time = "2025-09-27T18:37:31.177Z" },
+    { url = "https://files.pythonhosted.org/packages/6f/bc/4dc914ead3fe6ddaef035341fee0fc956949bbd27335b611829292b89ee2/markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42", size = 20543, upload-time = "2025-09-27T18:37:32.168Z" },
+    { url = "https://files.pythonhosted.org/packages/89/6e/5fe81fbcfba4aef4093d5f856e5c774ec2057946052d18d168219b7bd9f9/markupsafe-3.0.3-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b", size = 20585, upload-time = "2025-09-27T18:37:33.166Z" },
+    { url = "https://files.pythonhosted.org/packages/f6/f6/e0e5a3d3ae9c4020f696cd055f940ef86b64fe88de26f3a0308b9d3d048c/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758", size = 21387, upload-time = "2025-09-27T18:37:34.185Z" },
+    { url = "https://files.pythonhosted.org/packages/c8/25/651753ef4dea08ea790f4fbb65146a9a44a014986996ca40102e237aa49a/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2", size = 20133, upload-time = "2025-09-27T18:37:35.138Z" },
+    { url = "https://files.pythonhosted.org/packages/dc/0a/c3cf2b4fef5f0426e8a6d7fce3cb966a17817c568ce59d76b92a233fdbec/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d", size = 20588, upload-time = "2025-09-27T18:37:36.096Z" },
+    { url = "https://files.pythonhosted.org/packages/cd/1b/a7782984844bd519ad4ffdbebbba2671ec5d0ebbeac34736c15fb86399e8/markupsafe-3.0.3-cp39-cp39-win32.whl", hash = "sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7", size = 14566, upload-time = "2025-09-27T18:37:37.09Z" },
+    { url = "https://files.pythonhosted.org/packages/18/1f/8d9c20e1c9440e215a44be5ab64359e207fcb4f675543f1cf9a2a7f648d0/markupsafe-3.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e", size = 15053, upload-time = "2025-09-27T18:37:38.054Z" },
+    { url = "https://files.pythonhosted.org/packages/4e/d3/fe08482b5cd995033556d45041a4f4e76e7f0521112a9c9991d40d39825f/markupsafe-3.0.3-cp39-cp39-win_arm64.whl", hash = "sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8", size = 13928, upload-time = "2025-09-27T18:37:39.037Z" },
+]
+
+[[package]]
+name = "mdformat"
+version = "1.0.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "markdown-it-py" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/3f/05/32b5e14b192b0a8a309f32232c580aefedd9d06017cb8fe8fce34bec654c/mdformat-1.0.0.tar.gz", hash = "sha256:4954045fcae797c29f86d4ad879e43bb151fa55dbaf74ac6eaeacf1d45bb3928", size = 56953, upload-time = "2025-10-16T12:05:03.695Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/54/9a/8fe71b95985ca7a4001effbcc58e5a07a1f2a2884203f74dcf48a3b08315/mdformat-1.0.0-py3-none-any.whl", hash = "sha256:bca015d65a1d063a02e885a91daee303057bc7829c2cd37b2075a50dbb65944b", size = 53288, upload-time = "2025-10-16T12:05:02.607Z" },
+]
+
 [[package]]
 name = "mdurl"
 version = "0.1.2"
@@ -258,6 +559,123 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" },
 ]
 
+[[package]]
+name = "mergedeep"
+version = "1.3.4"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/3a/41/580bb4006e3ed0361b8151a01d324fb03f420815446c7def45d02f74c270/mergedeep-1.3.4.tar.gz", hash = "sha256:0096d52e9dad9939c3d975a774666af186eda617e6ca84df4c94dec30004f2a8", size = 4661, upload-time = "2021-02-05T18:55:30.623Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/2c/19/04f9b178c2d8a15b076c8b5140708fa6ffc5601fb6f1e975537072df5b2a/mergedeep-1.3.4-py3-none-any.whl", hash = "sha256:70775750742b25c0d8f36c55aed03d24c3384d17c951b3175d898bd778ef0307", size = 6354, upload-time = "2021-02-05T18:55:29.583Z" },
+]
+
+[[package]]
+name = "mkdocs"
+version = "1.6.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "click" },
+    { name = "colorama", marker = "sys_platform == 'win32'" },
+    { name = "ghp-import" },
+    { name = "jinja2" },
+    { name = "markdown" },
+    { name = "markupsafe" },
+    { name = "mergedeep" },
+    { name = "mkdocs-get-deps" },
+    { name = "packaging" },
+    { name = "pathspec" },
+    { name = "pyyaml" },
+    { name = "pyyaml-env-tag" },
+    { name = "watchdog" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/bc/c6/bbd4f061bd16b378247f12953ffcb04786a618ce5e904b8c5a01a0309061/mkdocs-1.6.1.tar.gz", hash = "sha256:7b432f01d928c084353ab39c57282f29f92136665bdd6abf7c1ec8d822ef86f2", size = 3889159, upload-time = "2024-08-30T12:24:06.899Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/22/5b/dbc6a8cddc9cfa9c4971d59fb12bb8d42e161b7e7f8cc89e49137c5b279c/mkdocs-1.6.1-py3-none-any.whl", hash = "sha256:db91759624d1647f3f34aa0c3f327dd2601beae39a366d6e064c03468d35c20e", size = 3864451, upload-time = "2024-08-30T12:24:05.054Z" },
+]
+
+[[package]]
+name = "mkdocs-get-deps"
+version = "0.2.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "mergedeep" },
+    { name = "platformdirs" },
+    { name = "pyyaml" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/ce/25/b3cccb187655b9393572bde9b09261d267c3bf2f2cdabe347673be5976a6/mkdocs_get_deps-0.2.2.tar.gz", hash = "sha256:8ee8d5f316cdbbb2834bc1df6e69c08fe769a83e040060de26d3c19fad3599a1", size = 11047, upload-time = "2026-03-10T02:46:33.632Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/88/29/744136411e785c4b0b744d5413e56555265939ab3a104c6a4b719dad33fd/mkdocs_get_deps-0.2.2-py3-none-any.whl", hash = "sha256:e7878cbeac04860b8b5e0ca31d3abad3df9411a75a32cde82f8e44b6c16ff650", size = 9555, upload-time = "2026-03-10T02:46:32.256Z" },
+]
+
+[[package]]
+name = "mkdocs-github-admonitions-plugin"
+version = "0.1.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "mkdocs" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/12/62/37f2080af26ec1d569bb21eb2a4d54f5ea54d36a92938abbbc37c6ab671b/mkdocs_github_admonitions_plugin-0.1.1.tar.gz", hash = "sha256:7f81520a0681b9955952d73b21ce99b923921830b6b6d1ace9b3fb95cd1fb61f", size = 5151, upload-time = "2025-06-02T09:45:12.321Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/e8/a3/8cfd5d9a651612b0d2a15176341b040d43d27b063e17684cb353ec7ce789/mkdocs_github_admonitions_plugin-0.1.1-py3-none-any.whl", hash = "sha256:824dc821764171943c1043c88218d4af0329693870ba9be657f890d484a0aa85", size = 5493, upload-time = "2025-06-02T09:45:11.068Z" },
+]
+
+[[package]]
+name = "mkdocs-llmstxt"
+version = "0.2.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "beautifulsoup4" },
+    { name = "markdownify" },
+    { name = "mdformat" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/ee/25/263ea9c16d1f95f30d9eb1b76e63eb50a88a1ec9fad1829281bab7a371eb/mkdocs_llmstxt-0.2.0.tar.gz", hash = "sha256:104f10b8101167d6baf7761942b4743869be3d8f8a8d909f4e9e0b63307f709e", size = 41376, upload-time = "2025-04-08T13:18:48.664Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/24/29/0a33f7d8499a01dd7fd0d90fb163b2d8eefa9c90ac0ecbc1a7770e50614e/mkdocs_llmstxt-0.2.0-py3-none-any.whl", hash = "sha256:907de892e0c8be74002e8b4d553820c2b5bbcf03cc303b95c8bca48fb49c1a29", size = 23244, upload-time = "2025-04-08T13:18:47.516Z" },
+]
+
+[[package]]
+name = "mkdocs-material"
+version = "9.7.7"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "babel" },
+    { name = "backrefs" },
+    { name = "colorama" },
+    { name = "jinja2" },
+    { name = "markdown" },
+    { name = "mkdocs" },
+    { name = "mkdocs-material-extensions" },
+    { name = "paginate" },
+    { name = "pygments" },
+    { name = "pymdown-extensions" },
+    { name = "requests" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/f1/cd/c05d3a530ba7934f144fb45f7203cd236adc25c7bdcc34673d202f4b0278/mkdocs_material-9.7.7.tar.gz", hash = "sha256:c0649c065b1b0512d60aad8c10f947f8e455284475239b364b610f2deb4d0855", size = 4097923, upload-time = "2026-07-17T16:21:33.156Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/ad/21/17c1bc9e6f47c972ad66fb2ac2568f99f90f1207eeb6fc3b34d094dba7b5/mkdocs_material-9.7.7-py3-none-any.whl", hash = "sha256:8ea9bb1737a5b524a5f9dcf2e1b4ebda8274ae3008aa7845720a97083bef708f", size = 9305438, upload-time = "2026-07-17T16:21:30.017Z" },
+]
+
+[[package]]
+name = "mkdocs-material-extensions"
+version = "1.3.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/79/9b/9b4c96d6593b2a541e1cb8b34899a6d021d208bb357042823d4d2cabdbe7/mkdocs_material_extensions-1.3.1.tar.gz", hash = "sha256:10c9511cea88f568257f960358a467d12b970e1f7b2c0e5fb2bb48cab1928443", size = 11847, upload-time = "2023-11-22T19:09:45.208Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/5b/54/662a4743aa81d9582ee9339d4ffa3c8fd40a4965e033d77b9da9774d3960/mkdocs_material_extensions-1.3.1-py3-none-any.whl", hash = "sha256:adff8b62700b25cb77b53358dad940f3ef973dd6db797907c49e3c2ef3ab4e31", size = 8728, upload-time = "2023-11-22T19:09:43.465Z" },
+]
+
+[[package]]
+name = "mkdocs-redirects"
+version = "1.2.3"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "mkdocs" },
+    { name = "properdocs" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/73/25/49725f78ca5d3026b09973f7a2b3a8b179cc2e8c15e43d5a13bc79f6b274/mkdocs_redirects-1.2.3.tar.gz", hash = "sha256:5e980330999299729a2d6a125347d1af78023d68a23681a4de3053ce7dfe2e51", size = 7712, upload-time = "2026-03-28T13:57:41.766Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/c6/90/871b1cddc01d2ba1637b858eeeabc2e3013dc8df591306b5567b98ef0870/mkdocs_redirects-1.2.3-py3-none-any.whl", hash = "sha256:ec7312fff462d03ec16395d0c001006a418f8d0c21cdf2b47ff11cf839dc3ce0", size = 6245, upload-time = "2026-03-28T13:57:40.466Z" },
+]
+
 [[package]]
 name = "msgpack"
 version = "1.2.1"
@@ -340,6 +758,33 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" },
 ]
 
+[[package]]
+name = "paginate"
+version = "0.5.7"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/ec/46/68dde5b6bc00c1296ec6466ab27dddede6aec9af1b99090e1107091b3b84/paginate-0.5.7.tar.gz", hash = "sha256:22bd083ab41e1a8b4f3690544afb2c60c25e5c9a63a30fa2f483f6c60c8e5945", size = 19252, upload-time = "2024-08-25T14:17:24.139Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/90/96/04b8e52da071d28f5e21a805b19cb9390aa17a47462ac87f5e2696b9566d/paginate-0.5.7-py2.py3-none-any.whl", hash = "sha256:b885e2af73abcf01d9559fd5216b57ef722f8c42affbb63942377668e35c7591", size = 13746, upload-time = "2024-08-25T14:17:22.55Z" },
+]
+
+[[package]]
+name = "pathspec"
+version = "1.1.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/5a/82/42f767fc1c1143d6fd36efb827202a2d997a375e160a71eb2888a925aac1/pathspec-1.1.1.tar.gz", hash = "sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a", size = 135180, upload-time = "2026-04-27T01:46:08.907Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f1/d9/7fb5aa316bc299258e68c73ba3bddbc499654a07f151cba08f6153988714/pathspec-1.1.1-py3-none-any.whl", hash = "sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189", size = 57328, upload-time = "2026-04-27T01:46:07.06Z" },
+]
+
+[[package]]
+name = "platformdirs"
+version = "4.11.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/e5/98/0bf930c4f97d0266b58a89e36c015f56232c52b5d2f207215d48cca9e8f7/platformdirs-4.11.2.tar.gz", hash = "sha256:3a2ae5fca3520a01ab1be8b45613537f52ddf5b5f6f53d88233892dfbf0cd82d", size = 32716, upload-time = "2026-08-10T15:48:06.092Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/49/e2/4e6eee633809c376c024821b91ade709cbfd040ec53939ffbcc292aa7eee/platformdirs-4.11.2-py3-none-any.whl", hash = "sha256:7f89089b6ea71bda7962953edcf784b2e2d9d285b40ad88be2bb75c6e9d82ab4", size = 23361, upload-time = "2026-08-10T15:48:04.855Z" },
+]
+
 [[package]]
 name = "prek"
 version = "0.4.12"
@@ -364,6 +809,29 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/46/1d/e2c0fc222904ef73df1739b11a83edc29e38bc4bc61259f2ca6d2f15abb0/prek-0.4.12-py3-none-win_arm64.whl", hash = "sha256:45e34a24fba4a4e4568682477158591698efc2375b8d1d418ae424691c4bd01b", size = 5632819, upload-time = "2026-08-03T11:28:31.743Z" },
 ]
 
+[[package]]
+name = "properdocs"
+version = "1.6.7"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "click" },
+    { name = "colorama", marker = "sys_platform == 'win32'" },
+    { name = "ghp-import" },
+    { name = "jinja2" },
+    { name = "markdown" },
+    { name = "markupsafe" },
+    { name = "packaging" },
+    { name = "pathspec" },
+    { name = "platformdirs" },
+    { name = "pyyaml" },
+    { name = "pyyaml-env-tag" },
+    { name = "watchdog" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/ec/29/f27a4e1eddf72ed3db6e47818fbafe6debbf09fd7051f9c1a007239b46ef/properdocs-1.6.7.tar.gz", hash = "sha256:adc7b16e562890af0e098a7e5b02e3a81c20894a87d6a28d345c9300de73c26e", size = 276141, upload-time = "2026-03-20T20:07:48.167Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/bd/4d/fc923f5c85318ee8cc903566dc4e0ebe41b2dfc1d2ecf5546db232397ed6/properdocs-1.6.7-py3-none-any.whl", hash = "sha256:6fa0cfa2e01bf338f684892c8a506cf70ea88ae7f3479c933b6fa20168101cbd", size = 225406, upload-time = "2026-03-20T20:07:46.875Z" },
+]
+
 [[package]]
 name = "pycparser"
 version = "3.0"
@@ -593,6 +1061,138 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" },
 ]
 
+[[package]]
+name = "pymdown-extensions"
+version = "11.0.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "markdown" },
+    { name = "pyyaml" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/21/a9/5f0c535ba3b08fe09270c16808e053a968868242ecbd5676d4e3a488bf28/pymdown_extensions-11.0.1.tar.gz", hash = "sha256:dd2905ae6fc5b75582fafb139a1266ffc754705efa902aa50067fa7ff4f94ec0", size = 857113, upload-time = "2026-07-02T17:59:22.955Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/d6/54/da572c98c0b77626a91b5d3b89f0231d8bff5125c225420908632f8b342d/pymdown_extensions-11.0.1-py3-none-any.whl", hash = "sha256:db3943a62bab7e03af1364f0c4083e64b91fb097675a4b6cceccfbe9a77e5eb2", size = 269455, upload-time = "2026-07-02T17:59:21.271Z" },
+]
+
+[[package]]
+name = "python-dateutil"
+version = "2.9.0.post0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "six" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/66/c0/0c8b6ad9f17a802ee498c46e004a0eb49bc148f2fd230864601a86dcf6db/python-dateutil-2.9.0.post0.tar.gz", hash = "sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3", size = 342432, upload-time = "2024-03-01T18:36:20.211Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/ec/57/56b9bcc3c9c6a792fcbaf139543cee77261f3651ca9da0c93f5c1221264b/python_dateutil-2.9.0.post0-py2.py3-none-any.whl", hash = "sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427", size = 229892, upload-time = "2024-03-01T18:36:18.57Z" },
+]
+
+[[package]]
+name = "pyyaml"
+version = "6.0.3"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/0d/a2/09f67a3589cb4320fb5ce90d3fd4c9752636b8b6ad8f34b54d76c5a54693/PyYAML-6.0.3-cp38-cp38-macosx_10_13_x86_64.whl", hash = "sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f", size = 186824, upload-time = "2025-09-29T20:27:35.918Z" },
+    { url = "https://files.pythonhosted.org/packages/02/72/d972384252432d57f248767556ac083793292a4adf4e2d85dfe785ec2659/PyYAML-6.0.3-cp38-cp38-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4", size = 795069, upload-time = "2025-09-29T20:27:38.15Z" },
+    { url = "https://files.pythonhosted.org/packages/a7/3b/6c58ac0fa7c4e1b35e48024eb03d00817438310447f93ef4431673c24138/PyYAML-6.0.3-cp38-cp38-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3", size = 862585, upload-time = "2025-09-29T20:27:39.715Z" },
+    { url = "https://files.pythonhosted.org/packages/25/a2/b725b61ac76a75583ae7104b3209f75ea44b13cfd026aa535ece22b7f22e/PyYAML-6.0.3-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6", size = 806018, upload-time = "2025-09-29T20:27:41.444Z" },
+    { url = "https://files.pythonhosted.org/packages/6f/b0/b2227677b2d1036d84f5ee95eb948e7af53d59fe3e4328784e4d290607e0/PyYAML-6.0.3-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369", size = 802822, upload-time = "2025-09-29T20:27:42.885Z" },
+    { url = "https://files.pythonhosted.org/packages/99/a5/718a8ea22521e06ef19f91945766a892c5ceb1855df6adbde67d997ea7ed/PyYAML-6.0.3-cp38-cp38-win32.whl", hash = "sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295", size = 143744, upload-time = "2025-09-29T20:27:44.487Z" },
+    { url = "https://files.pythonhosted.org/packages/76/b2/2b69cee94c9eb215216fc05778675c393e3aa541131dc910df8e52c83776/PyYAML-6.0.3-cp38-cp38-win_amd64.whl", hash = "sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b", size = 160082, upload-time = "2025-09-29T20:27:46.049Z" },
+    { url = "https://files.pythonhosted.org/packages/f4/a0/39350dd17dd6d6c6507025c0e53aef67a9293a6d37d3511f23ea510d5800/pyyaml-6.0.3-cp310-cp310-macosx_10_13_x86_64.whl", hash = "sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b", size = 184227, upload-time = "2025-09-25T21:31:46.04Z" },
+    { url = "https://files.pythonhosted.org/packages/05/14/52d505b5c59ce73244f59c7a50ecf47093ce4765f116cdb98286a71eeca2/pyyaml-6.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956", size = 174019, upload-time = "2025-09-25T21:31:47.706Z" },
+    { url = "https://files.pythonhosted.org/packages/43/f7/0e6a5ae5599c838c696adb4e6330a59f463265bfa1e116cfd1fbb0abaaae/pyyaml-6.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8", size = 740646, upload-time = "2025-09-25T21:31:49.21Z" },
+    { url = "https://files.pythonhosted.org/packages/2f/3a/61b9db1d28f00f8fd0ae760459a5c4bf1b941baf714e207b6eb0657d2578/pyyaml-6.0.3-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198", size = 840793, upload-time = "2025-09-25T21:31:50.735Z" },
+    { url = "https://files.pythonhosted.org/packages/7a/1e/7acc4f0e74c4b3d9531e24739e0ab832a5edf40e64fbae1a9c01941cabd7/pyyaml-6.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b", size = 770293, upload-time = "2025-09-25T21:31:51.828Z" },
+    { url = "https://files.pythonhosted.org/packages/8b/ef/abd085f06853af0cd59fa5f913d61a8eab65d7639ff2a658d18a25d6a89d/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0", size = 732872, upload-time = "2025-09-25T21:31:53.282Z" },
+    { url = "https://files.pythonhosted.org/packages/1f/15/2bc9c8faf6450a8b3c9fc5448ed869c599c0a74ba2669772b1f3a0040180/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69", size = 758828, upload-time = "2025-09-25T21:31:54.807Z" },
+    { url = "https://files.pythonhosted.org/packages/a3/00/531e92e88c00f4333ce359e50c19b8d1de9fe8d581b1534e35ccfbc5f393/pyyaml-6.0.3-cp310-cp310-win32.whl", hash = "sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e", size = 142415, upload-time = "2025-09-25T21:31:55.885Z" },
+    { url = "https://files.pythonhosted.org/packages/2a/fa/926c003379b19fca39dd4634818b00dec6c62d87faf628d1394e137354d4/pyyaml-6.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c", size = 158561, upload-time = "2025-09-25T21:31:57.406Z" },
+    { url = "https://files.pythonhosted.org/packages/6d/16/a95b6757765b7b031c9374925bb718d55e0a9ba8a1b6a12d25962ea44347/pyyaml-6.0.3-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e", size = 185826, upload-time = "2025-09-25T21:31:58.655Z" },
+    { url = "https://files.pythonhosted.org/packages/16/19/13de8e4377ed53079ee996e1ab0a9c33ec2faf808a4647b7b4c0d46dd239/pyyaml-6.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824", size = 175577, upload-time = "2025-09-25T21:32:00.088Z" },
+    { url = "https://files.pythonhosted.org/packages/0c/62/d2eb46264d4b157dae1275b573017abec435397aa59cbcdab6fc978a8af4/pyyaml-6.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c", size = 775556, upload-time = "2025-09-25T21:32:01.31Z" },
+    { url = "https://files.pythonhosted.org/packages/10/cb/16c3f2cf3266edd25aaa00d6c4350381c8b012ed6f5276675b9eba8d9ff4/pyyaml-6.0.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00", size = 882114, upload-time = "2025-09-25T21:32:03.376Z" },
+    { url = "https://files.pythonhosted.org/packages/71/60/917329f640924b18ff085ab889a11c763e0b573da888e8404ff486657602/pyyaml-6.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d", size = 806638, upload-time = "2025-09-25T21:32:04.553Z" },
+    { url = "https://files.pythonhosted.org/packages/dd/6f/529b0f316a9fd167281a6c3826b5583e6192dba792dd55e3203d3f8e655a/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a", size = 767463, upload-time = "2025-09-25T21:32:06.152Z" },
+    { url = "https://files.pythonhosted.org/packages/f2/6a/b627b4e0c1dd03718543519ffb2f1deea4a1e6d42fbab8021936a4d22589/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4", size = 794986, upload-time = "2025-09-25T21:32:07.367Z" },
+    { url = "https://files.pythonhosted.org/packages/45/91/47a6e1c42d9ee337c4839208f30d9f09caa9f720ec7582917b264defc875/pyyaml-6.0.3-cp311-cp311-win32.whl", hash = "sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b", size = 142543, upload-time = "2025-09-25T21:32:08.95Z" },
+    { url = "https://files.pythonhosted.org/packages/da/e3/ea007450a105ae919a72393cb06f122f288ef60bba2dc64b26e2646fa315/pyyaml-6.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf", size = 158763, upload-time = "2025-09-25T21:32:09.96Z" },
+    { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" },
+    { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" },
+    { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" },
+    { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" },
+    { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" },
+    { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" },
+    { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" },
+    { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" },
+    { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" },
+    { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" },
+    { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" },
+    { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" },
+    { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" },
+    { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" },
+    { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" },
+    { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" },
+    { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" },
+    { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" },
+    { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" },
+    { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" },
+    { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" },
+    { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" },
+    { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" },
+    { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" },
+    { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" },
+    { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" },
+    { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" },
+    { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" },
+    { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" },
+    { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" },
+    { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" },
+    { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" },
+    { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" },
+    { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" },
+    { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" },
+    { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" },
+    { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" },
+    { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" },
+    { url = "https://files.pythonhosted.org/packages/9f/62/67fc8e68a75f738c9200422bf65693fb79a4cd0dc5b23310e5202e978090/pyyaml-6.0.3-cp39-cp39-macosx_10_13_x86_64.whl", hash = "sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da", size = 184450, upload-time = "2025-09-25T21:33:00.618Z" },
+    { url = "https://files.pythonhosted.org/packages/ae/92/861f152ce87c452b11b9d0977952259aa7df792d71c1053365cc7b09cc08/pyyaml-6.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917", size = 174319, upload-time = "2025-09-25T21:33:02.086Z" },
+    { url = "https://files.pythonhosted.org/packages/d0/cd/f0cfc8c74f8a030017a2b9c771b7f47e5dd702c3e28e5b2071374bda2948/pyyaml-6.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9", size = 737631, upload-time = "2025-09-25T21:33:03.25Z" },
+    { url = "https://files.pythonhosted.org/packages/ef/b2/18f2bd28cd2055a79a46c9b0895c0b3d987ce40ee471cecf58a1a0199805/pyyaml-6.0.3-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5", size = 836795, upload-time = "2025-09-25T21:33:05.014Z" },
+    { url = "https://files.pythonhosted.org/packages/73/b9/793686b2d54b531203c160ef12bec60228a0109c79bae6c1277961026770/pyyaml-6.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a", size = 750767, upload-time = "2025-09-25T21:33:06.398Z" },
+    { url = "https://files.pythonhosted.org/packages/a9/86/a137b39a611def2ed78b0e66ce2fe13ee701a07c07aebe55c340ed2a050e/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926", size = 727982, upload-time = "2025-09-25T21:33:08.708Z" },
+    { url = "https://files.pythonhosted.org/packages/dd/62/71c27c94f457cf4418ef8ccc71735324c549f7e3ea9d34aba50874563561/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7", size = 755677, upload-time = "2025-09-25T21:33:09.876Z" },
+    { url = "https://files.pythonhosted.org/packages/29/3d/6f5e0d58bd924fb0d06c3a6bad00effbdae2de5adb5cda5648006ffbd8d3/pyyaml-6.0.3-cp39-cp39-win32.whl", hash = "sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0", size = 142592, upload-time = "2025-09-25T21:33:10.983Z" },
+    { url = "https://files.pythonhosted.org/packages/f0/0c/25113e0b5e103d7f1490c0e947e303fe4a696c10b501dea7a9f49d4e876c/pyyaml-6.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007", size = 158777, upload-time = "2025-09-25T21:33:15.55Z" },
+]
+
+[[package]]
+name = "pyyaml-env-tag"
+version = "1.1"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "pyyaml" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/eb/2e/79c822141bfd05a853236b504869ebc6b70159afc570e1d5a20641782eaa/pyyaml_env_tag-1.1.tar.gz", hash = "sha256:2eb38b75a2d21ee0475d6d97ec19c63287a7e140231e4214969d0eac923cd7ff", size = 5737, upload-time = "2025-05-13T15:24:01.64Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/04/11/432f32f8097b03e3cd5fe57e88efb685d964e2e5178a48ed61e841f7fdce/pyyaml_env_tag-1.1-py3-none-any.whl", hash = "sha256:17109e1a528561e32f026364712fee1264bc2ea6715120891174ed1b980d2e04", size = 4722, upload-time = "2025-05-13T15:23:59.629Z" },
+]
+
+[[package]]
+name = "requests"
+version = "2.34.2"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "certifi" },
+    { name = "charset-normalizer" },
+    { name = "idna" },
+    { name = "urllib3" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" },
+]
+
 [[package]]
 name = "rich"
 version = "15.0.0"
@@ -634,6 +1234,14 @@ source = { editable = "." }
 dev = [
     { name = "prek", marker = "python_full_version >= '3.12'" },
 ]
+docs = [
+    { name = "mkdocs", marker = "python_full_version >= '3.12'" },
+    { name = "mkdocs-github-admonitions-plugin", marker = "python_full_version >= '3.12'" },
+    { name = "mkdocs-llmstxt", marker = "python_full_version >= '3.12'" },
+    { name = "mkdocs-material", marker = "python_full_version >= '3.12'" },
+    { name = "mkdocs-redirects", marker = "python_full_version >= '3.12'" },
+    { name = "pyyaml", marker = "python_full_version >= '3.12'" },
+]
 release = [
     { name = "rooster", marker = "python_full_version >= '3.12'" },
 ]
@@ -642,6 +1250,14 @@ release = [
 
 [package.metadata.requires-dev]
 dev = [{ name = "prek", marker = "python_full_version >= '3.12'", specifier = "==0.4.12" }]
+docs = [
+    { name = "mkdocs", marker = "python_full_version >= '3.12'", specifier = ">=1.6.1" },
+    { name = "mkdocs-github-admonitions-plugin", marker = "python_full_version >= '3.12'", specifier = ">=0.1.1" },
+    { name = "mkdocs-llmstxt", marker = "python_full_version >= '3.12'", specifier = ">=0.2.0" },
+    { name = "mkdocs-material", marker = "python_full_version >= '3.12'", specifier = ">=9.7.7" },
+    { name = "mkdocs-redirects", marker = "python_full_version >= '3.12'", specifier = ">=1.2.3" },
+    { name = "pyyaml", marker = "python_full_version >= '3.12'", specifier = ">=6.0.3" },
+]
 release = [{ name = "rooster", marker = "python_full_version >= '3.12'", specifier = "==0.1.1" }]
 
 [[package]]
@@ -653,6 +1269,24 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/e0/f9/0595336914c5619e5f28a1fb793285925a8cd4b432c9da0a987836c7f822/shellingham-1.5.4-py2.py3-none-any.whl", hash = "sha256:7ecfff8f2fd72616f7481040475a65b2bf8af90a56c89140852d1120324e8686", size = 9755, upload-time = "2023-10-24T04:13:38.866Z" },
 ]
 
+[[package]]
+name = "six"
+version = "1.17.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz", hash = "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", size = 34031, upload-time = "2024-12-04T17:35:28.174Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl", hash = "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274", size = 11050, upload-time = "2024-12-04T17:35:26.475Z" },
+]
+
+[[package]]
+name = "soupsieve"
+version = "2.9.2"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/69/99/a6ca3beb3ccacb41fb3321d8a60e5566f9e6467601ef8eba6a17e1b89778/soupsieve-2.9.2.tar.gz", hash = "sha256:4a55d8cf158a9c2e587fa4922f1bbb91d68ac829e2d6f25403a85747c71daf74", size = 122445, upload-time = "2026-08-07T00:57:24.801Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/eb/dc/ad025c1ee131eba60c69f4dd5779b18fcf1e6b21a343e2162a84d5d133c7/soupsieve-2.9.2-py3-none-any.whl", hash = "sha256:8089a26fd974ca7a1f30276d3d8492ab266ab15af581642dfe8aa162e0c1c823", size = 37370, upload-time = "2026-08-07T00:57:23.524Z" },
+]
+
 [[package]]
 name = "tqdm"
 version = "4.68.3"
@@ -700,3 +1334,49 @@ sdist = { url = "https://files.pythonhosted.org/packages/55/e3/70399cb7dd41c10ac
 wheels = [
     { url = "https://files.pythonhosted.org/packages/dc/9b/47798a6c91d8bdb567fe2698fe81e0c6b7cb7ef4d13da4114b41d239f65d/typing_inspection-0.4.2-py3-none-any.whl", hash = "sha256:4ed1cacbdc298c220f1bd249ed5287caa16f34d44ef4e9c3d0cbad5b521545e7", size = 14611, upload-time = "2025-10-01T02:14:40.154Z" },
 ]
+
+[[package]]
+name = "urllib3"
+version = "2.7.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" },
+]
+
+[[package]]
+name = "watchdog"
+version = "6.0.0"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/db/7d/7f3d619e951c88ed75c6037b246ddcf2d322812ee8ea189be89511721d54/watchdog-6.0.0.tar.gz", hash = "sha256:9ddf7c82fda3ae8e24decda1338ede66e1c99883db93711d8fb941eaa2d8c282", size = 131220, upload-time = "2024-11-01T14:07:13.037Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/0c/56/90994d789c61df619bfc5ce2ecdabd5eeff564e1eb47512bd01b5e019569/watchdog-6.0.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:d1cdb490583ebd691c012b3d6dae011000fe42edb7a82ece80965b42abd61f26", size = 96390, upload-time = "2024-11-01T14:06:24.793Z" },
+    { url = "https://files.pythonhosted.org/packages/55/46/9a67ee697342ddf3c6daa97e3a587a56d6c4052f881ed926a849fcf7371c/watchdog-6.0.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:bc64ab3bdb6a04d69d4023b29422170b74681784ffb9463ed4870cf2f3e66112", size = 88389, upload-time = "2024-11-01T14:06:27.112Z" },
+    { url = "https://files.pythonhosted.org/packages/44/65/91b0985747c52064d8701e1075eb96f8c40a79df889e59a399453adfb882/watchdog-6.0.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:c897ac1b55c5a1461e16dae288d22bb2e412ba9807df8397a635d88f671d36c3", size = 89020, upload-time = "2024-11-01T14:06:29.876Z" },
+    { url = "https://files.pythonhosted.org/packages/e0/24/d9be5cd6642a6aa68352ded4b4b10fb0d7889cb7f45814fb92cecd35f101/watchdog-6.0.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:6eb11feb5a0d452ee41f824e271ca311a09e250441c262ca2fd7ebcf2461a06c", size = 96393, upload-time = "2024-11-01T14:06:31.756Z" },
+    { url = "https://files.pythonhosted.org/packages/63/7a/6013b0d8dbc56adca7fdd4f0beed381c59f6752341b12fa0886fa7afc78b/watchdog-6.0.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:ef810fbf7b781a5a593894e4f439773830bdecb885e6880d957d5b9382a960d2", size = 88392, upload-time = "2024-11-01T14:06:32.99Z" },
+    { url = "https://files.pythonhosted.org/packages/d1/40/b75381494851556de56281e053700e46bff5b37bf4c7267e858640af5a7f/watchdog-6.0.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:afd0fe1b2270917c5e23c2a65ce50c2a4abb63daafb0d419fde368e272a76b7c", size = 89019, upload-time = "2024-11-01T14:06:34.963Z" },
+    { url = "https://files.pythonhosted.org/packages/39/ea/3930d07dafc9e286ed356a679aa02d777c06e9bfd1164fa7c19c288a5483/watchdog-6.0.0-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:bdd4e6f14b8b18c334febb9c4425a878a2ac20efd1e0b231978e7b150f92a948", size = 96471, upload-time = "2024-11-01T14:06:37.745Z" },
+    { url = "https://files.pythonhosted.org/packages/12/87/48361531f70b1f87928b045df868a9fd4e253d9ae087fa4cf3f7113be363/watchdog-6.0.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c7c15dda13c4eb00d6fb6fc508b3c0ed88b9d5d374056b239c4ad1611125c860", size = 88449, upload-time = "2024-11-01T14:06:39.748Z" },
+    { url = "https://files.pythonhosted.org/packages/5b/7e/8f322f5e600812e6f9a31b75d242631068ca8f4ef0582dd3ae6e72daecc8/watchdog-6.0.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:6f10cb2d5902447c7d0da897e2c6768bca89174d0c6e1e30abec5421af97a5b0", size = 89054, upload-time = "2024-11-01T14:06:41.009Z" },
+    { url = "https://files.pythonhosted.org/packages/68/98/b0345cabdce2041a01293ba483333582891a3bd5769b08eceb0d406056ef/watchdog-6.0.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:490ab2ef84f11129844c23fb14ecf30ef3d8a6abafd3754a6f75ca1e6654136c", size = 96480, upload-time = "2024-11-01T14:06:42.952Z" },
+    { url = "https://files.pythonhosted.org/packages/85/83/cdf13902c626b28eedef7ec4f10745c52aad8a8fe7eb04ed7b1f111ca20e/watchdog-6.0.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:76aae96b00ae814b181bb25b1b98076d5fc84e8a53cd8885a318b42b6d3a5134", size = 88451, upload-time = "2024-11-01T14:06:45.084Z" },
+    { url = "https://files.pythonhosted.org/packages/fe/c4/225c87bae08c8b9ec99030cd48ae9c4eca050a59bf5c2255853e18c87b50/watchdog-6.0.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:a175f755fc2279e0b7312c0035d52e27211a5bc39719dd529625b1930917345b", size = 89057, upload-time = "2024-11-01T14:06:47.324Z" },
+    { url = "https://files.pythonhosted.org/packages/05/52/7223011bb760fce8ddc53416beb65b83a3ea6d7d13738dde75eeb2c89679/watchdog-6.0.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:e6f0e77c9417e7cd62af82529b10563db3423625c5fce018430b249bf977f9e8", size = 96390, upload-time = "2024-11-01T14:06:49.325Z" },
+    { url = "https://files.pythonhosted.org/packages/9c/62/d2b21bc4e706d3a9d467561f487c2938cbd881c69f3808c43ac1ec242391/watchdog-6.0.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:90c8e78f3b94014f7aaae121e6b909674df5b46ec24d6bebc45c44c56729af2a", size = 88386, upload-time = "2024-11-01T14:06:50.536Z" },
+    { url = "https://files.pythonhosted.org/packages/ea/22/1c90b20eda9f4132e4603a26296108728a8bfe9584b006bd05dd94548853/watchdog-6.0.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:e7631a77ffb1f7d2eefa4445ebbee491c720a5661ddf6df3498ebecae5ed375c", size = 89017, upload-time = "2024-11-01T14:06:51.717Z" },
+    { url = "https://files.pythonhosted.org/packages/30/ad/d17b5d42e28a8b91f8ed01cb949da092827afb9995d4559fd448d0472763/watchdog-6.0.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:c7ac31a19f4545dd92fc25d200694098f42c9a8e391bc00bdd362c5736dbf881", size = 87902, upload-time = "2024-11-01T14:06:53.119Z" },
+    { url = "https://files.pythonhosted.org/packages/5c/ca/c3649991d140ff6ab67bfc85ab42b165ead119c9e12211e08089d763ece5/watchdog-6.0.0-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:9513f27a1a582d9808cf21a07dae516f0fab1cf2d7683a742c498b93eedabb11", size = 88380, upload-time = "2024-11-01T14:06:55.19Z" },
+    { url = "https://files.pythonhosted.org/packages/5b/79/69f2b0e8d3f2afd462029031baafb1b75d11bb62703f0e1022b2e54d49ee/watchdog-6.0.0-pp39-pypy39_pp73-macosx_10_15_x86_64.whl", hash = "sha256:7a0e56874cfbc4b9b05c60c8a1926fedf56324bb08cfbc188969777940aef3aa", size = 87903, upload-time = "2024-11-01T14:06:57.052Z" },
+    { url = "https://files.pythonhosted.org/packages/e2/2b/dc048dd71c2e5f0f7ebc04dd7912981ec45793a03c0dc462438e0591ba5d/watchdog-6.0.0-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:e6439e374fc012255b4ec786ae3c4bc838cd7309a540e5fe0952d03687d8804e", size = 88381, upload-time = "2024-11-01T14:06:58.193Z" },
+    { url = "https://files.pythonhosted.org/packages/a9/c7/ca4bf3e518cb57a686b2feb4f55a1892fd9a3dd13f470fca14e00f80ea36/watchdog-6.0.0-py3-none-manylinux2014_aarch64.whl", hash = "sha256:7607498efa04a3542ae3e05e64da8202e58159aa1fa4acddf7678d34a35d4f13", size = 79079, upload-time = "2024-11-01T14:06:59.472Z" },
+    { url = "https://files.pythonhosted.org/packages/5c/51/d46dc9332f9a647593c947b4b88e2381c8dfc0942d15b8edc0310fa4abb1/watchdog-6.0.0-py3-none-manylinux2014_armv7l.whl", hash = "sha256:9041567ee8953024c83343288ccc458fd0a2d811d6a0fd68c4c22609e3490379", size = 79078, upload-time = "2024-11-01T14:07:01.431Z" },
+    { url = "https://files.pythonhosted.org/packages/d4/57/04edbf5e169cd318d5f07b4766fee38e825d64b6913ca157ca32d1a42267/watchdog-6.0.0-py3-none-manylinux2014_i686.whl", hash = "sha256:82dc3e3143c7e38ec49d61af98d6558288c415eac98486a5c581726e0737c00e", size = 79076, upload-time = "2024-11-01T14:07:02.568Z" },
+    { url = "https://files.pythonhosted.org/packages/ab/cc/da8422b300e13cb187d2203f20b9253e91058aaf7db65b74142013478e66/watchdog-6.0.0-py3-none-manylinux2014_ppc64.whl", hash = "sha256:212ac9b8bf1161dc91bd09c048048a95ca3a4c4f5e5d4a7d1b1a7d5752a7f96f", size = 79077, upload-time = "2024-11-01T14:07:03.893Z" },
+    { url = "https://files.pythonhosted.org/packages/2c/3b/b8964e04ae1a025c44ba8e4291f86e97fac443bca31de8bd98d3263d2fcf/watchdog-6.0.0-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:e3df4cbb9a450c6d49318f6d14f4bbc80d763fa587ba46ec86f99f9e6876bb26", size = 79078, upload-time = "2024-11-01T14:07:05.189Z" },
+    { url = "https://files.pythonhosted.org/packages/62/ae/a696eb424bedff7407801c257d4b1afda455fe40821a2be430e173660e81/watchdog-6.0.0-py3-none-manylinux2014_s390x.whl", hash = "sha256:2cce7cfc2008eb51feb6aab51251fd79b85d9894e98ba847408f662b3395ca3c", size = 79077, upload-time = "2024-11-01T14:07:06.376Z" },
+    { url = "https://files.pythonhosted.org/packages/b5/e8/dbf020b4d98251a9860752a094d09a65e1b436ad181faf929983f697048f/watchdog-6.0.0-py3-none-manylinux2014_x86_64.whl", hash = "sha256:20ffe5b202af80ab4266dcd3e91aae72bf2da48c0d33bdb15c66658e685e94e2", size = 79078, upload-time = "2024-11-01T14:07:07.547Z" },
+    { url = "https://files.pythonhosted.org/packages/07/f6/d0e5b343768e8bcb4cda79f0f2f55051bf26177ecd5651f84c07567461cf/watchdog-6.0.0-py3-none-win32.whl", hash = "sha256:07df1fdd701c5d4c8e55ef6cf55b8f0120fe1aef7ef39a1c6fc6bc2e606d517a", size = 79065, upload-time = "2024-11-01T14:07:09.525Z" },
+    { url = "https://files.pythonhosted.org/packages/db/d9/c495884c6e548fce18a8f40568ff120bc3a4b7b99813081c8ac0c936fa64/watchdog-6.0.0-py3-none-win_amd64.whl", hash = "sha256:cbafb470cf848d93b5d013e2ecb245d4aa1c8fd0504e863ccefa32445359d680", size = 79070, upload-time = "2024-11-01T14:07:10.686Z" },
+    { url = "https://files.pythonhosted.org/packages/33/e8/e40370e6d74ddba47f002a32919d91310d6074130fe4e17dabcafc15cbf1/watchdog-6.0.0-py3-none-win_ia64.whl", hash = "sha256:a1914259fa9e1454315171103c6a30961236f508b9b623eae470268bbcc6a22f", size = 79067, upload-time = "2024-11-01T14:07:11.845Z" },
+]

From a229fc705416ee90b7f5ef966a563ff6c0e718b1 Mon Sep 17 00:00:00 2001
From: William Woodruff 
Date: Tue, 18 Aug 2026 17:05:39 -0400
Subject: [PATCH 094/371] AGENTS: remove old script guidance (#27859)

Co-authored-by: Alex Waygood 
Signed-off-by: William Woodruff 
---
 AGENTS.md | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/AGENTS.md b/AGENTS.md
index 86874bef46..3b9212190b 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -179,4 +179,5 @@ Parts of `.github/workflows/release.yml` are generated by cargo-dist from `dist-
 - Run `cargo dev generate-all` after changing configuration options, CLI arguments, lint rules, or environment variable definitions, as these changes require regeneration of schemas, docs, and CLI references.
 - Don't prefix tests with `test_`.
 - Don't separate struct definitions from their `impl` blocks unless the `impl` is deliberately placed in a separate file, as for large structs.
-- Avoid running `uv run` for any scripts from the repository root unless you use `--no-project`, `--script` or similar. Using `uv run` from the Ruff repo root without these flags will build Ruff from source, which is very slow and usually unnecessary.
+- Write all self-contained Python scripts as PEP 723 scripts, with inline metadata.
+- When running a PEP 723 script, run it with `uv run