Add optional, droppable, and variadic-trailing parameters

AGENTS.md

@@ -22,19 +22,27 @@ Note that this only affects the main compilation tests;
 there is also `cntest` for nofib compilation tests, `catest` for application compilation tests,
 and `cwtest` for WASM compilation tests.
 
-Before finishing your work, run `hkmc2AllTests/test` to make sure the codebase compiles and all tests succeed.
+Before finishing work with retained code or test changes, run `hkmc2AllTests/test`
+to make sure the codebase compiles and all tests succeed;
+this covers `ctest`, so no need to run `ctest` separately.
+Full validation is not required for read-only investigations, documentation-only edits,
+and temporary experiments whose changes have all been reverted.
 
 After you are done fixing all the problems and all the tests pass,
 *you need to commit the resulting golden test output changes*.
 Any commit that does not include the latest changes to test outputs will fail the CI.
 
 Please also read the files in `.github/skills/hkmc2-difftests`.
 
-Use `.mls` diff tests for compiler behavior. Do not add Scala tests that duplicate
+Use `*.mls` diff tests for compiler behavior. Do not add Scala tests that duplicate
 that coverage or merely pin internal representation or assertion details. Reserve
 unit tests for properties diff tests cannot reliably observe, such as concurrency,
 cache isolation, bounded graph growth, or algebraic laws; state the extra coverage.
 
+Before adding new diff tests, make sure they are not already covered by existing tests.
+All new diff-test files you create should go into `hkmc2/shared/src/test/mlscript/slop/`
+unless there is a good reason to put them elsewhere.
+
 When you create a commit, author it under your (the agent's) own identity,
 not under the user's identity.
 Set this identity for the commit command only
@@ -44,6 +52,12 @@ Do not change repository-local or global Git identity configuration.
 
 ## Design Philosophy
 
+**Confirm language-design decisions before implementing them.** Propose changes
+to source syntax, language semantics, or public library contracts and obtain the
+user's confirmation unless that specific design is already explicitly approved.
+Authorization to fix compiler behavior does not by itself authorize introducing
+annotations, changing member naming or dispatch rules, or extending public APIs.
+
 You are working on an industry-strength compiler codebase.
 Although its test suite is extensive, tests alone are not enough to determine the quality of a change;
 the goal is not just to "make everything green".

README.md

@@ -184,6 +184,7 @@ and then use one of the following commands.
 - `hkmc2AppsTests/test` for running the applications compile and diff tests.
 - `hkmc2NofibTests/test` for running the nofib compile and diff tests.
 - `hkmc2WasmTests/test` for running the wasm compile and diff tests.
+- `hkmc2SlopTests/test` for running the AI-authored regression diff-tests in `hkmc2/shared/src/test/mlscript/slop`, which are excluded from the main diff-tests.
 - `hkmc2MostTests/test` for running all of the above.
 - `hkmc2AllTests/test` for running all default hkmc2 tests, including ScalaJS-compiled tests.
 - `~hkmc2DiffTests/Test/run` for running the test watcher,
@@ -194,6 +195,11 @@ and then use one of the following commands.
 Individual tests can be run with `-z`.
 For example, `~mlscriptJVM/testOnly mlscript.DiffTests -- -z parser` will watch for file changes and continuously run all parser tests (those that have "parser" in their name).
 
+Run `ctest` before running individual diff-tests, including `sdtest`,
+so the generated JavaScript dependencies are available.
+Also run `cwtest` before individual Wasm diff-tests, including those under `slop/wasm`.
+`hkmc2SlopTests/test` runs the main and Wasm compilation prerequisites automatically.
+
 #### Useful SBT commands aliases (use these for more pleasant development)
 
 The most useful command alias for development is `watch` for continuously running the test watcher, which:
@@ -207,6 +213,7 @@ or specify a string to filter the tests to run.
 
 - `ctest` aka "main compilation tests" for running specific tests in `hkmc2/shared/src/test/mlscript-compile`
 - `dtest` aka "main diff-tests" for running only the main diff-tests in `hkmc2/shared/src/test/mlscript`
+- `sdtest` for running only the diff-tests in `hkmc2/shared/src/test/mlscript/slop`
 - You can find other similar command aliases in `build.sbt`.
 
 The command aliases below are shorthand for running aggregated tests; they take no arguments.

build.sbt

@@ -93,18 +93,18 @@ lazy val hkmc2DiffTests = project.in(file("hkmc2DiffTests"))
     Test/run/fork := true, // so that CTRL+C actually terminates the watcher
   )
 
+val compileTestFixtures = taskKey[Unit]("Compile the MLscript fixtures needed by diff tests")
+
 /** Helper to create test subprojects that compile `.mls` files then run diff tests.
   * Each subproject depends on `hkmc2JVM` and `hkmc2DiffTests` for shared test infrastructure.
-  * When a compile runner is provided, `Def.sequential` guarantees it completes before diff tests start. */
+  * A named prerequisite lets suites share fixture compilation in aggregate runs.
+  * Diff tests wait for it before loading the generated modules. */
 def hkmc2TestSubproject(dirName: String, compileRunner: Option[String], diffRunner: String): Project = {
-  val testTask = compileRunner match {
+  val compileTask = compileRunner match {
     case Some(runner) =>
-      Def.sequential(
-        (Test / testOnly).toTask(s" hkmc2.$runner"),
-        (Test / testOnly).toTask(s" hkmc2.$diffRunner"),
-      )
+      (Test / testOnly).toTask(s" hkmc2.$runner")
     case None =>
-      (Test / testOnly).toTask(s" hkmc2.$diffRunner")
+      Def.task { () }
   }
 
   Project(dirName, file(dirName))
@@ -116,7 +116,9 @@ def hkmc2TestSubproject(dirName: String, compileRunner: Option[String], diffRunn
       libraryDependencies += "org.scalactic" %%% "scalactic" % scalaTestVersion,
       libraryDependencies += "org.scalatest" %%% "scalatest" % scalaTestVersion % "test",
 
-      Test / test := testTask.value,
+      Test / compileTestFixtures := compileTask.value,
+      Test / test := (Test / testOnly).toTask(s" hkmc2.$diffRunner")
+        .dependsOn(Test / compileTestFixtures).value,
 
       Test/run/fork := true, // so that CTRL+C actually terminates the watcher
     )
@@ -125,12 +127,23 @@ def hkmc2TestSubproject(dirName: String, compileRunner: Option[String], diffRunn
 lazy val hkmc2NofibTests = hkmc2TestSubproject("hkmc2NofibTests", Some("NofibCompileTestRunner"), "NofibDiffTestRunner")
 lazy val hkmc2AppsTests = hkmc2TestSubproject("hkmc2AppsTests", Some("AppsCompileTestRunner"), "AppsDiffTestRunner")
 lazy val hkmc2WasmTests = hkmc2TestSubproject("hkmc2WasmTests", Some("WasmCompileTestRunner"), "WasmDiffTestRunner")
+lazy val hkmc2SlopTests = hkmc2TestSubproject("hkmc2SlopTests", None, "SlopDiffTestRunner")
+  .settings(
+    // These worksheets load main-suite fixtures such as Iter.mjs and the Wasm.mjs
+    // runtime helper. Both must be generated before the slop tests start, even
+    // when an aggregate schedules the Wasm and slop suites as sibling tasks.
+    Test / test := (Test / test).dependsOn(
+      hkmc2JVM / Test / test,
+      hkmc2WasmTests / Test / compileTestFixtures,
+    ).value,
+  )
 
 lazy val hkmc2MainTests = project.in(file("hkmc2MainTests"))
   .settings(
     Test / test := (
       (hkmc2DiffTests / Test / test)
         .dependsOn(hkmc2JVM / Test / test)
+        .dependsOn(hkmc2SlopTests / Test / test)
     ).value
   )
 
@@ -141,6 +154,7 @@ lazy val hkmc2MostTests = project.in(file("hkmc2MostTests"))
         .dependsOn(hkmc2NofibTests / Test / test)
         .dependsOn(hkmc2AppsTests / Test / test)
         .dependsOn(hkmc2WasmTests / Test / test)
+        .dependsOn(hkmc2SlopTests / Test / test)
         .dependsOn(hkmc2JVM / Test / test)
     ).value
   )
@@ -152,6 +166,7 @@ lazy val hkmc2AllTests = project.in(file("hkmc2AllTests"))
         .dependsOn(hkmc2NofibTests / Test / test)
         .dependsOn(hkmc2AppsTests / Test / test)
         .dependsOn(hkmc2WasmTests / Test / test)
+        .dependsOn(hkmc2SlopTests / Test / test)
         .dependsOn(hkmc2JVM / Test / test)
         .dependsOn(hkmc2JS / Test / test)
         .dependsOn(hkmc2Benchmarks / Test / compile)
@@ -166,6 +181,7 @@ addCommandAlias("dtest", "hkmc2DiffTests/testOnly hkmc2.DiffTestRunner -- -z")
 addCommandAlias("ndtest", "hkmc2NofibTests/testOnly hkmc2.NofibDiffTestRunner -- -z")
 addCommandAlias("adtest", "hkmc2AppsTests/testOnly hkmc2.AppsDiffTestRunner -- -z")
 addCommandAlias("wdtest", "hkmc2WasmTests/testOnly hkmc2.WasmDiffTestRunner -- -z")
+addCommandAlias("sdtest", "hkmc2SlopTests/testOnly hkmc2.SlopDiffTestRunner -- -z")
 // Compilation tests
 addCommandAlias("ctest", "hkmc2JVM/testOnly hkmc2.CompileTestRunner -- -z")
 addCommandAlias("cntest", "hkmc2NofibTests/testOnly hkmc2.NofibCompileTestRunner -- -z")

doc/new-resolution-type-value-flow.md

@@ -36,6 +36,16 @@ alternative constraint sets or forming products of intersection candidates.
 They deliberately approximate type inference; diagnosing concrete incompatibilities
 such as `Str <: Int` belongs to type checking, not resolution.
 
+This distinction also applies to separate function signatures and callback
+layouts. Skipping an uncertain compatibility check must not skip its flows.
+Every callable component contributes its domain to the implementation parameters,
+and later curried lists remain connected even when an earlier layout differs.
+Each implementation parameter has a shared annotation endpoint that accumulates
+these components; replacing it with the last component would disconnect earlier
+constraints. Matching positions stay precise. Uncertain middle positions receive
+the union of their possible inputs, including typed rest elements. Rest slicing
+may widen lengths, but retains the deferred types of every possible element.
+
 ## Instance wrappers and interface observations
 
 `InstanceShape(DeclaredType)` describes an instance of a type. Parameter/result
@@ -135,7 +145,7 @@ input and output parts:
 | `out U` | `Nothing` | `U` |
 | `in T out U` | `T` | `U` |
 
-Declaration variance applies to an unqualified argument. A written wildcard
+Nominal declaration variance applies to an unqualified argument. A written wildcard
 supplies its own parts and overrides declaration variance. Comparing actual `a`
 with expected `b` installs `a.output <: b.output` and `b.input <: a.input`.
 Missing wildcard parts are actual top/bottom types, not inference holes.
@@ -189,12 +199,26 @@ type checker. The annotated-array case in `VarianceSoundness.mls` resolves
 normally and, with type checking disabled, records the unsafe runtime write.
 InvalML regressions check rejection of the corresponding concrete mismatch.
 
-Aliases bind both argument interpretations before interpreting their bodies at the
-enclosing polarity. This avoids applying the enclosing reversal twice:
-`Identity[T]`, for `type Identity[A] = A`, must behave like `T` in either polarity.
-Captured enclosing binders retain their own bindings. Nominal applications instead
-retain the argument pair composed with the application's polarity, since member
-signatures subsequently interpret that pair from the member's positive interface.
+Alias formals start a new substitution scope. References to already-bound
+enclosing parameters inside an argument still select their bounds according to
+their position in the enclosing signature. A fresh, unbound parameter remains
+itself. Apply declaration variance (or the written wildcard) to that interpreted
+argument, then select its endpoint according to its occurrence inside the alias body.
+For `type Source[out T] = {item: T}`, both `Source[A]` and
+`Source[in Nothing out A]` expose `A` in `item`, including when the application
+occurs in a function domain. The enclosing domain affects the interpretation
+of `A`, not whether the occurrence of `T` in `{item: T}` selects its output.
+
+The implementation keeps the alias body's enclosing polarity so captured outer
+binders retain their interpretation. It therefore reverses only the new formal
+bindings' stored endpoint pairs when that polarity is negative. A globally
+negative occurrence of `T` then selects the original output endpoint, while a
+further domain reversal inside the alias selects the original input endpoint.
+This also keeps `Identity[A]` equivalent to `A` in both polarities.
+Nominal applications retain their composed argument pairs directly: member
+signatures subsequently interpret them from the member's positive interface.
+`TransparentAliasPolarity.mls` covers shorthand and explicit projections;
+`VarianceAliases.mls` covers captured binders and nested identity aliases.
 
 `DeclaredType.positive` records this lexical polarity, independently of a later
 constraint's direction. `TypeShape.Wildcard` retains written parts;
@@ -208,6 +232,33 @@ Selections are cached by argument reference and polarity; selecting an existing
 selection is idempotent. Forwarding cycles retain subscriptions but stop repeated observation.
 For a fixed argument-reference set, selection adds at most two nodes per reference.
 
+### Iteration and synthesized applications
+
+Resolution is a best-effort check, not a type checker. Implicit spread observations
+must propagate known flows without rejecting an opaque or inconvenient protocol.
+They preserve every union and intersection component. Member lookup combines
+an intersection's interfaces, while constraints inspect each component independently. An unknown alternative joins the known elements; it never
+replaces them. Definitively completed iterator results supply no element, while
+a missing or unknown `done` still permits observing `value`.
+
+Each non-tuple spread operand owns one `IterationFlow`: stable sites for selecting
+`Symbol.iterator`, calling it, selecting and calling `next`, and reading `done`
+and `value`, plus one deferred element host. These observations use ordinary
+member scope transport and call instantiation. The implicit zero-argument calls
+omit compatibility diagnostics but retain argument and result flow. Tuple
+construction sees one unknown-length segment per operand, regardless of the
+number of incoming iterator or element alternatives. This prevents products of
+alternatives from multiplying through recursive spreads. Imported producers keep
+their canonical sites while publisher storage remains local to each consumer.
+
+Arrays use this same protocol through the prelude's symbol-keyed method. Direct
+Array element-bound observation remains useful for array patterns and rest
+constraints, where the operation describes Array storage rather than iteration.
+The `appliedTypes` table interns synthesized applications by constructor symbol
+and contextual arguments, sharing the mechanism for `Array[T]` and `Awaited[T]`.
+It reifies symbolic argument references before observation; caching their current
+values instead would lose later bounds and caller-specific substitutions.
+
 ## Shared bodies and contextual constraints
 
 The inferred body is a shared graph. Instantiated views carry flat original-binder
@@ -581,3 +632,54 @@ Worksheet coverage under `newres` includes `MutableArrays`, `ContextualInference
 generic call isolation, and escaping payloads. Deferred cases retain explicit
 regression expectations;
 see the [future-work reference](new-resolution-future-work.md).
+
+
+### Fresh property keys
+
+`@annotations.jsSymbol(path)` identifies a property by a stable path through modules and
+immutable vals. The terminal val's `@annotations.fresh` marker is a contract,
+not a type check or a proof about its initializer. Resolver key identity includes
+the receiver path, and direct immutable receiver aliases are normalized. Modules
+use their module symbol, independently of a companion function or class overload.
+Key recognition does not evaluate a val initializer or force alias expansion.
+
+Annotations on declared module members can be interpreted from the module's
+syntax before its value definitions complete. This matters for prelude bootstrap:
+`@fresh` must not require a runtime value from the declared
+`annotations` module. Builtin `Symbol` lookup is lazy so interpreting an unrelated
+annotation does not require `Symbol` to be present in an earlier prelude block.
+Completed overload sets are published as soon as their definitions are available,
+allowing a subsequent annotation to select through an earlier immutable val.
+
+Lowering records computed names as explicit class operands and selected keys as
+explicit path operands. Transformations therefore retain their imports and
+captures and substitute their symbols. Computed names are evaluated in the
+class's enclosing scope. Lifting cannot move such a class out of the invocation
+that evaluates its keys; doing so could reuse a key or read it before initialization.
+
+At block completion, resolution recovers an access path from the use site's
+lexical scope to the key's binding, following enclosing modules when necessary.
+Context updates preserve scope identity; entering a nested block creates a new
+scope. Term definitions retain their binding scope, as module definitions do.
+File bindings can also be reached through the compilation unit's existing default
+export or private export ABI. Foreign provenance alone does not make a local
+binding importable.
+
+The legacy resolver uses the same `AccessPath.findAccessPath` helper for implicit
+companion references. It retains the module overload at each step, so a companion
+of a constructor function is reached through its class value, including across
+files and after lifting.
+
+The recovered expression is part of the resolved reference and is cloned and
+lowered like other operands. Lowering does not maintain a separate inventory of
+available bindings. A member's resolved declaration cannot supply a factory-local
+key to an escaped instance's caller. Such accesses fail at the use site; nested
+closures can still capture the key. These checks do not change candidate flows.
+Overrides with a common key retain all their targets and share only the recovered
+key operand.
+
+Private members of classes and modules cannot supply keys across files: their
+generated accessor symbols are local to one emitted JavaScript module. Resolution
+reports the inaccessible private member at the selection, and code generation
+asserts this boundary as a compiler invariant. This restriction does not apply to
+private file-level vals, which use the compilation unit's private export ABI.