[wasi] Stand up CoreCLR-WASI library-test leg (per-app corerun relink)#130745
Draft
lewing wants to merge 21 commits into
Draft
[wasi] Stand up CoreCLR-WASI library-test leg (per-app corerun relink)#130745lewing wants to merge 21 commits into
lewing wants to merge 21 commits into
Conversation
Adds the infrastructure to run CoreCLR-WASI library tests on wasmtime via Helix, mirroring the browser-CoreCLR LibraryTestsCoreCLR legs. Follow-up to CoreCLR-WASI runtime tests (dotnet#130051); addresses dotnet#130129. Unlike Mono - which relinks dotnet.wasm per-app with wasi-sdk - CoreCLR-WASI ships a prebuilt corerun host in the runtime pack and loads managed assemblies from the wasmtime filesystem, so there is no per-app native relink. - WasiApp.InTree.props: RuntimeFlavor CoreCLR/Mono split (UseMonoRuntime); default invariant globalization for the CoreCLR bring-up. - WasiApp.InTree.targets: import WasiApp.CoreCLR.targets for RuntimeFlavor=CoreCLR. - WasiApp.CoreCLR.targets (new): reuse the shared WASI bundling with the native relink disabled; copy the prebuilt corerun and System.Private.CoreLib.dll (which ship in the pack's native/ dir) into managed/ so corerun finds the runtime via CORE_ROOT; skip the Mono-only _ReadWasmProps (wasm-props.json is not in the CoreCLR pack). - tests.wasi.targets: CoreCLR run command - managed/corerun WasmTestRunner.dll managed/<Test>.dll with -W exceptions=y and CORE_ROOT=/managed. - sendtohelix-wasi.targets: CoreCLR work-item prefix; gate Mono-only build-tooling helix payloads off for CoreCLR. - tests.proj: wasi CoreCLR smoke set (System.Runtime.Tests); gate the existing wasi smoke set to Mono. - New CI leg wasi-wasm-coreclr-library-tests.yml wired into runtime.yml (smoke-only, rolling alwaysRun, non-gating). The bundle build and run command were validated locally against a from-main wasi-wasm CoreCLR build: CoreCLR initializes and executes the managed entry point under wasmtime. Full green runs still depend on runtime-side fixes (interpreter callhelpers, corerun entry-assembly resolution). Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
…ath) Follow-up fixes validated by a local wasi-wasm CoreCLR build, getting the smoke test through to xunit discovery. - tests.wasi.targets: corerun resolves the managed entry assembly relative to its own directory, so pass it as "managed/WasmTestRunner.dll". - WasiApp.CoreCLR.targets: the SDK does not resolve a CoreCLR runtime pack for the wasi-wasm RID, so the self-contained publish omits the framework assemblies (only the app's own closure is bundled). Copy the framework implementation assemblies from the runtime pack lib dir into managed/, alongside corerun and System.Private.CoreLib.dll, without overwriting app-published assemblies. With this the bundle boots CoreCLR under wasmtime and runs WasmTestRunner + the XHarness xunit runner through test discovery of System.Runtime.Tests. Remaining failures are runtime-side (interpreter precode/callhelpers), not test infra. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
…moke leg The CoreCLR-WASI library-test smoke leg boots and reaches xunit discovery, but traps in the interpreter on a cold call through a portable entry point with a mismatched wasm function signature (dotnet#130634), which has no merged fix yet. Document that the leg is expected to stay red until dotnet#130634 lands. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
…y] types
The CoreCLR wasm reverse-P/Invoke thunk table is keyed by
"{Method}#{argCount}:{Assembly}:{Namespace}:{Type}". The generator
(PInvokeCollector) used Type.Namespace, which for a nested type returns the
enclosing namespace (e.g. "System.Tests" for
System.Tests.TimeZoneInfoTests+WindowsUILanguageHelper). The runtime lookup
(src/coreclr/vm/wasm/helpers.cpp GetHashCode -> GetFullyQualifiedNameInfo)
reports an empty namespace for nested types, so the emitted key never matched
at lookup time: LookupThunk returned null, EnsureCodeForUnmanagedCallersOnly
left the portable entry point without code, and GetActualCode asserted
(precode_portable.cpp:35) on the first cold ldftn of a nested UnmanagedCallersOnly
callback.
Match the runtime by emitting an empty namespace for nested types. With this,
a nested [UnmanagedCallersOnly] callback (e.g. the EnumUILanguages callback used
by globalization/culture code) resolves its reverse thunk and the interpreter
runs it instead of asserting.
Contributes to CoreCLR-WASI library test bring-up (dotnet#130129).
Co-authored-by: Copilot App <[email protected]>
Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Add a reference to dotnet#130739 documenting that the nested-type reverse-thunk key drops the enclosing-type chain (collisions are caught as a build-time duplicate-key error). The generator is expected to be superseded once R2R generates the reverse thunks directly. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
The interp smoke blocker was the nested-type reverse-thunk key mismatch (dotnet#130739), not the cold portable-entry-point timing (dotnet#130634). The leg needs the per-app relink for thunk coverage; dotnet#130634 only affects a future R2R-enabled leg. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Build the corerun host as a static archive (libWasiHost.a) alongside the executable, mirroring the browser BrowserHost-Static / libBrowserHost.a pattern. A per-app relink (WasiApp.CoreCLR.targets) links this archive with the app-generated callhelpers (replacing the baked coreclr_gen_static) so library/test [UnmanagedCallersOnly] reverse thunks are covered. Adds the libWasiHost.a platform-manifest entry so it ships in the runtime pack native dir. See dotnet#130129. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Replace the prebuilt-corerun copy with a per-app native relink so library/test
[UnmanagedCallersOnly] reverse thunks are covered (mirrors the browser CoreCLR
relink). The target:
1. runs ManagedToNativeGenerator (TargetOS=wasi) over the bundled managed
closure to emit the three callhelper .cpp tables,
2. compiles them with the wasi-sdk clang against the curated header set
(force-included coreclr_compat.h + vm/wasm + minipal includes, shared with
the browser CoreCLR build), and
3. relinks corerun from libWasiHost.a + the runtime-pack static archives + the
freshly compiled app callhelper .o (dropping the baked libcoreclr_gen_static.a)
via wasm-component-ld, declaring the wasi:http world with
-Wl,--component-type,WasiHttpWorld_component_type.wit.
The link order (incl. the duplicated libminipal.a and mid-list -lstdc++) mirrors
the corerun CMake link line, which was validated end-to-end by a manual relink
(9930 tests discovered and running under wasmtime -S http). Falls back to copying
the prebuilt corerun when WasiCoreCLRRelinkCoreRun=false.
See dotnet#130129.
Co-authored-by: Copilot App <[email protected]>
Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Two corrections after validating the relink end-to-end under wasmtime: - Wrap libWasiHost.a in --whole-archive/--no-whole-archive so main()/_start are pulled in. wasm-ld only includes archive members that satisfy a strong undefined reference, but main is referenced weakly by __main_void, so a plain archive link left main as undefined_weak and the host trapped at entry. - Compile the generated callhelpers with the full CoreCLR-WASI VM define set (from the runtime callhelper compile) rather than a sparse subset, so they build against the same ABI/struct layout as libcoreclr_static.a. With these, the per-app relinked corerun boots, coreclr runs, and xunit discovers and executes 9930 System.Runtime.Tests cases under 'wasmtime -W exceptions=y -S http' (managed exception dispatch through the interp EH + reverse UCO thunks works; remaining [FAIL]s are invariant-mode CultureNotFoundException test outcomes, not crashes). See dotnet#130129. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Refresh the stale 'prebuilt corerun' comments in tests.wasi.targets and the CI template now that WasiApp.CoreCLR.targets relinks corerun per-app (validated locally: discovers + runs the full System.Runtime.Tests suite under wasmtime). Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
The browser CoreCLR relink links libgcinfo_unix_wasm.a; the wasi relink deliberately does not, mirroring the authoritative wasi corerun CMake link line. Verified: that archive exports only the gcinfo ENCODER helpers (GcInfoArrayList, used when emitting GC info in JIT/crossgen), which the interpreter host never does; the gcinfo DECODER used by runtime GC stackwalks (TGcInfoDecoder<Wasm32GcInfoEncoding>) is folded into libcoreclr_static.a as weak symbols. corerun.exe omits the separate archive and runs the full CoreCLR-WASI runtime suite, confirming it is unneeded for the host. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Note the precise mechanism: libcoreclr_static.a does carry undefined references to the gcinfo encoder helpers (GcInfoArrayList), but they live in the emit path the interp/R2R host never reaches, so the linker gc-sections pass drops them and a successful link proves no retained code needs them. The shipped wasi corerun executable links identically (without the archive) and runs the full runtime suite. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Reference the browser build's coreclr_compat.h directly rather than copying it: a second copy would drift from the callhelpers ABI (the class of bug the nested-key fix addressed). Note the cross-tree reference is intentional and temporary; relocating the header to a neutral coreclr-adjacent location shared by browser + wasi, plus out-of-tree staging, is tracked with dotnet#128362. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
…style Compress the paragraph-length explanatory comments (relink overview, framework copy, coreclr_compat.h, gcinfo, whole-archive, define set) to the terse why-style used across the wasm build targets. Comment ratio drops from ~31% to ~23% (below the browser CoreCLR targets' 27%); load-bearing rationale (whole-archive pulls main, gcinfo encoder-vs-decoder, full define set for ABI parity) is kept, just shorter. Comment-only; relink still builds and runs the full System.Runtime.Tests suite under wasmtime. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
|
Azure Pipelines: Successfully started running 6 pipeline(s). 9 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
|
Tagging subscribers to this area: @JulieLeeMSFT, @BrzVlad, @janvorli |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR wires up an initial CoreCLR-on-WASI library tests lane (smoke scope: System.Runtime.Tests) by adding a per-app relink of the corerun host so app/test [UnmanagedCallersOnly] reverse-thunk callhelpers are generated and linked into the host, and by hooking that into Helix + CI.
Changes:
- Fix reverse P/Invoke thunk key generation for nested
[UnmanagedCallersOnly]callbacks so runtime lookup keys match. - Add a CoreCLR-specific WASI app-builder target set that generates callhelper sources and relinks
corerunper app usingwasi-sdk+wasm-component-ld. - Add smoke test selection + Helix/CI plumbing for a non-gating CoreCLR-WASI library-tests leg.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/tasks/WasmAppBuilder/coreclr/PInvokeCollector.cs | Align nested-type namespace handling for reverse-thunk key generation. |
| src/mono/wasi/build/WasiApp.InTree.targets | Select CoreCLR vs Mono WASI app-builder targets based on RuntimeFlavor. |
| src/mono/wasi/build/WasiApp.InTree.props | Configure WASI in-tree defaults for RuntimeFlavor (e.g., invariant globalization for CoreCLR bring-up). |
| src/mono/wasi/build/WasiApp.CoreCLR.targets | New: CoreCLR-WASI app-builder that copies framework bits and performs per-app corerun relink with generated callhelpers. |
| src/libraries/tests.proj | Add CoreCLR-WASI smoke project set (initially System.Runtime.Tests) and keep Mono-WASI smoke set intact. |
| src/libraries/sendtohelix-wasi.targets | Distinguish CoreCLR work items on the shared WASI queue and adjust payload staging for CoreCLR vs Mono. |
| src/installer/pkg/sfx/Microsoft.NETCore.App/Directory.Build.props | Add libWasiHost.a to the platform manifest so it ships in the runtime pack. |
| src/coreclr/hosts/corerun/CMakeLists.txt | Build/install a WASI host static archive (libWasiHost.a) for relink scenarios. |
| eng/testing/tests.wasi.targets | Teach WASI test runner args/env for CoreCLR (corerun, CORE_ROOT, EH proposal flag). |
| eng/pipelines/runtime.yml | Add a rolling/non-gating CoreCLR-WASI library-tests smoke leg template invocation. |
| eng/pipelines/common/templates/wasi-wasm-coreclr-library-tests.yml | New: pipeline template for CoreCLR-WASI Helix library tests (smoke-only bring-up leg). |
… issue link Copilot PR review flagged leftover comments from the earlier prebuilt-corerun approach. CoreCLR-WASI now relinks the corerun host per-app; update the comments in WasiApp.InTree.props/.targets and sendtohelix-wasi.targets accordingly. Also fix the CI template to reference dotnet#130740 (the nested-type thunk-key fix) instead of dotnet#130739 (which tracks the remaining enclosing-type collision limitation). Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
- Use the full --target=wasm32-unknown-wasip2 triple (matching the sibling Mono WasiApp.targets) for the callhelper compile and the corerun link, instead of the wasm32-wasip2 alias. Clang-equivalent (vendor defaults to unknown) so ABI-identical to the coreclr static archives, but avoids relying on the alias across wasi-sdk versions and matches the surrounding wasi build. - Filter the runtime-pack framework assemblies into a separate item list once, then run a single Copy on the remainder, instead of an item-metadata Condition on the Copy task (which forced per-file task batching). Relink still builds and runs the full System.Runtime.Tests suite (9930 cases) under wasmtime. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
WASM0066 (added in dotnet#130051) warns for any P/Invoke to a module that isn't statically linked / ignored / [WasmImportLinkage] / "*" / QCall, then skips it (by design: throw-if-called). Untrimmed library-test bundles pull in cross-platform test-support assemblies whose foreign-platform P/Invokes (kernel32, Security.framework, libsecret, OpenSsl, ...) are all unresolved on wasm, so the generator emits hundreds of WASM0066 for a by-design skip and CI's warn-as-error fails the build. Mono uses a different generator; browser-CoreCLR runs only curated Wasm.Build.Tests today - so the wasi library-test leg is the first to feed the generator a real untrimmed closure. Add WarnOnUnresolvedPInvokeModules (default true, preserving today's behavior for hand-authored apps where an unresolved P/Invoke is likely a bug). When false, the unresolved-module case logs a low-importance message instead of WASM0066. WasiApp.CoreCLR.targets sets it false for the library-test relink; browser's CoreCLR targets should adopt the same knob when its library-test leg runs untrimmed closures. Fixes the LibraryTestsCoreCLR_WASI leg build (validated under /p:MSBuildTreatWarningsAsErrors=true; suite still runs 9930 cases). Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
…nvariant - The missing-coreclr_compat.h error suggested CORECLR_VM_WASM_INCLUDE_DIR/ MINIPAL_INCLUDE_DIR, which only affect the -I include dirs, not where the compat header is found. Give the compat header a real override (CORECLR_COMPAT_HEADER, mirroring the include-dir overrides) and reword the error to name it, so the remediation is truthful for out-of-tree/pack builds (dotnet#128362). - Only link libSystem.Globalization.Native.a + the ICU static libs when !InvariantGlobalization, matching BrowserWasmApp.CoreCLR.targets and the generator (which ignores/omits the globalization P/Invoke module when invariant). Verified: the invariant default links clean without them and still runs the full suite (9930 cases). Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
…invariant The static ICU shim needs the host to preload icudt.dat before managed globalization initializes - GlobalizationNative_LoadICU() (the no-path entry the managed side calls on wasi) returns 0 unless the data was already set and falls back to invariant. Browser does this from its JS host (wasm_load_icu_data); the wasi corerun host had no equivalent, so ICU never worked on CoreCLR-WASI and everything ran invariant. corerun now calls GlobalizationNative_LoadICUData(<app>/icudt.dat) after coreclr_initialize. The symbol is declared weak: it is only linked by the non-invariant per-app relink (which links libSystem.Globalization.Native.a + ICU), so an invariant relink leaves it null and the preload is skipped. WasiApp.CoreCLR.targets stages icudt.dat into managed/ (next to the assemblies) when not invariant. Validated: a non-invariant System.Runtime.Tests bundle now loads ICU and runs with zero CultureNotFoundException failures (was 36+); remaining failures are genuine wasm/interp gaps (no-JIT RuntimeFeature, exception stack line), triaged separately. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
…iant) Now that the relinked corerun host preloads icudt.dat, flip the leg's default from invariant to full ICU globalization (both the app-build default in WasiApp.InTree.props and the run default in tests.wasi.targets). This eliminates the CultureNotFoundException failures (was 36+ under invariant) and gives real globalization coverage; the remaining failures are genuine wasm/interpreter gaps (no-JIT RuntimeFeature/JitInfo, exception stack line) to be triaged separately. Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Comment on lines
+41
to
+43
| <!-- Override the same-named target imported above so we do NOT bundle Mono's | ||
| dotnet.wasm. For the CoreCLR bring-up we run invariant-globalization by default, so | ||
| no ICU data is bundled; corerun is placed into managed/ by the relink/copy below. --> |
…ime.Tests With ICU working the leg's only failures are 3 genuine wasm-interpreter behavior differences (CI: Tests run 72926, Passed 72812, Failed 3, Skipped 111): - RuntimeFeatureTests.DynamicCode_Jit: the interpreter doesn't JIT-compile, so IsDynamicCodeCompiled is false. Extend the existing browser-CoreCLR ActiveIssue (dotnet#123011) from IsBrowser to IsWasm so it also covers wasi. - ExceptionTests.ThrowStatementDoesNotResetExceptionStackLine{Same,Other}Method: the interpreter reports different rethrow stack-line info. These are [ConditionalFact(HasAssemblyFiles)] - browser skips (no assembly files) but wasi has real .dll's so they run and fail. ActiveIssue for IsWasi+IsCoreCLR (dotnet#130796). Co-authored-by: Copilot App <[email protected]> Copilot-Session: ea71f7e7-5477-4e58-bf1a-ca27d4b23d74
Comment on lines
+41
to
+43
| <!-- Override the same-named target imported above so we do NOT bundle Mono's | ||
| dotnet.wasm. For the CoreCLR bring-up we run invariant-globalization by default, so | ||
| no ICU data is bundled; corerun is placed into managed/ by the relink/copy below. --> |
Member
|
I think we should manufacture proper corehost for this. It will be similar as src\native\corehost\apphost\static\CMakeLists.txt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft. Stands up a CoreCLR-WASI library-test leg (smoke scope:
System.Runtime.Tests), addressing part of #130129.What this does
Adds a per-app native relink of the
corerunhost so a test's own[UnmanagedCallersOnly]reverse P/Invoke thunks are covered — mirroring the browser-CoreCLR relink. The baked callhelpers table (libcoreclr_gen_static.a) only covers framework top-level UCO callbacks; library/test callbacks (e.g. a nestedWindowsUILanguageHelper.EnumUiLanguagesCallback) are unknown to it and trap at first use without the relink.PInvokeCollector.cs): nested-type reverse-thunk key mismatch — the generator used reflectionType.Namespace(enclosing namespace) while the runtime uses the empty metadata namespace for nested types, so the thunk lookup returned NULL →precode_portable.cpp:35assert. (Same two commits as [wasm] Fix reverse P/Invoke thunk key for nested [UnmanagedCallersOnly] types (CoreCLR + Mono) #130740; see below.)WasiHost-Static→libWasiHost.a(mirrors browser'sBrowserHost-Static), installed tosharedFramework+ platform-manifest entry so it ships in the runtime pack.src/mono/wasi/build/WasiApp.CoreCLR.targets):ManagedToNativeGenerator(TargetOS=wasi) → compile the generated callhelpers with the wasi-sdk clang → relinkcorerunfrom the runtime-pack static archives + the app callhelper.o(replacinglibcoreclr_gen_static.a) viawasm-component-ld. Thewasi:httpimport thatSystem.Net.*pulls in is declared with-Wl,--component-type(WasiHttpWorld_component_type.wit, as Mono does); wasmtime is given-S httpby the test targets.tests.wasi.targets,sendtohelix-wasi.targets,tests.projsmoke set, and awasi-wasm-coreclr-library-testsCI template (non-gating, rolling).Validation
Locally (macOS arm64, wasi-sdk + wasmtime): the relink builds green and the bundle discovers + runs the full suite:
Managed exception dispatch through the interpreter EH + reverse UCO thunks works with no precode assert. Remaining
[FAIL]s are invariant-globalization-modeCultureNotFoundException(legitimate test outcomes), not infrastructure failures.Relationship to other work
function signature mismatchon the first (cold) R2R call to a method whose native code is not yet on its portable entry point #130634 stays scoped to the R2Rcall_indirectpath (not this pure-interpreter leg).Tracked follow-ups (not in this PR)
wasi:httpcapability (no-http vs http-capablecorerun), and the longer-term "statify the app-declared imports" direction.coreclr_compat.hto a neutral shared location + stage the vm/minipal headers and the wit for out-of-tree/Helix runs (the relink currently uses in-repo fallbacks in-tree).-Wl,-u,__main_voidin place of--whole-archive, conditioning ICU on!InvariantGlobalization, and incrementalInputs/Outputson the relink target.Note
This pull request was authored with GitHub Copilot.