Shared compiler server: X# parser state lives on the CSharpCommandLineParser.Default singleton and is corrupted by concurrent compilations

[hpetriffer]

Shared compiler server: X# parser state lives on the CSharpCommandLineParser.Default singleton and is corrupted by concurrent compilations

X# version: 3.0.1.0 (Release), C:\Program Files (x86)\XSharp\bin\xsc.exe
Build: .NET SDK 10.0.400, dotnet build on a solution with ~48 projects, default parallelism, UseSharedCompilation at its default (/shared passed to xsc.exe)

Symptom

Sporadic, on a solution that builds cleanly when retried:

XSC : error XS9014: The dialect 'VO' requires a reference to the runtime DLLs XSharp.Core.DLL and XSharp.RT.DLL.   [AcsRepSDK.xsproj]
XSC : error XS9019: Compiler option 'vo13' (VO Compatible string comparisons) is not supported for dialect VO      [AcsRepSDK.xsproj]
XSC : error XS9019: Compiler option 'vo14' (Float literal Values) is not supported for dialect Core                [AcsImpSDK.xsproj]
XSC : error XS9019: Compiler option 'vo13' (VO Compatible string comparisons) is not supported for dialect Core    [AcsTabSSDK.xsproj]

Two things in there cannot come from the affected project's own command line:

• the runtime DLLs are referenced, yet XS9014 claims they are not
• the projects set <Dialect>VO</Dialect> and are invoked with /dialect:VO, yet the diagnostic reports dialect Core

Evidence from the binary log

The Xsc invocation for AcsRepSDK.xsproj, on MSBuild node 11:19:

09:04:48.794 11:19>CoreCompile:
  C:\Program Files (x86)\XSharp\bin\xsc.exe /shared /noconfig /dialect:VO
  ...
  /reference:c:\entwicklung\NuGetPackageRepository\xsharp.core\3.0.1\lib\Net8.0\XSharp.Core.dll
  /reference:c:\entwicklung\NuGetPackageRepository\xsharp.rt\3.0.1\lib\Net8.0\XSharp.RT.dll
  /reference:c:\entwicklung\NuGetPackageRepository\xsharp.vo\3.0.1\lib\Net8.0\XSharp.VO.dll
  /out:obj\AnyCPU\Debug\AcsRep.dll

and immediately afterwards, from the same node:

09:04:49.278 11:19>XSC : error XS9014: The dialect 'VO' requires a reference to the runtime DLLs ...

AcsImpSDK.xsproj and AcsTabSSDK.xsproj were verified the same way: /dialect:VO on the command line, all three runtime references present, dialect reported as Core.

The three failures are 36 ms apart — 09:04:49.278 (node 11), 09:04:49.304 (node 4), 09:04:49.314 (node 23) — i.e. three compilations in flight at the same moment.

Cause

CSharpCommandLineParser is a process-wide singleton:

// src/Roslyn/src/Compilers/CSharp/Portable/CommandLine/CSharpCommandLineParser.cs:27
public static CSharpCommandLineParser Default { get; } = new CSharpCommandLineParser();

The X# partial class adds mutable per-parse state to that same instance:

// src/Compiler/src/Compiler/XSharpCodeAnalysis/CommandLine/XSharpCommandLineParser.cs:16
private XSharpSpecificCompilationOptions options;

and Parse resets it in the middle of the method:

// src/Roslyn/src/Compilers/CSharp/Portable/CommandLine/CSharpCommandLineParser.cs:150
#if XSHARP
    this.ResetXSharpCommandlineOptions();
#endif

Every other piece of parse state in that method is a local. options is not: it is a field on a shared singleton, written by ParseXSharpArgument for each option, by SetOptionFromReference for each /reference, and read at the end by ValidateXSharpSettings.

With /shared, the compiler server serves several compilations concurrently in one process, all going through CSharpCommandLineParser.Default. One request's ResetXSharpCommandlineOptions() discards the half-built state of another. Both observed symptoms follow directly:

• RuntimeAssemblies is populated only from /reference file names (XSharpSpecificCompilationOptions.SetOptionFromReference). If those writes land on an object that is then replaced, withRT is false in ValidateXSharpSettings and XS9014 is raised even though the references were on the command line.
• Dialect falls back to the default of a freshly constructed XSharpSpecificCompilationOptions, which is Core — hence XS9019 naming a dialect the project never selected.

This also explains why the errors are sporadic and disappear on a retry: they need two X# compilations to overlap inside the server.

Workaround

dotnet build MySolution.slnx -p:UseSharedCompilation=false

/shared is emitted from that property (src/Compiler/src/Compiler/XSharpBuildTask/ManagedCompiler.cs:760). Serialising the compiler server away removes the overlap, at the cost of compiler start-up per project.

Suggested direction

The per-parse X# state should not live on the parser instance. Options, roughly in order of preference:

1. Thread an XSharpSpecificCompilationOptions instance through the parse call chain as a parameter, the way Roslyn handles all other parse state. Largest change, but it removes the shared mutable state entirely.
2. Failing that, make the field [ThreadStatic] or AsyncLocal<T>. Cheaper, but only correct as long as a single parse never hops threads.

Worth checking in the same pass: src/Compiler/src/Compiler/XSharpCodeAnalysis/VsParser/Stubs.cs:202 calls ResetXSharpCommandlineOptions() on a parser as well — if the language service shares an instance across concurrent parses, it has the same exposure.

fail.zip

[RobertvanderHulst]

Thanks for the report and analysis. I assume you have used CoPilot for this?
The last message aboud Stubs.cs is not relevant for the compiler. This file is only used when compiling the parser used inside VS.
But that parser could also suffer from the same issue with the Default CommandLineParser.