diff --git a/en/clice/design/command-resolve.md b/en/clice/design/command-resolve.md index 4d422d70c..da61535cd 100644 --- a/en/clice/design/command-resolve.md +++ b/en/clice/design/command-resolve.md @@ -18,13 +18,13 @@ This command records how the build system invokes the compiler during an actual Of these three problems, the first has the greatest impact on user experience: missing implicit information. When the language server cannot correctly obtain system header paths, users see errors on standard library headers -- `#include ` reports "file not found," or GCC built-in type traits are flagged as undeclared identifiers. In clangd's issue tracker, these problems are consistently among the most frequently reported ([clangd#1262](https://github.com/clangd/clangd/issues/1262), [clangd#1691](https://github.com/clangd/clangd/issues/1691)). -clangd's solution is the `--query-driver` flag: users manually specify which compilers need probing, and clangd then queries those compilers for system paths. This approach has two problems. First, it is manual -- users must know which compiler their project uses and correctly configure a glob pattern. Second, the flag is not independently useful -- it relies on existing CDB commands to trigger probing ([clangd#1219](https://github.com/clangd/clangd/issues/1219)). For cross-compilation and embedded development scenarios, users frequently need to debug their configuration repeatedly before clangd correctly identifies the toolchain. +clangd's solution is the `--query-driver` flag: users manually specify which compilers need querying, and clangd then queries those compilers for system paths. This approach has two problems. First, it is manual -- users must know which compiler their project uses and correctly configure a glob pattern. Second, the flag is not independently useful -- it relies on existing CDB commands to trigger querying ([clangd#1219](https://github.com/clangd/clangd/issues/1219)). For cross-compilation and embedded development scenarios, users frequently need to debug their configuration repeatedly before clangd correctly identifies the toolchain. -clice automates the entire command processing pipeline: after reading commands from the CDB, it automatically identifies the compiler family (GCC, Clang, MSVC, etc.), automatically probes the toolchain, and minimizes startup overhead through multi-level deduplication. Users do not need to manually configure any toolchain-related parameters. +clice automates the entire command processing pipeline: after reading commands from the CDB, it automatically identifies the compiler family (GCC, Clang, MSVC, etc.), automatically queries the toolchain, and minimizes startup overhead through multi-level deduplication. Users do not need to manually configure any toolchain-related parameters. ## Design -The core task of command processing is to convert the raw driver commands in the CDB into cc1 arguments consumable by the Clang frontend. This conversion involves four conceptual layers: argument classification, command separation, toolchain probing, and search path extraction. +The core task of command processing is to convert the raw driver commands in the CDB into cc1 arguments consumable by the Clang frontend. This conversion involves four conceptual layers: argument classification, command separation, toolchain querying, and search path extraction. ### Argument Classification @@ -32,15 +32,15 @@ When loading the CDB, each compilation option is classified into one of seven ca - **Discarded**: Options related to build artifacts that the language server does not need. These include output file (`-o`), compile mode (`-c`), dependency scanning (`-M` family), PCH building (`-emit-pch`), options the driver itself would ignore for this input, and options only GCC reads: a warning clang does not know in a GCC command (`-Wlogical-op`) and an LTO job count (`-flto=8`). -- **Codegen**: Options that only affect the code generation backend and do not affect semantic analysis. These include frame pointer (`-fomit-frame-pointer`), unwind tables, function and data sections (`-ffunction-sections`), debug info (`-g` family), etc. They do not change the AST, diagnostic output or toolchain probing results. +- **Codegen**: Options that only affect the code generation backend and do not affect semantic analysis. These include frame pointer (`-fomit-frame-pointer`), unwind tables, function and data sections (`-ffunction-sections`), debug info (`-g` family), etc. They do not change the AST, diagnostic output or toolchain querying results. > Note: `-O`, `-fPIC`, `-fstack-protector`, `-fsanitize=` and `-flto` are not in this category despite appearing code-generation-related. `-O` defines the `__OPTIMIZE__` macro, `-fPIC` and `-fstack-protector` define `__PIC__` and `__SSP__`, and `-fsanitize=address` affects `__has_feature(address_sanitizer)`: they alter preprocessor state and are therefore semantic options. `-flto` is one because the driver accepts `-fsanitize=cfi` and `-fwhole-program-vtables` only with it. -- **Diagnostics**: Warning-control options (`-Wall`, `-Wno-*`, `-Werror`). They change which diagnostics are emitted but neither the AST nor toolchain probing results. +- **Diagnostics**: Warning-control options (`-Wall`, `-Wno-*`, `-Werror`). They change which diagnostics are emitted but neither the AST nor toolchain querying results. -- **User-content**: Options that may differ per file but do not affect toolchain probing results. These include include paths (`-I`, `-isystem`, `-iquote`, `-idirafter`), macro definitions (`-D`, `-U`), and forced includes (`-include`, `-imacros`). Their meaning is "what this particular file additionally needs." +- **User-content**: Options that may differ per file but do not affect toolchain querying results. These include include paths (`-I`, `-isystem`, `-iquote`, `-idirafter`), macro definitions (`-D`, `-U`), and forced includes (`-include`, `-imacros`). Their meaning is "what this particular file additionally needs." -- **Semantic**: The remaining recognized options -- they affect compilation semantics and play a role in toolchain probing. Examples include `-std=c++20`, `-target`, `-march=`, etc. +- **Semantic**: The remaining recognized options -- they affect compilation semantics and play a role in toolchain querying. Examples include `-std=c++20`, `-target`, `-march=`, etc. - **Input**: The source file itself, which occupies an explicit slot in the parsed command rather than being a plain string among others. @@ -50,20 +50,20 @@ Classification is based on Clang's own option table (`OptTable`), using option I ### Structured Commands -After classification, each command is parsed **once** into a structured configuration (`CompileConfig`): an interned sequence of classified arguments in which the input file occupies an explicit slot. Identical configurations deduplicate to a single instance identified by a stable `ConfigID`; strings are interned for pointer-stable comparison. Everything downstream — toolchain probing, rendering, rules, identity — consumes the structured form; nothing re-parses command strings. +After classification, each command is parsed **once** into a structured command (`Command`): an interned sequence of classified arguments in which the input file occupies an explicit slot. The `CommandTable` holds every command clice knows — the databases' entries, the hand-written ones and the rules-applied ones — each once, identified by a stable `CommandID`; strings are interned for pointer-stable comparison. Everything downstream — toolchain querying, rendering, rules, identity — consumes the structured form; nothing re-parses command strings. Two derived views matter: -- **Rendering**: the same structured configuration can be rendered on demand as a driver command line (for probing or for handing agents a runnable command) or combined with probe results into frontend arguments. Rendering normalizes alias spellings, so identity is not disturbed by cosmetic variation in the CDB. -- **Entry identity**: the frontend-relevant view of the configuration, together with the input's slot position and working directory, is hashed into the entry's **identity hash** — the stable identity used for CDB diffing, index snapshot validation, and pinned context choices. +- **Rendering**: the same structured command can be rendered on demand as a driver command line (for querying or for handing agents a runnable command) or combined with query results into frontend arguments. Rendering normalizes alias spellings, so identity is not disturbed by cosmetic variation in the CDB. +- **Command hash**: the frontend-relevant view of the command, together with the input's slot position and working directory, is hashed into its **command hash** — the stable identity used for CDB diffing, index snapshot validation, and pinned context choices. -The classification also buys toolchain probing cache efficiency. Probing requires actually invoking the compiler driver (`g++ -E -dM -v`), which takes from a few milliseconds to a hundred or more. The probing result depends only on the driver and probe-relevant options -- user-content options (`-I`, `-D`) do not affect the system paths or target triple output by the driver. Therefore, regardless of what different `-I` paths files may have, files sharing the probe-relevant view share one probing result. In real projects, tens of thousands of files typically collapse to a few dozen distinct probe keys. +The classification also buys toolchain querying cache efficiency. Querying requires actually invoking the compiler driver (`g++ -E -dM -v`), which takes from a few milliseconds to a hundred or more. The querying result depends only on the driver and query-relevant options -- user-content options (`-I`, `-D`) do not affect the system paths or target triple output by the driver. Therefore, regardless of what different `-I` paths files may have, files sharing the query-relevant view share one querying result. In real projects, tens of thousands of files typically collapse to a few dozen distinct query keys. ### Compilation Database -`CompilationDatabase` holds the entries of every loaded **source** — a `compile_commands.json` registered by the configuration or discovered under the workspace. Each source loads, reloads and unloads independently; its entries are parsed and deduplicated into `ConfigID`s and kept in file order, because generators emit a file's several entries in a fixed configuration order. The database stores facts only: it does not decide which of a file's entries wins. +`CompilationDatabase` holds the entries of every loaded **source** — a `compile_commands.json` registered by the configuration or discovered under the workspace. Each source loads, reloads and unloads independently; its entries are parsed into the `CommandTable` and kept in file order, because generators emit a file's several entries in a fixed configuration order. The database stores facts only: it does not decide which of a file's entries wins. -Commands written by hand — a rule's `default_command`, the builtin fallback — go through the same normalization as an entry (`intern_command`), with the input slot synthesized at the end, so a declared command is probed, edited and rendered exactly like a database entry. +Commands written by hand — a rule's `default_command`, the builtin fallback — go through the same normalization as an entry (`Normalizer`), with the input slot synthesized at the end, so a declared command is queried, edited and rendered exactly like a database entry. ### Build @@ -98,15 +98,15 @@ The final command of a file is composed from those layers in a fixed order: the Patterns are compiled against absolute paths: a relative pattern is anchored at the configuration file's directory (`..` segments included), or at the workspace root for rules passed through `initializationOptions`; an absolute or `**`-led one matches as written. Rules carrying a `configuration` tag apply only while that tag is active; the distinct tags form the configuration menu, and the active one is resolved at startup from `--configuration`, the persisted selection, or `default_configuration` (see [Compilation Context](compilation-context.md#switching-build-configurations)). When any rule declares a source (a database or a `default_command`), discovery is off: the declaration is the whole intent, and a database sitting at the workspace root is not consulted. Only a configuration declaring no source at all falls back to [discovery](#discovery). -Configuration granularity is preserved throughout the pipeline: search-path extraction for dependency scanning is per effective command (different `-I` sets produce different search configs), while toolchain probing deduplicates further still, since user-content options do not affect probe results. +Configuration granularity is preserved throughout the pipeline: search-path extraction for dependency scanning is per effective command (different `-I` sets produce different search configs), while toolchain querying deduplicates further still, since user-content options do not affect query results. ### Toolchain -`Toolchain` converts driver-level commands into cc1 arguments. Its design centers on two core capabilities: +`Toolchain` converts driver-level commands into cc1 arguments: it queries the driver, in the way the command's compiler family takes (one file per compiler kind under `src/command/toolchains/`), and caches the answers. -**Compiler family identification.** `Toolchain` identifies the compiler family from the executable name -- the `CompilerFamily` enum includes GCC, Clang, MSVC, ClangCL, NVCC, Intel, and Zig. The identification logic handles various naming variants: version suffixes (`clang++-17`), architecture prefixes (`arm-none-eabi-g++`), and Windows `.exe` suffixes. The name decides only what speaks another language: `cl` and `clang-cl`, nvcc, and zig's two-word driver. Whether a GCC-style driver is GCC or clang underneath is told by its own answer to the probe, never by its name: `/usr/bin/c++` is clang on macOS, `emcc` and `zig cc` are clang. +**Compiler family.** A command's compiler family is read from the executable name when the command is normalized -- the `CompilerFamily` enum includes GCC, Clang, MSVC, ClangCL, NVCC, Intel, and Zig. The identification logic handles various naming variants: version suffixes (`clang++-17`), architecture prefixes (`arm-none-eabi-g++`), and Windows `.exe` suffixes. The name decides only what speaks another language: `cl` and `clang-cl`, nvcc, and zig's two-word driver. Whether a GCC-style driver is GCC or clang underneath is told by its own answer to the query, never by its name: `/usr/bin/c++` is clang on macOS, `emcc` and `zig cc` are clang. -**Caching strategy.** The cache has two layers. The **probe layer** caches the frontend each probe's job sets up, already cleaned of the probe's own input and of what clice's frontend cannot take, keyed by the probe-relevant shape of the command (driver, probe-relevant flags, input kind — `.c` and `.cpp` may trigger different driver rules; the working directory joins the key when the command is sensitive to it). The **synthesis layer** adds each command's own user content to it, per input kind, for command rendering to consume. Failed probes are cached too (negative caching), but transient failures are retried after a cooldown rather than being remembered forever. +**Caching strategy.** The cache has two layers. The **query layer** caches the frontend each query's job sets up, already cleaned of the query's own input and of what clice's frontend cannot take, keyed by the query-relevant shape of the command (driver, query-relevant flags, input kind — `.c` and `.cpp` may trigger different driver rules; the working directory joins the key when the command is sensitive to it). The **synthesis layer** adds each command's own user content to it, per input kind, for command rendering to consume. Failed queries are cached too (negative caching), but transient failures are retried after a cooldown rather than being remembered forever. ### Search Paths @@ -132,16 +132,16 @@ CDB loading uses simdjson for streaming JSON parsing, processing entries one by 3. Translate NVCC commands into an equivalent clang CUDA invocation 4. Filter out non-C/C++ files (e.g., `.rc`, `.asm`, `.def`) 5. Resolve relative file paths to absolute paths -6. Parse and classify each option once into the structured configuration, absolutizing relative include paths against `directory` -7. Deduplicate identical configurations into shared `ConfigID`s +6. Parse and classify each option once into the structured command, absolutizing relative include paths against `directory` +7. Deduplicate identical commands into shared `CommandID`s A database written inside Cygwin or MSYS2, by CMake run from their shell or a tool like it, spells its paths POSIX-style: `/cygdrive/c/proj` or `/c/proj`, and `/usr/bin/c++.exe` for the compiler. On Windows, the mount table of the Cygwin or MSYS2 runtime that wrote them maps them to Windows paths: the entry's directory and file, its compiler and the values of its path options. That runtime is the first on `PATH` whose table maps the first such entry's directory, and its compiler when it is spelled so, to files that exist: another runtime's table maps the directory under its own root, and Git for Windows' MSYS2, which maps an MSYS2 installation's drive paths alike, lacks its `/usr/bin/c++.exe`. Its `command` strings split as sh splits them. When no runtime on `PATH` maps the database, it is not read, and the user is told to put the right one there. The parsing process also recognizes the precompiled headers CMake passes to cc1 directly: a unit's `-Xclang -include-pch -Xclang ` and the `-Xclang -emit-pch` of the entry building it become structured options that no compile renders. The PCH on disk may come from another compiler, so clice builds its own from the command of the entry writing it (CMake's `cmake_pch.hxx.cxx`, also found through the `.gch` GCC writes beside a forced header), and every unit reading that PCH loads clice's build instead: the open file's preamble PCH chains onto it, and the background index parses over it. A header the PCH holds, opened on its own, compiles without it: the PCH's guards would skip the header's own text. The forced header the PCH stands for goes from its command too, as any forced include the compile enters a header through does (see [Compilation Context](compilation-context.md#automatic-self-containedness-detection)). -### Toolchain Probing +### Toolchain Query -Different compiler families use different probing strategies: +Different compiler families use different querying strategies, each in its own file under `src/command/toolchains/`: **GCC-style drivers** (gcc, clang, Apple clang, zig, emcc, cross and bare-metal GCC): the driver is asked once to preprocess an empty file of the input's language with the command's own flags, verbosely and printing its macros (`-E -dM -v`). The answer tells everything clice needs: @@ -150,7 +150,7 @@ Different compiler families use different probing strategies: Asked with the command's own flags, the driver answers for the command, and a GCC answers only when that compile succeeds: a flag that switches one of its defaults off (`-fno-stack-protector`), a `--sysroot` or a `--specs` file shows in its answer. CPU and ABI names clang does not know (GCC's `-mtune=intel`, `-mabi=lp64` on Darwin) are dropped from the cc1 command, and GCC's two-core form `-mcpu=cortex-a72.cortex-a53` keeps its first core. A universal build (`-arch arm64 -arch x86_64`) is read for its first architecture. -**Cygwin and MSYS2 compilers** on Windows (Cygwin's and MSYS2's own GCC, and toolchains built on Cygwin such as Andes' RISC-V GCC) answer as any GCC-style driver does, but in POSIX paths (`/usr/include`, `/cygdrive/c/...`), which clice's clang cannot open. The mount table of the runtime the driver loads maps them, read from the runtime's own `mount` for each probe. Of the runtimes (`cygwin1.dll`, `msys-2.0.dll`) beside the driver and on `PATH`, in the order Windows looks for a program's DLLs, that is the first whose table maps the program of the driver's job to a file that exists: taking the first alone would pick Git for Windows' MSYS2 where it comes ahead on `PATH` of the Cygwin a toolchain outside it loads. An assumed mapping would get them wrong, since Cygwin mounts `/usr/lib` at `/lib`, MSYS2 its drives at `/c`, and a user's `/etc/fstab` adds mounts of its own. The driver itself is found as Cygwin finds it: by a name without `.exe`, and through a Cygwin symlink (CMake records `/usr/bin/cc`, a link to `gcc.exe`). Such a compiler starts only with its runtime's `bin` directory on `PATH`; without it, the query fails with an error that says so, and [Editor Setup](../guide/editors.md#compiler-environment) sets it per editor. +**Cygwin and MSYS2 compilers** on Windows (Cygwin's and MSYS2's own GCC, and toolchains built on Cygwin such as Andes' RISC-V GCC) answer as any GCC-style driver does, but in POSIX paths (`/usr/include`, `/cygdrive/c/...`), which clice's clang cannot open. The mount table of the runtime the driver loads maps them, read from the runtime's own `mount` for each query. Of the runtimes (`cygwin1.dll`, `msys-2.0.dll`) beside the driver and on `PATH`, in the order Windows looks for a program's DLLs, that is the first whose table maps the program of the driver's job to a file that exists: taking the first alone would pick Git for Windows' MSYS2 where it comes ahead on `PATH` of the Cygwin a toolchain outside it loads. An assumed mapping would get them wrong, since Cygwin mounts `/usr/lib` at `/lib`, MSYS2 its drives at `/c`, and a user's `/etc/fstab` adds mounts of its own. The driver itself is found as Cygwin finds it: by a name without `.exe`, and through a Cygwin symlink (CMake records `/usr/bin/cc`, a link to `gcc.exe`). Such a compiler starts only with its runtime's `bin` directory on `PATH`; without it, the query fails with an error that says so, and [Editor Setup](../guide/editors.md#compiler-environment) sets it per editor. **MSVC / ClangCL**: cl or clang-cl is asked once to preprocess (`/EP`) a file naming `_MSVC_LANG` and `_MSC_FULL_VER`. They tell the C++ standard it compiles in, since what `/std:c++latest` means moves from one version to the next, and cl's version, where clang's own detection would take the newest toolset installed. Clang's driver in MSVC-compatible mode (`--driver-mode=cl`) then builds the cc1 command in that standard, with cl's version as `-fms-compatibility-version`. clang-cl's version is the one that same detection finds from the toolset its flags select (`/winsysroot`). cl hears only the command's `/std:`: the other flags are spelled for clang's cl mode, which cl rejects, and a standard or version the command sets in clang's spelling (a rule appending `-std=c++23`) stays clang's. @@ -160,13 +160,13 @@ Asked with the command's own flags, the driver answers for the command, and a GC > The external driver's version may be newer than clice's embedded LLVM version, and its cc1 output may contain options that clice does not recognize. During parsing, unknown options are silently dropped to ensure compatibility. -After probing completes, the temporary probe file's path and module-output-related flags (which reference the deleted temp file) are stripped from the results. If clice's resource dir differs from what the probe returned, all related paths are replaced to ensure the frontend uses builtin headers from a matching version; so is any other directory of clang builtin headers a search path names (another clang's `-isystem /lib/clang//include`, zig's `lib/include`). +After querying completes, the temporary query file's path and module-output-related flags (which reference the deleted temp file) are stripped from the results. If clice's resource dir differs from what the query returned, all related paths are replaced to ensure the frontend uses builtin headers from a matching version; so is any other directory of clang builtin headers a search path names (another clang's `-isystem /lib/clang//include`, zig's `lib/include`). -**Startup warm-up.** During the dependency scanning phase at server startup, all unique toolchain cache keys are collected and probed in parallel. Pipe draining and process waiting for probe subprocesses also run concurrently to avoid deadlocks from full pipes. Once warm-up completes, all subsequent toolchain queries hit the cache. +**Startup warm-up.** During the dependency scanning phase at server startup, all unique toolchain cache keys are collected and queried in parallel. Pipe draining and process waiting for query subprocesses also run concurrently to avoid deadlocks from full pipes. Once warm-up completes, all subsequent toolchain queries hit the cache. ### Search Path Extraction -After toolchain probing yields cc1 arguments, search path extraction iterates through these arguments, routing include path options by type into the four tiers. All paths are resolved to absolute paths and normalized (removing `.` and `..`). +After toolchain querying yields cc1 arguments, search path extraction iterates through these arguments, routing include path options by type into the four tiers. All paths are resolved to absolute paths and normalized (removing `.` and `..`). The `-iprefix` / `-iwithprefix` / `-iwithprefixbefore` options must be processed cooperatively in order of appearance: `-iprefix` sets a prefix, subsequent `-iwithprefix` prepends the prefix to its path and places it in the After tier, and `-iwithprefixbefore` places it in the Angled tier. @@ -214,7 +214,7 @@ After the four tiers are concatenated, deduplication begins from the Angled tier - **Cygwin symlinks among the headers are not followed.** A header Cygwin installs as a symlink (ncurses' `ncurses.h` in its `/usr/include`) is a file only Cygwin programs follow: clice's clang reads the link itself. Only the driver is followed through such a link. -- **Incomplete support for some compiler families.** A driver that answers the probe with neither a clang job nor a GCC one -- Intel's classic `icc`, other vendors' compilers -- gets clang's own defaults for the running machine, and the toolchain error says so. NVCC has dedicated probing (parsing `nvcc --dryrun` output into a clang CUDA invocation), but only with a GCC or Clang host compiler -- nvcc driving MSVC `cl` is not supported, and multi-architecture commands parse the newest architecture only. +- **Incomplete support for some compiler families.** A driver that answers the query with neither a clang job nor a GCC one -- Intel's classic `icc`, other vendors' compilers -- gets clang's own defaults for the running machine, and the toolchain error says so. NVCC has dedicated querying (parsing `nvcc --dryrun` output into a clang CUDA invocation), but only with a GCC or Clang host compiler -- nvcc driving MSVC `cl` is not supported, and multi-architecture commands parse the newest architecture only. - **GCC-only language parses as clang sees it.** Code that only GCC compiles -- GCC-only builtins and extensions, `__float128` on targets where clang lacks it, nested functions -- shows clang's errors, and a CPU name only GCC knows loses the macros of its features. The headers only GCC's private include directory holds (`omp.h`, `quadmath.h`) are not found: that directory comes with GCC's versions of clang's builtin headers. @@ -224,4 +224,4 @@ After the four tiers are concatenated, deduplication begins from the Angled tier - **MSVC-style option parsing.** On non-Windows systems, MSVC-style option prefixes (`/U`, `/D`, `/I`) must be handled specially to prevent Unix absolute paths (such as `/Users/...`) from being misparsed as MSVC options. This is currently resolved by dynamically adjusting option visibility based on the driver name, but edge cases may still exist. -- **Only the common compiler launchers are recognized.** ccache, sccache, distcc and icecc are taken off a command with their own options and environment assignments, and the compiler after them is the one clice asks. Any other launcher is taken for the compiler, so family identification and toolchain probing target the wrong binary; strip it from the compilation database. +- **Only the common compiler launchers are recognized.** ccache, sccache, distcc and icecc are taken off a command with their own options and environment assignments, and the compiler after them is the one clice asks. Any other launcher is taken for the compiler, so family identification and toolchain querying target the wrong binary; strip it from the compilation database. diff --git a/en/clice/design/compilation-context.md b/en/clice/design/compilation-context.md index 2ae577624..4e246be07 100644 --- a/en/clice/design/compilation-context.md +++ b/en/clice/design/compilation-context.md @@ -98,7 +98,7 @@ When a file needs to be compiled, the compilation context is automatically deter Beyond automatic resolution, clice also provides four LSP extension requests that let users explicitly query and switch compilation contexts: -**`clice/queryContext`** lists all available compilation contexts for a file. For headers, it returns the source files that can serve as hosts, ranked by relevance; for source files, it returns all CDB entries, using key flags from the compilation command (`-D`, `-std`, `-O`, etc.) as human-readable descriptions, each carrying a command hash identifying the entry. Results support pagination. For self-contained headers and source files, the context list is deduplicated by the entry's **identity hash**, computed over the frontend-relevant view of the effective command plus the input's slot position and working directory: two contexts with the same identity produce identical compilation results, so only the best-ranked representative is kept — otherwise a widely-included header like `` could list thousands of contexts. Non-self-contained headers are not merged this way: different hosts synthesize different prefixes, so every host is a distinct context. When the host's compile enters a header several times (a guard-less header), each place it does is returned as a separate entry distinguished by an occurrence number. +**`clice/queryContext`** lists all available compilation contexts for a file. For headers, it returns the source files that can serve as hosts, ranked by relevance; for source files, it returns all CDB entries, using key flags from the compilation command (`-D`, `-std`, `-O`, etc.) as human-readable descriptions, each carrying a command hash identifying the entry. Results support pagination. For self-contained headers and source files, the context list is deduplicated by the entry's **command hash**, computed over the frontend-relevant view of the effective command plus the input's slot position and working directory: two contexts with the same identity produce identical compilation results, so only the best-ranked representative is kept — otherwise a widely-included header like `` could list thousands of contexts. Non-self-contained headers are not merged this way: different hosts synthesize different prefixes, so every host is a distinct context. When the host's compile enters a header several times (a guard-less header), each place it does is returned as a separate entry distinguished by an occurrence number. **`clice/currentContext`** returns the context the file compiles under, as the listing names it, and whether the user chose it: the selection when there is one (host + occurrence, or command hash), else the context picked automatically. It returns no context only for a file that borrows none and has no entry of its own. diff --git a/en/clice/design/overview.md b/en/clice/design/overview.md index 6460e1556..43884e3dd 100644 --- a/en/clice/design/overview.md +++ b/en/clice/design/overview.md @@ -37,8 +37,9 @@ General-purpose utilities and infrastructure shared by all other modules. Commands read from the compilation database (CDB) are raw commands generated by the build system and cannot be fed directly to the Clang frontend -- they may contain options meant only for code generation, lack system header search paths, or include parameters irrelevant to a language server. This module parses each command once into a structured form that the rest of the system consumes. -- `CompilationDatabase`: Loads `compile_commands.json` and parses every entry once into an interned, classified argument structure identified by a stable `ConfigID`. On top of that it applies user configuration rules, synthesizes fallback commands for files without an entry, derives the input language, computes entry identity hashes, and renders driver or `-cc1` command lines on demand -- `Toolchain`: Queries system compilers for implicit compilation parameters (such as system header search paths), with a two-layer cache — probe results per driver invocation shape, and synthesized flag sets per input kind +- `CommandTable`: Every command clice knows, parsed once into an interned, classified argument structure identified by a stable `CommandID`, with what derives from one command: its input language, the precompiled headers it writes and reads, its command hash, its driver-level renders +- `CompilationDatabase`: Loads, reloads and diffs the `compile_commands.json` files, their entries normalized (`Normalizer`) into the `CommandTable`; the rules' edits apply on top of an entry's command (`apply_edits`) +- `Toolchain`: Queries system compilers for implicit compilation parameters (such as system header search paths), with a two-layer cache — query results per driver invocation shape, and synthesized flag sets per input kind — and renders the `-cc1` command lines; how each compiler kind is asked (GCC-style, cl, nvcc, clang's own driver) lives under `src/command/toolchains/` - `SearchConfig`: A four-tier model for header search paths (Quoted / Angled / System / After), matching Clang's internal search logic See [Compilation Command Resolution](command-resolve.md). diff --git a/en/clice/design/symbol-index.md b/en/clice/design/symbol-index.md index 1ebc9c65d..54e2e0ce9 100644 --- a/en/clice/design/symbol-index.md +++ b/en/clice/design/symbol-index.md @@ -160,7 +160,7 @@ The index is rebuilt from the records, on the thread pool, when the symbols merg ### Staleness Detection -Staleness detection determines whether a file needs to be re-indexed. Each indexed artifact records the identity and observed content version of its inputs; validation is delegated to the master's shared file table, which performs the same two-layer check used everywhere else — a (size, mtime) stat fast path against the file's last observation, then content-hash confirmation (see [Incremental Compilation](incremental-parse.md)). Re-indexing triggers only when input content actually changed; command changes are caught separately through the entry identity hashes recorded in the manifests. At startup the writer checks every file record against the disk and queues the units depending on a file that changed, on a place a failed lookup looked at that now holds a file, or on a changed input of a module or a precompiled header, and the units whose variants a rebuild of a shared file discarded; the stored shards serve until they are indexed again. +Staleness detection determines whether a file needs to be re-indexed. Each indexed artifact records the identity and observed content version of its inputs; validation is delegated to the master's shared file table, which performs the same two-layer check used everywhere else — a (size, mtime) stat fast path against the file's last observation, then content-hash confirmation (see [Incremental Compilation](incremental-parse.md)). Re-indexing triggers only when input content actually changed; command changes are caught separately through the command hashes recorded in the manifests. At startup the writer checks every file record against the disk and queues the units depending on a file that changed, on a place a failed lookup looked at that now holds a file, or on a changed input of a module or a precompiled header, and the units whose variants a rebuild of a shared file discarded; the stored shards serve until they are indexed again. ### Background Indexing Scheduling diff --git a/zh/clice/design/command-resolve.md b/zh/clice/design/command-resolve.md index f57fbb008..6d8a15620 100644 --- a/zh/clice/design/command-resolve.md +++ b/zh/clice/design/command-resolve.md @@ -18,13 +18,13 @@ g++ -std=c++20 -O2 -g -I../include -DNDEBUG -c src/foo.cpp 在这三个问题中，第一个问题对用户体验的影响最大：缺少隐式信息。当语言服务器无法正确获取系统头文件路径时，用户会遇到标准库头文件错误——`#include ` 报告 "file not found"，或者 GCC 内置类型特征被标记为未声明的标识符。在 clangd 的问题跟踪器中，这类问题一直是最常被报告的问题之一（[clangd#1262](https://github.com/clangd/clangd/issues/1262)、[clangd#1691](https://github.com/clangd/clangd/issues/1691)）。 -clangd 的解决方案是使用 `--query-driver` 参数：用户手动指定需要探测的编译器，随后 clangd 查询这些编译器以获取系统路径。这种方式有两个问题。首先，它需要手动配置——用户必须知道项目使用的是哪个编译器，并正确配置 glob 模式。其次，该参数无法独立发挥作用——它依赖 CDB 中已有的命令来触发探测（[clangd#1219](https://github.com/clangd/clangd/issues/1219)）。在交叉编译和嵌入式开发场景中，用户往往需要反复调试配置，clangd 才能正确识别工具链。 +clangd 的解决方案是使用 `--query-driver` 参数：用户手动指定需要查询的编译器，随后 clangd 查询这些编译器以获取系统路径。这种方式有两个问题。首先，它需要手动配置——用户必须知道项目使用的是哪个编译器，并正确配置 glob 模式。其次，该参数无法独立发挥作用——它依赖 CDB 中已有的命令来触发查询（[clangd#1219](https://github.com/clangd/clangd/issues/1219)）。在交叉编译和嵌入式开发场景中，用户往往需要反复调试配置，clangd 才能正确识别工具链。 -clice 实现了整个命令处理流程的自动化：从 CDB 读取命令后，它会自动识别编译器家族（GCC、Clang、MSVC 等）、自动探测工具链，并通过多级去重最大限度地降低启动开销。用户无需手动配置任何工具链相关参数。 +clice 实现了整个命令处理流程的自动化：从 CDB 读取命令后，它会自动识别编译器家族（GCC、Clang、MSVC 等）、自动查询工具链，并通过多级去重最大限度地降低启动开销。用户无需手动配置任何工具链相关参数。 ## 设计 -命令处理的核心任务是将 CDB 中的原始驱动命令转换为可供 Clang 前端使用的 cc1 参数。这个转换过程涉及四个概念层：参数分类、命令分离、工具链探测和搜索路径提取。 +命令处理的核心任务是将 CDB 中的原始驱动命令转换为可供 Clang 前端使用的 cc1 参数。这个转换过程涉及四个概念层：参数分类、命令分离、工具链查询和搜索路径提取。 ### 参数分类 @@ -32,15 +32,15 @@ clice 实现了整个命令处理流程的自动化：从 CDB 读取命令后， - **丢弃**：语言服务器不需要的构建产物相关选项，包括输出文件（`-o`）、编译模式（`-c`）、依赖扫描（`-M` 系列）、PCH 构建（`-emit-pch`）、驱动程序本身针对该输入会忽略的选项，还有只有 GCC 才读取的选项：GCC 命令里 clang 不认识的警告（`-Wlogical-op`）和 LTO 并行任务数（`-flto=8`）。 -- **代码生成**：只影响代码生成后端、不影响语义分析的选项。包括帧指针（`-fomit-frame-pointer`）、unwind 表（unwind tables）、函数与数据分段（`-ffunction-sections`）、调试信息（`-g` 系列）等。它们不会改变 AST、诊断输出或工具链探测结果。 +- **代码生成**：只影响代码生成后端、不影响语义分析的选项。包括帧指针（`-fomit-frame-pointer`）、unwind 表（unwind tables）、函数与数据分段（`-ffunction-sections`）、调试信息（`-g` 系列）等。它们不会改变 AST、诊断输出或工具链查询结果。 > 注意：`-O`、`-fPIC`、`-fstack-protector`、`-fsanitize=` 和 `-flto` 虽然看似与代码生成有关，但不属于此类。`-O` 会定义 `__OPTIMIZE__` 宏，`-fPIC` 和 `-fstack-protector` 会定义 `__PIC__` 和 `__SSP__`，`-fsanitize=address` 会影响 `__has_feature(address_sanitizer)`：它们会改变预处理器状态，因此属于语义选项。`-flto` 也属于语义选项，因为驱动程序只在有它时才接受 `-fsanitize=cfi` 和 `-fwhole-program-vtables`。 -- **诊断**：警告控制选项（`-Wall`、`-Wno-*`、`-Werror`）。它们会改变发出哪些诊断，但不会影响 AST 或工具链探测结果。 +- **诊断**：警告控制选项（`-Wall`、`-Wno-*`、`-Werror`）。它们会改变发出哪些诊断，但不会影响 AST 或工具链查询结果。 -- **用户内容**：可能因文件而异、但不影响工具链探测结果的选项。包括包含路径（`-I`、`-isystem`、`-iquote`、`-idirafter`）、宏定义（`-D`、`-U`）和强制包含（`-include`、`-imacros`）。它们表达的是“这个特定文件还需要什么”。 +- **用户内容**：可能因文件而异、但不影响工具链查询结果的选项。包括包含路径（`-I`、`-isystem`、`-iquote`、`-idirafter`）、宏定义（`-D`、`-U`）和强制包含（`-include`、`-imacros`）。它们表达的是“这个特定文件还需要什么”。 -- **语义**：其余可识别的选项——它们会影响编译语义，并在工具链探测中发挥作用。例如 `-std=c++20`、`-target`、`-march=` 等。 +- **语义**：其余可识别的选项——它们会影响编译语义，并在工具链查询中发挥作用。例如 `-std=c++20`、`-target`、`-march=` 等。 - **输入**：源文件本身，它在解析后的命令中占据一个明确的槽位，而非众多普通字符串中的一个。 @@ -50,20 +50,20 @@ clice 实现了整个命令处理流程的自动化：从 CDB 读取命令后， ### 结构化命令 -分类完成后，每条命令只解析**一次**，形成结构化配置（`CompileConfig`）：一个由已分类参数组成的驻留序列，其中输入文件占据明确的槽位。相同的配置会去重为单个实例，由稳定的 `ConfigID` 标识；字符串也会被驻留，以便通过稳定的指针进行比较。所有下游处理——工具链探测、渲染、规则和标识——都使用这种结构化形式；不会重新解析命令字符串。 +分类完成后，每条命令只解析**一次**，形成结构化命令（`Command`）：一个由已分类参数组成的驻留序列，其中输入文件占据明确的槽位。`CommandTable` 保存 clice 知道的每一条命令——数据库中的条目、手工编写的命令以及应用规则后得到的命令——每条只存一份，由稳定的 `CommandID` 标识；字符串也会被驻留，以便通过稳定的指针进行比较。所有下游处理——工具链查询、渲染、规则和标识——都使用这种结构化形式；不会重新解析命令字符串。 有两个重要的派生视图： -- **渲染**：同一份结构化配置可以按需渲染成编译器驱动程序命令行（用于探测，或向代理提供可运行的命令），也可以与探测结果组合成前端参数。渲染会规范化别名的拼写形式，因此 CDB 中仅形式上的差异不会影响标识。 -- **条目标识**：配置中与前端相关的视图，连同输入文件的槽位位置和工作目录，会共同哈希为条目的**标识哈希**——这是用于 CDB 差异比较、索引快照验证和固定上下文选择的稳定标识。 +- **渲染**：同一条结构化命令可以按需渲染成编译器驱动程序命令行（用于查询，或向代理提供可运行的命令），也可以与查询结果组合成前端参数。渲染会规范化别名的拼写形式，因此 CDB 中仅形式上的差异不会影响标识。 +- **命令哈希**：命令中与前端相关的视图，连同输入文件的槽位位置和工作目录，会共同哈希为它的**命令哈希**——这是用于 CDB 差异比较、索引快照验证和固定上下文选择的稳定标识。 -这种分类还能提高工具链探测的缓存效率。探测需要实际调用编译器驱动程序（`g++ -E -dM -v`），耗时从几毫秒到一百多毫秒不等。探测结果只取决于驱动程序和与探测相关的选项——用户内容选项（`-I`、`-D`）不会影响驱动程序输出的系统路径或目标三元组。因此，即使各文件的 `-I` 路径不同，只要探测相关视图相同，就会共用同一份探测结果。在实际项目中，上万个文件通常会归并为几十个不同的探测键。 +这种分类还能提高工具链查询的缓存效率。查询需要实际调用编译器驱动程序（`g++ -E -dM -v`），耗时从几毫秒到一百多毫秒不等。查询结果只取决于驱动程序和与查询相关的选项——用户内容选项（`-I`、`-D`）不会影响驱动程序输出的系统路径或目标三元组。因此，即使各文件的 `-I` 路径不同，只要查询相关视图相同，就会共用同一份查询结果。在实际项目中，上万个文件通常会归并为几十个不同的查询键。 ### 编译数据库 -`CompilationDatabase` 保存每个已加载**来源**的条目——来源即由配置注册或在工作区中发现的一份 `compile_commands.json`。每个来源独立加载、重新加载和卸载；它的条目被解析、去重为 `ConfigID`，并保持文件内的原有顺序，因为生成器输出同一文件的多个条目时会遵循固定的配置顺序。数据库只存事实：它不决定一个文件的哪个条目胜出。 +`CompilationDatabase` 保存每个已加载**来源**的条目——来源即由配置注册或在工作区中发现的一份 `compile_commands.json`。每个来源独立加载、重新加载和卸载；它的条目解析后存入 `CommandTable`，并保持文件内的原有顺序，因为生成器输出同一文件的多个条目时会遵循固定的配置顺序。数据库只存事实：它不决定一个文件的哪个条目胜出。 -手工编写的命令——规则的 `default_command`、内置的兜底命令——与条目经过同一套规范化流程（`intern_command`），并在末尾合成输入槽位，因此声明的命令在探测、编辑和渲染方面与数据库条目完全一致。 +手工编写的命令——规则的 `default_command`、内置的兜底命令——与条目经过同一套规范化流程（`Normalizer`），并在末尾合成输入槽位，因此声明的命令在查询、编辑和渲染方面与数据库条目完全一致。 ### 构建 @@ -98,15 +98,15 @@ clice 实现了整个命令处理流程的自动化：从 CDB 读取命令后， 模式针对绝对路径匹配：相对模式锚定在配置文件所在的目录（含 `..` 路径段），通过 `initializationOptions` 传入的规则则锚定在工作区根目录；绝对模式或以 `**` 开头的模式按原样匹配。带 `configuration` 标签的规则只在该标签生效时适用；不同的标签构成配置菜单，启动时按 `--configuration`、持久化的选择、`default_configuration` 的顺序决定哪个配置生效（参见[编译上下文](compilation-context.md#switching-build-configurations)）。只要有任何规则声明了来源（一份数据库或一条 `default_command`），发现流程就会关闭：声明即代表全部意图，不会再采用工作区根目录中的数据库。只有完全没有声明任何来源的配置才会回退到[发现](#discovery)。 -配置粒度在整个流水线中保持不变：依赖扫描的搜索路径提取按实际生效的命令进行（不同的 `-I` 集合产生不同的搜索配置），而工具链探测会进一步去重，因为与用户内容有关的选项不会影响探测结果。 +配置粒度在整个流水线中保持不变：依赖扫描的搜索路径提取按实际生效的命令进行（不同的 `-I` 集合产生不同的搜索配置），而工具链查询会进一步去重，因为与用户内容有关的选项不会影响查询结果。 ### 工具链 -`Toolchain` 将驱动级命令转换为 cc1 参数。它的设计围绕两个核心能力： +`Toolchain` 将驱动级命令转换为 cc1 参数：它按命令的编译器家族所要求的方式查询驱动程序（每种编译器各占 `src/command/toolchains/` 下的一个文件），并缓存得到的回答。 -**编译器家族识别。** `Toolchain` 根据可执行文件名识别编译器家族——`CompilerFamily` 枚举包括 GCC、Clang、MSVC、ClangCL、NVCC、Intel 和 Zig。识别逻辑可处理多种命名变体：版本后缀（`clang++-17`）、架构前缀（`arm-none-eabi-g++`）以及 Windows 的 `.exe` 后缀。名称只用来认出使用另一套命令行语法的驱动程序：`cl` 和 `clang-cl`、nvcc，以及由两个词组成的 zig 驱动程序。一个 GCC 风格的驱动程序底层究竟是 GCC 还是 clang，由它自己对探测的回答判断，从不看名称：`/usr/bin/c++` 在 macOS 上是 clang，`emcc` 和 `zig cc` 也是 clang。 +**编译器家族。** 命令的编译器家族在该命令规范化时从可执行文件名读出——`CompilerFamily` 枚举包括 GCC、Clang、MSVC、ClangCL、NVCC、Intel 和 Zig。识别逻辑可处理多种命名变体：版本后缀（`clang++-17`）、架构前缀（`arm-none-eabi-g++`）以及 Windows 的 `.exe` 后缀。名称只用来认出使用另一套命令行语法的驱动程序：`cl` 和 `clang-cl`、nvcc，以及由两个词组成的 zig 驱动程序。一个 GCC 风格的驱动程序底层究竟是 GCC 还是 clang，由它自己对查询的回答判断，从不看名称：`/usr/bin/c++` 在 macOS 上是 clang，`emcc` 和 `zig cc` 也是 clang。 -**缓存策略。** 缓存分为两层。**探测层**缓存每次探测的任务所设置的前端，其中已去掉探测自身的输入和 clice 前端无法接受的部分；它以命令中与探测相关的结构为键（驱动程序、与探测相关的标志、输入类型——`.c` 和 `.cpp` 可能触发不同的驱动规则；如果命令对工作目录敏感，工作目录也会纳入键中）。**合成层**按输入类型，在其上加入每条命令自身的用户内容，供命令渲染使用。失败的探测也会被缓存（负缓存），但暂时性失败会在冷却期后重试，而不会被永久记住。 +**缓存策略。** 缓存分为两层。**查询层**缓存每次查询的任务所设置的前端，其中已去掉查询自身的输入和 clice 前端无法接受的部分；它以命令中与查询相关的结构为键（驱动程序、与查询相关的标志、输入类型——`.c` 和 `.cpp` 可能触发不同的驱动规则；如果命令对工作目录敏感，工作目录也会纳入键中）。**合成层**按输入类型，在其上加入每条命令自身的用户内容，供命令渲染使用。失败的查询也会被缓存（负缓存），但暂时性失败会在冷却期后重试，而不会被永久记住。 ### 搜索路径 @@ -132,16 +132,16 @@ CDB 加载使用 simdjson 进行流式 JSON 解析，逐条处理记录： 3. 将 NVCC 命令转换为等效的 clang CUDA 调用 4. 过滤非 C/C++ 文件（例如 `.rc`、`.asm`、`.def`） 5. 将相对文件路径解析为绝对路径 -6. 将每个选项解析、分类一次并存入结构化配置，同时以 `directory` 为基准将相对包含路径转换为绝对路径 -7. 对相同配置进行去重，使其共享 `ConfigID` +6. 将每个选项解析、分类一次并存入结构化命令，同时以 `directory` 为基准将相对包含路径转换为绝对路径 +7. 将相同的命令去重，使其共享 `CommandID` 在 Cygwin 或 MSYS2 内部写出的数据库（由在它们的 shell 中运行的 CMake 或类似的工具写出）按 POSIX 风格书写路径：`/cygdrive/c/proj` 或 `/c/proj`，编译器则写作 `/usr/bin/c++.exe`。在 Windows 上，写出这些路径的 Cygwin 或 MSYS2 运行时用它的挂载表（mount table）把它们映射为 Windows 路径，包括条目的目录和文件、它的编译器，以及它的路径选项的值。这个运行时是 `PATH` 上第一个满足以下条件的运行时：它的挂载表能把首个此类条目的目录映射到实际存在的文件，该条目的编译器若也按这种风格书写，同样要映射到实际存在的文件。其他运行时的挂载表会把该目录映射到各自的根目录之下；Git for Windows 的 MSYS2 对盘符路径的映射虽与独立安装的 MSYS2 相同，却没有后者的 `/usr/bin/c++.exe`。这种数据库的 `command` 字符串按 sh 的规则拆分。如果 `PATH` 上没有运行时能映射该数据库，clice 不读取它，并提示用户把正确的运行时加入其中。 解析过程还会识别 CMake 直接传给 cc1 的预编译头文件：翻译单元的 `-Xclang -include-pch -Xclang `，以及构建该 PCH 的条目中的 `-Xclang -emit-pch`，都会成为结构化选项，任何编译都不会把它们渲染到命令中。磁盘上的 PCH 可能出自其他编译器，因此 clice 会用写出它的那个条目的命令自行构建一份（在 CMake 中即 `cmake_pch.hxx.cxx` 的条目；GCC 在强制包含的头文件旁写出的 `.gch` 同样能用来找到该条目），读取该 PCH 的每个翻译单元都改为加载 clice 构建的版本：打开文件的 Preamble PCH 以链式 PCH 的方式接在它之后，后台索引也在它之上解析。PCH 所含的头文件单独打开时，编译不使用该 PCH：否则 PCH 中的头文件保护会跳过该头文件自身的正文。与编译借以进入头文件的其他强制包含一样，PCH 所代表的那个强制包含的头文件也会从命令中去掉（见[编译上下文](compilation-context.md#automatic-self-containedness-detection)）。 -### 工具链探测 +### 工具链查询 -不同编译器家族使用不同的探测策略： +不同编译器家族使用不同的查询策略，每种策略各占 `src/command/toolchains/` 下的一个文件： **GCC 风格的驱动程序**（gcc、clang、Apple clang、zig、emcc、交叉编译与裸机 GCC）：只询问驱动程序一次，让它带着命令自身的标志，以输入文件的语言预处理一个空文件，同时输出详细信息并打印宏（`-E -dM -v`）。这一次的回答就包含了 clice 需要的一切： @@ -150,7 +150,7 @@ CDB 加载使用 simdjson 进行流式 JSON 解析，逐条处理记录： 询问时带着命令自身的标志，所以驱动程序回答的就是这条命令，而 GCC 只在这次编译成功时才会给出回答：关闭某项默认设置的标志（`-fno-stack-protector`）、`--sysroot` 或 `--specs` 文件都会体现在回答中。clang 不认识的 CPU 和 ABI 名称（GCC 的 `-mtune=intel`、Darwin 上的 `-mabi=lp64`）会从 cc1 命令中去掉，GCC 的双核写法 `-mcpu=cortex-a72.cortex-a53` 只保留第一个核。通用构建（`-arch arm64 -arch x86_64`）按其中第一个架构读取。 -**Cygwin 和 MSYS2 编译器**：Windows 上的这类编译器（Cygwin 和 MSYS2 自带的 GCC，以及基于 Cygwin 构建的工具链，例如 Andes 的 RISC-V GCC）像任何 GCC 风格的驱动程序一样作答，只是给出的是 POSIX 路径（`/usr/include`、`/cygdrive/c/...`），clice 的 clang 无法打开。这些路径由驱动程序加载的运行时用其挂载表映射，每次探测都从该运行时自己的 `mount` 读取挂载表。在驱动程序旁边和 `PATH` 上的运行时（`cygwin1.dll`、`msys-2.0.dll`）中，按 Windows 查找程序 DLL 的顺序，取第一个能把驱动程序所跑作业的程序映射到实际存在的文件的那个：只取第一个运行时的话，若 Git for Windows 的 MSYS2 在 `PATH` 上排在工具链所加载的、位于其外的 Cygwin 之前，就会选错。假定一套映射会出错：Cygwin 把 `/usr/lib` 挂载在 `/lib`，MSYS2 把驱动器挂载在 `/c` 这样的路径上，用户的 `/etc/fstab` 还会加上自己的挂载。驱动程序本身按 Cygwin 的方式查找：名称可以不带 `.exe`，也可以经由 Cygwin 的符号链接找到（CMake 记录的是 `/usr/bin/cc`，它是指向 `gcc.exe` 的链接）。这样的编译器只有在其运行时的 `bin` 目录位于 `PATH` 中时才能启动；否则查询会失败，错误信息会说明这一点，[编辑器设置](../guide/editors.md#compiler-environment)介绍了如何在各个编辑器中设置它。 +**Cygwin 和 MSYS2 编译器**：Windows 上的这类编译器（Cygwin 和 MSYS2 自带的 GCC，以及基于 Cygwin 构建的工具链，例如 Andes 的 RISC-V GCC）像任何 GCC 风格的驱动程序一样作答，只是给出的是 POSIX 路径（`/usr/include`、`/cygdrive/c/...`），clice 的 clang 无法打开。这些路径由驱动程序加载的运行时用其挂载表映射，每次查询都从该运行时自己的 `mount` 读取挂载表。在驱动程序旁边和 `PATH` 上的运行时（`cygwin1.dll`、`msys-2.0.dll`）中，按 Windows 查找程序 DLL 的顺序，取第一个能把驱动程序所跑作业的程序映射到实际存在的文件的那个：只取第一个运行时的话，若 Git for Windows 的 MSYS2 在 `PATH` 上排在工具链所加载的、位于其外的 Cygwin 之前，就会选错。假定一套映射会出错：Cygwin 把 `/usr/lib` 挂载在 `/lib`，MSYS2 把驱动器挂载在 `/c` 这样的路径上，用户的 `/etc/fstab` 还会加上自己的挂载。驱动程序本身按 Cygwin 的方式查找：名称可以不带 `.exe`，也可以经由 Cygwin 的符号链接找到（CMake 记录的是 `/usr/bin/cc`，它是指向 `gcc.exe` 的链接）。这样的编译器只有在其运行时的 `bin` 目录位于 `PATH` 中时才能启动；否则查询会失败，错误信息会说明这一点，[编辑器设置](../guide/editors.md#compiler-environment)介绍了如何在各个编辑器中设置它。 **MSVC / ClangCL**：先让 cl 或 clang-cl 预处理（`/EP`）一个写着 `_MSVC_LANG` 和 `_MSC_FULL_VER` 的文件，只问一次。由此得知它编译所用的 C++ 标准，因为 `/std:c++latest` 的含义随版本而变；还得知 cl 的版本，因为 clang 自己的探测会取已安装的最新工具集。随后由处于 MSVC 兼容模式（`--driver-mode=cl`）的 Clang 驱动构建 cc1 命令，使用这个标准，并把 cl 的版本作为 `-fms-compatibility-version`。clang-cl 的版本则是同一套探测从它的参数所选的工具集（`/winsysroot`）中得出的版本。cl 只会收到命令里的 `/std:`：其余参数是按 clang 的 cl 模式写的，cl 会拒绝；命令用 clang 写法设定的标准或版本（例如规则追加的 `-std=c++23`）仍以 clang 的读法为准。 @@ -160,13 +160,13 @@ CDB 加载使用 simdjson 进行流式 JSON 解析，逐条处理记录： > 外部驱动的版本可能比 clice 内嵌的 LLVM 版本更新，其 cc1 输出中可能包含 clice 无法识别的选项。解析时会静默丢弃未知选项，以保证兼容性。 -探测完成后，会从结果中移除临时探测文件的路径和与模块输出相关的标志（它们引用了已删除的临时文件）。如果 clice 的资源目录与探测返回的不一致，会替换所有相关路径，确保前端使用匹配版本的内置头文件；搜索路径指向的其他 clang 内置头文件目录（另一个 clang 的 `-isystem /lib/clang//include`、zig 的 `lib/include`）也同样替换。 +查询完成后，会从结果中移除临时查询文件的路径和与模块输出相关的标志（它们引用了已删除的临时文件）。如果 clice 的资源目录与查询返回的不一致，会替换所有相关路径，确保前端使用匹配版本的内置头文件；搜索路径指向的其他 clang 内置头文件目录（另一个 clang 的 `-isystem /lib/clang//include`、zig 的 `lib/include`）也同样替换。 -**启动预热。** 在服务器启动的依赖扫描阶段，会收集所有唯一的工具链缓存键并并行探测。探测子进程的管道排空和进程等待也会并发执行，以避免管道填满导致死锁。预热完成后，后续所有工具链查询都会命中缓存。 +**启动预热。** 在服务器启动的依赖扫描阶段，会收集所有唯一的工具链缓存键并并行查询。查询子进程的管道排空和进程等待也会并发执行，以避免管道填满导致死锁。预热完成后，后续所有工具链查询都会命中缓存。 ### 搜索路径提取 -工具链探测得到 cc1 参数后，搜索路径提取会遍历这些参数，按类型将 include 路径选项分入四个层级。所有路径都会解析为绝对路径并规范化（消除 `.` 和 `..`）。 +工具链查询得到 cc1 参数后，搜索路径提取会遍历这些参数，按类型将 include 路径选项分入四个层级。所有路径都会解析为绝对路径并规范化（消除 `.` 和 `..`）。 `-iprefix` / `-iwithprefix` / `-iwithprefixbefore` 三个选项必须按出现顺序配合处理：`-iprefix` 设置前缀，后续的 `-iwithprefix` 将前缀添加到其路径之前，并将结果放入 After 层级；`-iwithprefixbefore` 则将结果放入 Angled 层级。 @@ -214,7 +214,7 @@ CDB 加载使用 simdjson 进行流式 JSON 解析，逐条处理记录： - **不跟随头文件中的 Cygwin 符号链接。** Cygwin 以符号链接形式安装的头文件（例如 ncurses 的 `ncurses.h`，位于 Cygwin 的 `/usr/include` 中）只有 Cygwin 程序才会跟随：clice 的 clang 读到的是链接本身。clice 只对驱动程序跟随这种链接。 -- **部分编译器家族支持不完整。** 对探测既不给出 clang 任务、也不给出 GCC 任务的驱动程序——Intel 的经典编译器 `icc`、其他厂商的编译器——会使用 clang 针对当前机器的默认设置，工具链错误会说明这一点。NVCC 已有专门的探测逻辑（解析 `nvcc --dryrun` 输出并将其转换为 clang CUDA 调用），但仅支持以 GCC 或 Clang 作为宿主编译器——不支持由 nvcc 驱动 MSVC `cl`，多架构命令也只解析最新的架构。 +- **部分编译器家族支持不完整。** 对查询既不给出 clang 任务、也不给出 GCC 任务的驱动程序——Intel 的经典编译器 `icc`、其他厂商的编译器——会使用 clang 针对当前机器的默认设置，工具链错误会说明这一点。NVCC 已有专门的查询逻辑（解析 `nvcc --dryrun` 输出并将其转换为 clang CUDA 调用），但仅支持以 GCC 或 Clang 作为宿主编译器——不支持由 nvcc 驱动 MSVC `cl`，多架构命令也只解析最新的架构。 - **只有 GCC 接受的语言特性按 clang 的理解解析。** 只有 GCC 能编译的代码——GCC 专有的内建函数和扩展、在 clang 不支持它的目标上使用 `__float128`、嵌套函数——会显示 clang 的报错；只有 GCC 认识的 CPU 名称会丢失其特性对应的宏。只存在于 GCC 私有 include 目录中的头文件（`omp.h`、`quadmath.h`）找不到：那个目录里同时还有 GCC 版本的 clang 内置头文件。 @@ -224,4 +224,4 @@ CDB 加载使用 simdjson 进行流式 JSON 解析，逐条处理记录： - **MSVC 风格选项解析。** 在非 Windows 系统上，必须特别处理 MSVC 风格选项前缀（`/U`、`/D`、`/I`），避免 Unix 绝对路径（如 `/Users/...`）被误解析为 MSVC 选项。目前通过根据驱动名称动态调整选项可见性来解决，但边界情况仍可能存在。 -- **只识别常见的编译器启动器。** ccache、sccache、distcc 和 icecc 会连同它们自己的选项和环境变量赋值一起从命令中去掉，clice 询问的是它们后面的编译器。其他启动器会被当成编译器，导致家族识别和工具链探测以错误的二进制文件为目标；请从编译数据库的命令中去掉它。 +- **只识别常见的编译器启动器。** ccache、sccache、distcc 和 icecc 会连同它们自己的选项和环境变量赋值一起从命令中去掉，clice 询问的是它们后面的编译器。其他启动器会被当成编译器，导致家族识别和工具链查询以错误的二进制文件为目标；请从编译数据库的命令中去掉它。 diff --git a/zh/clice/design/compilation-context.md b/zh/clice/design/compilation-context.md index 04788e0df..bb4a8aa7e 100644 --- a/zh/clice/design/compilation-context.md +++ b/zh/clice/design/compilation-context.md @@ -98,7 +98,7 @@ const char* error_message(int code) { 除了自动解析外，clice 还通过四个 LSP 扩展请求让用户可以显式地查询和切换编译上下文： -**`clice/queryContext`** 列出文件的所有可用编译上下文。对于头文件，它返回可作为宿主的源文件，并按相关性排序；对于源文件，它返回所有 CDB 条目，以编译命令中的关键标志（`-D`、`-std`、`-O` 等）作为易读描述，每个条目还携带用于标识它的命令哈希。结果支持分页。对于自包含头文件和源文件，上下文列表会按条目的**身份哈希（identity hash）**去重；该哈希根据最终生效命令中与前端相关的部分，以及输入的槽位位置和工作目录计算得出。身份哈希相同的两个上下文会产生相同的编译结果，因此只保留排名最高的代表项——否则，像 `` 这样被广泛包含的头文件可能会列出数千个上下文。非自包含头文件不会以这种方式合并：不同宿主会合成不同的前缀，因此每个宿主都是独立的上下文。如果宿主的编译多次进入同一个头文件（未设包含防护的头文件），则每一处进入位置都会作为独立条目返回，并以出现序号（occurrence number）区分。 +**`clice/queryContext`** 列出文件的所有可用编译上下文。对于头文件，它返回可作为宿主的源文件，并按相关性排序；对于源文件，它返回所有 CDB 条目，以编译命令中的关键标志（`-D`、`-std`、`-O` 等）作为易读描述，每个条目还携带用于标识它的命令哈希。结果支持分页。对于自包含头文件和源文件，上下文列表会按条目的**命令哈希**去重；该哈希根据最终生效命令中与前端相关的部分，以及输入的槽位位置和工作目录计算得出。命令哈希相同的两个上下文会产生相同的编译结果，因此只保留排名最高的代表项——否则，像 `` 这样被广泛包含的头文件可能会列出数千个上下文。非自包含头文件不会以这种方式合并：不同宿主会合成不同的前缀，因此每个宿主都是独立的上下文。如果宿主的编译多次进入同一个头文件（未设包含防护的头文件），则每一处进入位置都会作为独立条目返回，并以出现序号（occurrence number）区分。 **`clice/currentContext`** 返回文件当前编译所用的上下文（名称与列表中一致），并说明它是否由用户选择：有用户选择时返回该选择（宿主 + 出现序号，或命令哈希），否则返回自动选出的上下文。只有既不借用上下文、自身也没有条目的文件才不返回上下文。 diff --git a/zh/clice/design/overview.md b/zh/clice/design/overview.md index 1a2755d02..8594b3828 100644 --- a/zh/clice/design/overview.md +++ b/zh/clice/design/overview.md @@ -37,8 +37,9 @@ clice 是一个全新的 C++ 语言服务器，从架构层面重新设计，旨 从编译数据库（CDB）读取的命令是构建系统生成的原始命令，不能直接交给 Clang 前端——它们可能包含仅用于代码生成的选项、缺少系统头文件搜索路径，或者包含与语言服务器无关的参数。本模块只需解析每条命令一次，即可将其转换为供系统其余部分使用的结构化形式。 -- `CompilationDatabase`：加载 `compile_commands.json`，将每个条目解析一次，转换为由稳定 `ConfigID` 标识的、经过驻留和分类的参数结构。在此基础上，它会应用用户配置规则，为没有条目的文件合成备用命令，推导输入语言，计算条目标识哈希，并按需生成 driver 或 `-cc1` 命令行。 -- `Toolchain`：查询系统编译器以获取隐式编译参数（例如系统头文件搜索路径），并采用两层缓存——按 driver 调用形式缓存探测结果，按输入类型缓存合成的选项集。 +- `CommandTable`：clice 知道的每一条命令，各自只解析一次，形成由稳定 `CommandID` 标识的、经过驻留和分类的参数结构，并附带由一条命令推导出的内容：输入语言、写出和读取的预编译头文件、命令哈希，以及 driver 层面的渲染结果。 +- `CompilationDatabase`：加载和重新加载 `compile_commands.json` 文件，并比对其新旧差异；文件中的条目经规范化（`Normalizer`）后存入 `CommandTable`；规则所做的编辑应用在条目的命令之上（`apply_edits`）。 +- `Toolchain`：查询系统编译器以获取隐式编译参数（例如系统头文件搜索路径），并采用两层缓存——按 driver 调用形式缓存查询结果，按输入类型缓存合成的选项集——还负责渲染 `-cc1` 命令行；每种编译器的询问方式（GCC 风格、cl、nvcc、clang 自身的 driver）位于 `src/command/toolchains/` 下。 - `SearchConfig`：头文件搜索路径的四层模型（Quoted / Angled / System / After），与 Clang 内部的搜索逻辑一致。 详见 [编译命令解析](command-resolve.md)。 diff --git a/zh/clice/design/symbol-index.md b/zh/clice/design/symbol-index.md index a153274d6..030d87d7b 100644 --- a/zh/clice/design/symbol-index.md +++ b/zh/clice/design/symbol-index.md @@ -160,7 +160,7 @@ Token 是模糊匹配在名称中可能走过的一条路径的 trigram，路径 ### 过期检测 -过期检测用于判断文件是否需要重新索引。每个索引产物都会记录其输入的标识以及观测到的内容版本；校验由主进程的共享文件表完成，它采用与其他各处相同的两层检查：先以文件最近一次的观测记录为基准走 (size, mtime) stat 快速路径，再通过内容哈希确认（见[增量编译](incremental-parse.md)）。仅当输入内容确实发生变化时，才会触发重新索引；命令变更则通过清单中记录的条目标识哈希单独检测。启动时，写入方把每条文件记录与磁盘核对，并把依赖以下情况的翻译单元排入队列：某个文件内容变了、某次失败查找看过的位置如今有了文件、模块或预编译头文件的某个输入变了，另外还有变体被共享文件的重建丢掉的翻译单元；在它们重新索引之前，已存的分片照常提供服务。 +过期检测用于判断文件是否需要重新索引。每个索引产物都会记录其输入的标识以及观测到的内容版本；校验由主进程的共享文件表完成，它采用与其他各处相同的两层检查：先以文件最近一次的观测记录为基准走 (size, mtime) stat 快速路径，再通过内容哈希确认（见[增量编译](incremental-parse.md)）。仅当输入内容确实发生变化时，才会触发重新索引；命令变更则通过清单中记录的命令哈希单独检测。启动时，写入方把每条文件记录与磁盘核对，并把依赖以下情况的翻译单元排入队列：某个文件内容变了、某次失败查找看过的位置如今有了文件、模块或预编译头文件的某个输入变了，另外还有变体被共享文件的重建丢掉的翻译单元；在它们重新索引之前，已存的分片照常提供服务。 ### 后台索引调度