From 66289bbfb34cc1c68ea438fd572fbe3cb675acac Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 21:07:47 +0800 Subject: [PATCH 1/2] fix(benchmarks)!: preserve TCP kernel sample onset Keep the first four qualifying snapshots and the latest four in each existing bounded history, preserving sampling and transport behavior. BREAKING CHANGE: opt-in tcp_kernel_sample_history records and the nested tcp_kernel_samples summary now declare schema 2 with first_4_latest_4 retention, sample ordinals and explicit policy metadata. Schema 1 raw histories remain tail-only evidence; consumers must interpret the recorded schema rather than relabel historical output. --- benchmarks/CONNECTIONS.md | 34 +++++- benchmarks/CONNECTIONS_CN.md | 24 +++- benchmarks/asio_connections.cpp | 40 ++++++- benchmarks/connections_kernel_samples.hpp | 53 ++++++--- docs/project/production-readiness.md | 25 ++++- docs/project/production-readiness_CN.md | 16 ++- docs/project/testing.md | 31 ++++++ docs/project/testing_CN.md | 26 +++++ tests/connection_kernel_samples.cpp | 129 ++++++++++++++++++++-- tests/connection_load_native_control.cpp | 9 +- 10 files changed, 351 insertions(+), 36 deletions(-) diff --git a/benchmarks/CONNECTIONS.md b/benchmarks/CONNECTIONS.md index 57808af..36eaeb8 100644 --- a/benchmarks/CONNECTIONS.md +++ b/benchmarks/CONNECTIONS.md @@ -310,14 +310,39 @@ deadlines or the existing one-second sampling cadence. The existing worker marker samples active, unfinished traffic sessions that have received DATA and have had no composed read or write completion for at least one second. This is an inactivity trigger, not evidence of a network failure. The -single IO worker queries at most 16 identities per tick, keeping eight samples -per identity in fixed storage below 32 KiB per process, in addition to the existing +single IO worker queries at most 16 identities per tick, keeping the first four +qualifying samples and latest four samples per retained identity in fixed storage below 32 KiB per process, in addition to the existing 16 MiB operation-trace budget (allocator overhead excluded). At each tick, observed progress, completion or close releases the retained slot before admitting new identities; persistent stalls retain their slots. Finite capacity can omit a connection. Overwritten samples, released histories, slot reuse and omitted eligible observations are counted. No new timer, thread or worker post is added. +Histories with up to eight observations keep every sample. On overflow, the +first four remain fixed and only the four-sample tail rotates; output stays in +observation order. `sample_ordinal` is one-based within that retained history, +`total_samples` counts its queries, and `overwritten_samples` counts the omitted +middle observations (`total_samples - 8` after overflow). Progress pruning or +slot reuse starts a new history and ordinal sequence; summary overwrite/query +counters remain cumulative, including histories later released. + +`tcp_kernel_sample_history` now uses `schema: 2`, with +`retention_policy: "first_4_latest_4"` and `head_sample_capacity: 4` / +`tail_sample_capacity: 4`. The nested `tcp_kernel_samples` summary independently +declares schema 2 and the same retention metadata; its enclosing +`operation_trace_summary` stays schema 1. Older schema 1 kernel histories used a +tail-only ring and remain that historical format; never relabel or interpret +those raw files as onset-preserving evidence. + +Kernel histories alone also carry `clock_domain: "local_steady_clock"`, +`clock_origin_us` and `clock_origin_unit: "microseconds_since_steady_clock_epoch"`. +The origin is the existing application `began` value converted to microseconds; +serialization performs no new clock or socket query. Add `sample_time_us` to +that origin to compare peer windows only when provenance establishes the same +host and steady-clock domain. These quantized values are not wall-clock time, +a cross-host clock mapping, or proof that peer observations were simultaneous. +Older records without an origin do not gain one retroactively. + On Apple platforms, samples query public `TCP_CONNECTION_INFO`: TCP state, flags, RTO in milliseconds, queued send bytes (including in-flight data), send/receive windows, congestion window, slow-start threshold and byte/retransmission counters. @@ -333,7 +358,10 @@ inside the existing bounded trace output. `operation_trace_summary` contains The final result adds `tcp_kernel_samples_enabled: true` only when enabled. Successful runs emit no sample history. `sample_time_us` is the periodic query start time in process-relative microseconds; it is distinct from operation-ring -freeze time. Sampling can miss transient states or be delayed by the IO worker. +freeze time. Sampling can miss transient states or be delayed by the IO worker. The retained +head begins with the earliest admitted qualifying query, after the one-second +inactivity threshold; it is not the onset of TCP recovery. Finite identity +admission and omitted middle samples can still leave gaps between peer windows. A retained state after reset cannot describe earlier recovery; use actual sampled state and time. Samples neither prove a root cause nor provide performance or capacity acceptance. Keep the raw diagnostic output local. diff --git a/benchmarks/CONNECTIONS_CN.md b/benchmarks/CONNECTIONS_CN.md index ffb6c67..72145a4 100644 --- a/benchmarks/CONNECTIONS_CN.md +++ b/benchmarks/CONNECTIONS_CN.md @@ -270,11 +270,29 @@ macOS 的 `tcp_info` 还包含 TCP 状态、窗口、发送缓冲字节数、收 现有工作线程标记检查活动且未完成的 traffic 会话：会话已经收到 DATA，且至少一秒没有 组合读写完成。此条件只表示应用操作无进展，不能证明网络故障。单个 IO 工作线程每次最多 -查询 16 个身份，每个身份保留 8 个样本；固定存储小于每进程 32 KiB，独立于原有 16 MiB +查询 16 个身份，每个身份保留最早 4 个合格样本与最新 4 个样本；固定存储小于每进程 32 KiB，独立于原有 16 MiB 操作历史预算，不含分配器额外开销。每次先释放已观察到进展、完成或关闭的会话槽位， 再接纳新身份；持续停滞的身份保留槽位。容量有限，不能保证每个连接都被采样。覆盖样本、 释放历史、槽位复用和未保留的合格观测次数均有计数，不增加计时器、线程或工作线程投递。 +历史不超过 8 次观测时保留全部样本；超出后固定保留最早 4 次，只轮换尾部 4 次， +输出仍按观测顺序排列。`sample_ordinal` 从该历史的 1 开始，`total_samples` 统计其查询次数， +`overwritten_samples` 统计省略的中段样本（超过容量后为 `total_samples - 8`）。 +进展清理或槽位复用会开始新的历史和序号；摘要的覆盖与查询计数仍累计包含已释放的历史。 + +`tcp_kernel_sample_history` 使用 `schema: 2`，并输出 +`retention_policy: "first_4_latest_4"`、`head_sample_capacity: 4`、`tail_sample_capacity: 4`。 +嵌套的 `tcp_kernel_samples` 摘要单独声明 schema 2 及相同保留元数据， +外层 `operation_trace_summary` 仍为 schema 1。旧 schema 1 内核历史使用仅保留尾部的环， +原始文件须保留原格式，不能重新标记或当作保留起始样本的证据。 + +仅内核历史还包含 `clock_domain: "local_steady_clock"`、`clock_origin_us`，以及 +`clock_origin_unit: "microseconds_since_steady_clock_epoch"`。起点由现有应用 `began` +转换为微秒，输出时不新增时钟或 socket 查询。只有运行凭据确认两端位于同一主机、 +使用同一 steady-clock 域时，才能用起点加 `sample_time_us` 比较两端窗口。 +这些量化值不是墙上时间，不建立跨主机时钟映射，也不能证明两端观测同时发生； +没有起点的旧记录不能补上这一字段后重新解释。 + Apple 平台通过公开 `TCP_CONNECTION_INFO` 查询 TCP 状态、flags、毫秒单位 RTO、发送队列 字节数（含在途数据）、收发窗口、拥塞窗口、慢启动阈值及字节／重传计数。其他平台输出 `tcp_info: null` 和 `unsupported_platform`；socket 已关闭、查询失败或返回结构过短时， @@ -284,7 +302,9 @@ Apple 平台通过公开 `TCP_CONNECTION_INFO` 查询 TCP 状态、flags、毫 槽位及操作记录是否成功输出。`operation_trace_summary` 包含 `tcp_kernel_samples` 预算／计数、 `tcp_kernel_histories_emitted` 和 `tcp_kernel_histories_omitted`，输出截断仍有明确标记。仅启用时，最终结果增加 `tcp_kernel_samples_enabled: true`。 成功运行不输出采样历史。`sample_time_us` 是周期查询开始时的进程相对微秒数，区别于操作 -历史冻结时间。采样可能错过瞬时状态，也可能因 IO 工作线程阻塞而延迟。reset 后的状态 +历史冻结时间。采样可能错过瞬时状态，也可能因 IO 工作线程阻塞而延迟。 +起始部分从超过一秒无进展阈值后、最早获准查询的合格观测开始，不是 TCP 进入恢复状态的时刻； +身份准入容量与被省略的中段样本仍可能造成两端窗口之间的缺口。reset 后的状态 不能描述此前的恢复过程，须按实际样本时间和状态解读。采样不能证明根因、性能或容量， 原始诊断输出仅留本地。 diff --git a/benchmarks/asio_connections.cpp b/benchmarks/asio_connections.cpp index b6285ba..8cedaf6 100644 --- a/benchmarks/asio_connections.cpp +++ b/benchmarks/asio_connections.cpp @@ -930,10 +930,11 @@ void application::trace_failure() { { ++kernel_histories_available; kernel_histories_emitted += output.append([&](std::ostream& out) { - out << "{\"event\":\"tcp_kernel_sample_history\",\"schema\":1,\"role\":" << std::quoted(o.role) + out << "{\"event\":\"tcp_kernel_sample_history\",\"schema\":" + << connection_load::tcp_kernel_samples::schema_version << ",\"role\":" << std::quoted(o.role) << ",\"run_id\":" << o.run << ",\"client_id\":" << value->state.owner.client << ",\"connection_id\":" << value->state.owner.connection << ','; - kernel_samples.write_history(out, value->state.owner); + kernel_samples.write_history(out, value->state.owner, began); out << '}'; }); } @@ -1525,6 +1526,41 @@ void self_test() { require(kernel_enabled.deadline == enabled.deadline && kernel_enabled.request_ms == enabled.request_ms && kernel_enabled.connections == enabled.connections && kernel_enabled.window == enabled.window && kernel_enabled.payload == enabled.payload); + // Synthetic queries only: exercise retention/JSON without sockets or load. + connection_load::tcp_kernel_samples retained(true); + unsigned kernel_queries{}; + for (unsigned index = 1; index <= 10; ++index) + { + retained.sample({7, 1, 1}, 1000000 + index, 1000000, 4, 2, 3, [&] { + ++kernel_queries; + return connection_load::tcp_kernel_info{}; + }); + } + std::ostringstream kernel_history, kernel_summary; + kernel_history << "{\"schema\":" << connection_load::tcp_kernel_samples::schema_version << ','; + retained.write_history(kernel_history, {7, 1, 1}, clock_type::time_point{std::chrono::microseconds{1234567}}); + kernel_history << '}'; + retained.write_summary(kernel_summary); + require(kernel_queries == 10 && + kernel_history.str().starts_with("{\"schema\":2,\"clock_domain\":\"local_steady_clock\",")); + require(kernel_history.str().find( + "\"clock_origin_us\":1234567,\"clock_origin_unit\":\"microseconds_since_steady_clock_epoch\"") != + std::string::npos); + require(kernel_history.str().find("\"head_sample_capacity\":4,\"tail_sample_capacity\":4") != std::string::npos); + require(kernel_history.str().find("\"total_samples\":10,\"overwritten_samples\":2}") != std::string::npos); + std::size_t kernel_position{}; + for (const auto ordinal : {1, 2, 3, 4, 7, 8, 9, 10}) + { + kernel_position = kernel_history.str().find("\"sample_ordinal\":" + std::to_string(ordinal) + + ",\"sample_time_us\":" + std::to_string(1000000 + ordinal), + kernel_position); + require(kernel_position != std::string::npos); + ++kernel_position; + } + require(kernel_history.str().find("\"sample_ordinal\":5,") == std::string::npos && + kernel_history.str().find("\"sample_ordinal\":6,") == std::string::npos); + require(kernel_summary.str().starts_with("{\"schema\":2,\"retention_policy\":\"first_4_latest_4\",")); + std::cout << "native control kernel retention schema 2 checks passed\n"; bool kernel_rejected{}; try { parse("off", false, true); } diff --git a/benchmarks/connections_kernel_samples.hpp b/benchmarks/connections_kernel_samples.hpp index d155bc7..de7d834 100644 --- a/benchmarks/connections_kernel_samples.hpp +++ b/benchmarks/connections_kernel_samples.hpp @@ -3,6 +3,7 @@ #include #include #include +#include #include #include #include @@ -80,6 +81,8 @@ class tcp_kernel_samples { public: static constexpr std::size_t session_capacity = 16, sample_capacity = 8, memory_limit = 32 * 1024; + static constexpr std::size_t head_sample_capacity = 4, tail_sample_capacity = 4; + static constexpr unsigned schema_version = 2; static constexpr std::int64_t inactivity_us = 1000000; explicit tcp_kernel_samples(bool enabled) @@ -140,7 +143,7 @@ class tcp_kernel_samples return; // Persistent stalls keep their slots; never query omitted identities. } value->completions = completions; - auto& event = value->events[value->total % sample_capacity]; + auto& event = value->events[event_index(value->total)]; event = {time_us, idle_us, next_receive, next_send, query()}; overwritten_ += value->total >= sample_capacity; ++value->total; @@ -159,24 +162,32 @@ class tcp_kernel_samples return data_ && find(owner); } - void write_history(std::ostream& out, identity owner) const { + void write_history(std::ostream& out, identity owner, std::chrono::steady_clock::time_point began) const { const auto* value = data_ ? find(owner) : nullptr; - out << "\"sample_phase\":\"periodic_io_worker\",\"time_unit\":\"microseconds_since_process_start\",\"samples\":" - "["; + out << "\"clock_domain\":\"local_steady_clock\",\"clock_origin_us\":" + << std::chrono::duration_cast(began.time_since_epoch()).count() + << ",\"clock_origin_unit\":\"microseconds_since_steady_clock_epoch\"," + "\"retention_policy\":\"first_4_latest_4\",\"head_sample_capacity\":" + << head_sample_capacity << ",\"tail_sample_capacity\":" << tail_sample_capacity + << ",\"sample_phase\":\"periodic_io_worker\",\"time_unit\":\"microseconds_since_process_start\"," + "\"samples\":["; if (value) { const auto size = (std::min)(value->total, std::uint64_t{sample_capacity}); + const auto head_size = (std::min)(size, std::uint64_t{head_sample_capacity}); + const auto tail_size = size - head_size; for (std::uint64_t index = 0; index < size; ++index) { if (index) { out << ','; } - const auto& event = value->events[(value->total - size + index) % sample_capacity]; + const auto ordinal = index < head_size ? index : value->total - tail_size + index - head_size; + const auto& event = value->events[event_index(ordinal)]; const auto& info = event.info; - out << "{\"sample_time_us\":" << event.time_us << ",\"no_completion_age_us\":" << event.idle_us - << ",\"next_receive\":" << event.next_receive << ",\"next_send\":" << event.next_send - << ",\"tcp_info\":"; + out << "{\"sample_ordinal\":" << ordinal + 1 << ",\"sample_time_us\":" << event.time_us + << ",\"no_completion_age_us\":" << event.idle_us << ",\"next_receive\":" << event.next_receive + << ",\"next_send\":" << event.next_send << ",\"tcp_info\":"; if (info.result == tcp_kernel_info::status::available) { out << "{\"state\":" << info.state << ",\"flags\":" << info.flags << ",\"rto_ms\":" << info.rto_ms @@ -204,11 +215,11 @@ class tcp_kernel_samples } out << '}'; } - out << "],\"overwritten_samples\":" << value->total - size; + out << "],\"total_samples\":" << value->total << ",\"overwritten_samples\":" << value->total - size; } else { - out << "],\"overwritten_samples\":0"; + out << "],\"total_samples\":0,\"overwritten_samples\":0"; } } @@ -221,15 +232,25 @@ class tcp_kernel_samples retained += value.used; } } - out << "{\"storage_bytes\":" << storage_bytes() << ",\"memory_limit_bytes\":" << memory_limit - << ",\"session_capacity\":" << session_capacity << ",\"sample_capacity\":" << sample_capacity - << ",\"retained_identities\":" << retained << ",\"released_histories\":" << released_ - << ",\"slot_reuses\":" << reused_ << ",\"omitted_eligible_observations\":" << omitted_ - << ",\"overwritten_samples\":" << overwritten_ << ",\"sampler_calls\":" << queries_ - << ",\"frozen\":" << (frozen_ ? "true" : "false") << '}'; + out << "{\"schema\":" << schema_version + << ",\"retention_policy\":\"first_4_latest_4\",\"head_sample_capacity\":" << head_sample_capacity + << ",\"tail_sample_capacity\":" << tail_sample_capacity << ",\"storage_bytes\":" << storage_bytes() + << ",\"memory_limit_bytes\":" << memory_limit << ",\"session_capacity\":" << session_capacity + << ",\"sample_capacity\":" << sample_capacity << ",\"retained_identities\":" << retained + << ",\"released_histories\":" << released_ << ",\"slot_reuses\":" << reused_ + << ",\"omitted_eligible_observations\":" << omitted_ << ",\"overwritten_samples\":" << overwritten_ + << ",\"sampler_calls\":" << queries_ << ",\"frozen\":" << (frozen_ ? "true" : "false") << '}'; } private: + static_assert(head_sample_capacity + tail_sample_capacity == sample_capacity); + + static std::size_t event_index(std::uint64_t ordinal) noexcept { + return ordinal < head_sample_capacity + ? ordinal + : head_sample_capacity + (ordinal - head_sample_capacity) % tail_sample_capacity; + } + struct event { std::int64_t time_us{}, idle_us{}; diff --git a/docs/project/production-readiness.md b/docs/project/production-readiness.md index e8fb925..b185675 100644 --- a/docs/project/production-readiness.md +++ b/docs/project/production-readiness.md @@ -103,8 +103,8 @@ The native control also offers default-off `--tcp-kernel-samples on` with `--operation-trace on` to retain periodic public Apple `TCP_CONNECTION_INFO` queries before failure cleanup. The existing one-second worker marker checks unfinished traffic sessions with DATA received and at least one second without a -composed read/write completion. It keeps eight samples for at most 16 identities -in less than 32 KiB of extra fixed storage; recovered/completed/closed sessions +composed read/write completion. It keeps the first four qualifying observations +and latest four for at most 16 identities in less than 32 KiB of extra fixed storage; recovered/completed/closed sessions release slots, while persistent stalls keep theirs. Omission and overwrite are explicit. Other platforms return null with an unsupported reason. Sample time is separate from the operation-ring freeze; delayed sampling can miss transient @@ -112,6 +112,22 @@ states. Selected Boost and Standalone Asio regressions include a controlled post-DATA stall that retains an ESTABLISHED sample before timeout cleanup. See [usage and limits](../../benchmarks/CONNECTIONS.md#opt-in-tcp-kernel-samples). +Kernel histories use schema 2 with `retention_policy: "first_4_latest_4"`; +the nested sampler summary also declares schema 2. Up to eight observations are +all retained; overflow omits the middle and preserves chronological head/tail +output with explicit sample ordinals and overwrite counts. The retained head is +the earliest admitted query after the one-second inactivity threshold, not the +onset of TCP recovery. Admission limits, omitted middle observations, worker +delay and non-simultaneous peer windows remain causal limits; retention alone +neither establishes a fault nor changes the load, polling or transport. +Kernel-history-only `clock_origin_us` converts the existing application `began` +to microseconds since its steady-clock epoch, with explicit +`clock_domain: "local_steady_clock"` and `clock_origin_unit` metadata. Origin plus +`sample_time_us` supports correlation only for the same host and clock domain; +it is not wall-clock time or a cross-host mapping, and adds no clock query. + + + ### Frame-boundary diagnostics The native control's pre-cancel `connection_diagnostic` includes a bounded @@ -145,7 +161,10 @@ A fixed five-run diagnostic series used the same host and workload above with sampling enabled: Boost 1.90 Release, TLS off, native source `605625dfea0efff90468771cc0839d03866a134c` (series revision `9200bfb` only adds documentation). All 5,000 connections reached ready/started in each run. These -consecutive outcomes are not an estimated failure probability. +consecutive outcomes are not an estimated failure probability. These historical +raw kernel records are schema 1 with tail-only retention. Their overwritten +onset observations cannot be recovered by relabeling them as schema 2; the +head/tail policy above has not been exercised by this five-run series. | Run | Finished connections per peer | Unfinished connections per peer | | --- | ---: | ---: | diff --git a/docs/project/production-readiness_CN.md b/docs/project/production-readiness_CN.md index 6c83d6f..91e2f5f 100644 --- a/docs/project/production-readiness_CN.md +++ b/docs/project/production-readiness_CN.md @@ -88,12 +88,24 @@ main 上的 [PR #26](https://github.com/OpenArkStudio/arknet/pull/26) 改善了 原生对照还提供默认关闭的 `--tcp-kernel-samples on`，要求同时启用 `--operation-trace on`， 用于在失败清理前保留公开 Apple `TCP_CONNECTION_INFO` 周期查询。现有一秒工作线程标记 检查已收到 DATA、尚未完成且至少一秒没有组合读写完成的 traffic 会话。额外固定存储 -小于 32 KiB，最多保留 16 个身份、每个身份 8 个样本；已有进展、完成或关闭的会话释放 +小于 32 KiB，最多保留 16 个身份、每个身份最早 4 个合格观测与最新 4 个样本；已有进展、完成或关闭的会话释放 槽位，持续停滞的身份保留槽位。遗漏和覆盖有明确计数，其他平台返回 null 并说明不支持。 采样时间区别于操作历史冻结时间，延迟采样可能错过瞬时状态。Boost 与 Standalone Asio 相关回归均通过，包括收到 DATA 后主动停止响应的受控测试：超时清理前保留了 ESTABLISHED 状态样本。参见[用法与限制](../../benchmarks/CONNECTIONS_CN.md#可选-tcp-内核采样)。 +内核历史使用 schema 2 与 `retention_policy: "first_4_latest_4"`，嵌套采样摘要也单独声明 +schema 2。不超过 8 次观测时全部保留；超出后省略中段，起始与尾部仍按时间顺序输出， +并提供样本序号和覆盖计数。起始部分是超过一秒无进展阈值后最早获准执行的查询， +不是 TCP 进入恢复状态的时刻。准入容量、中段省略、工作线程延迟及两端窗口不同步仍限制 +因果判断；仅改善保留方式不能确定故障，也不改变负载、轮询或传输。 +仅内核历史提供 `clock_origin_us`：将现有应用 `began` 转换为 steady-clock epoch 起的微秒， +并明确输出 `clock_domain: "local_steady_clock"` 与 `clock_origin_unit`。 +起点加 `sample_time_us` 仅用于同主机、同一时钟域的关联，不是墙上时间或跨主机映射， +也不新增时钟查询。 + + + ### 帧边界诊断 原生对照的取消前 `connection_diagnostic` 包含有界 `frame_boundary` 对象。 @@ -118,6 +130,8 @@ matcher 和原有 4,096 字节帧上限保持不变。规范的 9 字节前缀 固定五次诊断使用上表同一主机和负载，启用采样；Boost 1.90 Release，关闭 TLS。 原生源码为 `605625dfea0efff90468771cc0839d03866a134c`，系列运行版本 `9200bfb` 仅增加文档。每次均有 5,000 条连接就绪并开始；连续五次结果不能用于估算故障概率。 +这些历史原始内核记录为 schema 1，仅保留尾部。将其重新标记为 schema 2 无法恢复已覆盖的 +起始观测；这组五次运行并未使用上述起始／尾部保留方式。 | 运行 | 每端完成连接数 | 每端未完成连接数 | | --- | ---: | ---: | diff --git a/docs/project/testing.md b/docs/project/testing.md index 531283e..fb4c038 100644 --- a/docs/project/testing.md +++ b/docs/project/testing.md @@ -326,6 +326,37 @@ then across the requested restart rounds. Seven protocols with `--seconds 60` therefore request seven minutes of traffic in total, plus setup and teardown. Slow receivers test backpressure rather than throughput. +### Native TCP diagnostic retention + +The default-off kernel sampler keeps the first four qualifying observations and +latest four within its existing eight-sample, 16-identity, 32 KiB limit. This +preserves early qualifying evidence when a longer server tail would otherwise +replace it. Schema 2 kernel histories and their nested sampler summaries declare +`retention_policy: "first_4_latest_4"`; historical schema 1 tail-only records keep +their original interpretation. See [production limits](production-readiness.md) +and [diagnostic fields](../../benchmarks/CONNECTIONS.md#opt-in-tcp-kernel-samples). + +Select the existing native runner without connection load: + +```sh +cmake -S . -B build/diagnostic-retention -DARKNET_BUILD_TESTS=ON \ + -DARKNET_BUILD_EXAMPLES=OFF -DARKNET_BUILD_BENCHMARKS=ON \ + '-DARKNET_CHECK_TESTS=connection_kernel_samples;benchmark_asio_connections_self_test' +cmake --build build/diagnostic-retention --target arknet_check --parallel 1 +``` + +`connection_kernel_samples` covers zero through eight observations, repeated +overflow with fixed head/rolling tail, chronological ordinals, exact JSON +metadata, supplied steady-clock origin/units (including signed synthetic epochs), +omission/overwrite counters, progress pruning, slot reuse and inert +disabled/frozen modes. The native control `--self-test` uses synthetic queries +to check retention/JSON and unchanged opt-in argument validation. No kernel query +or real connection load is needed for these checks. They do not establish the +onset of TCP recovery, simultaneous peer state, a root cause or capacity. The +history-only origin is derived from existing `began`, without a clock query; +origin-plus-relative-time correlation requires the same host and clock domain, +not cross-host wall-clock comparison. + ## Local Verification The complete-suite source snapshot is `6c8418e`; its networking changes are diff --git a/docs/project/testing_CN.md b/docs/project/testing_CN.md index ea92ae8..46ee528 100644 --- a/docs/project/testing_CN.md +++ b/docs/project/testing_CN.md @@ -255,6 +255,32 @@ build/tests/arknet_tls_test --reporters=junit --out=build/tls.xml 单 context 四线程、四个 context 各一线程，再分给指定重启轮次。因此七种协议使用 `--seconds 60` 时，请求的流量阶段共七分钟，另加启动和收尾。慢接收者用例测试背压，不用于比较吞吐量。 +### 原生 TCP 诊断保留方式 + +默认关闭的内核采样器在原有每身份 8 个样本、16 个身份、32 KiB 限制内，保留最早 4 个 +合格观测与最新 4 个样本，避免更长的服务端尾部覆盖较早的合格证据。 +schema 2 内核历史及其嵌套采样摘要声明 `retention_policy: "first_4_latest_4"`； +历史 schema 1 仅保留尾部的记录保持原有解释。 +参见[生产限制](production-readiness_CN.md)与[诊断字段](../../benchmarks/CONNECTIONS_CN.md#可选-tcp-内核采样)。 + +通过既有原生运行器选择不需要连接负载的检查： + +```sh +cmake -S . -B build/diagnostic-retention -DARKNET_BUILD_TESTS=ON \ + -DARKNET_BUILD_EXAMPLES=OFF -DARKNET_BUILD_BENCHMARKS=ON \ + '-DARKNET_CHECK_TESTS=connection_kernel_samples;benchmark_asio_connections_self_test' +cmake --build build/diagnostic-retention --target arknet_check --parallel 1 +``` + +`connection_kernel_samples` 覆盖 0 至 8 次观测、固定起始与轮换尾部的多次溢出、 +按时间排列的序号、精确 JSON 元数据、传入 steady-clock 起点及单位（含正负合成 epoch）、 +遗漏及覆盖计数、进展清理、槽位复用， +以及关闭和冻结时不执行查询。原生对照 `--self-test` 使用合成查询验证保留方式、JSON +及未改变的可选参数校验；这些检查不查询内核，也不运行真实连接负载。 +它们不能确定 TCP 恢复的起点、两端同时状态、根因或容量。仅历史记录的时钟起点由现有 +`began` 派生，不增加时钟查询；起点加相对时间的关联要求同一主机、同一时钟域， +不能用于跨主机墙上时间比较。 + ## 本机验证 完整测试的源码快照为 `6c8418e`，其中的网络库改动已纳入性能测量版本 diff --git a/tests/connection_kernel_samples.cpp b/tests/connection_kernel_samples.cpp index c2f5b0b..b165349 100644 --- a/tests/connection_kernel_samples.cpp +++ b/tests/connection_kernel_samples.cpp @@ -1,7 +1,9 @@ // Copyright (c) 2026 OpenArkStudio. Distributed under BSL-1.0. See LICENSE. +#include #include #include #include +#include #include "vendor/doctest/doctest.h" @@ -17,7 +19,9 @@ constexpr identity first{7, 1, 1}, second{7, 1, 2}; std::string history(const tcp_kernel_samples& samples, identity owner = first) { std::ostringstream out; - samples.write_history(out, owner); + out << "{\"event\":\"tcp_kernel_sample_history\",\"schema\":" << tcp_kernel_samples::schema_version << ','; + samples.write_history(out, owner, std::chrono::steady_clock::time_point{std::chrono::microseconds{1234567}}); + out << '}'; return out.str(); } @@ -26,6 +30,22 @@ std::string summary(const tcp_kernel_samples& samples) { samples.write_summary(out); return out.str(); } + +std::vector numbers(const std::string& text, const std::string& field) { + const auto marker = '"' + field + "\":"; + std::vector result; + for (auto position = text.find(marker); position != std::string::npos; position = text.find(marker, position)) + { + position += marker.size(); + std::uint64_t value{}; + const auto parsed = std::from_chars(text.data() + position, text.data() + text.size(), value); + DOCTEST_REQUIRE(parsed.ec == std::errc{}); + DOCTEST_REQUIRE(parsed.ptr != text.data() + text.size()); + DOCTEST_CHECK((*parsed.ptr == ',' || *parsed.ptr == '}')); + result.push_back(value); + } + return result; +} } // namespace DOCTEST_TEST_CASE("kernel samples stay inert when off and before the inactivity threshold") { @@ -47,6 +67,70 @@ DOCTEST_TEST_CASE("kernel samples stay inert when off and before the inactivity DOCTEST_CHECK_FALSE(tcp_kernel_samples::stalled(true, true, false, true, 1000000)); DOCTEST_CHECK_FALSE(tcp_kernel_samples::stalled(true, false, true, true, 1000000)); DOCTEST_CHECK_FALSE(tcp_kernel_samples::stalled(true, false, false, false, 1000000)); + disabled.prune(first, false, 137); + disabled.release(first); + disabled.freeze(); + disabled.sample(first, 3000000, 2000000, 136, 65, 69, query); + DOCTEST_CHECK(calls == 0); + DOCTEST_CHECK(summary(disabled).find("\"storage_bytes\":0") != std::string::npos); +} + +DOCTEST_TEST_CASE("kernel retention schema preserves every qualifying sample from zero through eight") { + tcp_kernel_samples samples(true); + DOCTEST_CHECK(history(samples) == + "{\"event\":\"tcp_kernel_sample_history\",\"schema\":2,\"clock_domain\":\"local_steady_clock\"," + "\"clock_origin_us\":1234567,\"clock_origin_unit\":\"microseconds_since_steady_clock_epoch\"," + "\"retention_policy\":\"first_4_latest_4\"," + "\"head_sample_capacity\":4,\"tail_sample_capacity\":4,\"sample_phase\":\"periodic_io_worker\"," + "\"time_unit\":\"microseconds_since_process_start\",\"samples\":[],\"total_samples\":0," + "\"overwritten_samples\":0}"); + std::vector ordinals, times; + for (std::uint64_t count = 0; count <= 8; ++count) + { + if (count) + { + ordinals.push_back(count); + times.push_back(1999999 + count); + samples.sample(first, times.back(), 1000000, 136, 65, 69, [] { return tcp_kernel_info{}; }); + } + const auto text = history(samples); + DOCTEST_CHECK(numbers(text, "sample_ordinal") == ordinals); + DOCTEST_CHECK(numbers(text, "sample_time_us") == times); + DOCTEST_CHECK(numbers(text, "total_samples") == std::vector{count}); + DOCTEST_CHECK(numbers(text, "overwritten_samples") == std::vector{0}); + if (count == 1) + { + DOCTEST_CHECK( + text == + "{\"event\":\"tcp_kernel_sample_history\",\"schema\":2,\"clock_domain\":\"local_steady_clock\"," + "\"clock_origin_us\":1234567,\"clock_origin_unit\":\"microseconds_since_steady_clock_epoch\"," + "\"retention_policy\":\"first_4_latest_4\"," + "\"head_sample_capacity\":4,\"tail_sample_capacity\":4,\"sample_phase\":\"periodic_io_worker\"," + "\"time_unit\":\"microseconds_since_process_start\",\"samples\":[{\"sample_ordinal\":1," + "\"sample_time_us\":2000000,\"no_completion_age_us\":1000000,\"next_receive\":65,\"next_send\":69," + "\"tcp_info\":null,\"unavailable_reason\":\"unsupported_platform\"}],\"total_samples\":1," + "\"overwritten_samples\":0}"); + } + } + const auto metadata = summary(samples); + DOCTEST_CHECK(metadata.starts_with("{\"schema\":2,\"retention_policy\":\"first_4_latest_4\",")); + DOCTEST_CHECK(metadata.find("\"head_sample_capacity\":4,\"tail_sample_capacity\":4") != std::string::npos); + DOCTEST_CHECK(numbers(metadata, "sampler_calls") == std::vector{8}); +} + +DOCTEST_TEST_CASE("kernel history clock origin uses the supplied steady epoch in microseconds only") { + tcp_kernel_samples samples(true); + for (const auto origin : {std::int64_t{-1234567}, std::int64_t{0}, std::int64_t{1234567}}) + { + std::ostringstream out; + samples.write_history(out, first, std::chrono::steady_clock::time_point{std::chrono::microseconds{origin}}); + DOCTEST_CHECK(out.str().starts_with("\"clock_domain\":\"local_steady_clock\",")); + DOCTEST_CHECK( + out.str().find("\"clock_origin_us\":" + std::to_string(origin) + + ",\"clock_origin_unit\":\"microseconds_since_steady_clock_epoch\"") != std::string::npos); + } + DOCTEST_CHECK(summary(samples).find("clock_origin") == std::string::npos); + DOCTEST_CHECK(summary(samples).find("clock_domain") == std::string::npos); } DOCTEST_TEST_CASE("kernel samples retain pre-reset state and cannot be overwritten after freeze") { @@ -73,8 +157,11 @@ DOCTEST_TEST_CASE("kernel samples retain pre-reset state and cannot be overwritt info.rto_ms = 0; samples.sample(first, 14000000, 6000000, 136, 65, 69, query); samples.release(first); + samples.prune(first, false, 137); + samples.sample(second, 14000000, 6000000, 136, 65, 69, query); DOCTEST_CHECK(calls == 2); DOCTEST_CHECK(history(samples) == before); + DOCTEST_CHECK_FALSE(samples.contains(second)); } DOCTEST_TEST_CASE("kernel sample capacity preserves persistent stalls and reuses recovered slots") { @@ -100,26 +187,52 @@ DOCTEST_TEST_CASE("kernel sample capacity preserves persistent stalls and reuses DOCTEST_CHECK(samples.contains(first)); DOCTEST_CHECK(samples.contains(extra)); DOCTEST_CHECK_FALSE(samples.contains(second)); + DOCTEST_CHECK(numbers(history(samples, extra), "sample_ordinal") == std::vector{1}); + DOCTEST_CHECK(numbers(history(samples, extra), "sample_time_us") == std::vector{3000000}); DOCTEST_CHECK(history(samples).find("\"sample_time_us\":2000000") != std::string::npos); DOCTEST_CHECK(summary(samples).find("\"released_histories\":1") != std::string::npos); DOCTEST_CHECK(summary(samples).find("\"slot_reuses\":1") != std::string::npos); } -DOCTEST_TEST_CASE("kernel samples bound history and separate identities across client and run") { +DOCTEST_TEST_CASE("kernel samples preserve the first four and latest four across repeated tail wraps") { tcp_kernel_samples samples(true); - for (unsigned index = 0; index < 10; ++index) + for (std::uint64_t count = 1; count <= 20; ++count) { - samples.sample(first, 2000000 + index, 1000000 + index, 136, 65, 69, [] { return tcp_kernel_info{}; }); + samples.sample(first, 1999999 + count, 1000000 + count, 136, 65, 69, [] { return tcp_kernel_info{}; }); + if (count > 8) + { + const std::vector expected{1, 2, 3, 4, count - 3, count - 2, count - 1, count}; + auto times = expected; + for (auto& time : times) + { + time += 1999999; + } + DOCTEST_CHECK(numbers(history(samples), "sample_ordinal") == expected); + DOCTEST_CHECK(numbers(history(samples), "sample_time_us") == times); + DOCTEST_CHECK(numbers(history(samples), "overwritten_samples") == std::vector{count - 8}); + DOCTEST_CHECK(numbers(summary(samples), "overwritten_samples") == std::vector{count - 8}); + } } const auto text = history(samples); - DOCTEST_CHECK(text.find("\"overwritten_samples\":2") != std::string::npos); - DOCTEST_CHECK(text.find("\"sample_time_us\":2000000") == std::string::npos); - DOCTEST_CHECK(text.find("\"sample_time_us\":2000002") != std::string::npos); DOCTEST_CHECK_FALSE(samples.contains({8, 1, 1})); DOCTEST_CHECK_FALSE(samples.contains({7, 2, 1})); samples.release({8, 1, 1}); DOCTEST_CHECK(samples.contains(first)); DOCTEST_CHECK(text.size() < 8192); // Existing trace output's fixed record scratch ceiling. + samples.prune(first, true, 137); + DOCTEST_CHECK_FALSE(samples.contains(first)); + samples.sample(first, 9000000, 1000000, 137, 70, 74, [] { return tcp_kernel_info{}; }); + DOCTEST_CHECK(numbers(history(samples), "sample_ordinal") == std::vector{1}); + DOCTEST_CHECK(numbers(history(samples), "sample_time_us") == std::vector{9000000}); + DOCTEST_CHECK(numbers(history(samples), "overwritten_samples") == std::vector{0}); + DOCTEST_CHECK(numbers(summary(samples), "overwritten_samples") == std::vector{12}); + DOCTEST_CHECK(numbers(summary(samples), "slot_reuses") == std::vector{1}); + const auto after_progress = history(samples); + samples.freeze(); + samples.sample(first, 10000000, 2000000, 137, 70, 74, [] { return tcp_kernel_info{}; }); + samples.prune(first, false, 138); + samples.release(first); + DOCTEST_CHECK(history(samples) == after_progress); } DOCTEST_TEST_CASE("unavailable kernel samples emit null with a reason rather than zero metrics") { @@ -160,7 +273,7 @@ DOCTEST_TEST_CASE("largest kernel sample history fits the existing bounded recor connection_load::trace_output output; DOCTEST_CHECK(output.append([&](std::ostream& out) { out << "{\"event\":\"tcp_kernel_sample_history\",\"run_id\":7,\"client_id\":1,\"connection_id\":1,"; - samples.write_history(out, first); + samples.write_history(out, first, std::chrono::steady_clock::time_point::max()); out << '}'; })); DOCTEST_CHECK_FALSE(output.truncated()); diff --git a/tests/connection_load_native_control.cpp b/tests/connection_load_native_control.cpp index 3e54012..d0c5e81 100644 --- a/tests/connection_load_native_control.cpp +++ b/tests/connection_load_native_control.cpp @@ -516,7 +516,14 @@ void progress_then_stall(const std::string& executable) { const auto history = output.substr(history_position, output.find('\n', history_position) - history_position); const auto summary = output.substr(summary_position, output.find('\n', summary_position) - summary_position); const auto result = output.substr(result_position); - if (history.find("\"sample_phase\":\"periodic_io_worker\"") == std::string::npos || + if (history.find("\"schema\":2,") == std::string::npos || + history.find("\"clock_domain\":\"local_steady_clock\"") == std::string::npos || + history.find("\"clock_origin_us\":") == std::string::npos || + history.find("\"clock_origin_unit\":\"microseconds_since_steady_clock_epoch\"") == std::string::npos || + history.find("\"retention_policy\":\"first_4_latest_4\"") == std::string::npos || + counter(history, "head_sample_capacity") != 4 || counter(history, "tail_sample_capacity") != 4 || + counter(history, "sample_ordinal") != 1 || + history.find("\"sample_phase\":\"periodic_io_worker\"") == std::string::npos || history.find("\"run_id\":97,\"client_id\":3,\"connection_id\":1") == std::string::npos || counter(history, "sample_time_us") < 1000000 || counter(history, "no_completion_age_us") < 1000000 || counter(history, "next_receive") != 2 || counter(history, "next_send") != 3 || From 0c99f5558239c87e607d4acb2b6091159c41ef80 Mon Sep 17 00:00:00 2001 From: nick Date: Mon, 26 Oct 2026 21:44:46 +0800 Subject: [PATCH 2/2] docs(roadmap): defer TCP timeout investigation --- docs/project/roadmap.md | 10 ++++++++++ docs/project/roadmap_CN.md | 8 ++++++++ 2 files changed, 18 insertions(+) diff --git a/docs/project/roadmap.md b/docs/project/roadmap.md index e6a725f..8faebdb 100644 --- a/docs/project/roadmap.md +++ b/docs/project/roadmap.md @@ -23,6 +23,16 @@ for throughput and p99 charts, model comparisons and machine specifications. | WebSocket compression | TODO | Negotiate permessage-deflate; enforce decompressed limits and test adversarial frames. | | DTLS | TODO | Select a supported engine; define peer identity, retransmission and datagram limits. | +## Deferred TCP diagnostic TODO + +[TCP many-connection response timeout (#61)](https://github.com/OpenArkStudio/arknet/issues/61) +is the canonical tracker. Investigation is paused while framework delivery takes +priority. The completed three-run schema 2 campaign and earlier schema 1 evidence +keep their recorded source/format boundaries; raw logs remain local. +The root cause remains unresolved. These observations establish neither +performance/capacity acceptance nor a failure-rate estimate. Resume only after +explicit reprioritization and a bounded plan; see the [recorded limits](production-readiness.md). + ## HTTP Engine Boundaries [Boost.Beast](https://github.com/boostorg/beast#introduction) implements diff --git a/docs/project/roadmap_CN.md b/docs/project/roadmap_CN.md index bf42302..b8b746c 100644 --- a/docs/project/roadmap_CN.md +++ b/docs/project/roadmap_CN.md @@ -22,6 +22,14 @@ RUDP 与 ARQ／可选 FEC、原生单节点 MySQL 和 Redis 客户端、定时 | WebSocket 压缩 | TODO | permessage-deflate 协商、解压后上限和恶意帧测试。 | | DTLS | TODO | 选择可用引擎，定义对端身份、重传和数据报上限。 | +## 暂缓的 TCP 诊断 TODO + +[TCP 多连接响应超时（#61）](https://github.com/OpenArkStudio/arknet/issues/61) +是唯一跟踪入口。当前暂停调查，优先推进框架交付。已完成的三次 schema 2 诊断与早期 +schema 1 证据保留各自记录的源码及格式边界，原始日志仅留本地。 +根因仍未确定，这些观测不构成性能或容量验收，也不能用于估算故障率。 +仅在明确调整优先级并制定有界计划后恢复；参见[已记录的限制](production-readiness_CN.md)。 + ## HTTP 引擎边界 [Boost.Beast](https://github.com/boostorg/beast#introduction) 提供