Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions DECISIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -586,6 +586,24 @@ published by hand by the owner.
write to an allowed host fails the step. A page that writes on its own (telemetry) needs `WritableUrls`.
- **`BlockedRequest.RefusedWrite` is `null` for allow-list refusals**, so existing `BlockedRequest` equality is
unchanged.
## Preconditions

- **A run can wait for a deploy before it starts.** `JourneyPreconditions.WaitAsync` opens a page until a response header or an
element's text or attribute shows a value (exact, contains or regex). A deploy that has not landed, or whose cache has not
cleared, still shows the old value; failing there says so once, instead of failing every journey.
- **Every look is a new context.** A context has no cache, so a stale copy cannot satisfy the wait, and the run's user agent,
headers, allow list and read-only rule apply to it like to a journey, which keeps a monitor identifiable.
- **Header and element go through one browser path.** Chrome reports the main document's response (`Page.Response`: status and
headers) from the `Network` events every page already collects since the timeout diagnostics, so no second HTTP client and no
HTML parser is needed, and an element that script adds is seen as a visitor sees it.
- **`Timeout`, `Interval` and `Retries` are separate.** A look that works and shows another value is not an error: the wait goes on
until `Timeout` (default 10 s, every 500 ms). A look that fails (the page did not load in time, the connection dropped) counts
against `Retries` (default 2) and the wait gives up when they run out, so a dead origin fails in seconds and not after the
whole timeout. A host outside the allow list is refused before any look.
- **Not a journey step.** It is a run-level check the runner makes once, before any journey: `# @requires` and `wait` stay as they are.
- **What it cannot see:** it shows the value of the page it opened. A value that does not change between two deploys, such as a
version shared by a content-only deploy, passes at once.

## Run defaults

- **A harness sets the viewport, user agent and headers once per run.** A live site tells monitors from people by user
Expand Down
28 changes: 28 additions & 0 deletions docs/library/network.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,3 +48,31 @@ network har stop checkout.har
In a journey the file goes into the artifacts folder (`--artifacts`), and secrets typed during the run are replaced by
their `$env:` references. With the allow list on, each request waits for its verdict, which adds a little to
`blocked`; record with the guard off when the timings themselves are the point.

## Wait for a deploy

`JourneyPreconditions.WaitAsync` waits until a page shows a value before anything runs against it: a response header of the page's document, or
an element's text or attribute. A site whose deploy has not landed, or whose cache has not cleared, still shows the old value.

```csharp
var precondition = new JourneyPrecondition
{
Path = "/docs",
Selector = "meta[name='docs:build']",
Attribute = "content",
Expected = "1.74.1",
Timeout = TimeSpan.FromSeconds(10),
Retries = 2,
};

PreconditionResult result = await JourneyPreconditions.WaitAsync(browser, precondition, new JourneyOptions { BaseUrl = new Uri("https://docs-demo.elastic.dev/") });
if (!result.Passed)
Console.Error.WriteLine(result.Message);
```

Set `Header` instead of `Selector` to read a response header. `Match` is `Exact` by default; `Contains` and `Regex` are the others.
The wait looks again every `Interval` (500 ms) until `Timeout` (10 s). A look that fails, because the page did not load or the
connection dropped, counts against `Retries` (2), and the wait ends when they are used up. Each look is a new context, so nothing is cached, and
the options' allow list, user agent, headers and read-only rule apply.

`Page.Response` has the status and headers of the response that served the page's current document.
216 changes: 216 additions & 0 deletions src/Nullean.Cdp.Journeys/JourneyPreconditions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
using System.Globalization;
using System.Text.RegularExpressions;
using Nullean.Cdp.Protocol;

namespace Nullean.Cdp.Journeys;

/// <summary>
/// Something a site must show before journeys run against it: a response header or an element's text or attribute on one
/// page, equal to a value. A deploy that has not landed, or whose cache has not cleared, still shows the old value; waiting for
/// the new one first turns that into a clear failure and keeps it from showing up as a failure of every journey.
/// </summary>
public sealed record JourneyPrecondition
{
/// <summary>The page to open: an absolute URL, or a path that <see cref="JourneyOptions.BaseUrl"/> resolves.</summary>
public required string Path { get; init; }

/// <summary>A response header of the page's document. Set this, or <see cref="Selector"/>.</summary>
public string? Header { get; init; }

/// <summary>A CSS selector for an element, such as <c>meta[name='docs:build']</c>. Set this, or <see cref="Header"/>.</summary>
public string? Selector { get; init; }

/// <summary>With <see cref="Selector"/>: the attribute to read, such as <c>content</c>. The element's text when <c>null</c>.</summary>
public string? Attribute { get; init; }

/// <summary>The value to wait for.</summary>
public required string Expected { get; init; }

/// <summary>How <see cref="Expected"/> is compared with what the page shows.</summary>
public TextMatch Match { get; init; } = TextMatch.Exact;

/// <summary>How long to wait for the value in all. Default: 10 seconds.</summary>
public TimeSpan Timeout { get; init; } = TimeSpan.FromSeconds(10);

/// <summary>How long to wait between looks. Default: half a second.</summary>
public TimeSpan Interval { get; init; } = TimeSpan.FromMilliseconds(500);

/// <summary>
/// How many times a look that fails (the page did not load, the connection dropped) is tried again before the wait gives up. A look
/// that works and shows another value is not a failure: the wait goes on until <see cref="Timeout"/>. Default: 2.
/// </summary>
public int Retries { get; init; } = 2;

/// <summary>What the precondition looks at, in words: <c>the docs:build header of /docs</c>.</summary>
public string Subject =>
Header is not null
? $"the {Header} response header of {Path}"
: $"{(Attribute is null ? "the text" : $"the {Attribute} attribute")} of {Selector} on {Path}";
}

/// <summary>How waiting for a <see cref="JourneyPrecondition"/> ended.</summary>
/// <param name="Passed">The page showed the value.</param>
/// <param name="Message">Plain text: what matched, or what was expected and what was last seen.</param>
/// <param name="Attempts">How many looks it took.</param>
/// <param name="Elapsed">How long it waited.</param>
public sealed record PreconditionResult(bool Passed, string Message, int Attempts, TimeSpan Elapsed);

/// <summary>Waits for what a site must show before a run starts.</summary>
public static class JourneyPreconditions
{
private const int MaxSeen = 200;
private static readonly TimeSpan RegexTimeout = TimeSpan.FromSeconds(1);

/// <summary>
/// Opens the page in a fresh context (so nothing is cached) until it shows the value of <paramref name="precondition"/>, the wait
/// times out, or looks fail more often than <see cref="JourneyPrecondition.Retries"/> allows. The run's allow list, user agent,
/// headers and read-only rule of <paramref name="options"/> apply, so a monitor stays identifiable and cannot write.
/// </summary>
public static async Task<PreconditionResult> WaitAsync(
Browser browser,
JourneyPrecondition precondition,
JourneyOptions? options = null,
CancellationToken cancellationToken = default
)
{
ArgumentNullException.ThrowIfNull(browser);
ArgumentNullException.ThrowIfNull(precondition);
options ??= new JourneyOptions();
Validate(precondition);
var url = Resolve(precondition.Path, options);
var allowList = options.AllowList([]);
if (allowList.Refusal(url) is { } refusal)
return new PreconditionResult(false, $"Did not look at {url}: {refusal}.", 0, TimeSpan.Zero);

var started = System.Diagnostics.Stopwatch.StartNew();
var attempts = 0;
var failures = 0;
string? seen;
while (true)
{
attempts++;
try
{
seen = await LookAsync(browser, precondition, url, allowList, options, Remaining(precondition, started), cancellationToken).ConfigureAwait(
false
);
if (seen is not null && Matches(precondition, seen))
return new PreconditionResult(true, $"{Capitalise(precondition.Subject)} is {Quote(seen)}.", attempts, started.Elapsed);
}
catch (Exception e) when (IsLookFailure(e, cancellationToken))
{
failures++;
seen = null;
if (failures > precondition.Retries)
return new PreconditionResult(
false,
$"Could not look at {url} for {Describe(precondition)}: {failures} looks failed, the last with: {e.Message}",
attempts,
started.Elapsed
);
}

if (started.Elapsed + precondition.Interval >= precondition.Timeout)
return new PreconditionResult(false, Timeout(precondition, seen, attempts, started.Elapsed), attempts, started.Elapsed);
await Task.Delay(precondition.Interval, cancellationToken).ConfigureAwait(false);
}
}

private static TimeSpan Remaining(JourneyPrecondition precondition, System.Diagnostics.Stopwatch started)
{
var left = precondition.Timeout - started.Elapsed;
return left > TimeSpan.FromSeconds(1) ? left : TimeSpan.FromSeconds(1);
}

// What the page shows, or null when it shows nothing to compare: no such header, no such element.
private static async Task<string?> LookAsync(
Browser browser,
JourneyPrecondition precondition,
Uri url,
JourneyAllowList allowList,
JourneyOptions options,
TimeSpan budget,
CancellationToken cancellationToken
)
{
await using var context = await browser.NewContextAsync(cancellationToken).ConfigureAwait(false);
var page = await context.NewPageAsync(cancellationToken).ConfigureAwait(false);
page.NavigationTimeout = options.NavigationTimeout is { } navigation && navigation < budget ? navigation : budget;
await using var guard = options.GuardNetwork || options.ReadOnly
? await page.GuardNetworkAsync(options.GuardNetwork ? allowList.Allows : _ => true, options.GuardOptions(), cancellationToken).ConfigureAwait(false)
: null;
await JourneyRunner.ApplyDefaultsAsync(page, options, cancellationToken).ConfigureAwait(false);
await page.GotoAsync(url.AbsoluteUri, cancellationToken).ConfigureAwait(false);
if (precondition.Header is { } header)
return page.Response?.Header(header);
var element = page.Css(precondition.Selector!);
if (await element.CountAsync(cancellationToken).ConfigureAwait(false) == 0)
return null;
return precondition.Attribute is { } attribute
? await element.First.AttributeAsync(attribute, cancellationToken).ConfigureAwait(false)
: await element.First.TextAsync(cancellationToken).ConfigureAwait(false);
}

private static bool IsLookFailure(Exception e, CancellationToken cancellationToken) =>
e is TimeoutException or PageNavigationException or CdpException or CdpDisconnectedException ||
(e is OperationCanceledException && !cancellationToken.IsCancellationRequested);

private static bool Matches(JourneyPrecondition precondition, string seen) => precondition.Match switch
{
TextMatch.Contains => seen.Contains(precondition.Expected, StringComparison.Ordinal),
TextMatch.Regex => Regex.IsMatch(seen, precondition.Expected, RegexOptions.CultureInvariant, RegexTimeout),
_ => string.Equals(seen, precondition.Expected, StringComparison.Ordinal),
};

private static void Validate(JourneyPrecondition p)
{
if ((p.Header is null) == (p.Selector is null))
throw new ArgumentException(
"A precondition looks at a response header or at an element: set Header or Selector, not both and not neither.",
nameof(p)
);
if (p.Header is null && p.Attribute is not null && p.Attribute.Length == 0)
throw new ArgumentException("The attribute name is empty.", nameof(p));
if (p.Timeout <= TimeSpan.Zero || p.Interval <= TimeSpan.Zero)
throw new ArgumentOutOfRangeException(nameof(p), "The timeout and the interval are longer than zero.");
if (p.Retries < 0)
throw new ArgumentOutOfRangeException(nameof(p), "Retries are zero or more.");
if (p.Match == TextMatch.Regex)
{
try
{
_ = new Regex(p.Expected, RegexOptions.CultureInvariant, RegexTimeout);
}
catch (ArgumentException e)
{
throw new ArgumentException($"'{p.Expected}' is not a valid regular expression: {e.Message}", nameof(p), e);
}
}
}

private static Uri Resolve(string path, JourneyOptions options) =>
Uri.TryCreate(options.BaseUrl, path, out var url) && url.IsAbsoluteUri
? url
: throw new ArgumentException($"'{path}' is not a URL, and the options have no base URL to resolve it against.", nameof(path));

private static string Timeout(JourneyPrecondition p, string? seen, int attempts, TimeSpan elapsed) =>
string.Create(
CultureInfo.InvariantCulture,
$"Timed out after {elapsed.TotalSeconds:0.#}s waiting for {p.Subject} to {Describe(p)}: last saw {(seen is null ? "nothing" : Quote(seen))} in {attempts} {(attempts == 1 ? "look" : "looks")}."
);

private static string Describe(JourneyPrecondition p) => p.Match switch
{
TextMatch.Contains => $"contain {Quote(p.Expected)}",
TextMatch.Regex => $"match {Quote(p.Expected)}",
_ => $"equal {Quote(p.Expected)}",
};

private static string Quote(string value)
{
var flat = value.ReplaceLineEndings(" ");
return $"\"{(flat.Length > MaxSeen ? flat[..MaxSeen] + "..." : flat)}\"";
}

private static string Capitalise(string text) => char.ToUpperInvariant(text[0]) + text[1..];
}
8 changes: 5 additions & 3 deletions src/Nullean.Cdp.Journeys/JourneyRunner.cs
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,11 @@ internal NetworkGuardOptions GuardOptions()
};
}

internal JourneyAllowList AllowList(Journey journey)
internal JourneyAllowList AllowList(Journey journey) => AllowList(journey.AllowedHosts);

internal JourneyAllowList AllowList(IEnumerable<string> journeyHosts)
{
var hosts = journey.AllowedHosts.Concat(AllowedHosts);
var hosts = journeyHosts.Concat(AllowedHosts);
if (BaseUrl is { Scheme: "http" or "https" } web)
hosts = hosts.Append(web.IdnHost);
// "file:///x/site/" and "file:///x/site/index.html" both give /x/site.
Expand Down Expand Up @@ -290,7 +292,7 @@ private static async Task<JourneyFailure> FailureAsync(Run run, JourneyStep step
}

// The run's viewport, user agent and headers, set before the first step so the journey starts as the harness wants.
private static async Task ApplyDefaultsAsync(Page page, JourneyOptions options, CancellationToken cancellationToken)
internal static async Task ApplyDefaultsAsync(Page page, JourneyOptions options, CancellationToken cancellationToken)
{
if (options.Viewport is { } viewport)
await page.SetViewportAsync(viewport.Width, viewport.Height, viewport.Scale, cancellationToken).ConfigureAwait(false);
Expand Down
9 changes: 8 additions & 1 deletion src/Nullean.Cdp/Page.cs
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ internal Page(Browser browser, CdpConnection connection, string targetId, string
TargetId = targetId;
SessionId = sessionId;
ContextId = contextId;
_activity = new PageActivity(targetId);
}

internal Browser Browser { get; }
Expand All @@ -45,7 +46,13 @@ internal Page(Browser browser, CdpConnection connection, string targetId, string
internal int RecordingInteractions;

private readonly PageLog _log = new();
private readonly PageActivity _activity = new();
private readonly PageActivity _activity;

/// <summary>
/// The response that served the main frame's current document, with its status and headers, or <c>null</c> before one
/// arrived (a <c>file://</c> or <c>about:blank</c> page has none). A redirect shows the final response.
/// </summary>
public PageResponse? Response => _activity.Document;

/// <summary>The page's console calls since it was created or last cleared, oldest first.</summary>
public IReadOnlyList<ConsoleMessage> ConsoleMessages => _log.Console;
Expand Down
13 changes: 12 additions & 1 deletion src/Nullean.Cdp/PageActivity.cs
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,32 @@ namespace Nullean.Cdp;
/// What the page is waiting on and what already went wrong, kept so a timeout can say why it timed out: the requests still in
/// flight (oldest first), and the last failed or error-status ones. Holds URLs and counts only, never bodies or headers.
/// </summary>
internal sealed class PageActivity
internal sealed class PageActivity(string frameId)
{
private const int Shown = 5;
private const int Kept = 20;

private readonly ConcurrentDictionary<string, InFlight> _open = new(StringComparer.Ordinal);
private readonly ConcurrentQueue<string> _problems = new();
private volatile PageResponse? _document;

private readonly record struct InFlight(string Method, string Url, long Started);

/// <summary>The latest response for a document in the page's main frame, or <c>null</c> before one arrived.</summary>
public PageResponse? Document => _document;

public void OnEvent(CdpEvent e)
{
var p = e.Params;
if (p.StringOrNull("requestId") is not { } id)
return;
if (
e.Method == "Network.responseReceived" && p.StringOrNull("type") == "Document" && p.StringOrNull("frameId") == frameId && p.TryGetProperty(
"response",
out var main
)
)
_document = PageResponse.From(main);
switch (e.Method)
{
// A redirect reuses the id: the new hop replaces the old one.
Expand Down
26 changes: 26 additions & 0 deletions src/Nullean.Cdp/PageResponse.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
using System.Text.Json;

namespace Nullean.Cdp;

/// <summary>The response that served a page's document: its URL, status and headers.</summary>
/// <param name="Url">The URL the response came from.</param>
/// <param name="Status">The HTTP status code.</param>
/// <param name="Headers">The response headers by name, case-insensitive. A header sent more than once is joined with <c>, </c> (Chrome joins them with a newline).</param>
public sealed record PageResponse(string Url, int Status, IReadOnlyDictionary<string, string> Headers)
{
/// <summary>The value of <paramref name="name"/>, or <c>null</c> when the response has no such header.</summary>
public string? Header(string name) => Headers.TryGetValue(name, out var value) ? value : null;

internal static PageResponse From(JsonElement response)
{
var headers = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
if (response.TryGetProperty("headers", out var sent) && sent.ValueKind == JsonValueKind.Object)
foreach (var header in sent.EnumerateObject())
headers[header.Name] = (header.Value.GetString() ?? string.Empty).Replace("\n", ", ", StringComparison.Ordinal);
return new PageResponse(
response.StringOrNull("url") ?? string.Empty,
response.TryGetProperty("status", out var status) && status.ValueKind == JsonValueKind.Number ? status.GetInt32() : 0,
headers
);
}
}
Loading
Loading