Skip to content

Add Logs Ingestion API scenario with synthetic sample data - #296

Open
Austin McCollum (austinmccollum) wants to merge 4 commits into
microsoft:masterfrom
austinmccollum:logs-ingestion-scenario
Open

Add Logs Ingestion API scenario with synthetic sample data#296
Austin McCollum (austinmccollum) wants to merge 4 commits into
microsoft:masterfrom
austinmccollum:logs-ingestion-scenario

Conversation

@austinmccollum

Copy link
Copy Markdown

Summary

Adds a new scenario under Scenarios/How to collect data with the Logs Ingestion API/ to support the Logs Ingestion API portal tutorial.

Problem

The tutorial's inline sample data (~200 lines of Apache access logs) contained references to a real website (almhuette-raith.at), real page paths, and real user-agent strings — a privacy concern for published documentation.

Changes

File Description
README.md Describes the scenario, links to the tutorial, explains the sample data and generator script
Generate-SampleAccessLog.ps1 PowerShell script that generates synthetic Apache Combined Log Format entries with configurable count. Uses RFC 5737 documentation IPs (198.51.100.x, 203.0.113.x), contoso.example.com domains, and weighted response codes
sample_access.log Pre-generated 200-entry synthetic log file, ready to use with the tutorial

Design decisions

  • RFC 5737 IPs (198.51.100.x, 203.0.113.x) and RFC 2606 domains (example.com) — reserved for documentation, guaranteed non-routable
  • Weighted response codes — 65% 200, 8% 404, 8% 304, plus 301/400/401/403/500/502/503 — ensures the tutorial's where ResponseCode != 200 KQL filter produces meaningful output
  • Fixed random seed (42) in the generator for reproducibility
  • Varied user agents — desktop, mobile, bot, CLI — realistic mix without real device fingerprints

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new “Logs Ingestion API” scenario under Scenarios/How to collect data with the Logs Ingestion API/ to provide synthetic Apache access-log data and a generator script that can be used with the Azure Monitor Logs Ingestion API portal tutorial.

Changes:

  • Added a scenario README that links to the portal tutorial and explains how to use/generate sample access logs.
  • Added a PowerShell generator (Generate-SampleAccessLog.ps1) to create synthetic Apache Combined Log Format entries.
  • Added a pre-generated sample_access.log (200 entries) intended to be tutorial-ready.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 3 comments.

File Description
Scenarios/How to collect data with the Logs Ingestion API/README.md Scenario documentation and usage instructions for the sample/generator.
Scenarios/How to collect data with the Logs Ingestion API/Generate-SampleAccessLog.ps1 Synthetic log generator (currently has determinism/privacy mismatches vs PR description).
Scenarios/How to collect data with the Logs Ingestion API/sample_access.log Pre-generated sample log (currently contains real-domain UA URLs and many 10.0.* IPs).
Suppressed comments (1)

Scenarios/How to collect data with the Logs Ingestion API/Generate-SampleAccessLog.ps1:178

  • The $random = [System.Random]::new(42) variable is never used, so the generator output is not actually reproducible even though the script/comment claims a fixed seed. Switching the per-field selections (method/path/status/UA/referer and time delta) to use the seeded helper will make reruns deterministic.
$random = [System.Random]::new(42)  # Fixed seed for reproducibility
$entries = [System.Collections.Generic.List[string]]::new($Count)
$currentTime = $StartDate

for ($i = 0; $i -lt $Count; $i++) {

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants