MateusjsSilva · May 12, 2026 · May 11, 2026 · May 11, 2026 · May 11, 2026 · May 11, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,35 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [1.0.0-preview.3] - 2026-05-11
+
+### Added
+
+- **Durable audit outbox framework**: `IDurableAuditOutbox` contract with `DequeueBatchAsync`, `MarkProcessedAsync`, `MarkFailedAsync` for production-grade delivery guarantees.
+- **Audit outbox entries**: `AuditOutboxEntry` model tracking attempts, timestamps, errors, and dead-letter status for reliable retry logic.
+- **Outbox publisher abstraction**: `IAuditOutboxPublisher` for pluggable downstream delivery (SIEM, webhooks, event buses, etc.).
+- **Audit outbox dispatcher**: `AuditOutboxDispatcher` background service with configurable polling, exponential backoff, and dead-lettering.
+- **Metrics & diagnostics**: New diagnostic codes SF-CONFIG-013 (in-memory outbox outside Development) and SF-CONFIG-014 (durable outbox without publishers).
+- **Health checks**: `AuditOutboxHealthCheck` detects in-memory outbox usage in production.
+- **EF Core durable outbox** (`SensitiveFlow.Audit.EFCore.Outbox` package): `EfCoreAuditOutbox` with transactional guarantees (audit + outbox in single SaveChanges).
+- **Audit outbox diagnostics class**: `SensitiveFlowAuditDiagnostics` tracks enqueued/dispatched/failed/dead-lettered/pending record counts.
+
+### Deprecated
+
+- `InMemoryAuditOutbox` — use `AddEfCoreAuditOutbox()` or custom `IDurableAuditOutbox` for production.
+- `AddInMemoryAuditOutbox()` — same guidance as above.
+
+### Changed
+
+- Audit store decorator `OutboxAuditStore` now reports metrics via `SensitiveFlowAuditDiagnostics`.
+- `SensitiveFlowConfigurationValidator` now checks for in-memory outbox outside Development (SF-CONFIG-013).
+- Docs: `docs/audit.md` updated with at-least-once vs at-most-once delivery matrix and production-ready outbox patterns.
+
+### Security
+
+- Durable outbox prevents audit record loss on process restart or failure.
+- Transactional outbox pattern ensures audit writes and outbox enqueuing happen atomically.
+
 ## [1.0.0-preview.2] - 2026-05-10
 
 ### Added
@@ -15,7 +44,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - `SECURITY.md` with supported versions table, vulnerability reporting process, and security scope.
 - `CODE_OF_CONDUCT.md` (Contributor Covenant 2.1).
 - `RELEASE.md` with full versioning scheme and publishing process.
-- Legal disclaimer in README: "SensitiveFlow helps reduce accidental exposure of sensitive data, but it does not guarantee legal compliance or complete data protection by itself."
+- Disclaimer in README: "SensitiveFlow is a tool to help you manage sensitive data safely, not a guarantee of complete data protection by itself."
 - New README badges: Container Tests, NuGet Downloads, .NET 8 | 9 | 10.
 
 ### Changed
@@ -77,3 +106,5 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Documentation: getting-started, attributes, legal-bases, consent, audit, data-subject-rights, retention, data-map, incidents, ripd, international-transfer, efcore, aspnetcore, migration
 
 
+
+
diff --git a/Directory.Build.props b/Directory.Build.props
@@ -20,7 +20,7 @@
     <IncludeSymbols>true</IncludeSymbols>
     <SymbolPackageFormat>snupkg</SymbolPackageFormat>
     <GenerateDocumentationFile>true</GenerateDocumentationFile>
-    <Version>1.0.0-preview.2</Version>
+    <Version>1.0.0-preview.3</Version>
   </PropertyGroup>
 
   <ItemGroup Condition="'$(IsPackable)' != 'false'">

diff --git a/Directory.Packages.props b/Directory.Packages.props
@@ -6,9 +6,12 @@
   <ItemGroup Label="Microsoft">
     <PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="9.0.0" />
     <PackageVersion Include="Microsoft.Extensions.Logging.Abstractions" Version="9.0.0" />
+    <PackageVersion Include="Microsoft.Extensions.Logging" Version="9.0.0" />
     <PackageVersion Include="Microsoft.Extensions.Hosting.Abstractions" Version="9.0.0" />
     <PackageVersion Include="Microsoft.Extensions.DependencyInjection.Abstractions" Version="9.0.0" />
     <PackageVersion Include="Microsoft.Extensions.Options" Version="9.0.0" />
+    <PackageVersion Include="Microsoft.Extensions.Diagnostics.HealthChecks.Abstractions" Version="9.0.0" />
+    <PackageVersion Include="Microsoft.Extensions.Diagnostics.HealthChecks" Version="9.0.0" />
     <PackageVersion Include="Microsoft.AspNetCore.Http.Abstractions" Version="2.3.0" />
     <PackageVersion Include="Microsoft.EntityFrameworkCore" Version="9.0.0" />
     <PackageVersion Include="Microsoft.EntityFrameworkCore.Relational" Version="9.0.0" />

diff --git a/README.md b/README.md
@@ -1,24 +1,34 @@
 # SensitiveFlow
 
 <p align="center">
-  <img src="assets/logo.png" alt="SensitiveFlow logo" width="200" />
+  <img src="https://raw.githubusercontent.com/MateusjsSilva/SensitiveFlow/main/assets/logo.png" alt="SensitiveFlow logo" width="200" />
 </p>
 
 [![CI](https://github.com/MateusjsSilva/SensitiveFlow/actions/workflows/ci.yml/badge.svg)](https://github.com/MateusjsSilva/SensitiveFlow/actions/workflows/ci.yml)
 [![Container Tests](https://github.com/MateusjsSilva/SensitiveFlow/actions/workflows/container-tests.yml/badge.svg)](https://github.com/MateusjsSilva/SensitiveFlow/actions/workflows/container-tests.yml)
 [![CodeQL](https://github.com/MateusjsSilva/SensitiveFlow/actions/workflows/codeql.yml/badge.svg)](https://github.com/MateusjsSilva/SensitiveFlow/actions/workflows/codeql.yml)
 [![NuGet](https://img.shields.io/nuget/v/SensitiveFlow.Core?label=nuget)](https://www.nuget.org/packages/SensitiveFlow.Core)
-[![NuGet Downloads](https://img.shields.io/nuget/dt/SensitiveFlow.Core)](https://www.nuget.org/packages/SensitiveFlow.Core)
+[![NuGet Downloads](https://img.shields.io/nuget/dt/SensitiveFlow.Core)](https://www.nuget.org/packages?q=SensitiveFlow)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 [![.NET](https://img.shields.io/badge/.NET-8%20%7C%209%20%7C%2010-blue)](https://dotnet.microsoft.com)
 
-**SensitiveFlow** is a .NET library that brings observability and control to sensitive data. It focuses on runtime behavior -- automatic auditing, log redaction, and masking -- not compliance paperwork.
+**SensitiveFlow** is a .NET library that brings observability and control to sensitive data flows through automatic auditing, log redaction, JSON masking, and pseudonymization. Mark your data once, get automatic protection everywhere.
 
-> **Important:** SensitiveFlow helps reduce accidental exposure of sensitive data, but it does not guarantee legal compliance or complete data protection by itself. You are responsible for how you use these primitives in your application.
+> **Important:** SensitiveFlow is a tool to help you manage sensitive data safely, not a guarantee of complete data protection by itself. You are responsible for how you use these primitives in your application and for ensuring they meet your requirements.
 
 ## Why SensitiveFlow?
 
-Sensitive data flows through your application on every request: EF Core saves, HTTP responses, log lines. SensitiveFlow makes that flow visible and controlled at the infrastructure level, without requiring manual instrumentation.
+Sensitive data flows through your application constantly: EF Core database saves, HTTP JSON responses, structured logs, error messages. Without SensitiveFlow, that data is exposed everywhere—in logs, in responses, in backups, in error traces—often with no visibility into where it leaked or why.
+
+SensitiveFlow gives you:
+
+- **Automatic audit trail** — Every database change to a sensitive field is logged with who changed it, when, what changed, and why (from your request context). No manual instrumentation.
+- **Log redaction** — Your structured logger scrubs sensitive values *before* they reach any sink. A developer who accidentally logs a customer's email will never see it in the logs; it's redacted automatically.
+- **JSON response masking** — ASP.NET Core responses automatically mask sensitive fields in your JSON output without changing your serialization code.
+- **Reversible pseudonymization** — Replace real values with tokens while keeping them queryable. Trade a customer's email for `token_abc123` in logs and analytics, then recover the original when needed.
+- **Data retention & erasure** — Declare retention policies on fields; SensitiveFlow will anonymize or delete expired records on schedule.
+- **Data subject export** — Generate privacy reports showing all data associated with a customer in your system.
+- **Health checks** — Monitor whether your audit infrastructure is working; if your audit database fails, you'll know immediately instead of silently losing records.
 
 ## Packages
 
@@ -33,27 +43,26 @@ Sensitive data flows through your application on every request: EF Core saves, H
 | `SensitiveFlow.AspNetCore` | Middleware for actor/IP context enrichment | Preview |
 | `SensitiveFlow.Logging` | ILogger decorator for PII redaction in logs | Preview |
 | `SensitiveFlow.Diagnostics` | OpenTelemetry bridge (ActivitySource + Meter) for audit/logging spans & metrics | Preview |
+| `SensitiveFlow.HealthChecks` | Microsoft health checks for audit/token infrastructure | Preview |
 | `SensitiveFlow.Anonymization` | Masking, anonymization, pseudonymization, erasure, data export, and deterministic fingerprints | Preview |
 | `SensitiveFlow.Json` | `System.Text.Json` modifier that masks/redacts/omits annotated properties at serialization time | Preview |
 | `SensitiveFlow.Retention` | Retention metadata and expiration hook contracts | Preview |
 | `SensitiveFlow.Analyzers` | Roslyn analyzers for privacy anti-patterns | Preview |
 | `SensitiveFlow.Analyzers.CodeFixes` | Quick-fix providers for SF0001/SF0002 (wrap with `.MaskEmail()` / `.MaskPhone()` / `.MaskName()`) | Preview |
 | `SensitiveFlow.SourceGenerators` | Source generator that precomputes sensitive/retention member metadata | Preview |
 | `SensitiveFlow.TestKit` | xUnit conformance bases for `IAuditStore` / `ITokenStore` plus a `SensitiveDataAssert` leak-detection helper | Preview |
+| `SensitiveFlow.Tool` | `dotnet tool` command for discovery reports from annotated assemblies | Preview |
 
 ## Quick Start
 
-### 1. Install packages
+### 1. Install and annotate
 
 ```bash
-dotnet add package SensitiveFlow.Core
-dotnet add package SensitiveFlow.Audit
-dotnet add package SensitiveFlow.Audit.EFCore
-dotnet add package SensitiveFlow.EFCore
-dotnet add package SensitiveFlow.AspNetCore
+dotnet add package SensitiveFlow.AspNetCore.EFCore
+dotnet add package Microsoft.EntityFrameworkCore.SqlServer # or your EF Core provider
 ```
 
-### 2. Annotate your model
+Then mark your sensitive fields:
 
 ```csharp
 using SensitiveFlow.Core.Attributes;
@@ -62,65 +71,129 @@ using SensitiveFlow.Core.Enums;
 public class Customer
 {
     public Guid Id { get; set; }
-    public string DataSubjectId { get; set; } = string.Empty;
+    public string UserId { get; set; } = string.Empty;
 
     [PersonalData(Category = DataCategory.Identification)]
     public string Name { get; set; } = string.Empty;
 
     [PersonalData(Category = DataCategory.Contact)]
     public string Email { get; set; } = string.Empty;
 
-    [SensitiveData(Category = SensitiveDataCategory.Other)]
+    [SensitiveData(Category = SensitiveDataCategory.Financial)]
     [RetentionData(Years = 5, Policy = RetentionPolicy.AnonymizeOnExpiration)]
     public string TaxId { get; set; } = string.Empty;
 }
 ```
 
-### 3. Register a durable audit store
-
-`IAuditStore` is the persistence contract -- you own the implementation so audit records go
-exactly where your infrastructure requires (SQL, MongoDB, Azure Table Storage, etc.).
-
-For EF Core-backed audit storage, use `SensitiveFlow.Audit.EFCore`. It registers an
-`IAuditStore` that also implements `IBatchAuditStore`, avoiding one database roundtrip
-per sensitive field.
+### 2. Register and start
 
 ```csharp
-builder.Services.AddEfCoreAuditStore(options =>
-    options.UseSqlServer(builder.Configuration.GetConnectionString("AuditStorage")));
-builder.Services.AddAuditStoreRetry();
+using SensitiveFlow.AspNetCore.EFCore.Extensions;
+
+// Register SensitiveFlow
+builder.Services.AddSensitiveFlowWeb(options =>
+{
+    options.UseProfile(SensitiveFlowProfile.Balanced);
+    options.UseEfCoreStores(
+        audit => audit.UseSqlServer("Server=.;Database=Audit;..."),
+        tokens => tokens.UseSqlServer("Server=.;Database=Tokens;..."));
+
+    options.EnableEfCoreAudit();
+    options.EnableAspNetCoreContext();
+    options.EnableJsonRedaction();
+    options.EnableLoggingRedaction();
+    options.EnableValidation();
+    options.EnableHealthChecks();
+});
+
+// Wire the interceptor and middleware
+builder.Services.AddDbContext<AppDbContext>((sp, opt) =>
+    opt.UseSqlServer(connectionString)
+        .AddInterceptors(sp.GetRequiredService<SensitiveDataAuditInterceptor>()));
+
+var app = builder.Build();
+app.UseSensitiveFlow();
+app.MapHealthChecks("/health");
 ```
 
-> **Do not use an in-memory store in production.** Audit records must survive process
-> restarts. Losing audit history defeats the accountability the audit trail is meant
-> to provide.
+### 3. See the magic
 
-### 4. Register services
+Every time your code saves a customer, SensitiveFlow automatically:
 
-```csharp
-builder.Services.AddEfCoreAuditStore(options =>
-    options.UseSqlServer(builder.Configuration.GetConnectionString("AuditStorage")));
-builder.Services.AddAuditStoreRetry();
-builder.Services.AddSensitiveFlowEFCore();           // SensitiveFlow.EFCore
-builder.Services.AddSensitiveFlowAspNetCore();       // SensitiveFlow.AspNetCore
-builder.Services.AddSensitiveFlowLogging();          // SensitiveFlow.Logging
-```
+- **Creates an audit record** showing who changed what field, when, and from what to what
+- **Scrubs logs** — if a developer logs `email: {customer.Email}`, the actual email never appears in logs
+- **Masks JSON responses** — API endpoints return `"email": "[REDACTED]"` instead of the real address
+- **Tracks IP addresses** without storing them raw — instead stores a pseudonymized token
+- **Validates retention policies** — knows when data should expire or be anonymized
 
-### 5. Add the middleware
+No additional code needed. SensitiveFlow works at the infrastructure level.
 
-```csharp
-// Before UseAuthentication -- makes the pseudonymized IP token available to all handlers.
-app.UseSensitiveFlowAudit();
+### 4. Verify it works
+
+Check the `/health` endpoint to ensure your audit and token infrastructure are connected:
+
+```bash
+curl http://localhost:5000/health
 ```
 
-### 6. Wire the interceptor into your DbContext
+Success means your audit database is reachable and healthy.
 
-```csharp
-optionsBuilder.AddInterceptors(app.Services.GetRequiredService<SensitiveDataAuditInterceptor>());
+### Schema setup
+
+SensitiveFlow does not create tables automatically. Before your first write, create:
+
+- Your application tables (via EF Core migrations)
+- SensitiveFlow audit/token tables (via [SQL scripts](tools/migrations/) or EF Core)
+
+See [Database Providers](docs/database-providers.md) for per-provider setup and schema details. The samples create schema automatically on startup for convenience; production apps should use migrations or deployment scripts.
+
+## What You Get
+
+| Feature | Example | Why |
+|---------|---------|-----|
+| **Audit trail** | `AuditRecord: salary changed from $50k → $60k by [email protected] at 2025-06-15T10:30:00Z` | Know who changed sensitive data and when; essential for investigation and accountability |
+| **Log redaction** | `logger.LogInformation("Processing user: {Email}", email)` → logs show `Email: [REDACTED]` | Prevent accidental leaks in logs, error messages, monitoring systems, or backups |
+| **JSON masking** | `GET /api/employee/123` → `{"name":"John","salary":"[REDACTED]","ssn":"[REDACTED]"}` | API responses never expose sensitive fields even if you forget to scrub them in code |
+| **IP pseudonymization** | IP `203.0.113.42` → token `ip_token_xyz` in audit logs; recoverable via token store | Audit shows request origin without storing raw IPs in plaintext |
+| **Retention policies** | `[RetentionData(Years=7, AnonymizeOnExpiration)]` → field automatically anonymized after 7 years | Control how long sensitive data stays in your systems; delete when no longer needed |
+| **Data export** | Single API call to collect all data associated with a user | Generate comprehensive reports for users; useful for transparency and debugging |
+| **Data erasure** | Schedule erasure or call API to fully remove user's data from all tables | Clean removal of user data when requested; useful for testing and cleanup |
+
+## Going Deeper
+
+Once the basics work, explore:
+
+- **[Durable Outbox](docs/audit.md#outbox)** — guaranteed audit delivery even if database is temporarily unavailable
+- **[Retry & Backoff](docs/audit.md#retry)** — automatic exponential backoff with jitter for transient failures
+- **[Profiles](docs/policies.md)** — switch from `Balanced` to `Strict` (redact everything) or `Permissive` (trust some fields)
+- **[Custom policies](docs/policies.md)** — define your own classification rules
+- **[OpenTelemetry](docs/diagnostics.md)** — emit audit events as spans and metrics for observability
+- **[Caching token store](docs/anonymization.md#caching)** — speed up pseudonymization lookups with in-memory cache
+- **[Data subject reports](docs/anonymization.md#export)** — customize what fields appear in privacy reports
+
+See the [full documentation](docs/README.md) for all features.
+
+## Samples
+
+The repository includes runnable examples. After cloning:
+
+```bash
+cd samples/QuickStart.Sample
+dotnet run
+
+cd ../WebApi.Sample
+dotnet run
 ```
 
-Every `SaveChanges` on a field annotated with `[PersonalData]` or `[SensitiveData]` now
-produces an `AuditRecord` automatically.
+Each sample demonstrates different configurations: QuickStart keeps it minimal, WebApi shows outbox and diagnostics, MinimalApi shows routing integration, Console shows standalone usage, Redis shows distributed token store.
+
+## Advanced: Package-by-package setup
+
+The composition layer (`SensitiveFlow.AspNetCore.EFCore`) is recommended. For teams needing fine-grained control over which packages to install:
+
+See [Package Reference](docs/package-reference.md) for the full setup matrix and manual registration calls for each service.
+
+> **Do not use in-memory stores in production.** Audit records and token mappings must survive process restarts.
 
 ## Documentation
 
@@ -131,7 +204,7 @@ produces an `AuditRecord` automatically.
 
 ## Design Principles
 
-- **Runtime behavior over compliance paperwork** -- instruments what actually happens, not what should happen.
+- **Observable behavior over aspirational goals** -- instruments what actually happens with sensitive data, not what should happen in theory.
 - **Explicit metadata over implicit heuristics** -- every classification is opt-in via attributes.
 - **Composition over lock-in** -- each module is optional and independently testable.
 - **Safe defaults** -- the IP address is never stored raw; the log redactor strips sensitive values before they reach any sink.