Skip to content

rambudithe/EdgeSync.Polly

Repository files navigation

Polly.IoT

NuGet CI License: MIT

IoT-optimized resilience strategies for .NET using Polly v8.

Standard Polly strategies are designed for HTTP and RPC workloads. IoT device connectivity has fundamentally different characteristics that make standard Polly strategies ineffective or counterproductive in edge deployments.

Polly.IoT provides strategies specifically calibrated for MQTT and IoT edge connectivity patterns.


The Problem With Standard Polly for IoT

1. Reconnect Storm Amplification

Standard retry with jitter is not calibrated for MQTT broker reconnection. When hundreds of devices reconnect simultaneously after a shared connectivity event, standard exponential backoff still concentrates reconnection attempts in ways that overwhelm brokers serving large device fleets.

Polly.IoT solution: Device-count-aware jitter that automatically scales the spread window based on fleet size.

// Standard Polly — same jitter regardless of fleet size
builder.AddRetry(new RetryStrategyOptions { UseJitter = true });

// Polly.IoT — jitter scales with device count
builder.AddMqttReconnect(new MqttReconnectOptions
{
    DeviceCount = 250 // auto-tunes: JitterFactor = 0.8
});

2. No Offline Buffer Integration

Standard circuit breaker opens when failures exceed a threshold — but in IoT edge scenarios the correct behavior when the circuit opens is to activate offline buffering, not to fail fast.

Polly.IoT solution: Direct integration with offline buffer activation via callbacks.

3. Connectivity vs Broker Failure Distinction

IoT devices need to distinguish between:

  • Network connectivity loss → buffer locally and wait
  • Broker temporarily unavailable → circuit break and retry with backoff
  • Broker rejected the connection → alert and escalate

Standard Polly treats all failures identically.


Installation

dotnet add package Polly.IoT

Quick Start

// Assume you have an EdgeSyncBuffer for offline message storage
// github.com/rambudithe/EdgeSyncBuffer

var pipeline = new ResiliencePipelineBuilder()
    .AddIoTResilience(
        circuitBreakerOptions: new IoTCircuitBreakerOptions
        {
            ConnectivityFailureThreshold = 3,
            BrokerFailureThreshold = 5,
            BreakDuration = TimeSpan.FromSeconds(30),

            // Route to offline buffer on connectivity loss
            OnConnectivityFailure = async _ =>
            {
                await buffer.SetOfflineModeAsync();
                logger.LogWarning("Device offline — buffering locally");
            },

            // Flush buffer when connectivity restored
            OnCircuitClose = async () =>
            {
                await buffer.OnConnectivityRestoredAsync();
                logger.LogInformation("Connectivity restored — flushing buffer");
            },

            // Alert on broker rejection — do not retry
            OnBrokerRejection = async ex =>
            {
                await alertService.SendAsync(
                    $"Broker rejected connection: {ex.Message}");
            }
        },
        reconnectOptions: new MqttReconnectOptions
        {
            DeviceCount = 150,        // auto-tunes jitter for 150 devices
            InitialDelay = TimeSpan.FromSeconds(1),
            MaxDelay = TimeSpan.FromMinutes(5),
            MaxAttempts = int.MaxValue // retry indefinitely

            OnReconnectAttempt = async attempt =>
                logger.LogInformation("Reconnect attempt {Attempt}", attempt)
        })
    .Build();

// Use the pipeline for all MQTT operations
await pipeline.ExecuteAsync(async ct =>
{
    await mqttClient.ConnectAsync(connectOptions, ct);
});

Device-Count-Aware Jitter

The key innovation in Polly.IoT is automatic jitter calibration based on device fleet size.

Fleet Size JitterFactor Effect
1–10 devices 0.2 Minimal spread — small fleet
11–50 devices 0.4 Moderate spread
51–100 devices 0.6 Wide spread
100+ devices 0.8 Maximum spread — prevents storm

Why This Matters

In a deployment of 250 devices, a regional connectivity event causes all devices to attempt reconnection simultaneously. Without device-count-aware jitter:

T+0s:   250 devices attempt reconnection → broker overloaded
T+1s:   Broker rejects most connections → all devices retry
T+2s:   Same storm repeats → broker instability

With Polly.IoT (DeviceCount=250, JitterFactor=0.8):

T+0s:   Devices spread reconnection across 0–4 second window
T+0-4s: 60 devices/second reconnect → broker handles load
T+4s:   All devices reconnected → no storm

Failure Classification

The IoTCircuitBreakerOptions.FailureClassifier allows custom classification of exceptions:

var options = new IoTCircuitBreakerOptions
{
    FailureClassifier = ex => ex switch
    {
        MqttCommunicationException => IoTFailureKind.NetworkConnectivity,
        MqttConnectingFailedException { ReasonCode: 135 }
            => IoTFailureKind.BrokerRejection,  // Not authorized
        MqttConnectingFailedException
            => IoTFailureKind.BrokerUnavailable,
        _ => IoTFailureKind.Unknown
    }
};

Default classification uses exception message heuristics — works with most MQTT libraries without custom configuration.


Integration With EdgeSyncBuffer

Polly.IoT is designed as a companion to EdgeSyncBuffer. They address complementary problems:

Library Responsibility
Polly.IoT Connection resilience — retry, circuit break, jitter
EdgeSyncBuffer Message persistence — offline buffer, ordered replay

Together they provide complete IoT reliability:

// EdgeSyncBuffer handles message persistence
await using var buffer = new EdgeSyncBuffer<TelemetryEvent>(
    options: new EdgeSyncOptions { PersistPath = "/var/iot/events.jsonl" },
    uploadFn: UploadToCloudAsync);

// Polly.IoT handles connection resilience
var pipeline = new ResiliencePipelineBuilder()
    .AddIoTResilience(
        new IoTCircuitBreakerOptions
        {
            OnConnectivityFailure = _ => buffer.SetOfflineModeAsync(),
            OnCircuitClose = () => buffer.OnConnectivityRestoredAsync()
        },
        new MqttReconnectOptions { DeviceCount = fleetSize })
    .Build();

Related

  • EdgeSyncBuffer — Offline-first IoT message buffer
  • Polly — .NET resilience library (dependency)
  • MQTTnet — .NET MQTT library

License

MIT — see LICENSE

By Ram Budithe — github.com/rambudithe

About

IoT-optimized resilience strategies .NET — device-count-aware jitter,MQTT circuit breaker, and offline buffer integration using Polly v8

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages