A bounded queue makes disconnection manageable

Retention begins with a byte balance at the boundary that accepts production records. For each event class, measure the number of sources, their event rates in relevant production states, the serialised payload distribution and the design offline interval. The basic payload balance is B = N x r x s x T, where N is the source count, r is events per source per second, s is bytes per event and T is the offline interval in seconds. This is deterministic accounting. It does not predict how long a site outage will last, and it must not turn a national outage frequency into a site duration assumption.

Use separate rates for normal production, start-up, alarm bursts and recovery. A single shift average can conceal the very burst that fills the queue. Where records arrive in batches, retain both the batch count and the distribution of records per batch. Where payloads vary, calculate with an agreed upper design quantile and retain the observations behind it. The operator also needs an event-count budget, because a database can reach a row, inode or transaction limit before it reaches its byte limit. The measurement boundary remains the one established in the site survey: an event that never reaches a powered source or capture service cannot be rescued by downstream storage.[S03]

The paper's illustrative payload case uses N = 50 sources, r = 1 event per source per second, s = 500 bytes per event and T = 28,800 seconds, equivalent to an illustrative 8-hour outage. The live arrival rate is 50 x 1 x 500 = 25,000 bytes per second. The accumulated payload is 50 x 1 x 500 x 28,800 = 720,000,000 bytes, or 0.72 decimal GB, representing 1,440,000 illustrative events. Every input is assumed solely to demonstrate calculation C02. The persistent-store principle supports the architecture, but it does not supply these site inputs.[S08]

7.2 Provisioned storage must exceed serialised payload

The result is a payload subtotal, not a disk specification. A durable implementation also stores event metadata, indexes, replay state and integrity fields. Its database may maintain a write-ahead log, while its broker and operating system maintain their own persistence files. SQLite's WAL documentation explains that changes are appended to a separate log before checkpointing transfers them into the main database, so the main database file alone is not the full storage footprint.[S14] The system journal also needs configured limits and free-space treatment rather than permission to compete unnoticed with the production outbox.[S22]

Do not respond by applying a generic overhead multiplier. Benchmark the intended schema and software on the intended storage. Measure a representative event mix, index definitions, checkpoint behaviour, broker persistence, backup staging and log verbosity. Then calculate provisioned capacity = measured stored bytes for the design load + recovery workspace + operating reserve. Recovery workspace matters because a full partition is a poor place to perform integrity checks, compaction or restoration. Record the test version and configuration with the result. A later schema index or logging change can invalidate an apparently conservative estimate.

Images and other large attachments require a separate budget. Their byte rates can dominate structured events, yet their retention rules may differ. Keep a structured quality decision and its attachment reference in the event model, but account for the media store, its metadata and its recovery traffic independently. If the quality authority permits sampling, the retained decision must expose that policy rather than imply that every attachment remains available. This is a proposed evidence-completeness control, not a claim that sampling is acceptable for every inspection duty.

Retention also has a time dimension. A queue can have ample free bytes while its oldest mandatory event is already too old for the decision it must support. Capacity review should therefore include bytes used, events retained and oldest unacknowledged age. The design interval ends only when remote acceptance has been restored sufficiently to release custody, not when a modem first reports a signal. Unpredictable bandwidth and lossy channels make that distinction material to recovery planning.[S18]

A sender that cannot tell whether the previous attempt committed must either risk loss by moving on or risk duplication by trying again. For mandatory production records, GFT chooses controlled retransmission and makes the receiver idempotent. Idempotency means that repeating the same identified operation has the same intended business effect as performing it once. HTTP semantics use the same basic distinction for request methods, while also warning that idempotent intent does not erase every possible side effect or log entry.[S12] The factory contract must therefore define the business state that may change, not merely label a retry safe.

MQTT does not remove this obligation. Its at-least-once quality permits duplicates, and its other delivery handshakes operate within the messaging protocol rather than proving a downstream production transaction.[S19] A broker can accept a publication, the consumer can update a ledger, and the return acknowledgement can still be lost. If the next delivery appends a second production event, the queue has transported one physical event into two business actions. Protocol delivery and business idempotency are separate acceptance boundaries.

7.6 Identity must survive every retry

The producer assigns the event identity once and preserves it through every replay. A practical envelope includes the site, source, event identifier, boot identity and source sequence. RFC 9562 states that a UUID requires no central registration process, which makes it a candidate for local generation without an online allocator.[S31] That property does not excuse weak random generation, reused counters or accidental regeneration during retry. The implementation must test identity persistence across process restart, device replacement and restoration from backup. Clock interpretation remains a separate chapter; the queue needs stable identity even when event order is disputed.

At the remote consumer, deduplication and the business mutation belong in one atomic transaction. The receiver attempts to register the event key in a processed-event table with a uniqueness constraint. If registration succeeds, the same transaction applies the production or quality mutation and prepares the receipt. If the key already exists with the same payload digest, the receiver returns the earlier committed result without applying the mutation again. Atomicity is what keeps the deduplication mark and business state from diverging.[S13]

A repeated key with a different payload is not a harmless duplicate. It is an identity conflict. The receiver should preserve both the original committed state and the conflicting evidence, place the item in a quarantine state and notify the responsible reconciliation role. Automatically overwriting the first event would destroy history. Automatically accepting the second would double-count work. Silently dropping it would hide an implementation or integrity fault.

The receipt should identify what was committed, not merely what was received. For a batch, it can return committed event identifiers or a source-sequence watermark accompanied by a count and digest. A watermark is valid only across a contiguous, fully resolved range. Priority replay and quarantined records can create holes, so the sender must not treat the highest observed sequence as proof that every earlier sequence arrived. The response to a repeated batch should reproduce the committed result, allowing a sender whose earlier receipt was lost to finish reconciliation.

Idempotency does not establish causal order. A quality decision may arrive before the production event it references when queues drain by priority. The consumer must either resolve dependencies by source sequence, defer the dependent projection or record an explicit unresolved state. It must not invent a parent record or treat arrival order as event order. These are proposed application rules; the transport standard does not supply them.