Commissioning must challenge the system at its failure boundaries

Every continuity requirement needs a test row that another competent team can repeat. The row should name the requirement, the fault class, the initial state, the active load, the exact stimulus, the expected behaviour, the acceptance limit and the retained artefact. It should also identify the person authorised to inject the fault and the person allowed to release the system afterwards. NIST contingency and recovery guidance supports this link between priorities, recovery playbooks, testing and subsequent improvement.[S05][S06]

The initial state matters because an idle test can avoid the boundary that matters. A power cut after the queue has drained does not exercise recovery of a populated outbox. A broker restart without concurrent writes does not challenge the capture transaction. A backup restore without later live events does not expose replay ambiguity. The test record should therefore freeze the starting queue depth, active sources, write pattern, software state and pending update condition before the stimulus. These are test controls, not claimed factory measurements.

Expected behaviour should be written as invariants rather than impressions. If the local application acknowledged a durable commit, the acknowledged event must survive within the declared loss allowance. If acknowledgement was still in flight, the recovered state may contain either the complete committed event or no event, but never a torn business record. If a remote receiver retries delivery, repeated transport messages must not become repeated business actions. SQLite documents atomicity as all changes in a transaction occurring or none, while MQTT documentation expressly permits duplicates under at-least-once delivery. These are different boundaries and require different checks.[S13][S19]

The fault catalogue should cover application termination, operating-system crash, controlled interruption of the digital supply, WAN loss, LAN partition, clock error, storage saturation, interrupted update and stale backup restoration. The sequence should begin with isolated software faults, progress to target-hardware tests and finish with witnessed site exercises. A requirement that cannot yet be tested remains open. It must not be marked accepted because the corresponding component appears to work during normal operation.

Exhibit 51 positions the proposed experiments by coordination burden and diagnostic coverage. The scores are illustrative planning judgements, not measurements of risk or reliability.

The matrix is a sequencing aid, not a reason to defer a high-consequence test until after acceptance. A mandatory case belongs in the acceptance plan regardless of where its point falls. Conversely, a difficult test with narrow diagnostic value should not displace simpler checks that expose the same requirement more directly.

A continuity test must not create a process hazard in order to prove a data property. NIST's operational technology guidance explicitly places performance, reliability and safety requirements alongside security.[S03] The test designer should therefore separate the protected digital enclosure from production machinery, agree the safe mechanical state and obtain the responsible electrical and operations approvals before switching any physical supply. Simulation, a test rack or an isolated cabinet may answer the question without interrupting a live process.

The stimulus must also match the claim. Opening an upstream feeder would test much more than the edge record and could invalidate the exercise. A controlled interruption at the approved digital test boundary, preceded by instrumentation and a rollback procedure, produces clearer evidence. Where the required claim concerns machinery control rather than record keeping, this chapter's method is insufficient and the responsible operational technology engineering process must govern the work.

Simulation and physical exercises are complementary because they expose different assumptions. SQLite's testing documentation states that crash testing with real power failures is impractical and describes simulation using an alternative virtual filesystem to represent the database state following a crash.[S28] That approach allows a project to repeat transaction interruption, process termination, disk errors and network failures across many controlled states. It is particularly useful before equipment reaches the site, when a defect can still be corrected without disrupting operations.

A project harness should test the application, not merely invoke the database vendor's own suite. It should interrupt capture before validation, between event insertion and outbox insertion, before commit, after commit and before the client receives acknowledgement. Each interruption should restart the actual application logic and compare the resulting records with the predeclared manifest. The harness should also repeat replay with duplicates, reordered messages, malformed payloads and another connection loss before the first replay has finished.

Simulation cannot establish what the physical device does with a flush request. SQLite's corruption guidance discusses drives that do not honour synchronisation requests and warns that the database is resistant to corruption rather than immune.[S15] The Linux ext4 documentation states that write barriers enforce on-disk ordering of journal commits when volatile write caches are used.[S29] Consequently, the target-device test must include the selected storage media, controller, filesystem mount options, operating-system image and power path. A passing virtual filesystem test says nothing by itself about those components.

Database configuration also needs to be read back rather than inferred from a deployment file. SQLite documents that a transaction committed in WAL mode with synchronous=NORMAL might roll back after power loss or a system crash, even though consistency is retained.[S16] The commissioning result must therefore distinguish a structurally consistent database from conservation of every transaction that the application acknowledged as durable. The required setting is a design decision tied to the service contract, not a universal value to copy into every system.

Target-hardware exercises should repeat abrupt interruption during active writes, exhaustion of the protected supply, orderly shutdown and cold boot with external communications absent. They should record whether storage recovers without manual repair, whether services become ready in dependency order and whether the first new event receives the promised acknowledgement. Service supervision settings such as restart and stop timeouts are configuration inputs to version and exercise, not evidence that readiness has occurred.[S21]

The division of labour is clear. Simulation searches logic and timing states at low operational risk. Hardware exercises challenge the physical write path, transfer chain and boot sequence. Site exercises add the installed supply, network and operating team. None can be replaced by a supplier statement that the underlying components were tested in a different assembly.