Skip to content
GREEN FUTURETECH.
Explore

Research libraryFactories on a Weak Grid

Chapter 06 of 14 · Part II · Draft web edition

The local event must survive the acknowledgement

The transaction boundary, storage path and acknowledgement semantics decide whether a production record survives abrupt loss.

Sections
7
Exhibits
5
Minutes
40
Sources
26
EX 22 · Append commit and sync WAL (FULL). From Insert event and outbox in 1 tx (commit). Leads to Crash before ACK: retry deduplicated (link drop), Return durable local ACK to caller (fsync ok).

The cloth is woven from this chapter's 5 exhibits: each weft is one exhibit's own data, normalised; each warp group is a section. The probe reads the exhibit's own value where it crosses. Procedural visualisation, not a measurement.

The chapter as a draft: 7 sections, 4 exhibit wefts, one shuttle.

Lead exhibit · 22FlowillustrativeThe acknowledgement follows the durable transaction
Validate event andschemaInsert event andoutbox in 1 txCrash before commit:clean rollbackAppend commit andsync WAL (FULL)Crash before ACK:retry deduplicatedReturn durable localACK to caller

Read-outValidate event and schema. Starting point. Leads to Insert event and outbox in 1 tx (valid).

Method and source

Illustrative transaction and outbox sequence with crash boundaries before insert, before commit, after commit and before client acknowledgement. Guarantees follow SQLite atomic commit (S13), WAL synchronous modes (S16) and fsync behaviour (S0603).

Unit: transaction sequence. Status: illustrative. Sources: S13, S16, S0603. Illustrative figures are teaching examples, not measured factory results.

The chapter's warpBar: share of the reading · square: an exhibit

Section 6.0 · Opening · 1 min · no exhibit of its own · one movement Go to section ↓

Overture · 1 min

A factory event is captured only when the local node has validated its envelope, written both the business event and its pending-delivery outbox record within a single atomic transaction, and confirmed that the storage stack has flushed that commit to non-volatile media before returning an acknowledgement to the operator or machine controller.[S13][S16] In an industrial environment where 93.3 percent of surveyed Cameroonian firms reported electrical outages and experienced 10.4 outages in a typical month according to the 2024 World Bank Enterprise Survey, treating an in-memory buffer write or a transport socket handoff as an accepted record guarantees silent data loss whenever power fails between the acknowledgement and the physical disk flush.[S01][S24] Green Future Tech Limited (GFT), Hong Kong SAR, argues in its Cameroon textile programme (prepared 31 January 2024, sources refreshed 2026) that closing the gap between roughly 95 percent raw cotton export and the 50 percent local-processing target by 2030 requires point-of-work records that survive the exact moment of electrical interruption.[R06][R07] This chapter specifies the local transaction contract, the difference between database consistency and power-loss durability, the single-host storage rules for write-ahead logging, the boundary between message-broker queues and application commits, the partition reserve budget for bounded retention, and the fault-injection ledger required to prove recovery before production relies on the record.[S14][S20][S28]

¶ 01 · 9 sources

In an industrial environment where 93.3 percent of surveyed Cameroonian firms reported electrical outages…S01, S24

S13S16S01S24R06R07S14S20S28

6.13 min · 1 sub-steps · read with a related exhibit

An event envelope should carry enough context to recover

A bare production value such as a sewing bundle count, a loom pick total or a dye-bath temperature reading cannot be reconciled after an unscheduled reboot unless the event carries its own recovery metadata from the instant of creation.[S31][S32] When an edge gateway or line terminal loses mains supply and restarts without wide-area network reachability, its local operating system may come up with an unsynchronised real-time clock, a reset process counter and an unconfirmed transmission queue.[S03][S22] If the capture application relies on a central cloud database to assign auto-increment primary keys or ingestion timestamps, local operation either blocks completely during a backhaul outage or generates colliding local integers that overwrite history when connectivity returns.[S31] Data governance ownership, schema lifecycle approval and long-term archival retention rules belong to the wider governance framework in paper wp14; this chapter defines the nine mandatory envelope fields required at the local capture boundary so that any interrupted node can detect gaps, reject corrupted payloads and deduplicate replayed records without external assistance.[S13][S31]

6.2 Nine recovery fields separate replay from new production

6.2

Every production event written to the local store should be wrapped in a fixed contextual envelope before the database transaction begins.[S13][S31] Rather than trusting transport headers that disappear across protocol bridges, the capture service embeds nine fields directly into the committed record:[S19][S31]

¶ 02 · 3 sources

S13S31S19

  1. Event identifier (event_id): A locally generated 128-bit Universally Unique Identifier (UUID) formatted according to RFC 9562, requiring no central registration authority or online coordinator.[S31] Using time-ordered UUID version 7 (UUIDv7), which places a 48-bit Unix Epoch millisecond timestamp in the most significant bits followed by version, variant, optional monotonic counter and pseudorandom bits, preserves B-tree index locality during high-rate inserts while preventing identifier collisions across disconnected edge nodes.[S31]
  2. Source identifier (source_id): A stable, provisioned identifier naming the specific physical capture boundary, such as a garment bundle scanner station, a spinning-frame programmable logic controller bridge or a supervisor tablet assigned to a named production line.[S03][S31]
  3. Boot identifier (boot_id): A 128-bit identifier generated fresh each time the operating system kernel or capture daemon initialises, allowing downstream reconciliation to see immediately whether two events with low sequence numbers came from the same continuous run or sit on opposite sides of a power reset.[S22][S31]
  4. Per-source sequence number (source_seq): A strictly increasing 64-bit integer maintained per source_id and boot_id pair (or persisted across restarts in the local transaction store), which exposes missing records as arithmetic gaps even when the local clock jumps backward or forward during recovery.[S13][S32]
  5. Source event timestamp (event_time_utc): The instant at which the physical observation or operator action occurred, formatted under the RFC 3339 profile of ISO 8601 with an explicit Coordinated Universal Time (UTC) offset (Z or numeric offset), never an unqualified local wall-clock string.[S32]
  6. Local ingest timestamp (ingest_time_utc): The RFC 3339 UTC timestamp recorded when the local database transaction accepts the event, preserving the delay between peripheral sensing and local persistence whenever a scanner or controller buffers a reading briefly before handoff.[S13][S32]
  7. Clock quality indicator (clock_quality): An enumerated state flag recording whether the local system clock at event_time_utc was locked to an authenticated or verified local reference, running in bounded holdover after reference loss, or unsynchronised following a cold boot without battery-backed time.[S03][S32]
  8. Schema version (schema_version): An explicit integer or semantic version tag identifying the structure and engineering units of the payload so that queued offline events remain parseable even if an application update occurs before the backlog drains.[S05][S13]
  9. Payload integrity digest (payload_sha256): A cryptographic checksum or cyclic redundancy check computed over the canonical serialised payload bytes and key envelope headers at capture time, enabling the local store and remote consumer to distinguish application-level payload corruption from valid zero-value readings.[S13][S15]

Embedding these nine attributes inside the local transaction record means that an operator scanning a cut-work bundle on a sewing line in Douala or Garoua produces a self-describing evidence unit.[S03][S31] If a voltage dip restarts the station three seconds later, the next event carries a new boot_id and a verifiable source_seq continuity trail, allowing the performance-management system (PMS) proposed in GFT's Cameroon textile programme to separate genuine production stops from instrumentation restarts.[S03][S22]

¶ 01 · 3 sources

S03S31S22

6.34 min · 2 sub-steps · 1 exhibit

Write the event and outbox in one transaction

Capturing an event locally and queuing it for upstream transmission are two views of the same operational commitment, so they must succeed or fail as a single atomic database transaction.[S13][S0602] In SQLite, atomic commit guarantees that either all database modifications within a single transaction occur together or none of them occur, making writes across multiple tables appear instantaneous and simultaneous even when interrupted by an operating-system crash or power failure.[S13] When the local capture service writes the validated event into the local_events ledger table and simultaneously inserts a delivery pointer into the outbox_queue table inside one BEGIN IMMEDIATE ... COMMIT block, the node eliminates any state in which an event is recorded locally but forgotten by the uplink forwarder, or forwarded upstream while absent from the local shift ledger.[S13][S0602]

6.4 Dual writes open a split-brain window between storage and transport

6.4

Many industrial integration scripts violate atomicity by executing two separate actions: they insert a record into a local file or database and then publish a message over a Transmission Control Protocol (TCP) socket or message-broker client before returning success to the station.[S19][S24] That dual-write pattern creates two distinct failure windows under intermittent power.[S13][S15] If the process writes to the local database first and loses power one millisecond before publishing to the broker, the record exists on disk, but nothing in the communication path knows that it still needs to be transmitted once the wide-area network recovers.[S08][S13] Conversely, if the process publishes the event to a network socket first and crashes before the local database commit completes, the remote dashboard or downstream enterprise service may record a bundle completion that is missing from the local supervisor console, violating the single-record principle required for shop-floor accountability.[S03][S13]

¶ 02 · 6 sources

S19S24S13S15S08S03

Nor does a successful socket write prove that a remote or local peer has stored the event.[S24] As RFC 9293 specifies, TCP provides a reliable, in-order byte-stream service between network endpoints, not an application-layer transaction guarantee.[S24] A send() or write() call on a TCP socket returns as soon as bytes are copied into the local kernel socket buffer, long before the receiving application has parsed the payload, validated the schema or flushed a database transaction to non-volatile storage.[S24][S0603] By replacing dual writes with a single local database transaction that writes both local_events and outbox_queue, the edge node decouples point-of-work acknowledgement from network state.[S08][S13] An independent local forwarder process then reads committed rows from outbox_queue where delivery_state = 'PENDING', transmits them when connectivity permits, and updates delivery_state = 'ACKNOWLEDGED' only after receiving an explicit application-level commit receipt containing the matching event_id.[S12][S13]

¶ 03 · 5 sources

As RFC 9293 specifiesS24

S24S0603S08S13S12

6.5 Unacknowledged commits require deterministic client retry handling

6.5

Even with an atomic local transaction, a narrow failure window remains between the instant the storage device completes the commit and the instant the calling peripheral or operator terminal receives the local acknowledgement packet.[S12][S13] Suppose a barcode terminal at an inspection table submits event_id = 0192dce0-0002 to the local edge service, the SQLite transaction commits durably to flash storage, and the local access switch or terminal loses power before the HTTP or local inter-process response arrives.[S12][S13] When the terminal recovers, it cannot tell from the missing response alone whether the transaction aborted before commit or failed after commit.[S12][S13]

¶ 04 · 2 sources

…inspection table submits event_id = 0192dce0-0002 to the local edge serviceS12, S13

S12S13

If the terminal generates a brand-new random identifier on retry, the local database will insert a second row for the same physical garment bundle, inflating shift output.[S19][S31] To close this ambiguity, the peripheral or station client must retain the deterministic event_id (or the deterministic tuple of source_id, boot_id and source_seq) across its retry loop.[S12][S31] When the edge capture service receives the retried submission, its transaction checks the unique index on event_id (and on (source_id, boot_id, source_seq)).[S13][S31] If a row with identical payload_sha256 is already committed in local_events, the service does not insert a duplicate row or a duplicate outbox_queue entry; it simply returns a durable local acceptance receipt citing the original ingest_time_utc, making the local capture endpoint idempotent in the sense defined by RFC 9110.[S12][S13] If a retried event_id arrives with a different payload_sha256, the service logs an explicit integrity conflict rather than overwriting the prior record.[S13][S15]

¶ 05 · 5 sources

If a row with identical payload_sha256 is already committed in local_eventsS12, S13

S19S31S12S13S15

The transaction sequence in Exhibit 22 illustrates where crash points fall across validation, atomic insertion, write-ahead log synchronisation and caller acknowledgement.[S13][S16]

¶ 06 · 2 sourcesExhibit 22

S13S16

As Exhibit 22 shows, any interruption prior to the completion of the write-ahead log synchronisation step (n4) rolls back both the event and the outbox row cleanly upon reboot, leaving zero orphaned state.[S13][S16] Any interruption after n4 but before n6 preserves the committed event and outbox row on disk, while the idempotent retry rule on event_id prevents duplicate insertion when the caller resends the unacknowledged frame.[S12][S13] What this flow diagram cannot establish on its own is whether the underlying hardware honours the synchronisation call at step n4; that requires explicit database pragma configuration and storage verification.[S15][S16]

¶ 07 · 4 sources

…the write-ahead log synchronisation step (n4) rolls back both the event and the outbox…S13, S16

Exhibit 22

S13S16S12S15

6.64 min · 2 sub-steps · read with a related exhibit

Consistency does not guarantee the latest commit is durable

Related · chapter 097 paragraphs step it

Related exhibit 38 · chapter 09FlowillustrativeAn interrupted update returns to a known-good service
1. Stage bundle to /opt/gft/models/staging2. fsync + verify SHA-256 and signature3. Shadow warm-up on 50 golden frames4. Atomic symlink switch to Slot B (trial)5a. Pass 500-frame gate:bless Slot B5b. Fault or p99 breach:revert to Slot A

Read-out1. Stage bundle to /opt/gft/models/staging. Starting point. Leads to 2. fsync + verify SHA-256 and signature (Transfer complete).

Method and source

Illustrative A/B staging, cryptographic verification, health-gated trial, and automatic offline rollback workflow for edge inspection models. Synthesised from NIST SP 800-193, SP 800-184, SQLite atomic commit, and systemd supervision.

Unit: qualitative sequence. Status: illustrative. Sources: S04, S06, S13, S21. Illustrative figures are teaching examples, not measured factory results.

Read this exhibit in its chapter ↗
6.6 · Introduction · paragraph 1 of 7

A database engine that recovers to a structurally consistent state after a power cut can still silently discard the last several seconds of committed production transactions unless its synchronisation mode forces an input/output barrier on every commit.[S15][S16] Software teams frequently select SQLite Write-Ahead Logging (PRAGMA journal_mode=WAL) because it allows concurrent readers and writers and replaces random page rewrites with sequential log appends.[S14][S16] However, many deployments pair WAL mode with PRAGMA synchronous=NORMAL to avoid blocking the writer thread on disk flush operations, without realising that NORMAL in WAL mode sacrifices power-loss durability.[S14][S16]

6.7 Write-ahead log synchronisation modes trade speed against power-loss survival

In SQLite's traditional rollback-journal mode (DELETE, TRUNCATE or PERSIST), committing a transaction copies original unchanged pages into the rollback journal, flushes the journal to mass storage, writes modified pages into the main database file, flushes the database file, and then deletes, truncates or zeroes the header of the rollback journal to mark the commit instant.[S13][S0604] Write-Ahead Logging inverts that sequence: original pages remain untouched in the main database file while new page versions and a commit record are appended sequentially to a separate -wal file, and a later checkpoint operation transfers committed frames from the -wal file back into the main database file.[S14][S0604]

Because the -wal file is sequential, SQLite offers four distinct settings for PRAGMA synchronous: OFF (0), NORMAL (1), FULL (2) and EXTRA (3).[S16] Their exact failure boundaries under application crash versus operating-system crash or power loss are documented in the SQLite pragma specification:[S16]

  • synchronous=OFF (0): SQLite hands data to the operating system and continues immediately without issuing any fsync() or FlushFileBuffers() system call.[S15][S16] While transactions remain safe across an application-process crash if the operating system stays up, an operating-system crash or power loss before buffers reach non-volatile media will likely corrupt the database file in both rollback and WAL modes.[S15][S16]
  • synchronous=NORMAL (1): In WAL mode, SQLite synchronises the -wal file before each checkpoint, synchronises the main database file after each completed checkpoint, and synchronises the -wal header when reusing a reset log file, but it performs no sync operations during ordinary transaction commits.[S14][S16] Consequently, transactions committed in WAL mode with synchronous=NORMAL remain atomic, consistent and isolated, and survive application crashes, but a transaction committed with synchronous=NORMAL might roll back following a power loss or system crash.[S16] Between checkpoints (which default to running when the -wal file reaches 1,000 pages, or roughly 4 megabytes at the standard 4,096-byte page size), hundreds of acknowledged production events sitting only in volatile page cache can vanish cleanly on a power cut without raising a corruption error.[S14][S16]
  • synchronous=FULL (2): In WAL mode, SQLite invokes the Virtual File System xSync method (fsync() on Unix) on the -wal file after every single transaction commit in addition to the checkpoint syncs, making WAL transactions fully atomic, consistent, isolated and durable (ACID) across power loss provided the storage hardware honours the flush.[S14][S16] In rollback-journal mode, FULL prevents corruption across power loss, though on some filesystems an unlinked rollback journal directory entry can still reappear after an immediate power cut unless the containing directory is also synced.[S16][S0603]
  • synchronous=EXTRA (3): In WAL mode, EXTRA is identical to FULL.[S16] In rollback-journal DELETE mode, EXTRA adds an explicit sync of the directory containing the rollback journal after unlinking it, closing the Linux fsync(2) caveat where syncing a file descriptor does not automatically guarantee that its directory entry update has reached the disk surface.[S16][S0603]

For a factory service class that promises durable local acceptance at the moment an operator scans a bundle or logs a quality disposition, PRAGMA journal_mode=WAL with PRAGMA synchronous=NORMAL violates the contract.[S05][S16] A station screen that flashes green while its database runs WAL with synchronous=NORMAL is issuing an optimistic memory receipt, not a power-safe commitment.[S13][S16] Where the service class requires zero committed-record loss across abrupt mains failure (and where the UPS or protected circuit can still drop unexpectedly due to transfer failure or battery exhaustion), the local event store must run PRAGMA journal_mode=WAL with PRAGMA synchronous=FULL (or rollback mode with synchronous=EXTRA).[S05][S16]

6.8 Startup readback verifies effective database pragmas

Configuring durability requires more than placing a SQL string in an initialisation script, because SQLite deliberately generates no error message when an unknown or misspelled PRAGMA statement is issued; unknown pragmas are silently ignored.[S16] Furthermore, while PRAGMA journal_mode=WAL is persistent across database connections once succeeded, PRAGMA synchronous, PRAGMA foreign_keys, PRAGMA busy_timeout and PRAGMA wal_autocheckpoint are per-connection settings that revert to compile-time defaults whenever a database connection closes and reopens.[S14][S16] If a VFS lacks shared-memory primitives and the connection has not been placed into PRAGMA locking_mode=EXCLUSIVE prior to entering WAL, an attempt to set PRAGMA journal_mode=WAL silently leaves the database in its prior rollback mode and returns that prior mode string.[S14][S16]

6.106 min · 2 sub-steps · 1 exhibit

Keep a local database on a supported storage path

Step 1 of 6SQLite WAL FULL (S14, S16)

Exhibit 23 · sourcedPersistence claims have different failure boundaries

Multi-host sync (unsupported in WAL)Single-host durable commit (S13, S16 FULL)Disallowed network or volatile path (S14)Consistent or periodic-save only (S16, S20)Host coupling scope (0=network to 10=local) →Power-loss durability (0=RAM to 10=commit sync) →SQLite WAL FULL (S14…SQLite Rollback EXTRA…SQLite WAL NORMAL (S1…Mosquitto autosave (S…SQLite synchronous OF…WAL on network FS (S1…
Multi-host sync(unsupported in WAL)Single-host durable commit(S13, S16 FULL)Disallowed network orvolatile path (S14)Consistent orperiodic-save only (S16…Host coupling scope (0=network to 10=local) →Power-loss durability (0=RAM to 10=comm… →SQLite WAL FULL (S14…SQLite Rollback EXTRA…SQLite WAL NORMAL (S1…Mosquitto autosave (S…SQLite synchronous OF…WAL on network FS (S1…

SQLite WAL FULL (S14, S16)

SQLite WAL FULL (S14, S16): Host coupling scope (0=network to 10=local) 8.5, Power-loss durability (0=RAM to 10=commit sync) 9. Quadrant: Single-host durable commit (S13, S16 FULL) · sourced

Database transaction guarantees hold only when the database files reside on a local storage path whose locking, shared-memory, write-ordering and sector-overwrite semantics match the assumptions of the database engine.

SQLite Rollback EXTRA (S13, S16)

SQLite Rollback EXTRA (S13, S16): Host coupling scope (0=network to 10=local) 7.5, Power-loss durability (0=RAM to 10=commit sync) 8.5. Quadrant: Single-host durable commit (S13, S16 FULL) · sourced

SQLite WAL NORMAL (S16)

SQLite WAL NORMAL (S16): Host coupling scope (0=network to 10=local) 8.5, Power-loss durability (0=RAM to 10=commit sync) 4. Quadrant: Consistent or periodic-save only (S16, S20) · sourced

Mosquitto autosave (S20)

Mosquitto autosave (S20): Host coupling scope (0=network to 10=local) 7, Power-loss durability (0=RAM to 10=commit sync) 3. Quadrant: Consistent or periodic-save only (S16, S20) · sourced

SQLite synchronous OFF (S16)

SQLite synchronous OFF (S16): Host coupling scope (0=network to 10=local) 8, Power-loss durability (0=RAM to 10=commit sync) 1. Quadrant: Consistent or periodic-save only (S16, S20) · sourced

WAL on network FS (S14 disallowed)

WAL on network FS (S14 disallowed): Host coupling scope (0=network to 10=local) 1.5, Power-loss durability (0=RAM to 10=commit sync) 1.5. Quadrant: Disallowed network or volatile path (S14) · sourced

Method and source · Exhibit 23

Source-based mapping of documented persistence boundaries from SQLite atomic commit (S13), WAL same-host shared-memory constraints (S14), PRAGMA synchronous modes (S16) and Mosquitto mosquitto.db persistence (S20). Coordinates represent ordinal scope indices (0 to 10) for visual placement; exact source rules and GFT acceptance checks are tabulated in the text.

Unit: ordinal boundary index (0 to 10). Status: sourced. Sources: S13, S14, S16, S20.

6.10 · the bodyExhibit 23 above; its figures are illustrative where marked.

Database transaction guarantees hold only when the database files reside on a local storage path whose locking, shared-memory, write-ordering and sector-overwrite semantics match the assumptions of the database engine.[S13][S15] Placing an embedded transactional store on a convenience network share or on unverified consumer flash media can defeat even PRAGMA synchronous=FULL.[S14][S15]

6.11 Shared-memory indexing rules out network filesystems for local WAL capture

6.11

Plant integrators sometimes attempt to centralise resilience by mounting a Network File System (NFS) or Server Message Block (SMB) share from a back-office storage appliance onto multiple line-side terminals and pointing each terminal's SQLite database connection at the network share.[S14][S15] SQLite's documentation rules out this architecture explicitly: all processes using a WAL database must run on the same host computer, and WAL does not work over a network filesystem.[S14] To allow readers to locate page versions in the -wal file without scanning the entire log on every query, SQLite maintains a wal-index in a memory-mapped -shm file in the same directory as the database.[S14][S0604] Processes running on separate physical machines cannot share that memory mapping, and network filesystems frequently contain locking bugs or cache-coherency delays that break POSIX advisory locks and corrupt the database.[S14][S15]

¶ 02 · 3 sources

S14S15S0604

Even on a single host, concurrency boundaries require disciplined process design.[S14][S0602] In SQLite, writes are serialised so that only a single writer can modify a database at a time.[S14][S0602] While multiple local processes can open a WAL database concurrently, multi-connection write and checkpoint coordination introduces avoidable complexity and historical corner cases.[S14][S15] For example, on 3 March 2026 the SQLite team documented and fixed the WAL-reset race condition (present in versions 3.7.0 from July 2010 through 3.51.2 from January 2026, and fixed in 3.51.3 on 13 March 2026 with backports to 3.44.6 and 3.50.7), in which two separate connections concurrently writing and checkpointing at a tight timing boundary could leave a wal-index header field stale and cause a subsequent checkpoint to skip part of a committed transaction.[S14][S15] By confining all writes and checkpoints for the local event store to a single dedicated capture daemon process on the local host (or running a single-process connection with PRAGMA locking_mode=EXCLUSIVE, which also allows WAL to operate without shared memory if only one process accesses the file) and upgrading SQLite to at least version 3.51.3, the edge architecture eliminates cross-process write races entirely.[S14][S0602]

¶ 03 · 3 sources

on 3 March 2026 the SQLite team documented and fixed the WAL-reset race condition…S14, S15

S14S0602S15

Equally important, the main database file, its -wal write-ahead log, its -shm shared-memory index and any hot rollback -journal file form an indivisible state set.[S14][S15] If an administrator or faulty cleanup script deletes, renames or separates a hot -journal or -wal file from its database file after a power failure, SQLite cannot play back or recover the interrupted state on the next sqlite3_open(), resulting in silent transaction loss or immediate database corruption.[S14][S15]

¶ 04 · 2 sources

SQLite cannot play back or recover the interrupted state on the next sqlite3_open()S14, S15

S14S15

6.12 Filesystem barriers and flash controllers sit beneath database guarantees

6.12

Below the SQLite Virtual File System layer sit the operating-system filesystem and the physical mass-storage controller.[S13][S29] SQLite assumes by default (since version 3.7.9 in November 2011) that the underlying storage provides the powersafe overwrite property: when an application writes a range of bytes in a file, no bytes outside that range will change even if a crash or power failure occurs just before or during the write.[S13][S0601] On flash media where the minimum erasable block or sector (typically 4,096 bytes or larger) exceeds the byte range being modified, a controller lacking powersafe overwrite can suffer collateral sector damage during an interrupted writeback, corrupting adjacent database pages that were not even being modified.[S15][S0601] Even worse, SQLite's corruption guide notes that some low-cost flash memory controllers use wear-levelling logic that can inflict random filesystem damage across unrelated files if power is cut during any write, and that consumer USB flash sticks frequently report fsync() completion while data still sits in volatile controller RAM (visibly indicated by an LED that continues flashing for several seconds after COMMIT returns).[S15]

¶ 05 · 4 sources

…assumes by default (since version 3.7.9 in November 2011) that the underlying storage provides…S13, S0601

S13S29S0601S15

On Linux edge computers, the filesystem mount configuration must preserve write ordering down to the block device.[S29] The Linux kernel ext4 documentation explains that ext4 enables write barriers (barrier=1) and data=ordered mode by default.[S29] In data=ordered mode, ext4 forces all modified file data blocks out to the main filesystem prior to committing the associated metadata transaction to the journal, and write barriers enforce proper on-disk ordering of journal commits so that volatile disk write caches remain safe to use.[S29] By default, ext4 commits running metadata transactions every commit=5 seconds, whereas an explicit fsync() call from SQLite (synchronous=FULL) forces an immediate flush and barrier wait on the file descriptor.[S29][S0603] If an integrator mounts the partition with nobarrier (barrier=0) without a battery-backed storage controller, or uses data=writeback where metadata can commit before data blocks, an abrupt power loss can reorder writes across journal commits and defeat database recovery.[S15][S29]

¶ 06 · 3 sources

The Linux kernel ext4 documentation explains that ext4 enables write barriers (barrier=1) and data=ordered…S29

S29S0603S15

The source-based matrix in Exhibit 23 maps these documented persistence configurations across two operational dimensions: host coupling scope and power-loss durability.[S13][S14][S16][S20]

¶ 07 · 4 sourcesExhibit 23

S13S14S16S20

As Exhibit 23 shows, only single-host local configurations with explicit per-commit synchronisation (SQLite WAL FULL and SQLite Rollback EXTRA) occupy the upper-right quadrant where committed records survive abrupt power loss.[S13][S14][S16] Configurations in the lower-right quadrant (SQLite WAL NORMAL and Mosquitto autosave) remain on a supported local host but can lose recent committed events or queued messages when power drops between checkpoints or autosave intervals.[S16][S20] Placing WAL on a network filesystem sits in the lower-left disallowed zone because shared-memory indexing and network locking fail across hosts.[S14][S15] Because a two-dimensional plot uses ordinal placement indices (0 to 10) to visualise relative boundaries, the exact documented source statements and the corresponding GFT procurement and commissioning questions are set out in the comparison table below.[S13][S14][S16][S20]

¶ 08 · 5 sources

Because a two-dimensional plot uses ordinal placement indices (0 to 10) to visualise relative boundariesS13, S14, S16, S20

Exhibit 23

S13S14S16S20S15

Mechanism and sourceDocumented source behaviourFailure boundary under abrupt power lossGFT acceptance question (proposed control)
SQLite rollback journal with synchronous=EXTRA [^S13] [^S16] [^S0603]Original pages are flushed to a rollback journal before main database writes; EXTRA also syncs the containing directory after unlinking the journal [^S13] [^S16]Atomic, consistent, isolated and durable (ACID) across power loss if the storage device honours flush and powersafe overwrite [^S16] [^S0601]Does startup readback confirm synchronous=3 when rollback mode is selected, and is directory sync verified on the target filesystem? [^S16] [^S0603]
SQLite WAL mode on local disk [^S14]Appends changes to a -wal file and uses a memory-mapped -shm wal-index; requires all processes to run on the same host computer [^S14]Never supported over NFS or SMB network filesystems; requires SQLite 3.51.3 or later (or single-connection access) to avoid the multi-connection WAL-reset race [^S14] [^S15]Are the database, -wal and -shm files on a local block device with a single writer process running SQLite 3.51.3 or newer? [^S14]
SQLite WAL with synchronous=NORMAL [^S16]Syncs the -wal file before checkpoints and the main database after checkpoints, but omits fsync on ordinary transaction commits [^S14] [^S16]Preserves structural consistency across power loss, but transactions committed since the last checkpoint can roll back after power loss or hard reboot [^S16]Has the operator explicitly restricted NORMAL to non-critical diagnostic stores where post-crash rollback of recent events is acceptable? [^S05] [^S16]
SQLite WAL with synchronous=FULL [^S16]Issues an extra xSync (fsync) of the -wal file on every transaction commit in addition to checkpoint syncs [^S14] [^S16]Full ACID durability across power loss, bounded below by filesystem write barriers and honest hardware flush completion [^S15] [^S16] [^S29]Does the capture service verify journal_mode=wal and synchronous=2 on every connection open before issuing local acknowledgements? [^S16]
Linux ext4 mount and block device [^S15] [^S29] [^S0601]ext4 defaults to data=ordered, barrier=1 and commit=5 seconds; consumer flash sticks and non-powersafe controllers can lie about fsync or corrupt adjacent sectors [^S15] [^S29]If nobarrier is set without battery-backed cache or if flash media lies about flush completion, out-of-order writes can corrupt the database on power cut [^S15] [^S29]Does /proc/mounts confirm barrier=1 and data=ordered on industrial solid-state storage qualified via physical power-cut tests? [^S28] [^S29]
Eclipse Mosquitto broker persistence [^S19] [^S20]persistence defaults to false; when true, state is written to mosquitto.db on exit, SIGUSR1 or every autosave_interval (default 1,800 seconds) [^S20]In-memory queued messages and session updates accumulated since the last mosquitto.db save are lost if power fails before the next periodic save [^S20]Does the architecture rely on a transactional SQLite outbox for durability rather than assuming an MQTT PUBACK flushed mosquitto.db? [^S13][^S20]

6.134 min · 2 sub-steps · read with a related exhibit

Broker persistence is a separate configuration decision

In an edge architecture that uses Message Queuing Telemetry Transport (MQTT) to decouple local station publishers from local consumers or upstream bridges, an MQTT protocol acknowledgement is not proof that a message has been synchronously committed to non-volatile disk.[S19][S20] Treating a local message broker as a drop-in substitute for a transactional database confuses transport-layer session state with durable ledger storage.[S13][S19]

6.14 Periodic broker saves leave an unpersisted memory window

6.14

The OASIS MQTT Version 5.0 standard defines three Quality of Service (QoS) levels for message delivery between a client and a server (broker): QoS 0 (At most once, where messages are delivered according to the best efforts of the operating environment and message loss can occur), QoS 1 (At least once, where messages are assured to arrive via a PUBLISH and PUBACK exchange but duplicates can occur), and QoS 2 (Exactly once, where a four-part PUBLISH, PUBREC, PUBREL and PUBCOMP handshake assures that the message arrives once at the protocol layer).[S19] When a client connects with a persistent session (Clean Start set to 0 and a non-zero Session Expiry Interval in MQTT v5.0, or clean_session=False in MQTT v3.1.1), the specification requires the server and client to store session state so that undelivered QoS 1 and QoS 2 messages can be retried across network reconnections.[S19][S20]

¶ 02 · 2 sources

The OASIS MQTT Version 5.0 standard defines three Quality of Service (QoS) levels for…S19

S19S20

However, how and when a broker writes that session state to non-volatile disk is an implementation and configuration choice, not an automatic per-message fsync().[S20] The official mosquitto.conf manual page for Eclipse Mosquitto reveals three critical defaults that surprise teams who have not read the documentation:[S20]

¶ 03 · 1 source

S20

  1. Persistence is disabled by default: The persistence boolean option defaults to false, meaning that unless explicitly configured (persistence true with a valid writable persistence_location), Mosquitto stores all connection, subscription and queued message data in volatile memory only.[S20]
  2. Disk saves are periodic, not per-message: Even when persistence true is enabled so that state is written to mosquitto.db, Mosquitto writes the in-memory database to disk only when the broker closes cleanly, when it receives a SIGUSR1 signal, or at periodic intervals governed by autosave_interval.[S20] By default, autosave_interval is 1,800 seconds (30 minutes) when autosave_on_changes is false (or 1,800 accumulated subscription, retained-message and queued-message changes when autosave_on_changes is true).[S20]
  3. Per-client broker queues are bounded and drop silently on overflow: For disconnected persistent clients, Mosquitto holds at most max_queued_messages (defaulting to 1,000 QoS 1 or QoS 2 messages above those in flight) and max_queued_bytes (defaulting to 0), and queue_qos0_messages defaults to false.[S20] As the manual states, once the max_queued_bytes or max_queued_messages limit is reached, subsequent outgoing messages for that client are silently dropped.[S20] Similarly, if memory_limit is exceeded while an incoming message is processed, the message is dropped and the publishing client is disconnected.[S20]
List · 1 source

S20

Consequently, if a sewing-line terminal publishes a QoS 1 event to a local Mosquitto broker, receives a PUBACK packet in two milliseconds, and deletes its own copy of the event, an unscheduled power failure ten minutes later (before the 1,800-second autosave_interval elapses or when SIGKILL / abrupt mains loss prevents a clean broker exit) erases the message from the broker's volatile RAM.[S19][S20] Even if autosave_interval is lowered, rewriting the entire mosquitto.db snapshot file is an batch checkpoint mechanism, not an atomic per-event transaction log.[S13][S20]

¶ 04 · 3 sources

if a sewing-line terminal publishes a QoS 1 event to a local Mosquitto brokerS19, S20

S19S20S13

6.15 Transport quality of service does not equal application ledger durability

6.15

A second boundary error occurs between the broker and the downstream consumer application.[S19][S24] Suppose a local broker delivers a QoS 1 or QoS 2 message to a local PMS ingestion worker or an upstream cloud bridge.[S19] If the consumer client library sends PUBACK (or completes PUBCOMP) automatically upon invoking the application callback in memory, before the application has committed the event into its target database, a crash inside the consumer process loses the event permanently because the broker has already deleted its queued copy upon receiving the protocol acknowledgement.[S13][S19]

¶ 05 · 3 sources

…local broker delivers a QoS 1 or QoS 2 message to a local PMS…S19

S19S24S13

To maintain an unbroken durability chain across producer, broker and consumer, the architecture must enforce three separate boundaries:[S08][S13][S19]

¶ 06 · 3 sources

S08S13S19

  • Producer durability boundary: The station or edge capture service commits the event and its outbox_queue row into local SQLite (WAL, synchronous=FULL) before acknowledging the station action.[S13][S16] The producer's MQTT publisher reads from outbox_queue and retains the row in 'PENDING' state across broker restarts.[S08][S13]
  • Broker transport role: The local MQTT broker acts as a pub/sub router for live local subscribers (such as supervisor displays and maintenance alarm panels) with persistence true and bounded queue settings configured to aid fast session resumption, but the broker's mosquitto.db is never treated as the sole system of record for unforwarded production events.[S03][S20]
  • Consumer acceptance boundary: Downstream consumers disable automatic client-library PUBACK generation, execute an idempotent database transaction keyed by event_id (INSERT ... ON CONFLICT (event_id) DO NOTHING), commit that transaction durably, and emit both the protocol acknowledgement and an application-level receipt message (allowing the producer's outbox_queue to mark the event_id as 'ACKNOWLEDGED') only after the consumer commit succeeds.[S12][S13][S19]

6.176 min · 2 sub-steps · 2 exhibits

Full storage must trigger a visible and bounded policy

Step 1 of 7Raw payload (C02)

Exhibit 24 · illustrativeRetention must reserve space beyond event payload

01,0002,0003,0004,000720Raw payload(C02)+288Primary anddedup indexes+216Transactionaloutbox table+128WAL andcheckpoint he…+256Systemdjournal cap (…+1,352Online backupstaging (S17)+640Free-spacesafety reserve3,600Totalpartition bud…02,0004,000Raw payload (C02)720Primary and…+288Transactional…+216WAL and…+128Systemd…+256Online backup…+1,352Free-space…+640Total…3,600

Raw payload (C02)

Raw payload (C02): total 720. Running total 720 megabytes (MB, decimal) · illustrative

Persistent local storage protects the record during an outage only until the storage partition fills up…

Primary and dedup indexes

Primary and dedup indexes: adds 288. Running total 1,008 megabytes (MB, decimal) · illustrative

Transactional outbox table

Transactional outbox table: adds 216. Running total 1,224 megabytes (MB, decimal) · illustrative

Systemd journal cap (S22)

Systemd journal cap (S22): adds 256. Running total 1,608 megabytes (MB, decimal) · illustrative

Online backup staging (S17)

Online backup staging (S17): adds 1,352. Running total 2,960 megabytes (MB, decimal) · illustrative

Free-space safety reserve

Free-space safety reserve: adds 640. Running total 3,600 megabytes (MB, decimal) · illustrative

Total partition budget

Total partition budget: total 3,600. Running total 3,600 megabytes (MB, decimal) · illustrative

The method behind it: Worked example.

Method and source · Exhibit 24

Illustrative storage allocation starting from the 720 MB raw event payload baseline (C02: 50 sources, 1 event/s, 500 bytes, 8 h) and adding illustrative allowances for B-tree indexes, outbox rows, WAL and rollback journals (S14), system journals (S22), online backup staging (S17) and free-space reserve. Site sizing requires measured overhead.

Unit: megabytes (MB, decimal). Status: illustrative. Sources: S08, S14, S17, S22. Illustrative figures are teaching examples, not measured factory results.

6.17 · the bodyExhibit 24 above; its figures are illustrative where marked.

Persistent local storage protects the record during an outage only until the storage partition fills up; an unbounded queue or an unthrottled diagnostic log on a shared disk will eventually cause the database engine to fail with SQLITE_FULL and halt local capture.[S14][S22] Sizing the local disk solely from the raw byte length of the event payload ignores five additional consumers of non-volatile space that grow alongside the event table during an extended disconnection.[S14][S17][S22]

6.18 Checkpoints, journals and backups compete with raw event payloads

6.18

In the paper's calculation register (C02, examined further for uplink drain dynamics in Chapter 7), an illustrative site with 50 event sources generating 1 event per second per source at 500 bytes per event over an 8-hour (28,800-second) offline interval produces 1,440,000 events, or 720,000,000 bytes (720 decimal megabytes, 0.72 decimal gigabytes) of raw event payload.[S08] On an actual local filesystem, committing and maintaining those 1,440,000 events requires substantial additional space across six distinct structures:[S14][S17][S22][S0604]

¶ 02 · 5 sources

an illustrative site with 50 event sources generating 1 event per second per source…S08 · illustrative

S08S14S17S22S0604

  1. Primary and secondary B-tree indexes: Enforcing idempotency and sequence gap detection requires a unique index on event_id (128-bit UUID) and a composite index on (source_id, boot_id, source_seq) plus status indexing on outbox_queue.[S13][S31] Together with B-tree page fill factors and cell headers, index pages add an illustrative 35 to 45 percent above the raw payload size.[S14][S31]
  2. The transactional outbox table: Until the wide-area network recovers and downstream receipts clear the backlog, outbox_queue holds a row for every pending event (storing event_id, source_id, retry count, next-attempt timestamp and delivery state), adding an illustrative 150 bytes per event.[S08][S13]
  3. Write-ahead log (-wal) and temporary files: By default, SQLite checkpoints the -wal file automatically at 1,000 pages (about 4 megabytes), but it does not truncate the -wal file after a checkpoint unless PRAGMA journal_size_limit is configured, because overwriting an existing file is faster than appending.[S14][S16] Furthermore, if a poorly designed local dashboard or backup script holds a long-running read transaction open, checkpoint starvation prevents the checkpointer from resetting the -wal file, causing -wal to grow without bound until all readers close their transactions or a RESTART / TRUNCATE checkpoint forces completion.[S14][S16] SQLite can also create up to nine kinds of temporary files (rollback journals, super-journals, -wal files, -shm shared-memory files, statement journals, TEMP databases, view materialisations, transient indices and VACUUM databases), which must have reserved headroom on their target directory.[S0604]
  4. Operating-system and service journals (systemd-journald): When a backhaul fails, network retry daemons and connection watchdogs often emit verbose error logs.[S22] Under journald.conf, when Storage=persistent is active (the default), systemd-journald stores logs under /var/log/journal with SystemMaxUse= defaulting to 10 percent of the filesystem (capped at 4 gigabytes) and SystemKeepFree= defaulting to 15 percent (also capped at 4 gigabytes), and syncs non-critical logs every SyncIntervalSec=5m (5 minutes) while syncing CRIT, ALERT and EMERG messages immediately.[S22] Although RateLimitBurst= defaults to 10,000 messages per RateLimitIntervalSec=30s (multiplied by a base-2 logarithm factor of free disk space), journald never deletes active journal files during vacuuming, and if another process fills the disk after journal files are created, journald stops growing but does not remove existing archived files to restore SystemKeepFree=.[S22]
  5. Consistent online backup staging: As the SQLite Backup API documentation (S17) and corruption guide (S15) explain, copying a live database file with an external utility while a transaction is active or without its paired -wal / -journal file risks a corrupted backup.[S15][S17] Creating a safe local snapshot using sqlite3_backup_init() / sqlite3_backup_step() or VACUUM INTO 'filename' writes a complete second copy of the active database into staging storage before the file is moved to removable or secondary media, temporarily doubling the database footprint on disk.[S15][S17]
  6. Filesystem free-space reserve: Flash translation layers and ext4 multiblock allocators degrade sharply and risk ENOSPC during metadata allocation when a partition approaches 100 percent occupancy (ext4 reserves 2 percent or 4,096 clusters by default in reserved_clusters to prevent unexpected ENOSPC during delayed allocation).[S29]

The waterfall chart in Exhibit 24 reconciles these seven components for an illustrative 8-hour offline design budget starting from the 720-megabyte raw payload in calculation C02.[S08][S14][S17][S22]

¶ 03 · 4 sources

…seven components for an illustrative 8-hour offline design budget starting from the 720-megabyte raw…S08, S14, S17, S22 · illustrative

Exhibit 24

S08S14S17S22

As Exhibit 24 demonstrates, a 720-megabyte raw event payload requires an illustrative 3,600-megabyte (3.60 decimal gigabytes) dedicated partition budget once indexes (288 MB), the transactional outbox (216 MB), WAL and temporary file headroom (128 MB), a bounded systemd-journald allocation (256 MB), a complete online backup staging slot (1,352 MB, matching the combined 1,224 MB database plus 128 MB WAL allowance) and a 640-megabyte free-space reserve are included.[S14][S17][S22] Note that this 3,600-megabyte figure still excludes high-resolution camera frames from automated optical inspection; if defect images are buffered locally, they must be placed on a separate partition governed by its own ring-buffer policy so that image storage can never starve the transactional event ledger.[S03][S05]

¶ 04 · 5 sources

the transactional outbox (216 MB)S14, S17, S22 · illustrative

Exhibit 24

S14S17S22S03S05

To make the per-event database multiplier inspectable at the record level, Exhibit 25 breaks down the derived local database footprint per committed event before backup staging and system logs.[S13][S14][S31][S32]

¶ 05 · 4 sourcesExhibit 25

S13S14S31S32

Exhibit 25 shows why provisioning disk capacity from raw JSON or binary message length alone underestimates local database growth by nearly half: adding 200 illustrative bytes of index entries (event_id and composite sequence keys), 150 illustrative bytes for the outbox_queue coordination row, and 50 illustrative bytes of B-tree page slack and WAL frame headers raises each 500-byte event to 900 bytes inside the live SQLite store (1.80 times the raw payload) even before accounting for online backup copies or operating-system logs.[S13][S14][S31]

¶ 06 · 3 sources

adding 200 illustrative bytes of index entries (event_id and composite sequence keys)S13, S14, S31 · illustrative

Exhibit 25

S13S14S31

6.20 Watermark thresholds must protect production trace records before diagnostics

6.20

When an outage extends beyond the design envelope or a downstream receiver remains unreachable, storage exhaustion must follow a pre-approved priority policy rather than a first-come, first-failed crash.[S03][S05] First, the operating system should place /var/log/journal (with explicit SystemMaxUse= and SystemKeepFree= caps in journald.conf) and any camera image spool on separate partitions or logical volumes from the /var/lib/gft-ledger SQLite directory, so a runaway daemon log or image burst cannot consume the database's free blocks.[S05][S22] Second, the capture service must monitor free partition space, database page count (PRAGMA page_count; versus PRAGMA max_page_count;) and -wal file size continuously, running periodic PRAGMA wal_checkpoint(RESTART); or PRAGMA wal_checkpoint(TRUNCATE); during brief reader gaps so that dashboard queries never cause checkpoint starvation.[S14][S16]

¶ 07 · 5 sources

S03S05S22S14S16

Third, when the high watermark is crossed, the intake controller sheds load strictly by service class: discretionary debug traces and high-frequency ambient sensor readings are dropped first, while mandatory bundle completions, quality dispositions and safety-relevant maintenance alarms retain exclusive access to the remaining database reserve.[S03][S05] Crucially, the capture service must never silently overwrite or delete unacknowledged production trace events to free space.[S03][S05] Every shed diagnostic batch and any rejected production event must increment a persistent, monotonic rejected_event_count register stored in a fixed-size summary row and display an immediate visual banner on the local supervisor console instructing operators to switch to the controlled manual continuity ledger defined in Chapter 3.[S03][S05]

¶ 08 · 2 sources

S03S05

6.214 min · 1 sub-steps · 1 exhibit

A durable record needs a demonstrated recovery path

Exhibit 269 paragraphs step it

Exhibit 26RegisterillustrativeA known event ledger makes loss and duplication visible
Event ID and seq
0192dce0-0001 (seq 401)
Local commit
Committed (FULL)
Remote commit
Committed
Replay count
0
Audit classification
Verified normal delivery

Read-outEvent ID and seq: 0192dce0-0001 (seq 401). Local commit: Committed (FULL). Remote commit: Committed. Replay count: 0. Audit classification: Verified normal delivery

Open the full register (5 rows)
0192dce0-0001 (seq 401)Committed (FULL)Committed0Verified normal delivery
0192dce0-0002 (seq 402)Committed (FULL)Committed2ACK lost; retry deduplicated
0192dce0-0003 (seq 403)Rolled back (NORMAL)Missing0Power-loss rollback after ACK
0192dce0-0004 (seq 404)Aborted pre-commitMissing1Unacknowledged; re-sent as 405
0192dce0-0005 (seq 405)Committed (FULL)Pending outbox1Retained locally for replay
Method and source

Illustrative commissioning ledger comparing deterministic source sequence numbers and UUIDs (S31), UTC timestamps (S32) and atomic commit states (S13, S16). No factory test has been executed; values illustrate reconciliation logic.

Unit: illustrative audit ledger. Status: illustrative. Sources: S13, S16, S31, S32. Illustrative figures are teaching examples, not measured factory results.

6.21 · Introduction · paragraph 1 of 9

Claiming that a local capture node is crash-safe because it uses SQLite and an industrial solid-state drive is an unverified assumption until the exact software image, pragma configuration, filesystem mount and hardware storage path are subjected to repeatable fault experiments.[S06][S28] Commissioning must verify that interrupted transactions roll back cleanly, that acknowledged transactions survive on disk, and that post-recovery replay reconciles every single expected identifier against a deterministic test ledger.[S13][S28]

6.22 Simulated fault injection and physical power cuts test different assumptions

SQLite's own testing documentation explains why library-level verification and site hardware qualification play distinct roles: SQLite's core library (155.8 thousand source lines of C code as of version 3.42.0, backed by 92,053.1 thousand lines of test code across the TCL, TH3, SLT and dbsqlfuzz harnesses) achieves 100 percent branch and Modified Condition/Decision Coverage (MC/DC), but because it is impractical to run millions of crash tests using physical power failures, SQLite's own crash testing is performed in simulation.[S28] In the TCL crash harness, a custom Virtual File System intercepts writes, spawns a child process that crashes at random I/O points, and randomly reorders and corrupts unsynchronised writes to mimic a buffered filesystem; in the embedded TH3 harness, an in-memory VFS snapshots the filesystem after a set number of I/O operations, reverts to the snapshot, injects characteristic power-loss file damage, and runs PRAGMA integrity_check to confirm that every transaction either completed or rolled back completely.[S16][S28]

That simulated VFS testing proves the correctness of SQLite's internal journalling logic under its stated hardware assumptions, but it cannot test whether a specific factory edge computer's flash controller lies about fsync(), whether its Linux kernel ext4 mount honors write barriers, or whether an application developer accidentally left PRAGMA synchronous=NORMAL or an unawaited asynchronous promise in the capture service code.[S15][S16][S28][S29] Therefore, GFT's proposed commissioning method combines two complementary test tiers before any pilot line is accepted (noting that this white paper defines the verification method and does not claim that GFT has executed these tests at a client plant):[S06][S28]

  1. Software fault-injection exercises: While running a deterministic event generator at the design rate (for example, 50 simulated sources emitting sequential UUIDv7 identifiers and known payload_sha256 digests), the test harness terminates the capture process (SIGKILL), the local broker (SIGKILL) and the operating system (echo c > /proc/sysrq-trigger or virtual machine hard reset) at four specific points in the transaction lifecycle: before INSERT, after INSERT but before COMMIT, immediately after COMMIT returns inside the server process before the HTTP/IPC response is written to the socket, and during an active WAL checkpoint or online backup (sqlite3_backup_step()).[S13][S17][S28]
  2. Qualified physical power-cut exercises: Under the supervision of the site electrical engineer and within the equipment's rated switching envelope, the target edge computer and its local storage media undergo controlled mains and DC supply interruptions while actively committing events and running checkpoints.[S03][S28] After power returns and the system boots, the test harness executes PRAGMA integrity_check;, inspects journalctl for filesystem or block-layer I/O errors (ext4 journal replay warnings or barrier failures), and reconciles the local database and downstream consumer ledger against the generator's ground-truth log.[S16][S22][S29]

To make the pass/fail criterion unambiguous, the test harness compares four sets of identifiers for every run: the generator's sent ledger, the generator's locally acknowledged ledger (every event_id for which the capture endpoint returned a durable local ACK prior to power loss), the edge node's recovered local_events and outbox_queue tables, and the remote committed ledger after replay completes.[S13][S31] The illustrative audit table in Exhibit 26 demonstrates how five representative event outcomes are classified during commissioning reconciliation.[S13][S16][S31][S32]

In Exhibit 26, five distinct behaviours become immediately visible to the commissioning engineer:[S13][S16][S31]

  • Sequence 401 (0192dce0-0001): Committed locally under synchronous=FULL, acknowledged to the station, and forwarded to the remote ledger before the fault (0 replays); this is normal baseline operation.[S13][S16]
  • Sequence 402 (0192dce0-0002): Committed durably on disk before a fault severed the response path; the station retried twice using the same UUIDv7 (2 replays), and idempotent deduplication returned the original commit receipt without creating duplicate rows in either local_events or the remote ledger.[S12][S13][S31]
  • Sequence 403 (0192dce0-0003): Illustrates a critical acceptance failure that occurs when a database is misconfigured with WAL and synchronous=NORMAL: the capture service returned a local acknowledgement to the station, power failed before the next checkpoint sync, and the committed transaction rolled back on reboot, leaving a permanent gap (seq 403 missing locally and remotely) despite a positive acknowledgement.[S16] Any occurrence of an acknowledged event_id missing after recovery fails the durability gate.[S06][S16]
  • Sequence 404 and 405 (0192dce0-0004 and 0192dce0-0005): Sequence 404 was interrupted mid-transaction before commit and never acknowledged; upon restart, the station or operator re-submitted the work (or retained the original event_id depending on peripheral buffer capability), which committed durably as 0192dce0-0005 and sits in outbox_queue awaiting backhaul restoration.[S08][S13]

Once the local capture boundary is proven to commit every acknowledged event and its outbox entry atomically to a durable, bounded local store, the next engineering problem is managing the accumulated backlog across extended backhaul outages and draining it without overwhelming the restored link, which is the subject of Chapter 7.[S08][S13]

Check the record

The chapter keeps its sources within reach.

Every knot is a source this chapter cites; select one to read its record, or filter the register by grade. Publication dates and inherited-source limits stay explicit. Read the paper's full evidence summary.

26 sources · 378 citation marks on this pageMost cited: S13, 56 times

Grade AGrade BGrade CKnot size: citations on this page

All 26 records shown

R06 · Grade C · inherited record

Cameroon Trade Hub

Inherited evidence R06

About 95 percent of cotton fibre exported raw against 5 percent processed locally; industrialisation target of 50 percent local processing.

Publication date unrecorded. Inherited source-text record. The original publication has not been independently reopened for this web edition.

Chapter source notes

S13 SQLite. 2026-10-30.

S16 SQLite. 2026-10-30.

S01 World Bank. 2025.

S24 IETF / RFC Editor. 2022.

R06 Cameroon Trade Hub. Publication date unrecorded; inherited source-text check.

R07 Ecofin Agency. Publication date unrecorded; inherited source-text check.

S14 SQLite. 2026-10-30.

S20 Eclipse Mosquitto. 2026-10-30.

S28 SQLite. 2026-10-30.

S31 IETF / RFC Editor. 2024.

S32 IETF / RFC Editor. 2002.

S03 NIST. 2023.

S22 systemd project. 2026-10-30.

S19 OASIS. 2019-03-07.

S05 NIST. 2010.

S15 SQLite. 2026-10-30.

S0602 SQLite. 2022-04-18.

S08 IETF / RFC Editor. 2007.

S0603 Linux man-pages project. 2026-09-09.

S12 IETF / RFC Editor. 2022.

S0604 SQLite. 2025-06-12.

S06 NIST. 2016.

S29 Linux kernel project. 2026-10-30.

S0601 SQLite. 2025-05-31.

S17 SQLite. 2026-10-30.

S21 systemd. 2026-10-30.

Keep the thread · chapter 06 of 14

Chapter 07 picks up the thread.

Move the shuttle along the rack to preview any chapter: its sections, exhibits and reading time. Draft web edition. This research is not investment, legal or tax advice; illustrative models are not measured factory results.

The paper's rack · 14 boltsBolt length: reading time

07Part II: Keep the local record alive

A bounded queue makes disconnection manageable

Offline retention and recovery capacity must be designed together so a restored connection can clear its accumulated work.

  • 7.1Calculate retention from event production and outage duration
  • 7.3Separate the durable source queue from the uplink spool
  • 7.5Use idempotent ingestion to make duplicates harmless
  • 7.7Queue-drain capacity must exceed live arrivals
Reading time
24 min
Exhibits
4
Sections
7
Continue to chapter 07
← Previous: chapter 05, A working local network should outlast a failed backhaulContinue the threadChapter 07 · A bounded queue makes disconnection manageable

Find a thread.

Type to search.