Research libraryThe Industrial Data Spine
Chapter 04 of 14 · Part I · Draft web edition
A small set of enforced checks, visible exceptions and complete lineage is more useful than an unmeasured claim that the data are clean.
The cloth is woven from this chapter's 5 exhibits: each weft is one exhibit's own data, normalised; each warp group is a section. The probe reads the exhibit's own value where it crosses. Procedural visualisation, not a measurement.
Read-out
Illustrative scoring model (0 = untested, 1 = manual spot check, 2 = entry syntax check, 3 = automated gate with quarantine, 4 = versioned gate with full lineage) across four factory record sets and five quality dimensions, derived from ISO/IEC 25012 (S03), ISO/IEC 5259-3 (S04) and W3C DQV (S0401). Values are illustrative and not a measured GFT factory result.
Unit: illustrative coverage score (0 to 4). Status: illustrative. Sources: S03, S04, S0401. Illustrative figures are teaching examples, not measured factory results.
A factory record is fit for use only when its defects, omissions and transformations are explicitly measured and preserved before the record enters a management report or a training pipeline.[S01] In a small manufacturing team, declaring a dataset clean by inspection is an operational hazard. Unchecked capture errors do not vanish when rows are aggregated into shift summaries or fed into machine-learning features; they compound silently into distorted key performance indicators (KPIs), misallocated maintenance hours and brittle model predictions.[S18] The practical remedy is not an enterprise data-cleansing department. It is a compact set of automated and steward-enforced quality gates placed at capture and ingestion, paired with explicit quality states and end-to-end provenance links connecting every derived metric or prediction back to the shop-floor events that produced it.[S04] When a record fails a rule, the industrial data spine isolates the offending event in a visible quarantine queue and routes a repair task to the originating domain steward, keeping curated tables uncontaminated while preserving raw evidence.[S09]
This discipline is foundational for Green Future Tech Limited (GFT), Hong Kong SAR, in its three-part Cameroon textile programme covering feasibility, performance-management system design and digital strategy. According to the programme's inherited evidence base, prepared on 31 January 2024 and not refreshed in 2026, with original source publication dates unrecorded, about 95 percent of Cameroon's cotton fibre is exported raw and about 5 percent is processed locally, against a national target of 50 percent local processing by 2030.[R06][R07] Across existing domestic mills, a July 2023 report cited in that inherited register recorded capacity utilisation at 33.7 percent in Garoua, 48.9 percent at CICAM 1 and 10.83 percent at CICAM 2, with underutilisation costing about CFAF 2 billion in 2020.[R12] Those sector figures describe external industrial conditions rather than GFT operating results, yet they establish why data quality cannot wait. A factory closing such utilisation gaps on tight margins cannot base capital allocation, shift scheduling, quality disposition or demand forecasting on unverified spreadsheet entries or silently patched logs.[S36]
Where earlier chapters established why trusted records matter, who holds decision rights and how operational vocabularies define shared meaning, this chapter specifies how the data spine tests whether incoming records honour those contracts.[S02] It translates international data-quality and provenance standards into seven shift controls: turning quality dimensions into grain-specific tests, enforcing validation and quarantine at capture and ingestion, separating missing and estimated values from observations, versioning rules alongside schemas, recording provenance across KPIs and pipelines, improving capture iteratively through a quality register, and halting upstream data cascades before they corrupt models.[S03][S15]
4.1
Data quality is the degree to which stated characteristics of data fulfil explicit requirements for an intended task.[S01] Standardised frameworks such as ISO/IEC 25012 categorise data-quality requirements and measures so that data can be evaluated independently from the applications that process them.[S03] Similarly, the World Wide Web Consortium (W3C) Data Quality Vocabulary emphasises that publishing consistent quality measures allows a dataset consumer to judge fitness for purpose rather than relying on an unverified claim of completeness.[S0401] For analytics and machine learning, ISO/IEC 5259-3 extends this requirement across training, validation and operational production feeds.[S04]
ISO/IEC 5259-3 extends this requirement across training
S04
Standards catalogues list over a dozen quality characteristics, but a small team scoring every theoretical dimension across every table will produce administrative theatre rather than safer decisions.[S03] Within the evaluation approach of ISO/IEC 25012, this paper selects five operational dimensions as its practical test set: validity, completeness, consistency, timeliness and uniqueness.[S03] This five-part selection is an engineering design choice for small industrial teams, not a claim that international standards mandate only these five categories.[S04]
Each dimension answers a distinct question between shop-floor capture and curated storage:
An operational quality test is executable only when it specifies the grain of evaluation, the pass condition, the summary denominator and the owner who acts when the test fails.[S02] In ISO 22400-2, manufacturing KPIs require explicit formulas, time behaviours and boundaries; quality tests on underlying inputs need the same precision.[S38] Specifying that every closed corrective work order (grain: work_order_id) must contain a valid asset identifier, fault code, repair duration in minutes and restart timestamp before closure (owner: Maintenance Steward) turns an aspiration into a deterministic gate.[S02][S05]
Before building dashboards or training models, a factory should map active rule coverage across its core domains, as illustrated in Exhibit 15.[S03][S04]
The illustrative matrix in Exhibit 15 scores each intersection from 0 (no test) to 4 (versioned gate with quarantine and lineage), adapting ISO/IEC 25012, ISO/IEC 5259-3 and the W3C Data Quality Vocabulary.[S03][S04][S0401] In this illustrative baseline, customer orders and production logs reach scores of 3 to 4 on syntax validity and uniqueness because entry templates enforce primary keys and numeric cells. Quality inspections and maintenance tickets score 1 on completeness and consistency because operators record free-text notes on clipboards and transcribe them after the shift without cross-checking asset states or lot identifiers.[S05] What Exhibit 15 cannot establish is the actual defect rate of the records; a high coverage score proves only that a test runs, not that the shop-floor process is error-free. Its operational value is to stop a team from training a model on domains where completeness and consistency remain at level 1.[S04]
ISO/IEC 5259-3 and the W3C Data Quality Vocabulary
S03, S04, S0401 · illustrative
4.4
Capture raw event
Capture raw event. Starting point. Leads to Check entry syntax. · derived
Discovering a broken identifier or impossible unit weeks later during feature engineering is the costliest way to manage factory data.
Check entry syntax
Check entry syntax. From Capture raw event. Leads to Validate contract (Valid syntax). · derived
Validate contract
Validate contract. From Check entry syntax (Valid syntax), Log provenance and reprocess (Re-evaluate). Leads to Publish curated record (Pass gate), Isolate in quarantine (Fail rule). · derived
Publish curated record
Publish curated record. From Validate contract (Pass gate). End point. · derived
Isolate in quarantine
Isolate in quarantine. From Validate contract (Fail rule). Leads to Steward correction (Assign owner). · derived
Steward correction
Steward correction. From Isolate in quarantine (Assign owner). Leads to Log provenance and reprocess (Superseding event). · derived
Log provenance and reprocess
Log provenance and reprocess. From Steward correction (Superseding event). Leads to Validate contract (Re-evaluate). · derived
Derived operational control flow linking raw capture, entry syntax check, ingestion contract gate, quarantine isolation, steward remediation and provenance-logged reprocessing, based on ISO 8000-1 (S01), ISO/IEC 5259-3 (S04) and W3C PROV-O (S09).
Unit: . Status: derived. Sources: S01, S04, S09.
4.4 · the bodyExhibit 16 above; its figures are illustrative where marked.
Discovering a broken identifier or impossible unit weeks later during feature engineering is the costliest way to manage factory data.[S18] Upper-bound estimates from semiconductor manufacturing workshops indicate that data processing for a single application can reach 70 percent of total cost when teams repeatedly clean raw logs for each downstream tool.[S39] Under ISO 8000-1 and ISO/IEC 5259-3, validation belongs at two early boundaries: syntax enforcement at capture, followed by contract validation at ingestion before curated storage.[S01][S04]
…a single application can reach 70 percent of total cost when teams repeatedly clean…
S39
At capture, whether an operator types into a tablet at a weaving beam, scans a barcode on a dye lot or configures an edge gateway polling a controller, the interface should make syntactic invalidity impossible to submit.[S06] Picklists bound to master data prevent invented style codes, unit-locked inputs prevent kilogram weights from entering as bale counts, and barcode scans eliminate transposition errors.[S05]
Entry constraints cannot catch every defect. An edge buffer flushing messages after a network outage, a laboratory batch import or a cross-record check between production and maintenance can only be evaluated at ingestion.[S08] Dropping non-conforming records destroys physical evidence, whereas writing them directly into curated tables with unread warnings contaminates downstream KPIs and training sets.[S19] The solution is a formal quarantine gate between raw capture and curated publication, shown in Exhibit 16.[S01][S04][S09]
In Exhibit 16, every incoming record first lands in an immutable raw log so shop-floor evidence is preserved.[S09] Records passing syntax checks are tested against the active contract at ingestion, verifying foreign keys, units, state transitions and uniqueness.[S04] Passing records enter the curated store; failing records enter quarantine with the failed rule ID, contract version and rejection timestamp.[S01] While Exhibit 16 defines the control path, it cannot guarantee timely resolution without an enforced steward workload allocation and escalation routine.[S02]
When a record enters quarantine, a central analyst or IT administrator must never edit the number quietly in the database to clear an error.[S02] The analyst was not on the floor and cannot know whether an anomalous figure reflects a typo, a split lot or an unlogged fault.[S19] Moreover, silent downstream repair shields the originating department from its own capture defects, guaranteeing recurrence.[S18]
Under ISO 8000-150 role accountability, each quarantine rule routes a repair task to the originating domain steward.[S02] The task cites the record ID, failed rule (for example, illustrative check CONSISTENCY_04: output of 120 units exceeds maximum speed of 95 units per hour for style ST-104), shift and asset.[S38] The steward verifies the physical tally and submits a superseding correction event citing the raw event ID, reason code and role identity, which re-enters the contract gate in Exhibit 16 before curated release.[S09]
4.7
In factory spreadsheets and poorly designed databases, a blank cell, a sensor drop and a genuine zero-output hour often collapse into 0 or an untyped NULL coerced to zero.[S04] In manufacturing, that conflation is destructive. An hour in which a ring-spinning frame produced zero kilograms of yarn because it stopped for a scheduled bearing replacement differs fundamentally from an hour in which the frame ran normally but the clerk forgot the tally, or an hour in which a power dip dropped the gateway while the machine ran on backup generation.[S38]
…zero-output hour often collapse into 0 or an untyped NULL coerced to zero
S04
To prevent absence or approximation from masquerading as measured production, every value-bearing record and time bucket must carry an explicit quality-state attribute alongside its numerical value.[S0402] Following W3C Data on the Web Best Practices on documenting quality issues and data unavailability in metadata, every record is tagged with one of five mutually exclusive states:[S0402]
OBSERVED: Captured directly from the primary instrument, scan or verified tally within the timeliness window and passed all validation gates.[S01]MISSING: An expected observation for a scheduled interval or mandatory field did not arrive by cutoff; the value field remains empty rather than defaulting to zero.[S0402]ESTIMATED: Imputed using a documented, approved rule when primary measurement was unavailable, such as deriving fabric length from beam revolutions or allocating daily steam across shifts by run-time hours.[S41]CORRECTED: Superseded by an authorised steward correction with a recorded reason code and provenance link after failing a gate or reconciliation.[S09]NOT_APPLICABLE: The asset or line was formally unscheduled or outside the measure boundary during the interval, so no observation was expected.[S38]Storing quality states at the event level helps only if those states propagate through KPI formulas, management tables and feature pipelines.[S04] When an hourly KPI engine aggregates eight shift hours under ISO 22400-2, it must compute both the metric value and the quality-state composition across contributing intervals.[S38]
Quality-state propagation rule for operational aggregates and model features
event_id or hour_bucket) to shift, daily, weekly or lot-level KPIs and feature vectors.[S38]OBSERVED only when 100 percent of scheduled constituent intervals are OBSERVED or CORRECTED with a verified audit trail.[S04]ESTIMATED if any constituent interval is ESTIMATED and none are MISSING, carrying the estimated input share.[S41]INCOMPLETE if any scheduled constituent interval is MISSING, excluded from automated model ingestion, and displaying both the partial sum and missing-interval count without substituting zero.[S0402]OBSERVED, CORRECTED, ESTIMATED, MISSING and NOT_APPLICABLE intervals sum to the exact calendar capacity of the window.[S03]Worked example
Illustrative sewing-line availability and output distortion from conflating missing logs with zero production
Consider an illustrative 8-hour shift on sewing line Line S-02 scheduled for all 8 hours (480 minutes) at a target rate of 50 shirts per hour (400 shirts per shift), evaluated under ISO 22400-2 definitions.[S38] Assume the following illustrative sequence across the 8 one-hour buckets:
status: OBSERVED), with 60 run minutes per hour (300 minutes total; 250 shirts).[S38]status: OBSERVED, code MECH_JAM).[S38]status: MISSING).[S0402]Case A: Naive pipeline (missing buckets coerced to zero) In this illustrative naive calculation, a script treats the 2 missing hourly logs as 0 run time and 0 output across 8 scheduled hours, yielding 300 run minutes out of 480 (62.5% apparent availability) and 250 shirts (31.25 shirts per scheduled hour).[S38] Management sees an illustrative availability drop to 62.5 percent and 3 hours of stoppage, triggering a false maintenance escalation, while a forecasting model learns a downwardly biased output rate.[S19]
Case B: Governed quality-state spine (missing buckets isolated) Under the governed spine, the system retains MISSING on Hours 7 and 8 and refuses to coerce them to zero:[S0402]
75.0% illustrative coverage, with 2 hours MISSING isolated in quarantine).[S04]83.3%, not 62.5%).[S38]41.67 shirts per observed scheduled hour, or 50.0 shirts per running hour).[S38]87.5% availability and 350 shirts) before publication.[S04][S38]Interlude · the cloth half-woven
Woven: sections 4.0 to 4.7Still to weave: 4.10 to 4.19
Read-out
Inherited GFT programme evidence base prepared 31 January 2024. R06 and R07 were not refreshed in 2026 and their original publication dates are unrecorded. Figures describe sector-level baseline and policy ambition, not a current GFT factory measurement or causal productivity claim.
Unit: percent of cotton fibre. Status: sourced. Sources: R06, R07.
Read this exhibit in its chapter ↗4.10
Factory operations change continuously. A mill introduces blended yarn counts, recalibrates moisture sensors before the rainy season, updates defect thresholds for a new buyer or splits a downtime code.[S38] Overwriting validation rules and code lists in place without version control makes historical records uninterpretable, causing later audits to reject valid historical rows or misclassify passes.[S04]
Under ISO/IEC 5259-3 and W3C Data on the Web Best Practices, every validation rule set, range threshold and code list must be versioned alongside the data contract it enforces.[S04][S0402] Each gate evaluation stores not only pass or fail, but the tuple (schema_version, rule_set_version, evaluated_at).[S09]
Suppose an illustrative contract fabric_roll_inspection runs under schema_v1.1 and rules_v1.3 from Month 1 to Month 4, with valid cotton twill grammage set at 180 to 220 grams per square metre (g/m2). In Month 5, the mill runs a poplin order at an illustrative 135 g/m2. Rather than mutating rules_v1.3 globally, the owner and steward publish rules_v1.4, binding grammage bounds to the style master record (style_id -> [min_gsm, max_gsm]) from an explicit timestamp.[S02][S05] Historical rolls from Month 2 remain valid under rules_v1.3, new poplin rolls validate under rules_v1.4, and training queries spanning Months 1 to 6 inspect rule-version metadata to verify how each slice was screened.[S15]
Urgent situations occasionally require relaxing a strict gate temporarily, such as when a laboratory scale interface fails mid-week or a trial lot runs before its buyer style code enters the order table.[S05] If the gate offers only a binary choice between halting capture and permanently disabling the check, operational pressure will disable the check, leaving much of the suite inactive within months.[S18]
FK_BUYER_STYLE only for illustrative trial order TR-2026-09 on Line W-01).[S04]ESTIMATED or WAIVED_PENDING_RECONCILIATION, excluding them from automated model training until reconciled.[S0402]4.13
Raw event log
Raw event log. Starting point. Leads to Quality gate v1.2. · derived
Even when every event passes a versioned gate, raw and curated events are only the start of the chain.
Quality gate v1.2
Quality gate v1.2. From Raw event log. Leads to Curated event store (Validated). · derived
Curated event store
Curated event store. From Quality gate v1.2 (Validated). Leads to KPI calculation v2.0, Dataset snapshot. · derived
Shift management review
Shift management review. From KPI calculation v2.0 (KPI path). End point. · derived
Dataset snapshot
Dataset snapshot. From Curated event store. Leads to Pipeline and model v1.1. · derived
Pipeline and model v1.1
Pipeline and model v1.1. From Dataset snapshot. Leads to Disposition decision (Model path). · derived
Disposition decision
Disposition decision. From Pipeline and model v1.1 (Model path). End point. · derived
Derived provenance graph linking raw visibility event, versioned quality gate, curated record, KPI formula execution, shift review, frozen dataset snapshot, pipeline run and model decision, based on W3C PROV-O (S09), Google Cloud MLOps lineage requirements (S15) and W3C PROV Primer (S0403).
Unit: . Status: derived. Sources: S09, S15, S0403.
4.13 · the bodyExhibit 17 above; its figures are illustrative where marked.
Even when every event passes a versioned gate, raw and curated events are only the start of the chain. Before a figure reaches a review or a model outputs a defect probability, data undergo filtering, unit conversion, time-bucket joins, KPI calculation, snapshotting and feature transformation.[S15][S38] When an unexpected number appears or a model makes a false rejection, the team must walk backward from that output to the source records, code versions and approvals that created it.[S09]
prov:Entity): Physical or digital things with fixed aspects, such as an illustrative raw visibility event (raw_event_88412), a curated hourly table (curated_hourly_prod_2026W14), a frozen training snapshot (ds_vision_train_v03), or a model binary (model_defect_cnn_v1.1).[S09][S0403]prov:Activity): Processes occurring over a time interval that act upon or generate entities, such as an ingestion validation run, a steward correction, an ISO 22400-2 KPI job, or a training pipeline run.[S09][S38]prov:Agent): Persons, roles or services bearing responsibility for an activity or entity, such as Quality Steward, Production Owner, or ingestion-worker-v2.1.[S02][S0403]A unified data spine lets management reporting and machine-learning operations (MLOps) share the same upstream provenance graph rather than maintaining disconnected silos.[S15] Recording execution metadata for every ML pipeline run supports data and artefact lineage, reproducibility and comparison.[S15] Linking pipeline metadata to curated operational identifiers lets the factory reconstruct both a management KPI and a model decision down to the same raw events, as shown in Exhibit 17.[S09][S15][S0403]
In Exhibit 17, a single observation enters the raw log, passes Quality gate v1.2 and lands in the curated store with its provenance hash.[S09] Two auditable paths branch outward: KPI calculation v2.0 produces the shift management figure under ISO 22400-2, while a frozen Dataset snapshot feeds Pipeline and model v1.1 under S15 to support a live Disposition decision.[S15][S38] While Exhibit 17 cannot prove by itself that the KPI formula or feature code is conceptually sound, it allows the custodian to query forward (wasDerivedFrom) whenever a sensor fault is found, identifying every KPI report, snapshot and model candidate that absorbed the tainted interval.[S09][S15]
passes Quality gate v1.2 and lands in the curated store with its provenance hash
S09
To keep capture lightweight, the spine logs provenance at four checkpoints: (1) ingestion from raw batch to curated partition (schema_version, rule_set_version), (2) steward correction from quarantined event to superseding event (steward_role, reason_code), (3) KPI calculation (kpi_formula_version), and (4) MLOps execution (snapshot_id, pipeline_run_id, model_version_id).[S09][S15] Standard SQL queries across those four append-only tables satisfy ISO/IEC 5259-3 and NIST AI RMF reproducibility obligations without enterprise catalogue costs.[S04][S11]
4.16
The practical alternative follows the Greenhouse Gas (GHG) Protocol's Scope 3 guidance: collecting data, assessing data quality and improving data quality is an iterative process.[S41] Applied to factory records, the team captures essential events, assesses where defects threaten high-value decisions, and invests limited stewardship hours in fixing the top recurring failure modes.[S36][S41] A typo in an optional comment has near-zero consequence, whereas a dye-lot mismatch between the slasher-dryer sheet and weaving-beam ticket breaks lot genealogy (wp05 and wp16) and corrupts shade analysis (wp03).[S08] Each iteration follows four steps inside existing reviews (wp02):[S36][S41]
MISSING, ESTIMATED and CORRECTED records by domain and rule, plus steward repair hours.[S03][S04]wp03 or wp04), or threaten compliance (wp05 or wp16).[S11][S38]To make iteration visible, the steward maintains a live quality register, shown in Exhibit 18, linking recurring quarantine patterns to affected decisions, owners and verification windows.[S02][S19][S41]
Read-out
| QR-01 Production | MISSING | Shift OEE review | 14 | Prod Steward | Enable tablet offline queue alert (14d) |
|---|---|---|---|---|---|
| QR-02 Quality | INVALID | Roll disposition | 19 | Qual Steward | Lock defect picklist to v2.1 taxonomy (10d) |
| QR-03 Maintenance | INCONSISTENT | Downtime Pareto | 11 | Maint Steward | Reconcile stop time with loom state (21d) |
| QR-04 Production | LATE | Morning line balance | 22 | Line Lead | Move tally cutoff to shift handover (7d) |
| QR-05 Orders | DUPLICATE | Dye-lot scheduling | 6 | Order Steward | Enforce unique order-line key (14d) |
| QR-06 Quality | MISSING | Vision training set | 17 | Qual Owner | Require roll barcode scan at camera (21d) |
| QR-07 Sustainability | ESTIMATED | Batch energy intensity | 8 | Sust Steward | Install sub-meter on stenter frame (45d) |
| QR-08 Maintenance | MISSING | Spare-parts reorder | 9 | Maint Steward | Require part code on ticket close (14d) |
Illustrative twelve-issue factory quality register showing issue ID and domain, quality state, affected decision, illustrative weekly recurrence count, accountable owner and structural corrective action with verification window. Issue tracking and iterative improvement structure derived from Sambasivan et al. (S19), GHG Protocol iterative quality guidance (S41) and W3C DWBP metadata disclosure (S0402). All counts and windows are illustrative.
Unit: illustrative register entries. Status: illustrative. Sources: S19, S41, S0402. Illustrative figures are teaching examples, not measured factory results.
Read-out
| QR-09 Production | INCONSISTENT | Piece-weight yield | 12 | Prod Owner | Standardise tare weight on scale (10d) |
|---|---|---|---|---|---|
| QR-10 Quality | LATE | Lab shade approval | 7 | Lab Lead | Import spectrophotometer CSV live (28d) |
| QR-11 Orders | INVALID | Delivery promise date | 5 | Order Owner | Block past-date entry on amendment (7d) |
| QR-12 Sustainability | MISSING | Effluent pH compliance | 4 | Sust Owner | Add shift-end check before drain (14d) |
Illustrative twelve-issue factory quality register showing issue ID and domain, quality state, affected decision, illustrative weekly recurrence count, accountable owner and structural corrective action with verification window. Issue tracking and iterative improvement structure derived from Sambasivan et al. (S19), GHG Protocol iterative quality guidance (S41) and W3C DWBP metadata disclosure (S0402). All counts and windows are illustrative.
Unit: illustrative register entries. Status: illustrative. Sources: S19, S41, S0402. Illustrative figures are teaching examples, not measured factory results.
The twelve illustrative rows in Exhibit 18 turn abstract dimensions into engineering tasks.[S41] Illustrative issue QR-02 logs 19 weekly quarantine events for obsolete defect codes (status: INVALID), threatening roll disposition and wp03 evaluation sets; the fix is a 10-day task locking the picklist to taxonomy v2.1.[S04] Illustrative issue QR-07 discloses that 8 weekly batch energy allocations use ESTIMATED steam splits, guiding a 45-day stenter sub-meter installation.[S41][S0402] While Exhibit 18 cannot resolve cross-departmental trade-offs automatically, tracking remediation hours exposes hidden preparation costs: eliminating an illustrative 4 hours a week of manual reconciliation saves about 200 hours a year of recurring toil.[S02][S18][S39]
4.19
Related · chapter 1210 paragraphs step it
Read-out
Illustrative 24-week bi-weekly monitoring trace for a factory model. Week 6 shows a schema unit error repaired at the source; Week 12 shows a planned product-mix change requiring slice validation rather than retraining; Week 20 shows sustained concept drift after a process change that justifies a governed candidate update. Values are invented to demonstrate the diagnostic method, not measured factory performance.
Unit: illustrative drift index (alert threshold = 100). Status: illustrative. Sources: S17, S21. Illustrative figures are teaching examples, not measured factory results.
Read this exhibit in its chapter ↗Empirical evidence on AI failures reinforces why quality gates precede modelling. Studying high-stakes AI practitioners at ACM CHI 2021, Sambasivan et al. defined data cascades as compounding events causing negative downstream effects from data issues, finding them pervasive (92 percent prevalence in their interviewed sample), invisible, delayed, but often avoidable.[S19] Though that 92 percent figure describes their qualitative sample rather than a statistical survey of Cameroonian manufacturers, the mechanism is a direct warning for industrial MLOps.[S19]
Data cascades arise when upstream carelessness in calibration, labelling or schema evolution interacts with production over time.[S19] In textile inspection, literature notes that manual visual inspection is roughly 60 to 70 percent accurate and misses 20 to 30 percent of defects, whereas computer vision exceeds 90 percent accuracy across more than 40 defect types at speeds up to 60 metres a minute (inherited entry R29, prepared 31 January 2024, not refreshed in 2026, original date unrecorded).[R29] Yet validating a vision model against manual logs lacking a governed taxonomy or roll-encoder consistency gate makes the validation score an illusion.[S04][S19]
Three cascade paths are especially destructive in small factories:
wp04) or return analysis, outcomes arrive weeks after production.[S43] If order amendments or lot splits overwrote the original identifier instead of appending a provenance-linked event under EPCIS and PROV-O rules, delayed outcomes join to the wrong production record.[S08][S09]When quality gates and provenance graphs work together, diagnosing an anomaly becomes a fast lookup rather than a week-long forensic project.[S15] Following the CRISP-DM manufacturing lifecycle, evaluation and deployment remain coupled to data understanding and preparation.[S43] Before opening a modelling notebook to investigate a degraded forecast or erratic KPI, the spine runs three checks along the lineage graph:
MISSING, ESTIMATED or quarantined shares in feeding tables exceed the contracted threshold during the window?[S04]S38), master-data mapping (S05) or threshold change along wasDerivedFrom?[S09][S38]CORRECTED surge alter input or label distributions?[S02]Only when all three checks confirm stable capture, semantics and gates does the team treat the anomaly as process or concept drift under Part III.[S15][S43]
By making data fitness an explicit, versioned and provenance-backed property of every record, the factory completes Part I of the industrial data spine: decision rights are named, vocabulary is contracted, and quality gates guard curated storage.[S01][S02] Part II turns to the minimum durable architecture required to store, contextualise, protect and recover those records under factory constraints.
Check the record
Every knot is a source this chapter cites; select one to read its record, or filter the register by grade. Publication dates and inherited-source limits stay explicit. Read the paper's full evidence summary.
26 sources · 214 citation marks on this pageMost cited: S04, 25 times
Grade AGrade BGrade CKnot size: citations on this page
All 26 records shown
R06 · Grade C · inherited record
Inherited evidence R06
About 95 percent of cotton fibre exported raw against 5 percent processed locally; industrialisation target of 50 percent local processing.
Publication date unrecorded. Inherited source-text record. The original publication has not been independently reopened for this web edition.
R07 · Grade C · inherited record
Inherited evidence R07
SND30 restates the 50 percent local-processing target by 2030.
Publication date unrecorded. Inherited source-text record. The original publication has not been independently reopened for this web edition.
R12 · Grade C · inherited record
Inherited evidence R12
CICAM capacity utilisation, July 2023 report: Garoua 33.7 percent, CICAM 1 48.9 percent, CICAM 2 10.83 percent; underutilisation cost about CFAF 2 billion in 2020.
Publication date unrecorded. Inherited source-text record. The original publication has not been independently reopened for this web edition.
R29 · Grade C · inherited record
Inherited evidence R29
Manual defect inspection 60 to 70 percent accurate, missing 20 to 30 percent of defects; computer vision above 90 percent across more than 40 defect types; line-scan inspection up to 60 metres a minute.
Publication date unrecorded. Inherited source-text record. The original publication has not been independently reopened for this web edition.
c04-S01 · Grade A
ISO 8000-1:2022: Data quality, Part 1: Overview
The series defines which characteristics of data are relevant to data quality, specifies requirements applicable to those characteristics, and provides guidelines for improving data quality.
2022. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S02 · Grade A
ISO 8000-150:2022: Data quality, Part 150: Data quality management: Roles and responsibilities
This document specifies the key considerations for organizations that are establishing appropriate roles and responsibilities for data quality management.
2022. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S03 · Grade A
ISO/IEC 25012:2008: Software engineering, Software product Quality Requirements and Evaluation, Data quality model
Data quality requirements and data quality measures can be categorized with the data quality characteristics in 5.2 and used by an evaluation process to analyse data independently from other computer system's components.
2008. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S04 · Grade A
ISO/IEC 5259-3:2024: Artificial intelligence, Data quality for analytics and machine learning, Part 3: Data quality management requirements and guidelines
ISO/IEC 5259-3 defines the requirements and guidance for managing the quality of data used in analytics and machine learning (ML).
2024. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S05 · Grade A
IEC 62264-1:2013, Edition 2.0: Enterprise-control system integration, Part 1: Models and terminology
This part of IEC 62264 provides standard models and terminology for describing the interfaces between the business systems of an enterprise and its manufacturing operations and control systems.
2013. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S06 · Grade A
OPC Unified Architecture, Part 1: Overview and Concepts, OPC 10000-1, version 1.05.06
It provides a comprehensive framework for modelling information, secure communication, and ensuring interoperability between diverse systems and devices.
2025-10-22. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S08 · Grade A
EPCIS Standard, Release 2.0
The goal of EPCIS is to enable disparate applications to create and share visibility event data, both within and across enterprises.
2022. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S09 · Grade A
PROV-O: The PROV Ontology
It provides a set of classes, properties, and restrictions that can be used to represent and interchange provenance information generated in different systems and under different contexts.
2013-04-30. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S10 · Grade A
Current Standards Landscape for Smart Manufacturing Systems, NISTIR 8107
Standards are fundamental 1) to facilitate the delivery of the right information at the right time, 2) to enable actions based on that information and 3) to reduce risk of technology adoption and development.
2016-02-23. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S11 · Grade A
Artificial Intelligence Risk Management Framework, AI RMF 1.0, NIST AI 100-1
Human judgment should be employed when deciding on the specific metrics related to AI trustworthiness characteristics and the precise threshold values for those metrics.
2023-01-26. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S15 · Grade B
MLOps: Continuous delivery and automation pipelines in machine learning
Information about each execution of the ML pipeline is recorded in order to help with data and artifacts lineage, reproducibility, and comparisons.
2024-08-28. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S18 · Grade B
Hidden Technical Debt in Machine Learning Systems
Using the software engineering framework of technical debt, we find it is common to incur massive ongoing maintenance costs in real-world ML systems.
2015. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S19 · Grade B
Everyone wants to do the model work, not the data work: Data Cascades in High-Stakes AI
Data cascades are pervasive (92% prevalence), invisible, delayed, but often avoidable.
2021. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S36 · Grade B
The Missing Element of Firm Digitalization: Lessons for the EU Member States: Best practices for enhancing MSMEs Digitalization and Managerial Capacity Building
Embedding monitoring and evaluation throughout each step of the program design and implementation process is crucial for generating knowledge, tracking impact, and improving learning for the further improvement of interventions pursuing the digitalization of MSMEs.
2023. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S38 · Grade A
ISO 22400-2:2014: Automation systems and integration, Key performance indicators for manufacturing operations management, Part 2: Definitions and descriptions
The KPIs are presented by means of their formula and corresponding elements, their time behaviour, their unit/dimension and other characteristics.
2014. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S39 · Grade A
Artificial Intelligence with Open and Scaled Data Sharing in Semiconductor Manufacturing: Workshop Report
Also, manufacturers have reported that the cost of data processing for a single application can be as high as 70% of the total cost.
2025. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S41 · Grade B
Technical Guidance for Calculating Scope 3 Emissions, version 1.0
Collecting data, assessing data quality, and improving data quality is an iterative process.
2013. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S43 · Grade B
Ensuring the Robustness and Reliability of Data-Driven Knowledge Discovery Models in Production and Manufacturing
Briefly, CRISP-DM divides the data mining-related knowledge discovery process into six phases: 1) business understanding, 2) data understanding, 3) data preparation, 4) modeling, 5) evaluation and 6) deployment.
2021-06-14. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S44 · Grade B
Supporting SME companies in mapping out AI potential: a Finnish AI development case
One must remember that in IT investments, the decision to postpone, e.g. an AI project, is also valid if resources, skills, costs, or expected benefits are not met.
2024-07-27. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S0401 · Grade A
Data on the Web Best Practices: Data Quality Vocabulary
It does not provide a formal, complete definition of quality, rather, it sets out a consistent means by which information can be provided such that a potential user of a dataset can make his/her own judgment about its fitness for purpose.
2016-12-15. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S0402 · Grade A
Data on the Web Best Practices
Independently from domain-specific peculiarities, the quality of data should be documented and known quality issues should be explicitly stated in metadata.
2017-01-31. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗c04-S0403 · Grade A
PROV Model Primer
PROV defines a core data model for provenance for building representations of the entities, people and processes involved in producing a piece of data or thing in the world.
2013-04-30. Register access date: 2026-10-30. Source provenance is supplied by the paper evidence register.
Open the recorded source ↗S01 International Organization for Standardization. 2022.
S18 Advances in Neural Information Processing Systems. 2015.
S04 ISO/IEC. 2024.
S09 World Wide Web Consortium. 2013-04-30.
R06 Cameroon Trade Hub. Publication date unrecorded; inherited source-text check.
R07 Ecofin Agency. Publication date unrecorded; inherited source-text check.
R12 Business in Cameroon. Publication date unrecorded; inherited source-text check.
S36 World Bank. 2023.
S02 International Organization for Standardization. 2022.
S03 ISO/IEC. 2008.
S15 Google Cloud. 2024-08-28.
S0401 World Wide Web Consortium. 2016-12-15.
S38 International Organization for Standardization. 2014.
S05 International Electrotechnical Commission. 2013.
S08 GS1 AISBL. 2022.
S39 National Institute of Standards and Technology. 2025.
S06 OPC Foundation. 2025-10-22.
S19 Association for Computing Machinery, CHI 2021. 2021.
S0402 World Wide Web Consortium. 2017-01-31.
S41 GHG Protocol, World Resources Institute and World Business Council for Sustainable Development. 2013.
S11 National Institute of Standards and Technology. 2023-01-26.
S0403 World Wide Web Consortium. 2013-04-30.
S10 National Institute of Standards and Technology. 2016-02-23.
R29 MDPI; industry inspection literature. Publication date unrecorded; inherited source-text check.
S43 Frontiers in Artificial Intelligence. 2021-06-14.
S44 The Journal of Technology Transfer, Springer Nature. 2024-07-27.
Keep the thread · chapter 04 of 14
Move the shuttle along the rack to preview any chapter: its sections, exhibits and reading time. Draft web edition. This research is not investment, legal or tax advice; illustrative models are not measured factory results.
The paper's rack · 14 boltsBolt length: reading time
01Part I: Govern meaning before scale
The binding constraint on industrial AI is often the broken chain between work, record and decision, not the absence of an algorithm.
02Part I: Govern meaning before scale
A workable governance model fits into existing factory roles and makes every definition, exception and release somebody's explicit decision.
03Part I: Govern meaning before scale
Shared identifiers, event semantics and KPI formulas let unlike systems describe the same work without hiding their differences.
04Part I: Govern meaning before scale
A small set of enforced checks, visible exceptions and complete lineage is more useful than an unmeasured claim that the data are clean.
05Part II: Build the minimum durable spine
A durable architecture preserves source evidence, adds governed context once and lets each decision consume a controlled product.
06Part II: Build the minimum durable spine
A value becomes useful evidence only when the factory can establish what it describes, when it happened and which state or transformation it represents.
07Part II: Build the minimum durable spine
The data spine must protect production, people and legitimate access at the same time, with controls proportionate to a small team.
08Part II: Build the minimum durable spine
A factory proves resilience by operating through disconnection, restoring its records and leaving a vendor without losing its history.
09Part III: Control models as production change
The first model gate is operational and economic: nobody should train before the action, baseline, data rights and stop conditions are explicit.
10Part III: Control models as production change
A model becomes a release candidate only when another competent person can reconstruct what data, code, configuration and evaluation produced it.
11Part III: Control models as production change
Release authority follows demonstrated local fitness, safe fallback and accountable acceptance, not a strong aggregate validation score.
12Part III: Control models as production change
A useful monitor detects change, establishes whether it matters and routes a named person through a rehearsed response.
13Part IV: Prove value and scale deliberately
The spine earns further investment when people use it, decisions improve and net operational benefit survives a comparison with the baseline.
14Part IV: Prove value and scale deliberately
GFT should scale the spine only as ownership, meaning, recovery, reproducibility and net value are demonstrated in production.