Monitoring must trigger diagnosis, not automatic retraining

Once the factory verifies that incoming records are complete, timely, and contract-compliant, the monitoring system must help the team interpret changes in the statistical behaviour of inputs and outputs.[S17] In streaming and operational settings, concept drift describes unforeseeable changes in the underlying distribution of data over time.[S21] In manufacturing practice, however, not every distribution change breaks a model, and not every model failure is accompanied by a dramatic shift in marginal input distributions.[S1201] Treating all statistical distance metrics as equivalent retraining triggers wastes engineering capacity and destabilises production control.[S1202]

A diagnostic monitor separates three statistical phenomena because each points to a different physical or commercial event in the mill:[S21]

As shown in Exhibit 51, tracking a feature drift index alongside a prediction drift index over an illustrative 24-week operating window reveals why statistical movement requires contextual diagnosis rather than a reflex model update.[S17][S21]

In Exhibit 51, the illustrative monitoring trace crosses the alert threshold of 100 three times across 24 weeks, yet only the third excursion warrants retraining.[S17] At Week 6, both feature drift and prediction drift spike to illustrative indices of 145 and 138 because an upstream PLC tag scaling change corrupts a feed; repairing the ingestion mapping restores normal operation by Week 8 without touching the model.[S04] At Week 12, a scheduled changeover to a heavier fabric style pushes the illustrative feature drift index to 118, while prediction drift remains bounded at 44 and slice validation confirms that error rates on the heavier style stay within acceptance limits.[S1203] Only at Week 20, when a finishing-process modification alters both feature and prediction distributions and degrades verified inspection accuracy through Week 22, does the team authorise a governed retraining cycle that returns the indices to baseline by Week 24.[S13] This illustrative sequence demonstrates the diagnostic method; it cannot establish universal drift thresholds, which must be calibrated by human judgement on each site's historical variance.[S11]

A statistical distance metric cannot tell the model owner whether a shift is benign or malignant unless the spine links each prediction back to its operational context.[S1201] Because every prediction event in the governed spine carries site, line, asset, style, lot, and shift identifiers under the visibility-event contract established earlier in the paper, the monitoring job can decompose aggregate drift by operational slice.[S08] Before concluding that a model has decayed, the analyst checks whether the mix of styles, yarn counts, or machine states changed during the scoring window.[S38] As summarised in Exhibit 52, six common factory anomalies all produce visible movement in monitoring charts, but five of them are harmed or left unresolved by automatic retraining.[S21]

The comparison in Exhibit 52 shows why diagnosis must precede adaptation.[S21] Only the sixth row, where inputs pass all contract checks and matured ground truth confirms a persistent breakdown in the input-to-output mapping, justifies training a new candidate model.[S13] Even then, the table cannot tell a factory whether the newly trained candidate will outperform a simple rule comparator; that proof requires the full evaluation and staged-release discipline defined in the preceding chapters.[S43]

Textbook monitoring tutorials often assume that ground-truth labels arrive moments after a model makes a prediction, allowing the system to plot a live accuracy or error curve.[S21] In industrial operations, confirmed outcomes almost always lag behind the prediction event, sometimes by hours, days, or months.[S1202] When a small team ignores label delay and computes performance metrics over whatever partial outcomes happen to have arrived in the current week, it introduces severe survivorship and selection bias into its monitoring reports.[S19]

Every factory model faces a physical or commercial clock that governs when its predictions can be verified against reality:[S38]

Because operations managers cannot wait three months to discover that a model failed on day one, the monitoring design must pair delayed ground truth with immediate proxy indicators.[S17] Useful proxies include the distribution of predicted probabilities or quantities, the share of inputs falling in low-confidence regions near the decision boundary, the rate at which line operators or planners override the model's recommendation, and short-horizon physical checks such as immediate rework counts at the adjacent workstation.[S1202]

Each proxy carries a structural limitation that must be stated in the monitoring specification.[S11] A stable prediction distribution proves only that the model is emitting similar numbers, not that those numbers remain right when the process has shifted.[S1201] Conversely, a low operator override rate may reflect genuine agreement, or it may reflect automation complacency on a busy shift where workers click through recommendations without independent inspection.[S30] Proxies can therefore trigger an early diagnostic review or a protective fallback, but they must never be treated as confirmed ground-truth labels for automated retraining.[S19]