The manual and assisted paths should answer the same product question under the same acceptance conditions. Each pair needs one frozen brief, the same block and material references, the same required size range and the same physical and receiving-CAD checks. The reviewer should not demand a complete tech pack from manual work while accepting a rendered concept from assisted work. Equally, the assisted path should not receive extra procurement information or expert correction that the manual path is denied. Any necessary difference in support is part of the workflow and must be recorded as such.
NIST's AI Risk Management Framework calls for a human baseline when AI augments or replaces human activity. It also warns that controlled measurements can differ from operational settings.[S19] For this pilot, the baseline is not a generic industry lead time. It is the maker's own observed work on matched styles, with product mix and approval conditions retained. The report should preserve routine and held-out strata so a favourable average on easy variants cannot conceal weak performance on the cases that motivated qualification.
Order effects need a deliberate control. A technician who completes the manual path first may understand the brief better when starting the assisted path. A technician who sees a generated alternative first may anchor the subsequent manual design. Balancing the initial order across styles reduces the risk that one workflow systematically benefits from learning. It does not remove learning within a pair. The record should therefore retain order, elapsed time between attempts and any reused artefact. Where genuine independence is impossible, the limitation belongs in the interpretation rather than being hidden by the word matched.
The protocol should freeze model, integration and review rules before the evaluation batch. A partner may repair a broken connector, but the repair creates a new configuration whose results should not be pooled uncritically with the earlier one. Prompt or template adjustment on a routine development case is legitimate preparation. The same adjustment on a held-out evaluation case turns that case into part of development. Configuration identifiers and change dates let the reviewer reconstruct which system produced each result.
Every candidate and failed run belongs in the attempt log. Record the brief and output identifiers, model route, run status, validation outcome, expert help, correction reason and final disposition. Include runs that produce no usable image, programs that do not parse, exports that cannot be reopened and candidates abandoned after review. Selecting only visually impressive outcomes would remove precisely the review burden and technical failures the pilot must measure. NIST's generative-AI profile supports empirical capability checks, source verification and lifecycle monitoring, which reinforces the need for inspectable evidence rather than a curated demonstration.[S20]
Time capture should separate generation, technician touch time, expert assistance, queue time, physical sampling, buyer waiting and rework. A short generation event can coexist with a long approval calendar. Correction time should include diagnosis and rechecking, not only the visible edit. Partner effort supplied without an invoice still changes the resource boundary and should be identified. The aim is not stopwatch surveillance. It is to understand which activities are required to obtain an accepted specification and who performs them.
TailorCoPilot's TailorTrace research records discrete pattern states and explicit transformation operations.[S15] GFT can apply the same general principle to the pilot log: preserve the candidate, the technician's edit, the reason and the retest. This makes a manual rescue visible as work and as learning. It also prevents a corrected file from being presented as an unaided model success. The source reports a research system and user study, not a local training or productivity result, so the pilot must still observe its own corrections.
A matched comparison needs rules for missing evidence. If a timestamp fails, report the affected duration as unknown rather than replacing it with an assumption. If physical review is pending, the case has not completed the acceptance route. If a reviewer disagrees on a critical failure, retain both assessments and escalate under the release rule. These are evidence states, not inconveniences to be averaged away. The dataset should permit a reader to move from every summary value back to the case, revision and decision that produced it.