A passing test is only evidence for the configuration it ran on

Here’s a fictional release review I use to explain why a passing test needs context.

A controller board passes a thermal test on revision B with firmware 1.8. A week later, the release candidate uses revision C and firmware 1.9. Your dashboard still says “Thermal test: pass.” Before carrying that pass forward, check what actually changed and whether it affects the test.

Record three things separately: the configuration tested, the configuration intended for delivery, and the decision about whether the old evidence applies. The first two should be concrete enough for someone else to compare, for example board revision, firmware build, test setup, and the relevant operating limits. The third needs an owner and a reason.

For each difference, write one decision:

• Reuse the evidence if the responsible engineer can explain why the change does not affect the test condition or result.
• Run a targeted check if a specific changed part or behavior affects the evidence.
• Rerun the test when the impact is uncertain and the old evidence cannot support the release decision.

In this example, a changed regulator could alter the board’s thermal behavior. That does not tell you which test to repeat by itself. It tells you to inspect the regulator’s operating conditions and ask the verification owner to decide what evidence the release needs. Until they decide, mark the release evidence as pending.

Try this in your next release review: pick one result labeled pass and ask someone to show the tested build, the intended build, and the decision connecting them. If the team cannot reconstruct that decision from the record, the status needs more context.

Tymeline published a free example worksheet with fields for this review: engineering program review kit

Tymeline manages this publishing account.