The temptation to begin clean
Suppose a proposed profile adjustment makes a persona answer more quickly but also makes it more likely to state uncertain claims as facts. The change is rolled back. When the next experiment begins, a clean spreadsheet without that episode looks inviting. It also invites precisely the same mistake.
A failed trial is not a permanent verdict on the persona. The task mix may have changed, the comparison may have been weak, or a tool may have failed. Concealing the result prevents anyone from checking those alternatives. The failed intervention becomes an invisible influence rather than usable evidence.
The record that teaches
The older S16 architecture sketch describes before-state measurements, sandbox testing, after-state metrics, and human review, while stating that the loop was not fully implemented at the time. The newer self-reflection proposal records prior recommendations and actual outcomes, including unresolved results. Its companion math uses the prior review's real closing state rather than a wished-for target.
That distinction matters. If the requested change was never applied, the next review must not inherit it as fact. If a change was applied and later reversed, both actions belong in the lineage. A missing receipt is not permission to invent a good outcome.
Pause here: What would the next reviewer need to know to avoid repeating a failed change without treating that failure as the whole story?
Carry the failure honestly
A useful failure note says what was tried, what was expected, what was measured, what else may explain the result, and which conclusion remains uncertain. It protects the next reviewer from both repetition and premature certainty. It also lets the persona contest an interpretation that treats one bad trial as its defining flaw.
There is room for fresh beginnings in design. There is no need for amnesia. The next attempt can take a different route precisely because the earlier route remains on the map.
My answer: Preserve the failed trial, its limits, and its actual state changes. A new experiment should be free to differ, but it should not be forced to rediscover hidden evidence.