Keep a decision note after an AI experiment
Record the question, changed variable, observations and next decision so a useful experiment survives beyond the chat window.
A prototype conversation can end with several promising ideas and no clear record of what changed. A short decision note gives the next person a way to continue the investigation. This is our proposed template for independent builders, not a report of measured results from a community session.
Start with the question you were trying to answer. “Can a reviewer verify every date in the generated summary?” is specific enough to test. “Is this model better?” leaves the task, evidence and meaning of better undefined. Keep the question narrow enough that one observation could change your decision.
Record the baseline and the one deliberate change. The baseline might be a versioned instruction with a small set of synthetic notes. The change could require each proposed date to quote its supporting sentence. Keep other settings stable where possible and record differences you could not control.
Save the inputs and outputs you are allowed to retain. Record the configuration, application revision, attempt count and review method. Link to the existing bug report or task when one exists. A screenshot may help illustrate the result, but a readable input and output usually make the reasoning easier to revisit.
Write observations before conclusions. In a fictional exercise, three checked summaries preserve the date and one still invents a commitment. That supports a statement about those four checks. It does not establish reliability for all meetings, languages or document lengths. Note whether the reviewer knew which version produced each output.
The decision can be to continue, revise, revert or stop. Give one reason tied to the task. For example: keep the source quote because it helps review, but keep the draft behind approval while the invented-commitment case remains unresolved. Avoid turning an interesting response into permission for an external action.
Our compact note contains six headings: question, baseline, change, observations, decision and next check. Under the last heading, name an owner and the input or condition needed to resume. If nobody is taking it forward, write that plainly so a paused idea does not look like an active commitment.
Read the note a week later without opening the original conversation. Could you find the input, reproduce the intended comparison and explain why the decision was made? If not, add the missing reference rather than a longer narrative. This is an editorial working method; the related guides below cover preparing a reproducible failure and gathering feedback.