A short pair-review session for an AI prototype
Give a reviewer one task, observe where they hesitate and leave with a testable change. A suggested format for an informal session.
Ask a colleague to try one task while you watch. The purpose is to learn what they can understand and verify without your explanation. This proposed format works for an informal builder session; it is not a claim that a scheduled AI Krakow event or user study has taken place.
Choose a small job with a visible result. For a fictional prototype that turns notes into a follow-up draft, the task might be to create a draft and check which commitments remain unconfirmed. Prepare a synthetic example and reset the prototype so the reviewer starts from the intended state.
Before beginning, agree on the time available, the task, whether notes or a recording are allowed, and where feedback will be kept. Make it easy to decline recording. Tell the reviewer if a button would contact someone, spend money or change shared data, and disable those actions for the session where possible.
Give the task without teaching the interface. Ask the reviewer to describe what they are looking for if they are comfortable doing so. When they hesitate, write down the screen, the action and their words. “Could not find the source of this date” is more useful than “the interface needs to be clearer”.
Separate observations from explanations. The person may overlook a source link, distrust the output or misunderstand the task. Those are different hypotheses. Ask a neutral question such as “What did you expect here?” before proposing a fix or telling them what the button was meant to do.
W3C’s guidance on involving users explains the value of including people with disabilities in accessibility work. One colleague’s successful session cannot stand in for that range of experience or for standards-based evaluation. Record the participant and setup limits without collecting unnecessary personal details.
Finish with a small change card: observed difficulty, evidence, proposed change and the repeat task that would test it. For example, move the cited source beside the uncertain date, then ask a reviewer to verify that date again. A preference for a different colour may be worth noting, but it does not by itself explain the task failure.
Keep the initial notes and the changed version so the next conversation has a baseline. Share only material the participant agreed to share. If the session produces a useful reusable method, suggest it through the existing contact section. Reference checked September 20, 2026.