Turn the insight into a testable expectation
Research often ends with a recommendation: simplify a form, clarify a product description, change a support process, or address a concern raised in interviews. Before making the change, write down the problem it is meant to solve. Be specific about who experiences the problem and where it occurs. “New customers cannot tell which plan fits their needs” is easier to evaluate than “make the website better.”
Then describe what you expect to observe if the change helps. A clearer plan comparison might lead to fewer questions about plan differences, more visitors reaching a selection, or customers describing the options more accurately in a short follow-up interview. These signals are not interchangeable. One measures behavior, while another helps explain understanding. Choose measures that match the original insight rather than whichever numbers are easiest to retrieve.
Record a baseline and the conditions around it
A baseline is a record of what happened before the change. It might include the number of people completing a task, the questions received by support, or notes from observing customers use an existing page. Record how each measure was collected, the period it covers, and any limitations. A baseline built from a handful of interviews can reveal recurring confusion, but it should not be treated as a precise estimate of how common that confusion is.
Also note what else could affect the outcome. A pricing update, a new traffic source, a revised support script, or a seasonal shift in demand could change the results even if your research-informed adjustment has no effect. You may not be able to control every factor, but documenting them makes a later comparison more honest. If the change will be rolled out gradually, consider whether an unchanged group or version can provide a useful point of comparison, provided the groups are reasonably comparable.
Choose a small set of meaningful measures
Use a primary measure that reflects the intended improvement, then add a few supporting measures. For a revised help article, the primary measure might be whether readers can complete the task it explains. Supporting measures might include related support questions and feedback on what remains unclear. Decide in advance what direction of change would count as encouraging evidence, and what result would prompt another look.
Include a guardrail: something you do not want to make worse while improving the main outcome. A shorter form may increase submissions but produce requests that staff cannot act on because essential details are missing. A simpler comparison page may speed up decisions but leave customers unsure about important differences. Checking both the intended benefit and possible trade-offs prevents a single appealing number from carrying the whole decision.
Compare results without overclaiming
Once the change is live, collect evidence in a way that resembles the baseline where possible. Use the same definitions for outcomes, similar customer groups, and a period long enough to include typical use rather than only the first burst of attention. Break results down when it matters: an overall improvement could hide a worse experience for first-time visitors or people using a particular device.
Look for a pattern across measures and listen for explanations. If task completion improves and interview participants can explain the new wording, those observations reinforce one another. If the numbers move but customers still describe the same confusion, investigate further. Small samples, uneven traffic, and simultaneous changes can make apparent differences misleading. In those cases, say that the result is promising or unclear rather than claiming the change caused an improvement.
Make a decision and keep the learning
Close the loop by choosing an action: keep the change, adjust it, test an alternative, or return to the earlier version. Record the decision alongside the original expectation, the evidence collected, and the main uncertainties. If results are mixed, identify the next question rather than repeating the whole research project by default. For example, you might need to learn why a revised page helps returning customers but not newcomers.
A useful evaluation does not have to deliver a simple win. It should help the team distinguish between an idea that addressed the problem, an idea that needs refinement, and a problem that was misunderstood. Keeping that record makes future decisions easier to explain—and helps the next research-informed change start from what you have already learned.
