Most experiments leave behind a screenshot and a phrase such as “version B won”. A few months later, nobody remembers what changed, which audience saw the test or whether the result repeated. The team then tests the same idea again or turns a circumstantial signal into a permanent rule.

A decision log is a short record linking an experiment to the business decision. It does not replace detailed data, but makes learning searchable and comparable.

Record the situation before the change

Note what happens today, which problem is observed and what evidence supports it. State the period, audience, journey and measurement system. Future teams will then avoid comparing different contexts as though they were identical.

Write the hypothesis and mechanism

The hypothesis should say which change is expected to affect which behaviour and why. “We will test a new CTA” explains nothing. “A clearer description of the next step will increase completed applications because it reduces uncertainty” can be assessed and challenged.

Define the main outcome in advance

Choose one primary measure and a few guardrails. An experiment may increase clicks while reducing enquiry quality. Also record a stopping condition, time window and external factors that would make interpretation difficult.

Describe exactly what was implemented

Save the copy, creative, targeting, URLs, dates and page version. Note changes made during the experiment. “New design” is insufficient to reproduce the experiment or understand what changed.

Separate result, interpretation and decision

The result is what was observed. The interpretation is the possible explanation. The decision is what the team will do. Keep them in separate fields to avoid inflating certainty. If the sample is small or conditions changed, another test may be more appropriate than universal rollout.

Close with an owner and review date

Specify who implements the decision, who monitors consequences and when a check follows full rollout. An experiment that briefly improves a metric may have a different effect over a longer period.

The decision log framework is an original methodology developed by DigitalNow.

DIGITALNOW EDITORIAL TEAM

Practical guidance from DIGITALNOW, part of VNG Digital Group.

Editorial policy