Segmentation and analytics

A/B test: which variant works better

A/B test (split test) is a comparison of two variants of an offer, email or page on random groups of players. The winner is the variant that does better on a metric chosen in advance, usually deposits or NGR.

Yevhen SychovOctober 2026

Example

The variant with the better result on the main metric winsTwo subject lines, 10,000 players each: opens 18% and 21%, deposits 4.1% and 4.0%. The first subject line wins: on deposits, not on opens.

Rules

  • One change at a time: the subject line, the offer or the game. Otherwise you cannot tell what worked.
  • Random groups from the same segment, at the same time.
  • Pick the metric in advance: deposits, NGR or players won back from churn. Opens and clicks are only a hint.
  • Enough players and time: a small difference on small groups is often random. Do not stop the test on day one, when a variant is "already winning".

A/B test vs control group

An A/B test compares variants with each other. A control group compares a campaign with no campaign at all. A good test keeps both: variant A, variant B and a group that gets no message.

Common mistakes

  • Judging offers on deposits alone. A generous offer almost always brings more deposits. Make the decision on NGR minus the bonus cost.
  • Never retesting the winner. A month later the effect changes, so strong decisions get rechecked.
  • Testing small details. Button colour moves the needle less than the game in the offer or the channel of the message.
All glossary terms