Skip to content

Test three or more versions at once

Compare several ideas in one test: three headlines, four hero pictures or two layouts next to the original. You add variants C, D and more to an existing A/B test, and the RL module finds the best one among all of them. Start with Setting up A/B tests if the element has no variant B yet.

Before you start

  • The element already runs an A/B test with the pills A and B on its control bar.

Set up

  1. Click the pill of the version you want to build on. The new variant starts as a copy of it.
  2. Click More and choose Create variant. A new pill, C, appears.
  3. Change variant C.
  4. Repeat steps 1 to 3 for each further version.
  5. Click Save on the toolbar.

What you see

The control bar shows one pill per version, from A up to Z at most. All versions share the conversion you chose under Experiment settings.

At first every version gets about the same share of visitors, a third each with three versions. As results come in, the RL module sends more visitors to the leader. Weaker versions still get some visitors, so a late riser can catch up. Adding a version later is fine: it joins on the next page view.

How much traffic you need

More versions split the same visitors, so each one collects impressions more slowly. The report's Conversion Score is (conversions + 1) divided by (impressions + 2). With 20 impressions, one extra conversion moves the score by about 4.5 percentage points. With 200 impressions, it moves it by about half a point. Wait until the scores stop jumping before you trust them.

Smart sort is multivariate too

In smart sort, every item of an accordion or tabs element is a version of its own. The RL module ranks the items again for each visitor, with the strongest ones usually on top.

If you don't see the new variant

  • The new variant copies the wrong version. A new variant copies the version you were viewing. Remove it, click the pill you want to copy, and create the variant again.
  • A letter seems to be skipped. New letters follow the highest letter in use. Removing B while C exists makes the next variant D.
  • The scores of all versions stay close together. Traffic is shared by many versions. Remove the weakest ones to give the rest more visitors.

What's next?

Best practices for running experiments to continue exploring DXPR Builder.

Something wrong or missing on this page? Report it or edit the page.