All Collections
A/B testing in sequences
A/B testing in sequences

Wondering how Reply performs A/B testing of your email bodies and subject lines and how you can track results and act on the outcome?

Reply Team avatar
Written by Reply Team
Updated over a week ago

Sequences let you organize A/B testing in the Email step, and you can create unlimited variations of it. For A/B tests, Reply splits the contacts in your sequence as evenly as possible, with each variant going out to approximately 50%. The system selects contacts that receive different variations at random.

For example, if you have A and B variants for Step 1 and import a list of 200 contacts in one batch, the system will automatically send variant A to 100 and variant B to the remaining 100. Each imported batch is randomized separately.

For system to split contacts between variants evenly - all contacts will need to be imported in one batch and delay between sending of each email shouldn't be over 150 seconds. Another setting which can impact the fact that contacts are not allocated even is throttling in the safety set up of email account. Please make sure that email throttling frequency limit is not set to higher than 9 minutes.

However, if contacts are manually added to the sequence that contains an A/B testing step, and there are also time intervals between adding subsequent contacts, the system may distribute these prospects unevenly across options A and B.

Note: Since A/B testing is random, you won’t know who exactly will receive options A and B. It is also random for individual steps in a sequence - so, for example, contacts who received email A in Step 1 may not necessarily receive email A in Step 2.

Creating a variant
On the Steps tab of your sequence, click Add variant under one of the sequence steps:

Cloning, editing, removing an existing variant
Hover over an existing variant and click on the needed action:

Pausing and resuming an existing variant
Sequences let you pause an existing variant or several variants of the Email step. Note that at least one variant must remain active. To pause or resume a variant, hover over it with a mouse cursor and click on the needed action:

Viewing performance of A/B variants
To see how your variants of the Email step perform, open the Stats tab, scroll down to the Brief/Detailed section and enable the A/B toggle. 

Did this answer your question?