# A/B Testing

> Create, launch, and understand page experiments in Lexsis

Canonical URL: https://0.0.0.0:3005/guides/ab-testing

Run a simple page experiment to learn which version of a storefront page
performs better.

An A/B test sends visitors to different page versions, records what they do,
and compares the results. In Lexsis, each page version is called an
**experiment arm**.

## Before you start

Prepare two or more published or ready-to-publish Lexsis pages that you want
to compare.

Good experiments usually change one meaningful idea:

- A headline or value proposition
- A hero image
- A call-to-action
- The order of sections
- A complete page layout

Keep one page as the **control**. This is the version you want to use as the
baseline for comparison.

## Create an experiment

1. Open [A/B Testing](https://app.trylexsis.com/storefront/experiments) in your workspace.
2. Create an experiment and add the pages you want to compare.
3. Choose the control page.
4. Set the traffic allocation. A 50/50 split is a good default.
5. Add a short hypothesis so you remember what the test is trying to learn.
6. Prepare the experiment.

During preparation, Lexsis checks the selected pages and saves an immutable
version of each one. This prevents a page edit from changing the test while
it is running.

## Preview versus production

Before the experiment starts, the page buttons are **Preview** links. They let
you inspect each prepared page without sending normal visitors through the
experiment.

The production experiment URL becomes available only after you start the
experiment. That URL is the link to share with visitors or campaigns.

If the page says **No production URL** while the experiment is prepared, the
experiment has not been activated yet. Click **Start experiment** after all
arms show that their artifacts are ready.

## Start routing

When you click **Start experiment**, Lexsis publishes the prepared page arms
and enables the experiment link.

Visitors are assigned using the allocation you chose. The assignment is
sticky, which means a returning visitor should continue to see the same arm
instead of switching between pages.

The control page remains part of the experiment. It is the comparison point
for the other page versions.

## Pause, resume, and stop

### Pause

Pausing stops new experiment assignments. Lexsis keeps the experiment data
already collected, so you can resume later without losing the history.

### Resume

Resuming allows new visitors to enter the experiment again using the existing
allocation. Existing visitors keep their recorded assignment where possible.

### Complete or abort

Completing an experiment closes its active test period. Aborting stops the
experiment without declaring a winner. Existing analytics and audit history
remain available.

Do not complete an experiment just because one arm is temporarily ahead. Wait
until it has enough eligible traffic and runtime to make the comparison useful.

## How Lexsis measures the test

Lexsis separates raw events from the visitors who are eligible for the
experiment.

| Metric | Meaning |
|---|---|
| **Eligible sessions** | Visitor sessions that were successfully assigned to an experiment arm |
| **Exposures** | The recorded event showing that a visitor was eligible to see an arm |
| **Page views** | Visits recorded for the experiment page |
| **Conversions** | Completed purchases reported for eligible experiment sessions |
| **CVR** | Conversions divided by eligible sessions |
| **Revenue** | Revenue reported for eligible experiment orders |
| **AOV** | Average value of eligible experiment orders |
| **Section reach** | The percentage of visitors who reached a page section |
| **Held 3s+** | Visitors who stayed with a section visible for at least three seconds |
| **CTA clicks** | Clicks on tracked calls to action |
| **Scroll depth** | How far visitors moved through the page |

The experiment denominator is based on valid eligible exposures, not every
request to the page. This prevents visitors who were never assigned to an arm
from changing the result.

For Shopify purchases, the completed-order webhook is the source of truth.
Repeated webhook deliveries are deduplicated, so retrying the same order
should not count it twice.

## Reading the results

Use the **Overview** tab to understand the experiment state and the
**Analytics** tab to compare the page arms.

The results area may show the following states:

- **Evaluation unavailable** — the evaluation service has not produced a
  result yet, or there is not enough eligible data.
- **Collecting data** — the experiment is running and gathering observations.
- **Candidate** — one arm is showing a promising result, but review the
  sample and runtime before making a decision.
- **Inconclusive** — the available data does not clearly favor an arm.
- **Futile** — the current data suggests the experiment is unlikely to reach
  a useful result under its current conditions.

Early in an experiment, it is normal for runtime, sessions, and effect
threshold values to be unavailable. The evaluation updates after the worker
has processed eligible ledger data.

## Understanding page sections

The page-sections view shows which parts of each page visitors reached and
interacted with.

- A filled bar means visitors reached the section.
- A stronger bar indicates visitors stayed with it longer.
- A CTA marker means visitors clicked a tracked call to action.
- A dashed section means no recorded visitor reached it yet.
- A high drop-off marker shows where visitors most often stopped scrolling.

“Not reached” does not mean the section is broken. It means the current
traffic has not produced a recorded view of that section yet.

## Why results can be delayed

Events are captured in the browser and sent through Lexsis analytics ingest.
The dashboard then combines those events with exposure and order records.

Results can take time to appear when:

- The experiment has just started.
- Visitors have not reached the lower sections of the page.
- Consent or browser privacy settings prevent an event.
- A Shopify order webhook has not arrived yet.
- The evaluation worker has not completed its next update.

Use the dashboard state and activity history as the source of truth. Browser
network requests are useful for troubleshooting, but they are not the results
themselves.

## A practical testing workflow

1. Prepare the pages and confirm every arm says **ready**.
2. Open each Preview link and check the page on desktop and mobile.
3. Start the experiment.
4. Open the production experiment URL in a private browser window.
5. Visit both arms using separate sessions.
6. Check that page views, sections, scroll events, and CTA clicks appear.
7. Complete a test checkout if you are validating purchase reporting.
8. Review the experiment after the evaluation status updates.
9. Pause the experiment if you need to stop new assignments.

## Troubleshooting

### I only see Preview links

The experiment is prepared but not active. Start the experiment first. A
production URL is not created for a prepared-only experiment.

### Routing did not become active

Confirm that every arm is ready, the experiment has a valid allocation, and
the production experiment route is available. If the problem continues, check
the activity history and contact Lexsis support with the experiment name.

### I see page views but no conversion

Conversions require an eligible exposure and a completed order. Confirm that
the Shopify `orders/paid` webhook is registered and that the order was made
after the visitor entered the experiment.

### I see “Evaluation unavailable”

This is expected before the first eligible evaluation update. If it remains
after the experiment has collected traffic, check the experiment activity
history and whether the evaluation worker is running.

### A lower section shows no data

Visitors may not have scrolled far enough to reach it. Wait for more traffic
before deciding that the section has no engagement.

## Current limits

The current experiment experience uses fixed traffic allocation and does not
automatically promote a winner. Lexsis keeps the decision with you so you can
review the result and choose what to publish next.

The experiment also needs enough runtime and eligible sessions before the
evaluation can make a useful recommendation.

## Related pages

- [Judge.me Reviews](/guides/judge-me-reviews) — ingest, curate, and snapshot review collections for experiment arms
- [Page management](/tools/page-management)
- [Analytics](/tools/analytics)
- [Tracking Tags & Page Analytics](/tracking)