A/B Testing — Reference
The A/B Testing surface is the split-test experiment plane for SGEN. It owns the catalog of experiments configured on a site, the per-experiment variant definitions (the page or component variations being compared), the stable variant-assignment layer that decides which visitor sees which variant, the conversion-tracking layer that records the events an experiment counts as success, and the statistical-significance gate that holds the result back until the sample is large enough to trust. Anything that asks "ship variant A or variant B based on measured outcome" traces back to a record managed here.
This page is a reference for platform engineers and integrators who need to understand the surface before scripting against it, extending the assignment model, or scoping an experiment program. Customer-facing how-tos for running tests live in the customer docs set; this page describes the shape of the surface, not the steps to drive it.
Overview
A/B Testing lives under the Experiments module in SG-Admin. The module renders four primary views — the experiment list (every configured experiment with state and current sample size), the experiment editor (a target page picker, the variant editor, the conversion-event picker, the traffic-split allocation, and the stop conditions), the run dashboard (the current per-variant sample size, conversion count, conversion rate, and the significance-gate state), and the result archive (concluded experiments with the winning variant and the captured run statistics).
The module is paired by convention with the Pages, Forms, Feature Flags, and Activity Log surfaces. Variants reference page records by id; conversion events reference form-submission records, page-view records, or named custom events; the variant-assignment layer reads the per-visitor hash the Feature Flags module also uses; every experiment state transition writes an Activity Log entry. The A/B Testing module owns the experiment and the variant assignment; the consuming surfaces own the records the experiment points at.
A second surface — the variant assignment cookie — is the read-side contract surfaced to the public frontend. When a visitor first loads a page that is part of an active experiment, the assignment layer computes a deterministic per-visitor variant (using a stable hash of the visitor identifier and the experiment slug), writes the assignment into a long-lived cookie, and surfaces it to the page renderer. Subsequent loads read the cookie — the same visitor always sees the same variant for the same experiment.
Where it lives in SG-Admin:
- Sidebar: SG-Admin → Experiments
- URL prefix:
/sg-admin/experiments/ - View templates:
application/views/Admin/Experiments/
┌──────────────────────────────────────────────────────────────────────┐│ SG-Admin → Experiments [+ New experiment]│├──────────────────────────────────────────────────────────────────────┤│ Experiment Variants Sample Significance State ││ ───────────────────────── ──────── ────────── ──────────── ─────││ homepage-hero-headline A | B 4,210 Reached Live ││ pricing-page-layout A | B|C 1,830 Pending Live ││ checkout-cta-color A | B 8,995 Reached Done ││ trial-signup-form A | B|C 612 Pending Live ││ ││ [⋯ Edit] [⋯ Run dashboard] [⋯ Conclude] [⋯ Archive] │└──────────────────────────────────────────────────────────────────────┘Actions
The A/B Testing surface exposes the following operations. Each is described by what it does to the data and to the experiment lifecycle, not by its internal method name.
List and search
Returns the experiment records visible to the current operator, paginated, with name, variant count, current sample size, significance-gate state, and lifecycle state columns. Supports column sort, free-text filter on experiment name and target page, and per-page count. Driven by the data-table contract used across SG-Admin modules — responses arrive as a row collection plus a total count, suitable for direct binding to a paged grid.
Create experiment
Opens an empty experiment form. Required fields at minimum: experiment name, target page, variant definitions (at least two variants), conversion event, traffic-split allocation (must sum to 100), and a minimum-sample threshold the significance gate uses. Optional fields cover a maximum-runtime stop, a notes field, and a per-variant baseline link for the control variant. On submit, the surface validates that variants are non-empty and traffic splits sum correctly, persists the record, and returns the new identifier.
Edit experiment
Loads an existing experiment into the same form shape used for create, pre-populated. Submit replaces the stored configuration. Edits to traffic splits or variant definitions on a live experiment reset the captured sample to zero — the surface returns a structured warning and requires explicit confirmation, since a mid-run variant change invalidates the data captured so far.
Start experiment
Transitions the experiment from draft to live. The variant-assignment layer becomes active for the target page — new visitors receive a variant cookie and the variant page renders in place of the control. Existing visitors without a cookie are assigned on their next page load. The surface validates that variant definitions are complete and the conversion event is configured before allowing the transition.
Pause experiment
Suspends the variant-assignment layer without discarding captured data. New visitors receive the control variant for the duration of the pause; existing assignments remain in the cookie and are honored when the experiment resumes. Pause is the fast path for an operator who needs to halt an experiment while investigating a problem without losing the run-to-date sample.
Resume experiment
Reverses a pause. Variant assignment resumes from the captured state; cookied visitors keep their prior variant.
Conclude experiment
Transitions the experiment from live to done. An operator picks the shipped variant (typically but not always the variant with the higher conversion rate). The shipped variant becomes the default page render for all visitors; assignment cookies are no longer read for that experiment. The surface validates that the significance gate has either been reached or has been explicitly overridden with a recorded reason.
Override significance gate
A dedicated write path for the case where an operator needs to conclude an experiment before the significance gate is reached. Requires a reason note that is stored on the experiment record and on the audit-trail entry. The surface surfaces a clear warning that the conclusion is statistically unsupported.
View run dashboard
Returns the per-variant sample size, conversion count, conversion rate, confidence interval, and the current significance-gate state for a single live experiment. The dashboard is refreshed on demand — it is not a long-poll surface, the operator reloads to see updated numbers.
Archive experiment
Marks the record inactive without removing it from the database. Archived experiments disappear from the default list view but remain queryable via the archive filter. Captured run data is retained.
Permanent delete
Hard-removes the record. Available only after an archive. Captured run data is removed with the record. This path is irreversible.
Data model
An experiment record carries the following fields. Field names below are the conceptual shape — the on-disk column names match closely but are not contractually stable across releases.
| Field | Type | Notes |
|---|---|---|
id | integer | Primary key. Stable across edits. |
slug | string | Unique. Used in the variant-assignment cookie name. |
name | string | Display name. Not unique. |
target_page_id | integer | Resolves against the Pages module. |
variants | ordered list | Each variant: { slug, label, page_id_or_inline_override, traffic_share }. Minimum 2. |
conversion_event | string | One of: page view, form submission, custom event slug. |
min_sample_threshold | integer | The significance gate is not "reached" until each variant exceeds this. |
max_runtime_days | integer | Optional. Auto-pauses past the cutoff. |
lifecycle_state | enum | draft, live, paused, done, archived. |
shipped_variant_slug | string | Set on conclude. Null until then. |
significance_override_reason | string | Set if conclude bypassed the gate. Null otherwise. |
created_at | timestamp | Set on create, immutable. |
updated_at | timestamp | Touched on any edit. |
started_at | timestamp | Set on first transition to live. |
concluded_at | timestamp | Set on transition to done. |
Run-data attribution: per-variant sample counts and conversion counts are stored on a separate run-data record keyed by experiment id and variant slug. Run-data is append-only during a live experiment and immutable once the experiment is done.
EXPERIMENT RECORD├── id integer primary key├── slug string unique, drives cookie name├── name string display├── target_page_id integer Pages module reference├── variants ordered [{slug, label, page_ref, traffic_share}]├── conversion_event string page-view | form-submission | custom slug├── min_sample_threshold integer significance-gate floor per variant├── max_runtime_days integer optional auto-pause cutoff├── lifecycle_state enum draft | live | paused | done | archived├── shipped_variant_slug string set on conclude└── timestamps created_at, started_at, concluded_at, updated_at↓ per visitor per page loadVARIANT ASSIGNMENTcookie: sgen-ab-<slug> = <variant_slug>stable per visitor; same visitor → same variant↓ on conversion eventRUN-DATA RECORDper (experiment_id, variant_slug): sample, conversions, last_conversion_at(append-only during live; frozen on done)Permissions
Access to the A/B Testing surface is gated at two layers.
Layer 1 — admin gate. Every action under SG-Admin passes through the platform's standard admin access check at request entry. An unauthenticated request never reaches the A/B Testing surface.
Layer 2 — per-action capability. Within SG-Admin, each A/B Testing action checks a capability associated with the calling operator's role. The default role configuration ships with three roles — Administrator, Editor, Viewer — and the capability map is:
| Capability | Administrator | Editor | Viewer |
|---|---|---|---|
| List experiments | ✔ | ✔ | ✔ |
| Create experiment | ✔ | ✔ | — |
| Edit experiment (draft state) | ✔ | ✔ | — |
| Edit experiment (live state) | ✔ | — | — |
| Start / pause / resume | ✔ | ✔ | — |
| Conclude experiment | ✔ | ✔ | — |
| Override significance gate | ✔ | — | — |
| View run dashboard | ✔ | ✔ | ✔ |
| Archive | ✔ | — | — |
| Restore archived | ✔ | — | — |
| Permanent delete | ✔ | — | — |
Self-protection rules. An experiment cannot reference itself recursively (no variant can point at the same experiment's target page through a chain of overrides). An experiment with a variant whose traffic share is zero cannot be transitioned to live. The surface returns a structured rejection in each case.
Audit. Every write — create, edit, start, pause, resume, conclude, override, archive, restore, delete — emits an Activity Log entry. The entry records the acting operator, the target experiment, the change shape, and the lifecycle transition. Significance-gate overrides carry the operator-supplied reason on both the experiment record and the audit entry.
LIFECYCLE TRANSITIONSdraft ────start────▶ live ────conclude────▶ done ────archive────▶ archived▲ │ ▲ ││ │ pause │ │ delete│ ▼ │ ▼│ paused ──resume──┐ │ (removed)│ │ │ ││ └─cancel─┐ │ ││ ▼ │ │└─────────reject (P3 fail)─◀──┴─────┘ ││Significance-gate states (orthogonal): │- "Pending" = sample below min_threshold │- "Reached" = sample ≥ min_threshold │- "Override" = concluded with explicit reason ─┘Related references
- Pages — Reference. Variants reference page records by id. Soft-deleted pages still render through cached variant assignments until the assignment cookie expires or the experiment is concluded.
- Forms — Reference. Form-submission conversion events resolve against form records. Form deletion mid-experiment surfaces a warning on the run dashboard.
- Feature Flags — Reference. The variant-assignment layer reuses the stable per-visitor hash that the Feature Flags module uses for percentage rollouts. Same visitor, same hash, deterministic across both surfaces.
- Activity Log — Reference. Every experiment lifecycle transition emits a one-line entry. Conclude entries include the shipped variant slug.
- Settings — Reference. Owns the role definitions referenced by the capability map and the Activity Log retention policy that governs audit-entry lifetime.
/docs/a-b-testing.