You can run a trustworthy PostHog A/B test on Shopify without installing a dedicated testing app. The difficult part is not creating the feature flag. It is implementing every variation safely in the correct Shopify theme, proving that each one works, and keeping test traffic out of the live store until the setup is approved.
This guide shows the workflow we use to prepare Shopify experiments for review: define the change, duplicate the live theme, choose the right experiment container, resolve selectors across every affected template, add deterministic variation previews, validate the unpublished theme, and create the matching PostHog experiment as a draft.
The result is not merely a script that swaps content. It is an unpublished Shopify implementation that another person can preview, verify, and approve without editing the live theme or accidentally entering the experiment.
What You Need Before You Start
- A working PostHog project.
- PostHog installed on the Shopify storefront so
window.posthogis available. - Shopify access that allows you to duplicate, customize, and edit themes.
- A clear description of what is being tested, the variations, what changes in each variation, and where those changes appear.
- A canonical feature flag key and variation keys that will be used identically in Shopify and PostHog.
- A downstream business event or funnel that can measure the experiment.
You do not need to know every final selector before starting. Finding and validating the real selectors in the theme source and rendered storefront is part of the workflow.
Step 1: Understand the Shopify Change
Before touching Shopify, answer four questions:
- What is being tested?
- Which variations exist?
- What changes in each variation?
- Where does each change appear in the storefront?
Keep the implementation narrow enough that the result is interpretable. A treatment can bundle several coordinated changes when that bundle represents one deliberate strategy, but the experiment should not accidentally combine unrelated edits.
Resolve the canonical feature flag key before coding. Use that exact key as the experiment slug in Shopify. For example:
Experiment: 026 - Product Page - Description Key Points
Feature flag: 026-product-page-description-key-points
Variations: control, test
For three or more variations, use stable machine keys such as control, test_group_1, and test_group_2. Descriptive labels belong in PostHog; the machine keys should remain short and unchanged after exposure begins.
Step 2: Duplicate the Current Live Shopify Theme
Never implement experiment code directly on the live theme. There is no exception for a small copy change or a script that appears easy to reverse.
In Shopify admin, go to:
Sales channels -> Online Store -> Themes
- Identify the current live theme.
- Open its actions menu and select Duplicate.
- Give the duplicate a clear experiment name.
- Record the unpublished theme name and theme ID.
- Open Customize on the duplicate and confirm that you are not editing the live theme.
You can reuse an existing experiment theme only after confirming that it is unpublished and still contains the intended baseline. Before changing a theme file or Custom HTML block, read its current contents and preserve unrelated code. Established stores often reuse one experiment container for several tests.
Step 3: Choose the Correct Experiment Code Location
Use the narrowest standard location that covers the experiment:
- Product-page experiment: reuse the product template’s existing Custom HTML section. Many established themes already have one for experiment scripts.
- Other template-specific experiment: use that template’s Custom HTML or equivalent Custom Liquid section.
- Site-wide experiment: use the header or footer according to where the change belongs.
- Global early-running code: use
theme.liquidonly when the code genuinely must run everywhere or before the normal sections render.
A Custom HTML section can remain visually hidden while serving as the script container, but verify that the theme still renders and executes its contents. The container must not introduce visible spacing or content.
Step 4: Resolve Every Template and Selector
Do not assume that one product template, section ID, or selector represents the entire storefront.
- List every Shopify template and template suffix where the experiment should run.
- Find the relevant section, block, code setting, or target element in each template.
- Open a representative storefront URL for every affected template.
- Compare the IDs in the theme source with the IDs rendered in the browser DOM.
- Confirm that every selector matches the intended element exactly once.
- Record template-specific selector or mount differences instead of forcing one selector everywhere.
Watch for Shopify’s dynamic ID suffixes
Some Shopify themes append a dynamic suffix to a fixed source ID, commonly separated by a double underscore. For example, an ID defined as Block-description can render as Block-description__abc123.
/* The rendered ID stays unchanged */
#Block-description
/* The rendered ID becomes Block-description__<dynamic-suffix> */
[id^="Block-description__"]
Use the starts-with selector only after confirming that the theme actually generates the suffix. When more than one element could match, scope the selector to the nearest stable container.
Swap tests and existence tests need different control logic
- Swap test: the existing control element and treatment element both have selectors.
- Existence test: control preserves the existing storefront and may not need its own selector. The treatment introduces or reveals the new element.
- DOM-mutation test: control leaves the current element unchanged while the treatment changes its copy, attributes, order, or behavior.
Do not duplicate a control element merely to satisfy a generic script structure. The correct control behavior is always to preserve the existing storefront.
Step 5: Implement the Variations Safely
Keep each experiment in one clearly bounded block. The implementation should:
- Use the PostHog feature flag key as its slug.
- Use variation keys that exactly match PostHog.
- Leave the storefront unchanged for
control. - Apply treatments idempotently so repeated evaluation cannot duplicate or compound the change.
- Fall back to control when PostHog is unavailable, returns no value, or returns an unknown value.
- Use bounded retries when Shopify renders the target asynchronously.
- Namespace added classes, IDs, data attributes, and logs with the experiment slug.
- Preserve unrelated scripts in a reused experiment container.
The following simplified pattern handles a prebuilt control and treatment, bounded waiting, control fallback, and deterministic URL previews. Replace the flag key and selectors with values verified in your theme:
When control has no dedicated element, leave its selector list empty and make sure treatment content is hidden safely by default. More complex experiments can mutate the existing DOM instead, but the same rules still apply: one bounded block, idempotent behavior, control fallback, and exact key matching.
Step 6: Add URL Parameters That Force Every Variation
Do not rely on random assignment or browser-console overrides for review. Build deterministic previewing into the experiment script.
Use the feature flag key as the URL parameter name and a valid variation key as its value:
?026-product-page-description-key-points=test
Combine it with Shopify’s unpublished-theme parameter:
https://example.com/products/example-product
?preview_theme_id=123456789
&026-product-page-description-key-points=test
A valid forced variation should take precedence over PostHog. Apply it and return before requesting a feature flag so preview sessions do not request or record an experiment assignment. Invalid values should be ignored so the script continues to normal PostHog evaluation and control fallback.
Create a complete preview URL for control and every treatment on every affected template.
Step 7: Validate and Report the Unpublished Theme
Read every modified file back from the unpublished theme and verify that the experiment slug appears only where intended. Then preview control and every treatment on a representative URL for every affected template.
Visual and functional QA
- The forced variation matches the URL parameter.
- Only the intended variation appears.
- The surrounding storefront remains unchanged.
- Selectors resolve correctly on every template.
- Repeated execution does not duplicate the change.
- Mobile and desktop layouts work.
- Product forms, add to cart, and checkout entry still work.
- No relevant browser-console errors appear.
- PostHog events remain available during normal, non-forced evaluation.
What to include in the handoff
- Unpublished theme name, theme ID, and confirmed unpublished status.
- Base theme preview URL.
- Full preview URLs for control and every treatment.
- A Shopify code-editor link for every modified file.
- Affected templates and representative storefront URLs.
- Selectors, insertion locations, and any dynamic-ID handling.
- Fallback behavior, validation results, and unresolved issues.
- An explicit statement that the theme is unpublished and nothing has launched.
A useful Shopify code-editor link follows this format:
https://admin.shopify.com/store/<store-handle>/themes/<theme-id>/editor?key=<theme-file-path>
Step 8: Create the Matching PostHog Experiment
Create the PostHog experiment only after the unpublished Shopify implementation validates. Keep it as a draft and verify that its feature flag key and variation keys exactly match the theme code.
Choose Match by Device for normal Shopify storefront traffic
Most Shopify storefront visitors are anonymous and identified through a browser or device cookie. For that architecture, set PostHog’s Match by setting to Device.
Use User only when the store has a verified persistent account identity, participants are consistently identified before the flag is evaluated, and the same external ID returns them to their original PostHog profile across sessions and devices.
PostHog person records, distinct_id values, and $identify events do not prove that User matching is appropriate. Verify the real identity architecture before configuring release conditions. When anonymous and authenticated traffic are mixed without guaranteed continuity, use Device unless the experiment is explicitly restricted to reliably identified users.
Complete the experiment configuration
- Keep the variation split stable, usually 50/50 for an A/B test.
- Use rollout percentage to control release risk instead of changing variation weights.
- Add release conditions only after identity and eligibility are verified.
- Evaluate the flag at the user-visible treatment boundary.
- Use a semantic exposure event when feature flag evaluation does not prove that the visitor could actually see the change.
- Choose a primary metric with the correct eligible denominator.
- Document overlapping experiments that affect the same population or storefront surface.
Purchase is usually the strongest business outcome for a revenue experiment, while add to cart or checkout started can provide useful secondary evidence. Do not automatically use $feature_flag_called as the first funnel step: it is valid exposure only when the flag is evaluated for an eligible visitor at the moment the tested experience becomes available.
What Happens After the Setup Is Approved?
The setup workflow ends with a validated unpublished theme and an unlaunched PostHog experiment. Publishing and launch are separate approval-gated actions.
- Obtain explicit approval for the validated unpublished theme.
- Reconfirm that the approved theme has not changed.
- Publish it.
- Verify the live storefront, affected actions, event capture, and exact feature flag.
- Launch the PostHog experiment only after the live implementation is confirmed.
Do not let “QA is complete” become automatic permission to publish. The unpublished preview handoff exists so the implementation can be reviewed before it becomes production code.
Common Mistakes to Avoid
Editing the live theme directly
Every experiment starts on an unpublished duplicate. Small changes are not exempt.
Using one selector without checking every template
Product template suffixes and page-builder variants can render different sections or IDs. Verify representative URLs individually.
Applying a wildcard to every Shopify ID
Use a starts-with selector only when the rendered DOM actually adds the dynamic __ suffix.
Making the experiment script global without need
A product-page experiment belongs in the product template’s experiment container, not automatically in theme.liquid.
Letting forced previews request PostHog assignments
Apply a valid forced variation and return before evaluating the PostHog feature flag.
Using Match by User because PostHog has person records
Anonymous cookie-based Shopify traffic should normally use Device. User requires a verified persistent identity architecture.
Running overlapping experiments on the same element
Each experiment needs its own bounded block, feature flag key, variation keys, and namespace. Avoid competing changes on the same surface unless interaction is explicitly part of the design.
Why This Workflow Produces Better Shopify Experiment Data
A trustworthy result begins before the first production visitor is assigned. Safe theme isolation prevents accidental changes. Verified selectors prevent missing or mixed treatments. Deterministic previews make review reproducible. Correct identity matching prevents unstable assignment. Exposure and metric checks make the PostHog result correspond to the experience visitors could actually see.
That is the difference between changing a Shopify page and running an experiment that can support a business decision.
Frequently Asked Questions
Can I run a PostHog A/B test on Shopify without an A/B testing app?
Yes. PostHog can assign experiment variations while Shopify renders or changes the corresponding storefront experience. The theme implementation still needs safe selectors, fallback behavior, deterministic previews, and validation.
Can I add the experiment directly to the live Shopify theme?
No. Build and validate it on an unpublished duplicate of the current live theme. Publish only after the preview has been reviewed and approved.
Where should product-page experiment code go?
Use the product template’s existing Custom HTML section when available. Reuse the established experiment container and preserve any unrelated scripts already inside it.
When should experiment code go in theme.liquid?
Only when it genuinely needs to run globally or before normal header, footer, and template sections render. Template-specific experiments should remain template-scoped.
How do I target a Shopify ID with a dynamic suffix?
Compare the source ID with the rendered DOM. If Shopify changes Block-description to Block-description__abc123, use [id^="Block-description__"] and scope it when necessary. Do not use that wildcard unless the suffix is actually present.
Does control always need its own CSS selector?
No. In an existence test, control may simply leave the current storefront unchanged while the treatment adds or reveals a new element. In a swap test, both control and treatment can have selectors.
How do I force a variation on an unpublished theme?
Implement a query parameter whose name is the feature flag key and whose value is a valid variation key. Combine it with Shopify’s preview_theme_id parameter and make the script apply the forced value before PostHog evaluation.
Should a normal Shopify experiment use Match by Device or User?
Use Device for anonymous or cookie-based storefront tracking. Use User only when a verified persistent account identity reliably restores participants to their original PostHog profile and the flag is evaluated after identification.
What happens when PostHog does not load?
The storefront should remain in control. An analytics or feature flag failure must not break the buying experience.
Can multiple experiments share one Custom HTML section?
Yes. Give every experiment its own bounded block, feature flag key, selectors, and namespace. Remove or change only the intended block and avoid experiments that compete over the same element.
What is the best primary metric for a Shopify experiment?
Use the most decision-relevant downstream outcome supported by the store’s traffic and buying cycle. Purchase is strongest for revenue tests; checkout started and add to cart can provide useful secondary evidence. Make sure the denominator contains visitors who were eligible and genuinely exposed.
How do I know the unpublished implementation is ready?
Every variation renders correctly on every affected template, selectors match once, repeated execution is safe, storefront actions still work, console errors are clear, the complete preview URLs are shareable, and the theme is still unpublished.
Subject: Great guide, but a few clarifications needed for Step 5 & Multiple Tests
Hi Amin,
Thanks for this guide! It’s one of the few that actually breaks down the coding side of PostHog for Shopify.
However, I ran into a few confusing spots while implementing this, specifically regarding Step 5 and setting up multiple concurrent tests. I thought I’d share them here to help you update the post or help future readers:
1. Confusion on “Control” IDs in Step 5
The text in Step 5 says: “Replace ‘variant-one-element’ with the actual CSS IDs of the control.”
However, the code snippet uses control: [] and test_group_1.
It would be helpful to clarify that there are two types of tests:
Swap Tests (e.g., changing a headline): You must put the Original Element ID inside control: [‘old-id’] and the New Element ID in test_group_1.
Existence Tests (e.g., adding a new review widget): If you are adding a brand new element that didn’t exist before, the control array should be left empty: control: []. I was stuck looking for a control ID for a new element until I realized I didn’t need one!
2. Running Multiple Tests
Please also explain how we’re going to do multiple tests on one page. If you can explain that with example, it would be very beneficial.
Thanks again for the resource!