If you’re running a Shopify store and want to set up an A/B test, you’re in the right place.
In this post, I’ll walk you through a clear, step-by-step process and share exactly how we did it on our own site to help you avoid any mistakes.
Before you begin, ensure the following requirements are met to successfully setup PostHog A/B test on Shopify:
- You have an active PostHog account with valid login credentials.
- PostHog is already installed on your Shopify website. If not, follow this PostHog installation guide for Shopify or read this article to set it up quickly.
- This method only works when your Shopify elements have unique and static CSS IDs. Make sure your test elements are clearly identifiable to allow PostHog to track them accurately.
Step 1: Duplicate the Theme You Want to Test
By duplicating the page, you create a safe working copy that allows you to experiment freely without affecting the original version.
In your Shopify admin, go to:
Sales Channels → Online Store → Themes
- Find your current live theme and click Actions → Duplicate to create a copy.
- Rename the duplicated theme to reflect that it’s your test version (e.g., “Test Theme – New Layout”).
- Click Customize on the duplicated theme and make the desired changes to the specific templates.
- Important: Do not delete or modify the original (control) version. Keep it unchanged so you can accurately compare performance later.
- For any test-specific elements, ensure they are hidden by default using (display: none;) in their custom CSS. This ensures they only appear on designated test pages and do not affect the live experience.
Step 2: Create the Experiment in PostHog
Log into PostHog and go to Experiments via the left sidebar. Click the New Experiment button.
Fill in the required fields:
We use this general format for experiment name:
[Experiment number] – [Step of the Funnel/Page] – [Test Hypothesis]
For the feature flag key, start with a version of the experiment name—then make it code-friendly by converting it to lowercase, removing spaces, and abbreviating or numbering the funnel step/page.
Take a moment to read this article for a clearer understanding of what you should do.
Add your control and test variants under the Variants section.
Click Save as draft when finished.
Click here for a step-by-step guide on how to create an experiment in PostHog.
Step 3: Fetch the Variant Elements
Collaborate with the developer or client to determine which parts of the page will change during the test. These elements should have unique CSS IDs or data attributes to make them easily targetable.
In this step, all elements that need to be hidden, displayed, or modified must be clearly identified and documented to ensure everything is organized and ready for implementation in the script.
To locate the correct CSS selectors, use your browser’s Inspect tool (press F12) and examine the elements under the “Elements” tab. Each relevant section should ideally have a unique identifier. If CSS IDs or attributes are not present or visible in the HTML, reach out to the developer to have them added or clarified.
Step 4: Insert the A/B Test Script
We use a custom script to run the A/B test in Shopify.
In the theme editor, scroll to the bottom of the page and add a new Custom Content section just above the footer. Inside it, you’ll find an HTML block; click it to open the editor in the sidebar.
Insert the following script there:
We use custom code to hide the control elements and display the test elements based on the assigned variant.
If you’re running multiple test variants, you should create additional variants in PostHog and name them accordingly (e.g., Test Group 1, Test Group 2, etc.). Then, update your code by adding additional conditional blocks after the first test group to handle each variant separately.
The script will require adjustments based on your test setup — these are explained in the next step.
By clicking on the newly added section in the sidebar, you can access its Custom CSS settings. To make the section invisible and ensure it doesn’t affect the page layout, apply the following CSS:
Don’t forget to save your changes.
Step 5: Adjust your script to match the content of the page
Update the script from last step with the real CSS selectors. Here are the variants:
- Replace “YOUR_FEATURE_FLAG_KEY” with the actual Feature Flag Key from your PostHog experiment.
- Replace “variant-one-element” and “variant-two-element” with the actual CSS IDs of the control and test elements.
This ensures that the control is hidden and the correct variation is shown when the test group is active.
Step 6: Quality Check the Variants
Start by previewing the theme and verifying that the test is set up correctly. To test each variant, use Feature Flag Key in the page URL as follows:
Format: [your page url]/?[Feature Flag Key]=[variant name]
For example, we’ve built a dedicated test page on our website to run an A/B test. In this experiment, we have one control group and two test variations: Test Group 1 and Test Group 2.
The only difference between these three versions is the heading, which allows us to isolate the impact of this specific element. To keep things clear, each version displays a heading that identifies the group (e.g., the control page shows “Control Group” as its heading).
When a user visits the test page, they are randomly directed to one of the three versions. We track user behavior and performance across these variants using PostHog.
To verify that each variant is loading correctly, simply add the Feature flag key parameter to the end of the URL like the upper format. For example:
Test Group 1:
https://99ways.io/experiment-test-page/?001-about-page-Heading-test=test_group_1
Repeat this for each variant to ensure they render as expected.
Step 7: Publish Your A/B Test on PostHog
Once all tests are confirmed to work correctly:
Set the edited theme as Current Theme in Shopify.
To launch the experiment as part of your effort to setup PostHog A/B test on Shopify, you need to define the primary metric first:
Open your saved experiment in PostHog.
Scroll to Primary metrics and click Add primary metric.
Choose Funnels as the metric type and define your steps (e.g., Feature flag called → Form Visit).
Adjust the Attribution type, set the Conversion window limit, and save.
Scroll to the top and click the Launch button.
Your experiment is now live and will start tracking performance based on your chosen metric.
Frequently asked qusetions
Do I need a developer to set up a PostHog A/B test on Shopify?
You don’t strictly need a developer, but it helps a lot.
If your theme already uses clear, static CSS IDs and you’re comfortable editing theme code and custom HTML sections, you can follow the guide and do it yourself.
If elements don’t have IDs, or you’re using complex custom sections or third-party apps, you’ll probably need a developer to add stable IDs and help adjust the script safely.
Think of it this way: if touching theme code makes you nervous, involve a dev.
Can I run a PostHog A/B test without duplicating my Shopify theme?
Technically yes, but it’s not recommended.
The duplicate theme gives you:
A safe environment to test visual or structural changes.
A clean Control vs. Variant separation.
The ability to roll back instantly by switching the active theme.
Running experiments directly on the live theme increases the risk of breaking something or accidentally pushing an experiment change to 100% of traffic.
What if my Shopify elements don’t have unique CSS IDs?
Then your first job is to create them.
Ask your developer to add unique IDs or data attributes to the elements you want to test (buttons, sections, headings, forms, etc.).
Avoid relying on generic selectors like
.btnor.heading– they’ll likely target multiple elements.Once IDs exist, update your script so it targets those specific IDs.
Without stable selectors, your A/B test will be fragile and hard to debug.
Will this PostHog A/B test method slow down my Shopify store?
Not if implemented correctly.
PostHog’s snippet is lightweight; the extra logic in your custom HTML section only shows/hides existing elements.
The section itself is hidden with CSS (
display: none;), so it doesn’t affect layout or CLS.The real risk is bad code (heavy libraries, blocking scripts). The approach described in your article uses simple, targeted JS and CSS, which should have negligible impact.
If performance matters, run a before/after check in Lighthouse after implementing the test.
How do feature flags in PostHog actually control variants on Shopify?
Feature flags in PostHog decide which variant name each user gets (e.g., control, test_group_1, test_group_2).
Your custom script:
Calls PostHog’s feature flag API using
"YOUR_FEATURE_FLAG_KEY".Receives the assigned variant.
Uses that value to hide or show specific elements on the Shopify page (via CSS IDs / selectors).
PostHog handles randomization and consistency; the script just translates that decision into what the user sees.
Can I run multiple PostHog experiments on Shopify at the same time?
Yes, but be careful.
Each experiment must use its own feature flag key.
Make sure experiments don’t target the same element (e.g., two tests both changing the hero section).
Keep your scripts organized: ideally, one structured script that checks multiple flags, rather than random snippets scattered around.
If multiple experiments overlap on the same area, your results will be noisy and hard to trust.
How long should I run a PostHog A/B test on Shopify?
You stop when you have statistically useful data, not when you’re bored of waiting.
As a rough rule of thumb:
Run for at least one full business cycle (e.g., 2–4 weeks) so you cover weekdays, weekends, and typical promo patterns.
Aim for a minimum number of conversions per variant (not just visitors). If one variant only has a handful of sales, don’t trust the result.
Use PostHog’s experiment results and confidence indicators as a guide, but anchor your decision in sample size + business judgment.
What’s the best primary metric for a Shopify A/B test in PostHog?
It depends on what the page does, but generally:
Product pages / collections: add to cart → checkout → purchase (purchase is usually the primary metric).
Landing pages / offers: key action (e.g., “Add to cart” or “Start checkout”) as primary, purchase as secondary.
In PostHog you’ll typically:
Choose Funnels as metric type.
Define steps like “Feature flag called → Add to cart → Purchase”.
Use purchase as the primary conversion for revenue-related tests.
Don’t pick vanity metrics like page views; they won’t change your decisions.
How do I know if my PostHog A/B test is tracking correctly?
You verify at three levels:
Visual: Use the URL trick with
?[feature_flag_key]=[variant_name]to force each variant and confirm the right elements show/hide.Event-level: In PostHog, check that page views and key events are being captured for your test page.
Experiment-level: Open the experiment and confirm traffic is flowing into all variants and the primary metric is incrementing.
If any variant shows 0 traffic or 0 conversions after a reasonable period, something is misconfigured (flag key, selectors, or theme not published).
Can I use this method with builders like PageFly, GemPages, or Shogun?
Yes, but it’s more fragile.
These builders often generate dynamic, nested markup where IDs/classes change or are reused.
You still need stable selectors; otherwise, your script might stop working after a builder update.
Ideally, ask the builder or a developer to add stable IDs to key elements, then target those from your script.
If the builder makes that impossible, you might be better off testing at the section level or using a different testing strategy.
Does this A/B testing method work with multilingual or multi-currency Shopify stores?
Yes, with caveats:
The logic is driven by feature flags + CSS selectors, not language or currency directly. As long as the DOM structure and IDs are consistent across languages/currencies, your test will behave the same.
If different translations or currencies use different templates or layouts, you may need separate experiments or extra logic per locale/currency.
Document which templates and markets the experiment is meant to cover before launching.
Can I use PostHog A/B tests on Shopify to analyze traffic sources or devices?
Yes.
PostHog tracks events with properties like URL, referrer, UTM parameters, browser, and device. Once your experiment is running, you can:
Break down experiment results by UTM source / campaign.
Compare mobile vs. desktop behavior for each variant.
This is a big advantage vs. basic Shopify tests: you can see if a variant wins overall but loses badly on mobile or on a specific traffic source.
What are the most common mistakes when setting up PostHog A/B tests on Shopify?
The big ones:
No duplicate theme: editing the live theme directly and breaking layout or mixing experiment changes with other edits.
Unstable selectors: targeting generic classes instead of unique IDs, so the test breaks when the theme updates.
Wrong feature flag key: mismatch between the PostHog experiment key and what you use in the script or URL.
No visual QA: not forcing each variant via URL and checking them manually before launch.
Bad primary metric: optimizing for clicks or views instead of actual revenue-related actions.
Avoid these and your tests will be cleaner, more reliable, and easier to scale.
