Before you begin, ensure the following requirements are met to successfully setup PostHog A/B test on Shopify:
- You have an active PostHog account with valid login credentials.
- PostHog is already installed on your Shopify website. If not, follow this PostHog installation guide for Shopify or read this article to set it up quickly.
- This method only works when your Shopify elements have unique and static CSS IDs. Make sure your test elements are clearly identifiable to allow PostHog to track them accurately.
Step 1: Duplicate the Theme You Want to Test
In your Shopify admin, go to:
Sales Channels → Online Store → Themes
- Find your current live theme and click Actions → Duplicate to create a copy.
- Rename the duplicated theme to reflect that it’s your test version (e.g., “Test Theme – New Layout”).
- Click Customize on the duplicated theme and make the desired changes to the specific templates.
- Important: Do not delete or modify the original (control) version. Keep it unchanged so you can accurately compare performance later.
- For any test-specific elements, ensure they are hidden by default using (display: none;) in their custom CSS. This ensures they only appear on designated test pages and do not affect the live experience.
Step 2: Create the Experiment in PostHog
Log into PostHog and go to Experiments via the left sidebar. Click the New Experiment button.
Fill in the following fields:
- Name your experiment: Use this general format: [Experiment number] – [Step of the Funnel/Page] – [Feature Name]
- Feature flag key: Also structured, removing spaces, and keeping everything lowercase for easier implementation in code., e.g., [Test number]-[Experiment name]
- Variants: Add your control and test variants
Take a moment to read this article for a clearer understanding of what you should do.
Click Save as draft when finished.
Click here for a step-by-step guide on how to create an experiment in PostHog.
Step 3: Fetch the Variant Elements
Collaborate with the developer or client to determine which parts of the page will change during the test. These elements should have unique CSS IDs or data attributes to make them easily targetable.
In this step, all elements that need to be hidden, displayed, or modified must be clearly identified and documented to ensure everything is organized and ready for implementation in the script.
To locate the correct CSS selectors, use your browser’s Inspect tool (press F12) and examine the elements under the “Elements” tab. Each relevant section should ideally have a unique identifier. If CSS IDs or attributes are not present or visible in the HTML, reach out to the developer to have them added or clarified.
Step 4: Insert the A/B Test Script
Use a custom script to run the A/B test in Shopify.
In the theme editor, scroll to the bottom of the page and add a new Custom Content section just above the footer. Inside it, you’ll find an HTML block; click it to open the editor in the sidebar.
Insert the following script there:
We use custom code to hide the control elements and display the test elements based on the assigned variant.
If you’re running multiple test variants, you should create additional variants in PostHog and name them accordingly (e.g., Test Group 1, Test Group 2, etc.). Then, update your code by adding additional conditional blocks after the first test group to handle each variant separately.
The ** ** are indicating the variant part of the script.
The script will require adjustments based on your test setup — these are explained in the next step.
By clicking on the newly added section in the sidebar, you can access its Custom CSS settings. To make the section invisible and ensure it doesn’t affect the page layout, apply the following CSS:
Don’t forget to save your changes.
Step 5: Adjust your script to match the content of the page
Update the script from last step with the real CSS selectors. Here are the variants:
- Replace **Feature Flag** with the actual Feature Flag Key from your PostHog experiment.
- Replace **TG1_CSS_ID** and **TG2_CSS_ID** with the actual CSS IDs of the control and test elements.
This ensures that the control is hidden and the correct variation is shown when the test group is active.
Step 6: Quality Check the Variants
Start by previewing the theme and verifying that the test is set up correctly. To test each variant, use Feature Flag Key in the page URL as follows:
Format:
[your page url]/?[Feature Flag Key]=[variant name]
For example, we’ve built a dedicated test page on our website to run an A/B test. In this experiment, we have one control group and two test variations: Test Group 1 and Test Group 2.
The only difference between these three versions is the heading, which allows us to isolate the impact of this specific element. To keep things clear, each version displays a heading that identifies the group (e.g., the control page shows “Control Group” as its heading).
When a user visits the test page, they are randomly directed to one of the three versions. We track user behavior and performance across these variants using PostHog.
To verify that each variant is loading correctly, simply add the testslug parameter to the end of the URL like the upper format. For example:
Test Group 1:
https://99ways.io/experiment-test-page/?001-about-page-Heading-test=test_group_1
Repeat this for each variant to ensure they render as expected.
Step 7: Publish Your A/B Test on PostHog
Once all tests are confirmed to work correctly:
Set the edited theme as Current Theme in Shopify.
To launch the experiment as part of your effort to setup PostHog A/B test on Shopify, you need to define the primary metric first:
Open your saved experiment in PostHog.
Scroll to Primary metrics and click Add primary metric.
Choose Funnels as the metric type and define your steps (e.g., Feature flag called → Form Visit).
Adjust the Attribution type, set the Conversion window limit, and save.
Scroll to the top and click the Launch button.
Your experiment is now live and will start tracking performance based on your chosen metric.
