Landing Page A/B Experiment Analysis

Landing pages play a critical role in digital marketing performance. Small changes in layout, messaging, or user experience can directly affect conversion rates and revenue outcomes.

For an e-commerce business, selecting the winning landing page version should not be based only on visual preference or raw conversion volume. A proper decision requires statistical validation to confirm whether observed differences are meaningful and unlikely to be explained by random variation.

This project evaluates both conversion performance and spending behavior to determine whether Landing B should replace Landing A as the primary version.

• Does Landing B generate a higher conversion rate than Landing A?

• Do users who convert on Landing B spend more than users who convert on Landing A?

• Are traffic source and conversion behavior statistically associated?

• Does user type influence conversion performance?

• Which landing page version should be recommended based on statistical evidence?

• Landing B outperformed Landing A in both conversion rate and average spending among converted users. Landing A achieved a conversion rate of 12.57%, while Landing B achieved 15.96%, representing an absolute lift of approximately 3.38 percentage points and a relative improvement of roughly 27%.

• The two-proportion z-test confirmed that the conversion rate difference between both landing pages is statistically significant. This means the observed improvement is unlikely to be explained by random chance.

• The spending analysis also showed a statistically significant difference between converted users from Landing A and Landing B. Converted users exposed to Landing B generated higher average spending, making Landing B stronger not only in conversion rate but also in revenue potential.

• Traffic source was statistically associated with conversion behavior. Organic generated the largest absolute number of conversions due to its scale, while Email showed the highest relative conversion rate.

• User type, however, did not show a statistically significant relationship with conversion. Although new users generated more conversions in absolute terms, this was mainly because they represented a larger share of total traffic.

• Landing B should be implemented as the primary landing page version because it demonstrated stronger conversion performance and higher average spending among converted users.

• The business should continue monitoring Landing B after implementation to confirm that the uplift remains stable under real operating conditions.

• Organic and Ads should remain important acquisition channels due to their high conversion volume. However, Email deserves further analysis because it showed the strongest relative conversion rate.

• The company should avoid prioritizing campaigns only based on user type, since the analysis did not find a statistically significant relationship between new/returning users and conversion.

• As a next step, the experiment could be extended by analyzing revenue per visitor, acquisition cost by channel, device-level performance, regional performance, and post-conversion behavior such as retention or repeat purchases.

• A/B testing analysis

• Hypothesis testing

• Two-proportion z-test

• Independent samples t-test

• Chi-square test for categorical association

• Data validation and experiment review

• Conversion rate analysis

• Business-oriented statistical interpretation

• Data visualization with Python

• Communication of results for decision-making