[fusion_builder_container hundred_percent=”yes” overflow=”visible”][fusion_builder_row][fusion_builder_column type=”1_1″ background_position=”left top” background_color=”” border_size=”” border_color=”” border_style=”solid” spacing=”yes” background_image=”” background_repeat=”no-repeat” padding=”” margin_top=”0px” margin_bottom=”0px” class=”” id=”” animation_type=”” animation_speed=”0.3″ animation_direction=”left” hide_on_mobile=”no” center_content=”no” min_height=”none”][latexpage]
Often times, when running your first A/B tests, you will notice that the data given by your testing tool differs, sometimes markedly, from the data you see in Google Analytics. Your Traffic and Conversion Rate numbers will be different, because each tool measures each metrics in a different way.
I’ll explain quickly what the differences are and how you can extract from GA the numbers you need to plan your tests accurately.
Traffic is measured in different ways
Google Analytics: traffic is measured in sessions
Google Analytics measures traffic by Sessions. A session starts when a visitor sees any page on your website and ends when they leave your website or when they are not active on your website for 30 mins (but they can navigate away and come back within 30 mins and it will continue the same session). The identification of this user is cookie-based, so if she changes browsers or devices, unless you use advanced configuration with Universal Analytics, it will be counted as a new session.
You can see that if a the same visitor comes to your website multiple times a day but all more than 30 mins apart, multiple sessions will be counted for the same visitor.
GA uses the number of sessions as its default metric when estimating your traffic. There are a few other parameters that define a session and may make a small difference. You find more info from Google on this: How a session is defined in Analytics?
A/B Testing tools: traffic is measured in Unique Visitors
All A/B testing tools measure traffic in terms of Unique Visitors. For once, this metric means (nearly) exactly what it says: each unique visitor is counted only once. The only caveat is that testing tools are using cookie-based methods of tracking, so if the same person visits your website from her laptop and then from a phone, she will be counted as 2 Unique Visitors. But as long as users use the same browser, they are counted only once.
And the Conversion Rate calculated is based on Traffic numbers
As a reminder, the generic definition of Conversion Rate can be expressed as such:
\[
\text{Conversion Rate} = \frac{\text{Conversions}}{\text{Traffic}}
\]
How Google Analytics computes the Conversion Rate you see in GA
The Traffic number used is the total number of sessions during the period considered. The Conversions number used is the number of sessions during which a conversion event occurred. For example, if you are measuring your e-commerce CR, then it is the number of sessions where a visitor made at least one purchase. The same computation is used for Goals Conversion Rates. So if we consider your e-commerce CR, Google Analytics measures it as such:
\[
_\text{Google Analytics}\text{Conversion Rate} = \frac{\text{Number of Sessions with at least 1 Transaction}}{\text{Total number of Sessions}}
\]
Be careful: the number used for Conversions is not the Total Transactions number as often indicated on blogs, etc. It is the number of Sessions during which at least 1 Transaction occurred. This means that if the same user makes multiple transactions within 1 session, only one Conversion will be counted. On the other hand, if the same user makes 2 transactions in 2 separate sessions the same day, then 2 Conversions will be counted.
How Testing tools computes the Conversion Rate they use in test results
The Traffic number used is the number of Unique Visitors tested. The Conversions number is the number of Unique Visitors that converted once for the goal you are measuring:
\[
_\text{A/B Testing tool}\text{Conversion Rate} = \frac{\text{Number of Unique Visitors converting at least once}}{\text{Total number of Unique Visitors}}
\]
For e-Commerce CR, this means that if the same visitor makes multiple purchases, only the first will be counted as a Conversion.
Which Conversion Rate should you use?
That depends ultimately on the type of website you are running, but one thing is sure: if you want to estimate the time to complete a test (for example using our planning calculator), you should not use Google Analytics’ definition of Traffic or Conversion Rate, because this is not what will be used by your testing tool. Instead, use the following Custom Report in Google Analytics to get your Unique Visitor numbers (named “Users”) and use it to plan your tests.
Photo credit: Leon Riskin on Flickr[/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]