All onboarding welcome emails that were A/B split tested saw between 14.31-16.33% increase in CTRs (a 67-166% performance increase depending on the email) across the series of three. This was through cleaner and more concise email designs, ensuring the information communicated in each took into account the timeline of a member's onboarding journey on the website and questions they may start to have in their early days.
We also saw a small increase in activations (first cashback transaction).
-
Client
TopCashback USA
Myself and one other UX designer worked on the entire project from start to finish, including the research, testing, and design.
Background
When a user signs up to the site, they receive a series of 3-4 emails as part of a welcome campaign series. This is meant to inform how to use the service and some inspirational brands and offers to get them started.
We were tasked to improve and modernise the onboarding "Welcome" email series for the TopCashback service.
Main Goal: Increase CTRs of the emails (Clickthrough rates)
Secondary Goal: Increase activation rates of these new members (first transaction)
Research and implementation
We mapped out what the typical sign-up journey was like for a member, gathering data on the existing engagement and activation rates through quantitative data and custom email reports.
After understanding the email sends journey, we realise that the first email was sent to the new member whilst they are still browsing the website for the first time. Therefore we needed to ensure the email did not include too much information about how the service worked as this would be covered already as part of the normal on-boarding on the site. To avoid cognitive overload, we determined a simpler first welcome email to reassure them was a potential.
By the time the second email arrived, it would be a couple of days later, and we took this opportunity to answer any questions they may have had about the service after signing up. These included our top most FAQs from member feedback. We also presented three popular brands that TopCashback has on site as a means of inspiration and to form trust.
The third email was sent a few days later, and we decided to showcase in an interesting visual way the variety of categories they could get cashback on, even on items they did not know they could do. This was based on quantitative data of popular categories as well as a member survey.
This timeline was important to understand because it would determine the content theme per email-- this was insight from our research around the 5 levels of awareness.
We looked at competitors for inspiration in terms of content design, but we also took this opportunity to clean up the emails and keep them focused -- Something that was obvious over the years of more and more things being added to the Welcome emails, it needed to be toned down. We applied general UX laws including Miller's Law on all three emails so as not to overwhelm the member whilst still communicating useful and important information.
This project involved several workshops (some with stakeholders), looking at existing quantitative data, information hierarchy, grouping, wireframes, etc
Testing
We performed an A/B split of all three emails versus their live/control counterparts. The split was simply based on the Member ID numbers (odds vs evens). The test lasted 2 weeks.
Results
The tests revealed an overall large increase in CTRs which was the main goal.
1st Email: +14.31% CTR (a 67% performance increase, conclusive)
2nd Email: +15.85% CTR (a 155% performance increase, conclusive)
3rd Email: +16.33% CTR (a 166% performance increase, conclusive)
Activation rates have a longer timeline allowed to them, but the Insights team concluded from early numbers that there was overall a small correlation between viewing these new emails vs transacting within the next 24 hours.
The massively successful new series was rolled out on the USA site and soon after to the UK, Germany, and China sites.
BEFORE (Left) / AFTER (right)
Above: Test of Email 1 of Original/Control (left) vs New Variation (right)
Above: Test of Email 2 of Original/Control (left) vs New Variation (right)
Above: Test of Email 3 of Original/Control (left) vs New Variation (right)