project overview

D2C members come from various sources including Betterup.com, search engines, and SEM landing pages, showing different levels of interest in coaching. Although some express specific intent through search terms like "career coach" or "communication skills," we don't currently use this data during onboarding. Currently, all customers go through the same onboarding process, which assumes limited coaching experience and a preference for a generalist coach, regardless of how they arrived.In this experiment we aim to test customizing onboarding for our users to increase retention.

problem

BetterUp D2C offers Coaching at good value, but members cancel before 35 days because a meaningful connection between coaches and users has not been established.

hypothesis

I hypothesized we would drive higher conversion and retention by leveraging source information to customize the onboarding and coaching experience, thereby connecting customers directly with coaches who aligned with their search query.

success looks like

Primary: Subscription conversion 
Secondary: 30-day retention

requirements

Implement Communication
Coaching as the first test because it is the most searched term by users
Maintain current UI because it is tied to the B2B business and is a high lift to change

my approach

For this project, I stayed tight with the Product Manager and fostered cross-collaboration within the D2C organization, ensuring unified efforts and information sharing. This led to a seamless, personalized user journey, leveraging the diverse strengths of the team.

funnel

First, I identified our efforts as being placed in the Engagement (Landing Page) and Consideration (Onboarding) portions of the marketing funnel. This helped to set context for where our users would be in the big picture of the flow.

‍

Second, I looped in engineering to ensure what we were designing would work from a development perspective as they would be implementing the updates to web and onboarding. We continued touch points through the project to the end.

Funnel focus: Engagement and Consideration

web

The Landing Page work would be covered by the web team who I maintained consistent communication with to ensure alignment with the most relevant high-ranking search terms while developing this flow. Majority of the traffic comes in from desktop and this would be built out there and on mobile.

Landing page focused on high-ranking search terms | desktop view

coach development

Next, I collaborated closely with the coach development manager to define Communication Coaching topics that aimed to delve into the specific requirements of each customer. These were Communication Effectiveness, High Stakes Conversations, and Presenting with Confidence.

Communication Effectiveness

High Stakes Conversations

Presenting with Confidence

onboarding flow

From there, I reviewed the control onboarding flow and identified where to make adjustments to better accommodate our new users coming in from the communications landing page.

‍

I saw opportunity in 3 places:

‍

Low-lift: Switching from videos to static testimonial quotes about communication coaching is a low-risk, high-value change due to historically low engagement with the videos
‍
Medium-lift: Provide an escape hatch in case users do not see Communication Coaching as a match 
‍
High-lift: Streamlined the customer journey, offering four distinct Communication Coaching paths, an option for unlisted communication topics, and the ability to choose the general onboarding flow if preferred

results

We saw a +90% improvement in CVR over the control experience. 
‍
Treatment 1-day Subscription CVR is showing a nearly 100% improvement (p < 0.01) over control. Control seemed to perform unusually poorly compared to baseline Comm LP conversion, so dug in a bit deeper here. 
‍
Treatment outperformed baselines on its own, both of which outperformed control (note: ignore July baseline numbers, there were issues with LP events so CVR looks inflated)