Autopilot
Let the best invest for you.
Overview
Autopilot is a consumer mobile app that automatically copies and executes trades for users within their own brokerage.
During my time at Autopilot as UX Researcher, I led our initiatives to increase retention by 7% and scale product operations. Due to collective growth initiatives, the ARR increased from $1M to $4M.
To date, Autopilot has over 400,000 downloads in the App Store, trending Top 100 over 10 consecutive weeks, and approaches $5M ARR.
Highlights
Increased recovered revenue rates by 127% MoM and MRR by 6% as UX Research Lead on payment collection recovery with Design and Engineering
Increased user engagement and increased retention by 7% through the redesign of pilot activation flows and custom push notifications via discovery decks and A/B testing
Improved conversion rates for incomplete activations by 3% through the optimization of pilot selection processes and the introduction of personalized pilot recommendations based on user financial goals
July — August 2023
Recovered Revenue Initiative — Increasing MRR by 6%
One of the first projects I initiated at Autopilot was implementing stronger revenue recovery mechanisms and optimizing failed subscription payment notifications. This was a relatively high business impact initiative due to successfully recovered failed payments.
Team
UX Researcher, Product Operations
Sarah Bai
Executive Stakeholders
Brian Schardt, CEO and Co-Founder
Developers
Scott Schardt, Co-Founder
Michael Horn, Senior iOS Engineer
Product & Design
Carter Strickling, Head of Product & Design
Problem
Users were not fixing their failed subscription payments. The majority of users that received any initial revenue recovery communications did not take any action, and we did not have full insights into all the payment failure reasons and why users seemed to have a low action rate or incentive to fix their payment method.
I queried failed payment reasons from a sample of hundreds of incomplete payments of active subscribers. While the details of this data are confidential, some of the unsuccessful payments were due to insufficient funds, bank rejection, invalid account info or expired card.
We needed to create communications with actionable tasks for the user to ensure service continuation and proper payment. Additionally, filtering all the 30+ decline codes from our payment processor, some of the reasons did not warrant direct user action such as “do not honor”.
After a survey group and measuring top payment-related requests, we also found out users wanted to be able to manage their payment better as in delete outdated methods and switch cards to a valid card.
Goals
Discover and understand the primary causes for failed payments and failed recovery of payments for an enduring solution
Proactively help users ensure continuation of trading service
Increase recovered payments by communicating actionable tasks in an accessible way
Make it easier for users to fix failed payments
Value
Revenue recovery — increased revenue by recovering failed subscription payments
Retention — successful subscription payments and renewals bolstered lifetime value
Delight — smooth continuation of service and proactive communication enhanced the user’s experience and expectations of Autopilot
Process
Understand the scope of the problem and why users were not updating their failed payment method — Since this was a large unscoped user state with many subsets, similarly to the Capital project of suspected fraud transaction user states, we needed to understand the many subsets of failed payments and decide which reasons were actionable on the user.
Understanding the User
Why are users not responding to the current communications about failed payments? What is the rate of fixed recovery?
Which reasons were an intentional failed payment?
What is the current state of any communications to notify the user? What is the cadence of these notifications and how do users feel about the delivery overall?
Sort and analyze the failed payment cases to prioritize specific failed reasons — Insufficient funds was a fairly common reason for failed payments and it was relatively low user effort to fix. We used this framework to prioritize communication for users, measuring impact by number of instances, likelihood of action and user intent, and the scope of the action itself for payment recovery.
Design communication notifications and app improvements for scalability — Based on the user-actionable failed payment reasons, we needed to map certain notifications with CTAs to the failed payment reason. The method of communication for each reason needed to be standardized and we needed to test notification designs to optimize for recovery and action rates.
Monitor solution performance and adjust for further optimizations
Method
We pulled and sorted a meaningful sample size of failed payments from a recent subscription cycle. I used SQL to query the failed payments from Redash and then pulled the file to organize the failed payment reasons.
I worked with our Engineering and Executive Stakeholders to analyze the current automated failed subscription payment notifications that were set up through Stripe, our payment processor. We were also able to deep dive into the rate of failed payments over the last few billing periods which allowed me to build a stronger case for project prioritization.
We discovered that there was no contextual explanation for failed payment notifications and therefore no specific action for users to take to fix the failed payment.
This was the only user notification for all failed payment reasons when I researched the current state of declined payment communications. I initiated a project that overhauled our communications templates and conversion rates, a separate but related research initiative.
We mapped out the failed payment reasons from Stripe and the proposed actions, if any, for the user. See the snippet below for some of the failed payment reasons. I grouped them by similar user CTA so that we could organize automated communications as the next step.
For example, the grouped failed payment reasons below generally required bank intervention, unlike a different group of decline codes including insufficient_ funds when the user could fix the failed payment on their own.
Initial mapping of decline codes to CTAs and communication messages. We grouped CTAs by decline code.
Next, we optimized communication methods and copy for rate of action and successful recovery. To do this, we beta-tested contextual notifications as well as A/B tested CTAs and other content in each type of notification.
We also reviewed specific user tickets on failed payment method updates and pushed the changes on those users as a usability study (N > 15).
Due to this usability test, I also noted several user comments about the app feature to edit payment methods. Many users had expressed confusion for:
Which payment method was active for payment subscription
The ability to update or remove a payment method. At the time, there was no indication in the UI that payment methods could be switched through a tap
This was the UI for payment methods before our updates. There was no indication of the currently active card, or switching and removing payment methods.
Outcome
We launched contextual payment recovery communications, payment method feature improvements, and several internal backend improvements in how payment methods were synced.
This included:
1. New app features — remove payment methods, switch payment methods, and set new default payment tags. This improvement significantly improved user self-serve payment method update requests, as well as clarified to the user the card on file which gave them peace of mind. One measure of this improvement included a meaningful decrease in our ticket requests for manual payment method changes. Actual metric redacted, contact for more information.
2. In-app dismissible banner — since our users' average app engagement rate was relatively high as a baseline, our users were more likely to act for in-app messages. We needed to over-communicate the importance and urgency of the user's failed payment so we surfaced a banner that redirected to Payment Settings.
3. Contextual push notifications — we already implemented trade alert notifications so the payment method push notification needed to be distinct and more actionable over more regular alerts. Contextual messages needed to be concisely enough to communicate the specific cause and CTA for the user's failed payment.
4. Automated contextual email notifications with repeat reminders — we moved our failed payment notifications in-house, which allowed us more control over the messaging, design, and contextual CTAs. Timing was optimized through A/B testing.
5. Backend updates for payment handling — we improved how we synced our subscription payment statuses with our payment processor so that new payment methods added were automatically the default.
We added a default active payment method indicator and a feature to remove payment methods so that users were more empowered to manage their financial connections on Autopilot with peace of mind.
Result Metrics
MRR increased 6%
Recovered revenue increased 127% MoM
Number of failed payments decreased by 36%
Some improvements are redacted, contact me for more info
Challenges and Learnings
The greatest challenge was routing all of the payment failure reasons from the payment processor API and translating the root causes to decide how to handle each reason. Since some failed payments were due to deliberate customer blocks, those users shouldn’t expect an urgent CTA.
The user state grouping and communications needed to make sense for all 30+ applicable payment failure reasons.
Valuable User Insights
Users may actually switch payment methods often and want to keep more than one card on file. This was often due to spend restricted cards, which also gleans more on our user demographics. We improved designs to make it easier to switch methods, and clarified the default card.
If a user does not react to the first two notifications in general, there is a larger chance of lost revenue. We optimized our notification timing and became less spammy.
Emails may be a better channel for actual intent of action. Push notifications could be too ephemeral and the required action may be delayed or forgotten. We improved our copy to express more urgency for push notifications.