An AI model that predicts the optimal time to send a marketing message. Marketing teams use this technology to personalize the messaging experience for their consumers and drive brand loyalty and conversions.
My role
Model development, experimentation, and integration. Led the ideation and creation of the supporting machine learning model, including customer experimentation (A/B testing models) and integration of the model's output with backend systems.
Customer discovery lead. Recruited customers for discovery sessions to evaluate business viability and usability risks, to ensure the ML solution drove value for the marketer and a great experience for their end user.
Solution champion. Worked with marketing to build launch materials including blog posts and webinars, and toured with the Airship team for the Elevate roadshow, presenting on stage in NYC, Atlanta, Santa Monica, SF, and Toronto.
Execution
Predicting the best time to send started out as a hack week project. It gained internal support and interest across Airship teams, from sales (“I can sell that tomorrow!”) to customer success (“customers want more recommendations!”).
With backing from stakeholders, development happened in three phases: customer discovery, model design, and productization.
Customer discovery. We spoke with several customers about how they decide when the best time to send a message might be. Every customer we spoke with was interested in a “best time” model, for the chance to save themselves time and potentially increase the open rate of the messages they sent.
Model design. We took several different approaches to developing this model, and our concept of the problem we were solving evolved over time. The goal became predicting when a user will open an app by learning from their previous opening behavior.
Productizing the solution. With the model ready to emit user-level predictions, the next step was feeding those predictions into the Airship backend system. We instrumented the model to integrate send time predictions into both the Airship messaging pipeline (“send to a user's best hour”) and Airship's data product line (“show me a distribution of users' best hours”).
Challenges
Translating local time to global time. Since Airship customers exist across the world, I worked with engineering to ensure global timezone translations worked with backend system requirements.
Don't just tell me, show me. To overcome the natural skepticism marketers have toward AI, we built dashboards showing user behavior alongside the model's predictions.
Scaling the model with customer usage. As we onboarded more customers, we ran into scaling issues processing predictions for a growing number of users. We solved this by only submitting updates for predictions that had actually changed.
Achievements
Improved user open rates. Through A/B tests with marketing teams, we demonstrated that the send time optimization model improved user open rates by up to 14%.
Quick adoption by media and retail customers. Marketing teams went all in, sending 100% of their marketing messages via automated predictions within three months of launch.
Unexpected benefit: audience traffic optimization. Since predictions are individualized to each user's behavior, the model naturally spreads messaging across the hours of the day — a major benefit for large retail apps.