Introduction
In the highly competitive mobile gaming industry, acquiring and retaining profitable users is crucial for success. The freemium model—offering games for free while monetizing through in-app purchases (IAP) and advertising—dominates the industry but presents challenges in predicting customer lifetime value (LTV) and customer churn due to highly skewed revenue distributions and diverse user behaviors. A small percentage of users, often less than 2%, generate the majority of revenue, making accurate predictions essential for optimizing user acquisition (UA) strategies and enhancing financial planning.
Recent research has explored various machine learning (ML) approaches to tackle these prediction challenges by uncovering patterns in user behavior, monetization, and churn. This white paper updates previous insights by incorporating findings from recent studies, including the work of Santos and Afonseca (2020), Chen et al. (2018), and Jang et al. (2020), who conducted in-depth analyses of LTV and churn prediction using ML in free-to-play mobile games. In a followup blog post we show how you can leverage these predictions to actually acquire better users when advertising on Google Ads and Meta Advertising.
Understanding the Freemium Challenge
The Freemium Model and Revenue Skewness
The freemium model allows users to download and play games at no cost, monetizing through IAPs and advertisements. This model leads to:
High Data Volume: Massive amounts of user interaction data are generated.
Skewed Revenue Distribution: A small fraction of users (often referred to as "whales") contribute disproportionately to total revenue.
Complex User Behavior: Diverse user engagement patterns make it challenging to predict which users will become high-value customers or are at risk of churning.
Importance of Predicting Customer Lifetime Value and Churn
Customer Lifetime Value (LTV) represents the total revenue a user is expected to generate over a specific period. Accurate LTV predictions enable:
Optimized User Acquisition: Focusing marketing efforts on the most profitable user segments.
Improved Financial Planning: Better budgeting and resource allocation.
Personalized Experiences: Tailoring in-game offers and experiences based on predicted user value enhances engagement and monetization.
Customer Churn Prediction involves identifying users who are likely to discontinue using the game. Accurate churn predictions enable:
Enhanced User Retention: Implementing targeted strategies to retain at-risk users.
Cost Savings: Reducing the higher costs associated with acquiring new users compared to retaining existing ones.
Revenue Stability: Maintaining a steady revenue stream by minimizing user attrition.
Machine Learning Approaches to LTV and Churn Prediction
Overview of ML Models
Several ML models have been applied to predict LTV and churn in freemium games:
Parametric Models: Statistical models like Pareto/NBD that predict future purchases based on past behavior.
Linear Models: Including Linear Regression, Ridge Regression, and Lasso Regression.
Tree-Based Models: Decision Trees, Random Forests, and Gradient Boosted Machines (GBM).
Neural Networks: Multilayer Perceptrons (MLP), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory networks (LSTMs), and Attention Networks (ANs).
Support Vector Machines (SVMs): Used for classification tasks in churn prediction.
Hybrid Approaches: Combining classification and regression techniques.
Findings from Recent Research
Study by Santos and Afonseca (2020)
Santos and Afonseca conducted an analysis of a real dataset from the mobile game Castle Crush, provided by Wildlife Studios, a leading mobile gaming company based in Brazil. Their study aimed to predict user LTV after 180 days of gameplay, using only the first seven days of user activity data.
Methodology:
Data Exploration: Performed exploratory data analysis (EDA) to understand user behavior and feature relationships.
Feature Selection: Identified key features influencing LTV.
Model Implementation: Evaluated several regression ML algorithms, including Lasso, Ridge, and Multilayer Perceptron (MLP).
Evaluation Metrics: Used error metrics such as Root Mean Squared Error (RMSE) and R-squared (R²) to assess model performance.
Key Findings:
Feature Importance:
Purchase-Related Features: Number of purchases and net revenue per purchase were the most significant predictors of LTV.
Accumulated Metrics: Cumulative net revenue and purchases over the first seven days showed strong correlation with LTV.
Average Ticket Value: Net revenue per purchase (average purchase value) was highly correlated with LTV.
Static User Features: Features like country and platform had low correlation with long-term LTV but provided useful business insights.
In-Game Events and Actions: Actions such as starting or winning battles showed low correlation with LTV.
Model Performance:
Lasso and Ridge Regression: These linear models performed best across most error metrics, with Lasso standing out due to its simplicity and high performance with paying users.
Multilayer Perceptron (MLP): Performed well but was slightly outperformed by Lasso and Ridge in terms of error metrics.
Tree-Based Models: Random Forest and XGBoost did not perform as well in explaining the variance in LTV, possibly due to overfitting or high correlation among features.
Data Imbalance Handling:
Balancing the Dataset: The authors addressed data imbalance by selecting an equal number of paying and non-paying users, creating a balanced training set. This approach improved model performance without the computational complexity of synthetic sampling techniques like SMOTE.
Feature Engineering:
Creation of New Features: Developed accumulated features and ratio-based features such as net revenue per purchase to enhance model input.
Feature Selection Criteria: Applied correlation thresholds to select features that significantly contribute to LTV prediction while avoiding overfitting.
Interpretability:
Model Simplicity: Lasso regression provided interpretable coefficients, making it easier to understand the impact of each feature on LTV prediction.
Study by Chen et al. (2018)
Chen and colleagues investigated the use of deep learning models, particularly Convolutional Neural Networks (CNNs), in predicting customer LTV in video games. They compared the performance of CNNs with traditional parametric models like Pareto/NBD, using data from the freemium mobile game Age of Ishtaria, a role-playing game with millions of players worldwide.
Methodology:
Data Utilization: Analyzed extensive player data, including in-game actions and behavioral records.
Model Implementation: Evaluated CNNs and Deep Multilayer Perceptron (DNN) models against parametric models like Pareto/NBD and its variants.
Input Features:
For CNNs: Used raw sequential time-series data of player behavior (e.g., daily logs), allowing the model to learn directly from raw data without feature engineering.
For DNNs and Parametric Models: Used engineered features derived from RFM (Recency, Frequency, Monetary value) analysis and player transaction history.
Evaluation Metrics: Assessed models using error measures such as Root Mean Squared Logarithmic Error (RMSLE), Normalized Root Mean Square Error (NRMSE), and percentage error.
Key Findings:
Model Performance:
CNNs and DNNs: Deep learning models significantly outperformed parametric models in predicting LTV, with CNNs showing the best performance.
Parametric Models: Models like Pareto/NBD and its extensions were less accurate, especially in predicting high-spending users.
Advantages of CNNs:
No Need for Feature Engineering: CNNs can process raw time-series data directly, reducing computational time and complexity.
Better Prediction of High-Value Users: CNNs were particularly effective in predicting the LTV of "whales," the small percentage of users who generate a large portion of revenue.
Implications for High-Value Users:
Early Detection: CNNs' superior performance aids in the early identification of potential high-value players, allowing for targeted retention strategies.
Operational Efficiency: Ability to handle large datasets efficiently makes CNNs suitable for production environments in the gaming industry.
Challenges with Parametric Models:
Underestimation of Top Spenders: Parametric models struggled to accurately predict the expenditure of high-value users.
Limited Use of Data: Reliance on RFM variables limits the models' ability to capture complex user behavior patterns.
Study by Jang et al. (2020)
Jang and colleagues focused on churn prediction in mobile games, emphasizing the importance of minimizing churn due to the high costs associated with acquiring new users compared to retaining existing ones. They introduced a novel concept called the "churn vector" to improve the accuracy of churn prediction.
Methodology:
Churn Vector Concept: Defined the churn vector as the normalized number of days remaining until a user churns divided by the total number of days the user has played. This approach accounts for each user's unique usage period.
Churn Vector=Days Until ChurnTotal Days Played\text{Churn Vector} = \frac{\text{Days Until Churn}}{\text{Total Days Played}}Churn Vector=Total Days PlayedDays Until Churn
Data Utilization: Collected data from 800 real users over eight months from a mobile idle game called "Maze X Brave." Data included login logs, stage clears, resource acquisitions, and character acquisitions.
Model Implementation: Evaluated various machine learning algorithms:
Traditional ML Models: LASSO, SVM, Decision Tree (DT), Random Forest (RF), Gradient Boosted Machine (GBM).
Neural Networks: Multilayer Perceptron (MLP), Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory networks (LSTM), Attention Networks (AN).
Evaluation Metrics: Assessed models using accuracy for classification tasks and R-squared (R²) scores for regression tasks.
Key Findings:
Churn Vector vs. Churn Day:
Improved Accuracy: The churn-vector-based approach significantly improved churn prediction accuracy compared to traditional day-based methods.
Higher R² Scores: Models using the churn vector achieved higher R² scores in regression tasks.
Model Performance:
Attention Networks (AN): Achieved the highest accuracy of 96.6% in predicting churn, outperforming other models.
Neural Networks: In general, neural network-based algorithms outperformed traditional ML models in churn prediction.
Algorithm Independence: The churn-vector method improved prediction accuracy regardless of the underlying algorithm used.
Practical Implications:
Personalized Retention Strategies: The churn vector allows for grouping users based on their churn risk level, enabling targeted promotions.
Early Intervention: By accurately predicting when users are likely to churn, companies can implement strategies to retain them before they leave.
Advantages of Churn Vector:
Accounts for Individual Usage Patterns: Provides a normalized measure that considers each user's play history.
Improved Prediction for Diverse User Bases: Particularly useful in games where users have varying engagement durations.
Implications for User Retention:
Strategic Promotions: By identifying users with high churn vectors, companies can offer tailored incentives to retain them.
Resource Allocation: Focus retention efforts on users most at risk of churning, optimizing resource use.
Practical Implications for User Acquisition and Retention
Optimizing Marketing Spend
Accurate LTV and churn predictions enable marketers to:
Allocate Budgets Effectively: Invest more in channels and campaigns that attract high-LTV users and retain existing valuable users.
Bid Strategically: Adjust bids in real-time based on predicted user value and churn risk.
Target High-Value Segments: Focus on user segments with the highest revenue potential and those at risk of churning, informed by early purchase behavior and churn vectors.
Enhancing Retention and Monetization
Personalized Offers: Use LTV and churn predictions to tailor in-game promotions and offers to individual users, increasing engagement and spending.
Early Identification of At-Risk Users: Implement targeted strategies for users identified early by churn prediction models, such as exclusive offers or personalized content.
User Engagement Strategies: Identify key moments in early gameplay (e.g., first seven days) to encourage purchases and reduce churn risk.
Data Handling and Feature Engineering
Leverage Deep Learning Advantages:
Process Raw Data: Utilize CNNs to handle raw sequential data without extensive feature engineering.
Capture Complex Patterns: Benefit from deep learning models' ability to detect intricate user behavior patterns that traditional models might miss.
Implement Churn Vectors:
Normalize Churn Risk: Use churn vectors to account for individual user engagement periods.
Enhance Prediction Accuracy: Improve model performance across various algorithms by incorporating churn vectors.
Handle Data Imbalance: Balance datasets to improve model learning, especially important in freemium games where paying users and churn events are minorities.
Iterative Model Improvement: Continuously refine models with new data and features to adapt to changing user behaviors.
Recommendations for Implementing ML-Based LTV and Churn Prediction
Start with Exploratory Data Analysis: Understand the data distributions, correlations, and key features influencing LTV and churn.
Consider Deep Learning Models: Implement CNNs for LTV prediction and attention networks (AN) for churn prediction to leverage their strengths in processing raw data and capturing sequential patterns.
Leverage Purchase and Behavior Data: Incorporate both transaction history and in-game behavior logs to enrich the input data.
Implement Simple Models for Interpretability: Use models like Lasso regression for insights into feature importance, complementing deep learning models.
Implement Churn Vectors: Use churn vectors to improve churn prediction accuracy and enable personalized retention strategies.
Engineer Accumulated Metrics: Create features that capture cumulative user behavior over the initial days post-installation.
Balance the Dataset: Address data imbalance by balancing the number of paying and non-paying users and considering the frequency of churn events.
Avoid Overfitting: Be cautious with highly correlated features and consider dimensionality reduction techniques.
Regularly Update Models: Retrain models with new data to maintain accuracy over time.
Conclusion
Machine learning offers powerful capabilities for predicting customer lifetime value and churn in freemium mobile games. The integration of findings from recent studies, such as those by Santos and Afonseca (2020), Chen et al. (2018), and Jang et al. (2020), underscores the importance of both purchase-related features and advanced modeling techniques in LTV and churn prediction.
Deep learning models, particularly Convolutional Neural Networks for LTV prediction and Attention Networks for churn prediction, demonstrate superior performance in predicting key metrics that drive revenue and user retention. The introduction of innovative concepts like the churn vector provides new avenues for enhancing prediction accuracy and developing effective retention strategies.
By focusing on early user behavior and leveraging advanced ML models, game developers and marketers can enhance their user acquisition and retention strategies, targeting users with higher revenue potential and those at risk of churning. Combining the interpretability of simpler models with the predictive power of deep learning allows for actionable insights and effective decision-making.
As the mobile gaming industry continues to evolve, leveraging machine learning for LTV and churn prediction will remain critical components in optimizing user acquisition, maximizing profitability, and delivering personalized user experiences.
About TensorOps
TensorOps is a leading tech consulting company specializing in artificial intelligence and machine learning solutions. We empower businesses to harness the power of data through innovative AI strategies and applications.
References
Chen, P. P., Guitart, A., Fernández del Río, A., & Periáñez, Á. (2018). Customer Lifetime Value in Video Games Using Deep Learning and Parametric Models. 2018 IEEE International Conference on Big Data (Big Data), 2134-2140.
Jang, K., Kim, J., & Yu, B. (2020). On Analyzing Churn Prediction in Mobile Games. [Conference/Journal], [Pages if available].
Santos, G. V. de C., & Afonseca, M. de A. (2020). User Lifetime Value Prediction Using Machine Learning in a Free-to-Play Mobile Game. Bachelor's Thesis, Escola Politécnica da Universidade de São Paulo.
Tapper, T. (2022). Using Machine Learning to Predict Customer Lifetime Value of Players in a Freemium Mobile Game: Effect of Seasonal Features. Master's Thesis, Aalto University School of Business.
Sifa, R., et al. (2015). Predicting Purchase Decisions in Mobile Free-to-Play Games.
Sifa, R., et al. (2018). Customer Lifetime Value Prediction in Non-Contractual Freemium Settings.
For more insights on leveraging machine learning for user acquisition and retention, visit our blog at TensorOps Blog.
Comments