Adoption, Stickiness, Growth, Product Engagement
One of the biggest challenges in Learning and Development is measuring the Return On Investment (ROI) of learning. While there are more than eight models used in Learning and Development to evaluate learning ROI, including the very effective Kirkpatrick New World Model and the Philips model, product-specific Key Performance Indicators (KPIs) can complement measurement insights from these models by providing information about the learner behavior and engagement with the learning product. Such data will enable you and your L&D team to evaluate your various learning products in terms of their value to the learner, allocate your limited resources accordingly in enhancing existing products that learners like, sunsetting the ones that learners never engage with, and improving the ones that show potential.
At the enterprise level, you can demonstrate the value your learning products deliver to learners and help you articulate the ROI of learning to your organization. In this article, we adopt and adapt 4 KPIs from software development and explore the critical business questions each KPI provides an answer to inform data-driven business decisions for your L&D team. These 4 metrics are stickiness, adoption, growth, and the Product Engagement Score.
Product Stickiness
This KPI answers a fundamental business question: do learners keep coming back to the product? The primary way to answer this question is by measuring the number of users returning to the product consistently. This KPI can be calculated as a ratio depending on the frequency of customer engagement with the product. First, you have to define your daily active users (DAU), weekly active ysers (WAU), and monthly active users (MAU). Next, let’s say you want to calculate the stickiness of your learning experience platform, where monthly learners will likely return weekly at first; you can calculate the WAU/MAU. As stickiness increases, you can calculate the monthly users who return daily or DAU/MAU. Apart from knowing how many learners return, you can also use stickiness to build your engagement and internal marketing strategies to address based on the return patterns of your most frequent users.
Product Adoption
This KPI answers a foundational business question: are learners adopting the product? Adoption is a key KPI because it helps you measure how many learners engage with your learning product. Here, again you will track the active users daily, weekly, or monthly over time. So, you will essentially depict a trend line of your daily active users, weekly active users, and monthly active users. You can use the learning product adoption data to track the impact of your newly rolled-out learning products, as well as the impact of your marketing campaigns. Are learners coming back to the product after the campaign ended?
Product Growth
This KPI answers a critical business question: are new learners coming to the product faster than existing learners are abandoning it? This KPI measures the net impact of your learner acquisition and retention efforts. You can increase growth by adding new learners and retaining the existing ones. Pendo uses the quick ratio to measure how efficiently the product is growing. This ratio can be calculated as follows: add the total number of new learners to the existing learners and divide by the number of dropped learners. You will be able to track growth from month to month and use this metric to allocate resources to those learning products that demonstrate the highest growth.
Product Engagement Score
This KPI answers a critical business question: how are learners engaging with the product overall? The Product Engagement Score (PES) can provide leaders in your organization with one go-to metric to discern how users engage with the learning product. The Product Engagement Score can be calculated as the average of the adoption, stickiness, and growth metrics of your learning products. And while taking the average is not always a good approach because averages can muddle or skew the overall picture, in this case, the PES is a practical way to discern how your learning product is doing because you can easily see which of the three metrics (adoption, stickiness, or growth) is lowering your average. To calculate the PES, you must add the adoption, stickiness, and growth scores, divide them by 3, and then multiply that total by 100 to get your PES.
Again, the practical power of the PES is that you have a single number to review for product engagement, while at the same time you can dig into the trends and data of each of the three metrics to discern where things need to improve and how quickly so that you improve your learner experience. The PES can also be used to complement your Net Promoter Score, which tells you how happy learners are overall with your learning product. Tracking both scores can be powerful because you can let your leadership know not only whether learners are happy with each learning product but also how frequently and consistently they engage with it. Research by Pendo showed that PES correlates to retention and can help you predict which learners will return to the learning product, which ones will engage with new products, and which ones will lag behind. This information can be powerful as you try to deepen and cultivate the culture of learning in your organization.
Conclusion
The maxim “if you can measure it, you can manage it” applies to any industry and product, including Learning and Development. Measuring how much and how often learners engage with your learning products can illuminate your efforts both at the operational and strategic levels. At the operational level, it can help you discern which learning products learners find most popular and which products they visit more often, which in turn can enable you to design impactful marketing campaigns and allocate limited resources accordingly. At the strategic level, you can demonstrate product adoption, stickiness, and growth, and present your learning Product Engagement Score to management. Importantly, you can leverage these metrics in your reporting and justification of investment in L&D by demonstrating learner overall engagement with your learning products over time.
