You can now tell management in a way that would be familiar as to how the prediction can help the business, i.
e improving conversion rate.
Now management can compare this metric to their current process, and your project now has clear business implications.
Translating recall to capture rateSuppose now you know there is a group of 100 high value visitors that spends a lot more than the usual customer.
The prediction model does the same prediction: whether a visitor is of high value or not, and follow-ups are targeted towards these individuals.
The model manages to include 80 of the high value customers that is our target segment.
That would mean a capture rate of 80 percent with regards to the 100 people that we are trying to target.
Well, the 80 percent capture rate is actually the recall of the model!To put it simply, precision is the ability to get the highest ratio of high value customers, recall is the ability to not miss out on high value customers .
Notice that, with different quality of clients, our focus becomes different.
In the first case, we are more interested in converting as many visitors to buyers as possible with as little effort, whereas in the second case we aim to capture as many of the target segment as possible.
Lets look at a different example.
In the case of a bank launching a campaign offering attractive personal loan packages, you would like to extend your package to as many potential loan payers as possible, while avoiding loaning out to potential defaulters.
To keep things simple I did not consider other cost such as cost of campaigning or cost of alienating existing customers with unequal treatment.
Revisiting the previous analogy:Precision is the ability to get the highest ratio of loan payers in the campaign, recall is the ability to not miss out on loan payers.
Alternatively, precision aims to not mistake anyone as payer.
Recall aims to not mistake anyone as defaulters.
Your precision is the hit rate of loan payers while recall is the capture rate of loan payers.
The higher the precision the less likely it is to recruit defaulters, but the potential client pool becomes smaller.
The higher the recall, the larger the potential pool of clients but the higher the risk of recruiting defaulters.
The balance of our recall and precision levels is a matter of risk appetite.
Are we willing to accept more risk of recruiting bad clients in order to capture more potential clients?.Crucially these are the key points that businesses are focusing on.
Aligning with Business ObjectivesThe strategic objective of businesses could be anything from identifying potential clients, capturing fraud, maximizing campaign effectiveness or reducing churn.
To communicate prediction model results effectively, we should align with the metrics business leaders are looking at: conversion rate, churn rate, fraud incident rate, capture rate, hit rate, etc.
Hence instead of saying your model achieved a recall or precision of a certain percentage, say you managed to achieve an improved capture rate or hit rate.
The model metrics aligns perfectly with the business objective!While data scientists are capable of producing the required results, its the subsequent communication mismatch that can be confusing to decision makers.
Nailing the business metric is crucial for maximizing the benefits gained from a robust prediction algorithm.
I hope I managed to introduce a different perspective for using precision and recall to bridge the gap between business users and data science.
Thanks for reading!.. More details