What machine learning does in retention, and who should wait

A retention model does one narrow thing: it ranks your customers by how likely each is to leave, or to come back after going quiet. It does not write your offer, set your discount or decide who you may contact. Most small businesses should wait for one, and do the same ranking by hand until the hand work becomes the bottleneck.

I build and sell this kind of work, so weigh my view accordingly. I will say plainly where I think it is too early.

What does a retention model actually produce?

Two numbers per customer, from two separate models. The first is the chance that an active customer leaves in the next few weeks. The second is the chance that a customer who has already gone quiet comes back. They lead to different actions, and trying to fit one model to both tends to produce a model that predicts nothing well.

Think of a doctor who notices from your test results that something is changing before you have any symptoms. The model reads the same kind of signs in customer behaviour: orders getting smaller, visits getting rarer, a longer gap than usual.

In the casino system I work on, the model is a type called gradient boosting, which works well on tables of numbers and whose reasons can be explained to an analyst. The details are in the iGaming retention playbook. You do not need the details to decide whether you need one.

What does the output look like?

The output is a ranked worklist. A score alone is of little use to a team. What helps is a list that is:

• Sorted by expected value. Say one customer has a 90% chance of leaving and spends $100 a month, and another has a 50% chance and spends $10,000. The second one comes first. These are example numbers.
• Grouped by suggested action. Hold, win back, or keep an eye on. People think in actions.
• Explained. Each row shows the top reasons for the flag, so a person can decide whether to trust it.
• Respecting permission. Customers who must not be contacted appear in a separate lane for checking, never in the outreach queue.

Notice what is missing: the offer, the script and the channel. In my view those belong to the owner, because they depend on your margin, your rules and your customers.

What does a model need before it works?

Mostly boring things. This is where I see projects fail.

• Enough history. The sources I rely on ask for at least six months of clean customer history, so the model sees customers who stayed and customers who left.
• Raw records. If you feed it summaries that were computed over a customer's whole history, including the period it is meant to predict, it learns from the future. It looks brilliant on old data and fails on new customers. It is like marking an exam with the answer key beside you.
• Clean accounts. Test accounts, staff orders and bots must be removed first.
• Someone to act. A ranked list nobody works is an accurate forecast and no money.

For a mid-size operator, the write-ups I have read put a first deployment at roughly six to twelve weeks, depending on how clean the data is and how custom the platform is. If the data is not ready after the first phase, stopping is a good outcome. I would rather tell a client that than sell them a model.

Why not just ask a chatbot to score customers?

Because a wrong score costs real money before anyone sees it, and a chatbot cannot show its reasoning in a way an analyst can check. Every language model produces wrong answers some of the time. A confident wrong score means a discount given to the wrong customer, or a good customer left to drift.

A language model has a real place in retention: drafting message copy, or summarising a customer's history for a person before a call. Anywhere a human reviews the output before it costs money, it is fine. Anywhere the output touches money first, I keep it out. I make the same point in build or buy a churn model, where I also list what keeping a model alive costs.

What does it cost to keep a model running?

More than the first build suggests. Customer behaviour drifts as your products, prices and customer mix change, so the model needs retraining, weekly on a rolling window as a default and monthly at the outside. Somebody has to watch for drift, keep versions and fix the pipeline when your systems change. That is a standing job for a person or a small team.

If nobody has that job, the model decays and the team stops trusting the list. A model nobody trusts is the most expensive spreadsheet you own.

Who should wait?

You should wait if any of these is true.

• You have no "next due" dates yet.
• You have under six months of dependable records, or most of them lack dates.
• Nobody owns returning customers.
• You have never compared a campaign with a group you did not contact.

In each case the cheaper step comes first, and it will teach you more. A hand-made list can do most of what a small model does: sort customers by days past their usual gap, weigh them by what they spend, and call the top ten.

How would you know a model works?

Run it in shadow mode first: your team sees the flags but changes nothing. After a few weeks, check whether the flagged customers did leave more often than the others. Then run a controlled test with a group nobody touches and compare. The method is the same one you can use today without a model. I would be sceptical of any vendor who quotes a lift before seeing your data. I include myself.

What can you do today?

Build the poor man's model. In your customer list, add a column that shows days since last purchase divided by the customer's usual gap. A value of 1 means on time, 2 means twice as late as usual. Sort by that column, multiply by what the customer spends, and look at the top ten.

Call three of them. If the list makes sense to you, you already know what a model would try to do, and whether you need one.