Nobody Asks for a £14,999 Loan

The most commercially valuable analysis I have ever done contained no model. No adstock, no saturation curve, no priors, no counterfactual. It was a bar chart, a crude animation I built because I had watched a TED talk, and one line of small print underneath a rate table.

This is a real project from early in my career, deliberately disguised. The client is not named, and every number, table and chart below comes from a synthetic dataset built to behave like the original. The story is true; the data is simulated.

The Brief Was a Model

I was the new analyst on a marketing mix modelling project for a UK retail bank. Its parent group had diversified into financial services, which was the fashion at the time. If enormous volumes of cash are already moving through your business, lending some of it out starts to look obvious. The bank sold credit cards, personal loans, insurance and savings. My part of the project was loans.

The brief was conventional. Weekly loan applications as the dependent variable, the usual media variables, the usual controls: price, seasonality, competitor activity. Two things about the project turned out to matter more than the model did.

The first was that price comparison sites had become the dominant route to a personal loan. Consumers no longer walked into a branch; they typed in an amount and a term and read a ranked list of APRs. So one of my first real jobs was writing VBA to scrape those sites every week and capture every lender's advertised rate at every loan amount, along with their position on the results page. It gave us rate variables and rank variables over time, which are genuinely good model inputs. It also, entirely by accident, gave us the evidence for something else.

The second was the shape of the data delivery. Applications did not arrive as a weekly total. They arrived by requested amount. In a given week: 129 applications for £8,000, 51 for £8,500, one for £8,450, and so on down every value a human being had typed into a form.

Nobody had asked for it in that form. It was simply how the extract came out of the source system. I have thought about that a great deal since.

Everybody Asks for a Round Number

Loans were priced in three bands, each with its own representative APR: small loans below £7,500, medium loans in the middle, and large loans at the top. Medium was where the war was. Rates in that band had fallen from around 8% to as low as 4.4%, and a new headline number appeared most weeks. Small loans were expensive and nobody was fighting over them.

Faced with a column I had not expected, I did the obvious thing and plotted the distribution.

The first thing it showed was the band structure, which was reassuring rather than interesting: the cheapest band carried by far the most volume. The second thing was much more interesting. Within a band, the distribution was nothing like smooth. It was a comb.

People ask for £8,000. They ask for £10,000. They ask for £7,500, because they have worked out that is where the cheap band starts. They do not ask for £8,145. Here are the ten most requested amounts in year 1:

Ten amounts, nine multiples of £1,000 plus the round number where the cheap band began, accounted for 47% of the year's applications. Everything that was not a multiple of £500 accounted for about 7% of volume in total, spread so thinly across the twenty-four thousand remaining possibilities that any particular one turned up once or twice in three years. £14,999 came up twice.

On its own that is a mildly entertaining fact about human psychology. Keep hold of it, because it is half of the finding.

Borrowing Hans Rosling's Trick

Around this time I watched Hans Rosling's TED talk, the one where sixty years of development statistics come alive as bubbles racing across a scatterplot. What stuck with me was not the data. It was that he had made time visible, and that the story was completely invisible in any single frame.

I had three years of weekly data and a distribution I had only ever looked at as an average. So I spent an evening teaching VBA to redraw the chart week by week. It was an ugly animation. It was also the most useful thing I did on that project.

This figure needs JavaScript. The short version: applications cluster hard on round numbers, and part-way through year 2 the £15,000 bar falls over.

Averages hide time. An animation makes change impossible to ignore, and what I saw in year 2 was a single bar falling over.

The Bar That Fell Over

Applications for exactly £15,000 ran at around 150 a week for the first ten months. In one week they more than halved, and over the following four months they bled down to fewer than ten.

A 94% fall, which then sat there, flat, for eighteen months.

This figure needs JavaScript. The short version: total applications rose 48% over the same eighteen months in which £15,000 applications fell 94%.

That chart contains the uncomfortable part of the story. Over precisely the same period, total loan applications rose 48%. We had cut the medium-band rate hard, climbed the comparison tables where the volume was, and applications were at a record. The £15,000 hole was about 140 applications a week sitting inside a series that had just gained more than a thousand. The model was fitting a growing dependent variable. Every dashboard was green.

A collapse is invisible when it is smaller than the growth around it. That is not a data quality problem or a tooling problem. It is a granularity problem, and the only defence is looking at the data at the level where the customer actually made a decision.

Comparing the neighbours made it worse, and also made it undeniable. £12,000, £13,000 and £14,000 all roughly doubled over those eighteen months, exactly as you would expect after a big rate cut in their band. £15,000 fell 94%. Whatever had happened was specific to one amount, and it was not the market losing interest in borrowing fifteen thousand pounds.

£14,999

Because of the scraping job, I already had the answer sitting in a spreadsheet. Our position on the comparison table at £15,000 had gone from second of fifteen lenders to twelfth. That matters enormously, because those tables are brutally top-heavy: the first three rows take roughly six applications in ten, and twelfth place takes about one in a hundred.

Then I read the small print underneath our own rate table. Medium loans: £7,500 to £14,999.

Same lender, same week, same customer. One pound apart, 2.2 points of APR apart, ten places apart on the page.

Every competitor defined medium loans as £7,500 to £15,000 inclusive. We defined them as £7,500 to £14,999. So £15,000 fell into our large-loan band, which had sat out the rate war entirely and was still priced at 6.6%. Every rival was showing a customer asking for £15,000 a keen medium-band rate. We were showing them a large-loan rate, and they were choosing accordingly.

And nobody asks for £14,999.

This is why the two halves have to be held together. If demand had been smooth across amounts, the boundary would have cost a rounding error: whatever sliver of customers happened to want between £14,999 and £15,000. Because demand is a comb with a tooth on every round number, the boundary was sitting directly on top of one of the biggest teeth in the whole distribution. £15,000 was our second most requested amount.

This figure needs JavaScript. The short version: our quoted rate stepped up 2.2 points at exactly £15,000, while the rest of the market stepped up £1 higher.

Plotted against amount, our own rate curve had inverted. A customer asking for more money was quoted a materially worse price, which is not a thing any pricing committee would ever knowingly sign off. But nobody was looking at the curve. Pricing was managed as three numbers in three bands, and in that view everything was fine. The bands were competitive. It was the join between them that wasn't.

Charm Pricing, Pointed the Wrong Way

Retailers have understood the psychology in this story for about a century. £14.99 outsells £15.00 by considerably more than a penny's worth of extra value, because we read prices left to right and stop concentrating after the first digit or two. Fourteen-something feels like a different order of thing from fifteen. The effect is called left-digit bias, and charm pricing is the entire industry built on top of it: the 99 exists to keep the leading digit down.

What happened to this lender was the same bias seen from the other side of the table.

In a shop, the seller writes the number and the buyer reads it, so left-digit bias is something you can exploit on purpose, and everybody does. On a comparison site for loans, the buyer writes the number and the seller has to decide how to price whatever gets typed in. The same mental habit is still running, but it now shows up in the quantity rather than the price: people ask for £15,000 because round thousands are what we hold in our heads, not £14,850 or £15,200. Instead of a pricing tactic, the bias becomes a demand curve made of spikes.

Seen that way, the boundary was charm pricing in reverse, and I suspect that is genuinely how it came to be written. £14,999 has exactly the shape of £14.99, three nines tucked under a round number, and a band that runs "up to £14,999" sounds like it reaches almost all the way to fifteen thousand. It does. That is precisely the problem. Charm pricing puts the nines just below a threshold to win the customer's attention. This band edge put the nines just below a threshold and excluded the most popular request in the entire distribution.

There is a happier consequence of demand arriving in spikes next to a step in price, which is that it makes elasticity unusually easy to see. Between £14,999 and £15,000 there is no plausible difference in the customer: same week, same site, same intent, one pound apart. There is a 2.2 point difference in the price they were quoted. A sharp discontinuity in price with essentially nothing else changing across it is the setup that every elasticity estimate wants and almost never gets, because normally you have to separate price from everything else that happened that week, whereas here the boundary does the separating for you. Estimating elasticities from band edges, thresholds and other administrative accidents is a rich enough subject that it deserves a post of its own, which is on my list.

Why It Had Never Mattered Before

Here is the part I find most instructive, years later. That boundary had been in the pricing table for years and had never cost a penny. In year 1 it was actively helping. At £15,000 we quoted the large-band rate of 6.6% against competitors' medium rates that started at 6.4%, which put us second on the page. At £14,999 we quoted 6.9% and ranked third. The odd boundary was worth a place.

Nothing about the boundary broke. The market moved and left it behind. While the gap between our medium and large rates was 0.3 points, the edge was invisible. When the rate war pulled medium down by 2.5 points and left large untouched, the very same edge became a cliff.

A structural feature of your business is only as safe as the conditions it was set under. Bands, tiers, thresholds, cutoffs and qualifying criteria are all decisions someone made against a market that no longer exists. Nothing alerts you when they go stale, because nothing about them has changed.

Making the Case

I took it to my manager, who did the right thing with an excited junior analyst and asked me to build a short deck for the client. Three choices are the reason it landed as analysis rather than as enthusiasm.

We haircut for substitution. Some people who wanted £15,000 would have shrugged and asked for £14,000 instead, so not all of the lost volume was genuinely lost. The neighbouring amounts were the natural control group, and they said the effect was real: the demand had overwhelmingly gone to other lenders rather than down a notch. We still assumed we would only recover between 60% and 85% of it.

We converted applications into loans, then into value. An application is not a sale. We took it through acceptance and drawdown to funded advances, then to net present value per loan after cost of funds, expected credit losses, aggregator commission and servicing.

We put the cost in. Moving £15,000 into the medium band means earning 4.4% on it rather than 6.6%, including from the handful of customers still coming through. It also means the cliff does not disappear. It moves to £15,001, where some £16,000 demand will now round down to £15,000 to get the cheaper rate. We sized that at roughly £100,000 a year and said so on the slide.

The point of the cautious column is that it was still overwhelming. Even assuming we recovered only three fifths of the volume at the worst conversion rate we had ever seen, the change was worth £22m of incremental lending and about £0.7m of value a year, against an implementation cost of editing one number in one pricing table.

What Happened

We presented it. The client changed the band definition the same day.

Applications at £15,000 came back the following week and settled above where they had been before the collapse, around 225 a week against the old 150, because the whole market for a £15,000 loan was bigger at 4.4% than it had been at 6.6%. In the fifteen weeks left in that financial year it put roughly 3,200 additional applications through the door, and it was the single largest reason the loans book hit its target.

It remains the fastest implementation of a recommendation I have ever been involved in, for the obvious reason: it required one person to change one number.

Incremental, But Not New

It is worth being precise about what this change actually did, because "incremental" routinely gets used for two things that behave very differently.

The bank gained roughly 215 applications a week. Almost none of that was demand it had brought into existence. The people who wanted to borrow £15,000 had wanted to borrow £15,000 the whole time. They had been looking at a page where eleven other lenders appeared cheaper, and they had quite reasonably gone to those lenders. Changing the band definition created no appetite for a loan whatsoever. It made the bank visible to appetite that was already forming, and it stopped competitors harvesting that appetite unopposed.

To the client's P&L this distinction is irrelevant, because £22m of lending is £22m of lending however it arrives. To anyone interpreting the result it matters a great deal. Harvesting existing demand and creating new demand have different ceilings, since capture is bounded by demand that already exists and creation is not. They have different competitive dynamics, because capture is close to zero-sum and rivals eventually notice and respond. They decay differently too: a structural fix like this one holds until somebody reprices, whereas created demand usually needs continuous spend to sustain.

What makes this a useful teaching case is that the recovery contains both, in a ratio you can actually read off. Of the 225 applications a week we ended up with, about 150 was demand that had existed at the old prices and had been going elsewhere, which is pure capture. The remaining 75 or so existed only because a £15,000 loan now cost two points less than it used to across the whole market. That part is real category expansion, and we did not cause it. The rate war did. Our contribution was being on the list when it showed up. A further slice of the apparent recapture was never incremental to us in the first place, because some of those customers had shrugged, asked for £14,000 instead and borrowed from us anyway, which is exactly what the substitution haircut in the business case was there to strip out.

My view now is that "is it incremental?" is close to unanswerable as posed, and that the useful version specifies three things: incremental to what, measured against which alternative, and lasting how long. On all three counts a boundary fix and a brand campaign are different species of result, even when they land on the same line of the P&L. I make that case properly in a separate post on what incrementality actually means.

The Most Valuable Analysis I Have Done Had No Model In It

A career and a lot of Bayesian machinery later, that is still true, and I think it is worth being precise about why.

The model was not the problem. It answered the question it was asked, and answered it well. But it could not have found this. £15,000 applications were about 5% of the dependent variable, and they fell in the same weeks the rate war was lifting everything else, so the net movement in the total was upwards. What was left for the model to explain was a series going the right way, comfortably attributable to the rate cuts we had actually made. Above all, the model had no concept of a loan amount at all, because the first thing we did with the data was sum over that column. The finding lived in a dimension the modelling dataset had aggregated away before the model ever ran.

The lesson is not that models are overrated. It is narrower and more useful than that: the modelling dataset is not the data. Aggregation is a modelling choice, and like every modelling choice it destroys something. Look at what you are about to throw away before you throw it away.

Three habits came out of this that I still use twenty years on.

• Plot every dimension you were given, including the ones nobody asked for. The most valuable column in that extract was there by accident.
• Make time visible. Animate it, or facet it, or small-multiple it. An average over three years is a machine for hiding something that happened in year 2.
• Check your competitive position at the granularity of the customer's decision. Not by channel, not by product, not by band, but at the level where the person actually chose.

And Then It Vanished Into the Base

There is a second lesson in this story and it took me much longer to appreciate, because it happened slowly and to somebody else.

Twelve months on, the recovered £15,000 volume was no longer an achievement. It was the base.

Year-on-year targets are set off last year's actuals, so everything that worked last year becomes the number you have to beat this year. A one-off structural fix is the worst-behaved possible input to that system. It delivers a step change rather than a trend, it has no ongoing spend to point at, and by the next planning cycle it is indistinguishable from "the business". Within two cycles the executives who had approved it had moved on, and the ones who replaced them inherited a healthy-looking loans book and a growth target set off it, with no way of knowing that a slice of the base they were being measured against came from one line of small print.

That is not carelessness, it is the default. Organisations are diligent about recording spend and diligent about recording outcomes. They almost never record decisions, and the decision is the only thing that explains the step.

Two defences, both cheap:

• Keep a change register. A dated log of every structural change you make to pricing, band definitions, eligibility rules, fees and customer journeys, with the expected effect and who signed it off. It costs an hour a month and it means next year's analyst can explain the step in the data instead of inventing a reason for it.
• Put the change in the model. A step variable dated to the day it went live does two jobs at once. It stops the effect being credited to whatever media happened to be running that week, and it keeps the event in an artefact that survives the people who remember it. When somebody asks why year 2 finished so strongly, the answer should be in the model, not in a former colleague's memory.

The measurement failure and the memory failure are the same failure on different timescales. Both come from tracking what you spent instead of what you changed.

Where Else This Lives

This particular £1 is long since fixed, but the shape of the problem is everywhere. You need two ingredients: a business rule that is discontinuous, and customer demand that clusters. Wherever a spike in the second sits next to a step in the first, there is either a windfall or a leak.

• Free delivery over £50, with basket values piling up at exactly £50. This one usually works in your favour, which is precisely why it is worth measuring.
• Age and mileage bands in insurance, where quotes jump on a birthday or at 10,000 miles and applicants round their answers.
• Per-seat software tiers that change price at ten users, when almost every company asks for ten.
• Credit score and affordability cutoffs, where a hard threshold meets a lumpy applicant distribution.
• Minimum spend for a promotion, a loyalty tier, or free returns.
• Anywhere a competitor's boundary differs from yours, by any amount at all.

The diagnostic takes an afternoon. Plot the distribution of the quantity the customer chooses, at the finest granularity you have, and find the spikes. Then plot your price on the same axis, or your acceptance rate, or your ranking, or your delivery promise, and find the steps. Compare the two charts.

No model needed. That is rather the point.