The model estimates the probability that a restaurant concept at a given location survives to 4 years, against a citywide base rate of 61.2%. It is an L2-regularised (ridge) logistic regression fit on 12,332 matured location attempts — each kept only once its 4-year outcome is actually observable — with a post-hoc calibrator that adjusts the probability scale without touching the ranking, and that ships only because it improved calibration on held-out years it was never fit on (see below). Every figure in this section is read from the shipped model artifact — the exact file the product scores with.

Read this honestly. An AUC of 0.670 is modest discrimination — this model is directional, not deterministic, and should never be the only input to a lease decision. What it does well is separate the extremes: the gap between top-decile and bottom-decile survival is 47 points, which is a large and actionable difference even when any single prediction carries real uncertainty.

• Out-of-time holdout. The evaluation set is split by time, not at random. Random splits leak the future into training and flatter the model; a time split does not. 2,423 attempts are held out.
• Bootstrap confidence intervals on every reported figure, so the precision of each estimate is visible rather than implied.
• Holm-Bonferroni correction across all 20 candidate hypotheses. Testing many features and reporting the winners produces false discoveries; correcting for the number of tests is what prevents that.
• Every shipped feature is ablated. Each of the 4 features is removed, the model refit, and the difference measured on the held-out years with a paired bootstrap interval. One of them — serviceSig, the concept's own track record — carries the model: removing it costs 0.078 AUC. The others are kept on a different basis. Two of them contribute nothing measurable to ranking but hold the probability scale straight at the weak end, and removing either makes the estimate for a marginal location several points more optimistic than it should be. One is simply too small an effect for a holdout this size to resolve, which is a statement about the test rather than the feature, and is recorded as such.
• The calibrator is kept on evidence, not taste. It was fitted on one held-out slice and evaluated on a later one, where it cut the average gap between stated probability and outcome from 4.5 to 3.5 points across 918 attempts. A retrain where that evidence does not hold ships without one.
• Rejected variants are recorded. A conformal prediction interval on the outcome was built and benched — verdict: correct and useless: for a single binary outcome the 90% prediction set is {closed, open}. Shipped the rate interval instead. And an entire second model class, fit on roughly four times the training signal, failed to beat this one out-of-time; that negative result gets its own section below rather than a footnote.

Every feature considered for the model, whether or not it worked. 10 of 20 hypotheses survive Holm correction at α = 0.05 — and 7 of those 10 are negative results: features that were significant precisely because they made out-of-time performance worse. Only 3 helped, the largest by 0.0098 AUC.

Restaurant density and transit proximity are the clearest examples. Both are intuitively appealing, both were tested properly, and both reduced predictive accuracy. They are not in the model.

2 of the 3 that helped are not in the model yet: they have cleared validation but not a retrain. This table records what was tested, not what ships. The Shipped marker is read from the production model file, so it cannot drift from what is actually running.

Δ AUC is the change in out-of-time AUC from adding the feature. Negative means the feature made the model worse. “Not significant” means the result did not survive correction for the number of hypotheses tested — it is not evidence the feature has no effect, only that this data cannot establish one.

This is the most consequential result on the page, and it is a negative one.

The shipped model trains on 12,332 matured attempts — one row per restaurant, kept only once its four-year outcome is actually observable. That throws away every restaurant too young to have reached the horizon, which is a lot of them. So a second model was built to stop throwing them away: a discrete-time hazard model that expands each restaurant into yearly observations, letting a still-open two-year-old contribute real evidence instead of being dropped. It trains on 47,655 attempt-year rows from 14,691 attempts — roughly four times the training signal.

It did not do better. Out-of-time AUC 0.634 against the binary model's 0.635 measured at the same time — a dead heat, correctly benched. The shipped model has since improved to 0.670 on other grounds, which only widens the gap.

Quadrupling the training data moved nothing. That is strong evidence the limit here is not how much history has been collected — it is how much the available signals can say. No larger sample and no fancier model closes that gap, because the gap is not made of data volume. The only routes past it are genuinely new information about a location or an operator, or a second market to test whether any of this generalises at all.

It is published here rather than left in a file because a negative result of this size changes what is worth building next, and because a research programme that only reports what worked is not one whose positive results should be trusted.

• Chicago only. The model is fit entirely on City of Chicago business-license data and has not been validated in any other market.
• License data lags reality. Chicago renewals routinely lag six to twelve months, so a business that has closed can still appear active. Active status carries a 365-day grace window for exactly this reason, which means recent closures are systematically under-counted.
• Cuisine is inferred, not declared. It is classified from business names, license activity codes, and Google categories. It is accurate in aggregate and wrong for individual records.
• The score is optimistic at the weak end, by about 6 points. AUC measures ordering and cannot tell you whether an absolute probability is correct, so the estimate is checked separately against what actually happened on the held-out years. Across all bins the average gap between estimate and outcome is 4.4 points, but it is not evenly spread: the lowest-scoring locations survive somewhat less often than the number suggests, while the highest-scoring ones are accurate to within a point or two. Treat a low score as at least as bad as it reads, never better.
• Modest discrimination. An AUC near 0.670 supports ranking and comparison. It does not support confident single-location verdicts.
• Survivorship in the source. Attempts are reconstructed from license records, so operators who never obtained a license are invisible to the model.
• An era assumption sits under every score. The fit includes an opening-year term so a 2002 opening is compared fairly against a 2021 one. At prediction time that term is pinned to a single year — 2021, the most recent opening the model can score an outcome for. The choice matters: across the plausible range of that one constant the estimate for an average concept moves from 68% to 60%. It is pinned inside the range the model was trained on rather than extrapolated past it, and the most recent matured cohorts are checked against their own predictions — the largest gap is 6.3pp. This assumption is invisible to AUC, which is why it is stated here rather than left to a validation figure to catch.

Fee, investment and unit-economics figures for 75 franchise brands are read from each brand's own current Franchise Disclosure Document, as registered with the Wisconsin Department of Financial Institutions — a franchise registration state that publishes complete FDDs publicly. Every figure is read against a cited page of the filing rather than parsed, because FDD table layouts vary enough between franchisors that a parser cannot be trusted with a number that ends up in a cost estimate.

Item 19 is optional, and 26 of these brands decline it entirely. Where it does exist there is no required format, so the disclosed population differs sharply between brands — franchised outlets versus company-operated, one build format versus the whole system, captive venues versus street-front, and samples ranging from two outlets to twelve thousand. Every figure therefore carries a description of what it covers, and that description travels with the number wherever it is shown. Payback estimates are refused outright when a brand's Item 7 and Item 19 describe different populations, since dividing one into the other yields a confident-looking number with no meaning.

Chicago outcomes for these brands are computed from the same licence history and the same corrected closure labels as the survival model above, so they are directly comparable to every other survival figure in the product. They are descriptive: a brand's Chicago record reflects where it chose to open and who operated those locations, and is not evidence that the brand causes survival.

Registry-backed franchise detection was tested as a model improvement and rejected — see the hypothesis ledger above. It corrects a real labelling gap but does not improve out-of-time accuracy.

License and inspection data refreshes daily; Google verification and rent bands refresh monthly, on a separate cadence. Both timestamps are shown so neither is mistaken for the other. Franchise registrations renew annually and are re-swept quarterly.

Chicago licenses

never

Google enrichment

never

Every shipped model is identified by a content hash — 8549203495b2b57b for the model serving today — and that hash is recorded alongside every scored decision, so any score a customer acted on can be traced to the exact model that produced it.

The model retrains quarterly, after the monthly data refreshes have settled. A retrain ships only with a published validation delta against the outgoing version — if accuracy does not hold out-of-time, the old model keeps serving.

Issued reports are never retroactively changed. A report is a record of what a specific model version said on a specific date; when the model updates, existing reports stand as issued, and re-running the same address produces a new report under the new version stamp.