Forecasting rodent-inspection failures before they happen

Introduction

Ardenus’s research team trained a calibrated LightGBM model - benchmarked against XGBoost and CatBoost - to forecast whether a given New York City location will fail its next rodent inspection. Trained entirely on public data, it reaches a ROC-AUC of about 0.84 on held-out inspections, with a precision-recall AUC of about 0.72 on the rare positive class.

• Target: a binary forecast - will a location’s next inspection record rat activity?
• Data: three public NYC datasets - rodent inspections, 311 rodent complaints, and PLUTO property records.
• Model: a single calibrated LightGBM classifier, chosen after benchmarking LightGBM, XGBoost, and CatBoost.
• Result: ROC-AUC ≈ 0.84 (PR-AUC ≈ 0.72) on inspections the model never saw during training.

Inspections are reactive. Risk is predictable.

New York City’s Department of Health and Mental Hygiene inspects properties for rodent activity across the five boroughs and logs the outcome of every visit. Inspection capacity is finite, so most visits are a response to a problem that is already visible. The question we set out to answer is whether the pattern of past inspections and the fixed characteristics of a place carry enough signal to estimate, ahead of time, which locations are most likely to fail their next inspection - so that attention can be pointed proactively rather than reactively.

We framed this as a supervised classification problem: given everything knowable about a location before an inspection, predict whether that inspection will record rat activity.

The prediction target

The label, has_rat_activity, marks an inspection positive when its official result records rat activity (“Rat Activity” or “Rat Activity and Failed”) and negative when it “Passed.” Mouse-only findings and treatment or monitoring visits are excluded rather than counted, so this is strictly a rat model. Positives are rare - fewer than one inspection in ten records rat activity - which makes it a genuinely imbalanced problem and shapes how we measure success below.

The label is the outcome of an inspection itself, and every feature is computed strictly from information available before that inspection: a location’s own history can never include the visit being predicted. Applied at serving time to a location’s current state, the model then estimates whether its next inspection will find rats.

The data

The work is grounded in three public New York City open datasets:

• Rodent inspections - the City’s open DOHMH Rodent Inspection dataset, which supplies the target and a location’s inspection history, keyed to a tax lot.
• 311 rodent complaints - service requests filtered to rodent reports, which supply recent complaint activity in and around each lot.
• Property records - the City’s PLUTO tax-lot data, which supplies building type, age, size, land use, and zoning.

Every one of these is publicly available, keyed to the public Borough-Block-Lot identifier. To be explicit about the civic tie: the connection to New York City is its open data, and nothing more. The City did not commission, review, or endorse this work; the model runs only in a shadow setting that drives no real inspection; and there is no customer, complainant, or account-level data anywhere in the pipeline.

Feature engineering

The production model uses sixty features, organised into the families that both field experience and the public-health literature point to:

• Inspection history - a location’s prior rat rate, how recently it was last inspected, how recently it last showed activity, and how many clean inspections it has strung together. Past behaviour is the single most informative signal about future behaviour.
• Built environment - building age and type, unit counts, lot and floor area, land use, and zoning, drawn from PLUTO. Older and mixed-use structures tend to offer more harbourage.
• 311 complaint signals - rolling counts, recency, and velocity of nearby rodent complaints. These are a core input, not a footnote: complaint activity is one of the strongest short-horizon signals the model has.
• Neighbourhood and geography - the surrounding area’s rat rate and inspection and complaint density, aggregated over a hexagonal spatial grid, plus ZIP code, borough, and community board.
• Season - calendar features, because rat activity is strongly seasonal.

One note on rigor: because complaint volume partly reflects who is likely to report as much as where rats actually are, we treat the 311 signals as evidence to weigh, not ground truth, and monitor per-borough error to watch for the geographic reporting differences that complaint data can introduce.

Why gradient-boosted trees

The data is tabular. It mixes numeric fields with high-cardinality categoricals such as ZIP code and building class, carries nonlinear interactions between them, and has missing values. That is precisely the regime where gradient-boosted decision trees are the strongest general-purpose choice, so we benchmarked three of them - LightGBM, XGBoost, and CatBoost - under hyperparameter search, against simple baselines (logistic regression, a historical per-lot rate, and the majority class).

LightGBM won and ships. At near-identical discrimination to XGBoost, it held a clear advantage on fairness across boroughs, so the final model is a single LightGBM classifier wrapped in an isotonic-regression calibrator - one calibrated model, not an ensemble of algorithms. The calibrator matters: it makes the model’s probabilities trustworthy, so a score of 0.7 means roughly a seventy-percent chance rather than an arbitrary ranking.

Validation

Because inspections are spread across space and time, the evaluation has to respect that structure. Shuffling inspections at random and splitting them would let a location’s later visits leak into training and flatter the score. Instead the data is split in time - train on the earliest window, validate on the next, test on the most recent - with calendar gaps between the splits, so the test set is genuinely later inspections the model never saw.

Both figures are AUC scores - short for the area under a curve, a single number between 0 and 1 that summarises how well the model separates the two outcomes across every possible decision threshold rather than at one arbitrary cut-off. ROC-AUC - the area under the receiver-operating-characteristic curve - has a clean reading: it is the probability that, handed one location that will fail and one that will pass, the model assigns the higher risk to the one that fails. A score of 0.5 is a coin flip, no better than guessing, and 1.0 is a perfect ranking, so 0.84 means the model puts the failing location first roughly five times out of six. PR-AUC - the area under the precision-recall curve - asks the stricter question that matters when positives are rare: of the locations it flags as high-risk, how many truly fail, weighed against how many of the real failures it catches.

On that held-out set the model reaches a ROC-AUC of about 0.84. We also report a precision-recall AUC of about 0.72, which better reflects performance on the rare positive class than a single accuracy figure would - on an imbalanced problem, a model that simply predicted “no rats” every time would look deceptively accurate while being useless. The result holds up under one-, three-, and six-month forward holdouts and on lots the model had never seen, and it collapses to chance when the labels are shuffled - the negative control that tells you the signal is real rather than leaked.

What the signal looks like

The signals the model leans on hardest are a location’s own inspection history, recent 311 complaint activity in and around it, and the surrounding neighbourhood’s rat rate, together with the building and geographic characteristics of the lot. That a purely data-driven model rediscovers what public-health researchers have documented for years - that recent activity and neighbourhood context are the best predictors of future activity - is itself a form of validation.

Limitations and responsible use

• The labels reflect where the City actually inspected - a complaint-driven, non-random slice of the city, not a census of every rat - so the model learns from an inspection footprint, and its live numbers should be read with that selection in mind.
• Complaint signals can encode reporting patterns as well as infestation. That is monitored, not ignored.
• A forecast is a probability, not a verdict. A high-risk score means “worth looking at sooner,” never “infested.”
• Risk here is a property of buildings and infrastructure, not of the people who live nearby. This work should never be used to stigmatise a neighbourhood.
• It is research on public data, run only in a shadow setting that drives no real inspection, and the City of New York was not involved in producing it.

From open data to operations

This study is a public-data proof of a capability Ardenus applies every day for operators: take scattered, messy, real-world data, unify it into one model, and turn it into a decision made before the truck rolls. The same discipline shown here - a carefully defined target, leakage-safe validation, calibrated probabilities, and honest limitations - is what we bring to the systems our customers already run. You can read more of our research and writing on the Ardenus articles hub.