Missing values

prep_min is missing on 152 of 900 rows. scikit-learn estimators refuse to fit on NaN, so you have to do something — and which something is a more consequential decision than it looks.

Ask why it's missing first

Before choosing any fix, look at what the missingness correlates with:

missing = deliveries["prep_min"].isna()

deliveries.loc[missing,  "late"].mean()    # 0.329
deliveries.loc[~missing, "late"].mean()    # 0.201

Rows with no prep time are 32.9% late; rows that have one are 20.1%. That gap is not an accident. The value goes missing when the stall is slammed — nobody stops to log a prep time during the rush — and the rush is exactly when deliveries run late.

So the absence of the value is one of the more informative things in the row. Hold onto that; everything below follows from it.

The three kinds of missing

• MCAR — missing completely at random. A sensor dropped packets. Dropping rows is safe, imputing is safe, nothing is biased.
• MAR — missing at random given the other columns. Older devices log less. Imputation using the other columns works well.
• MNAR — missing depends on the value itself, or on the target. Ours. This is the common case in real data, and the one where the standard advice hurts.

You can't determine which you have from the data alone — it takes knowing how the data was collected. But the check above (does the missingness correlate with the target?) catches a lot of MNAR quickly.

Dropping the rows

X.dropna()      # 900 → 748 rows
# and the late rate falls from 0.2222 to 0.2005

You lose 17% of your data, which is bad enough. Worse, you lose it selectively — the rows you dropped were disproportionately the late ones, so every number downstream is now computed on a dataset that under-represents the thing you're trying to predict.

Nothing in your pipeline will tell you this happened. Dropping is only safe under MCAR, and that has to be checked rather than assumed.

Imputing

from sklearn.impute import SimpleImputer

SimpleImputer(strategy="median")          # skewed columns — the safe default
SimpleImputer(strategy="mean")            # symmetric columns
SimpleImputer(strategy="most_frequent")   # categorical columns
SimpleImputer(strategy="constant", fill_value=0)

Measured on our data, 5-fold AUC with the full feature set:

median                 0.8590
mean                   0.8592
constant, fill 0       0.8173

Median and mean are indistinguishable. Filling with 0 costs 0.042, because it isn't a neutral placeholder — it's a claim that those deliveries needed no preparation at all, and the model believes it.

Median is the better default for the same reason as in the matplotlib distributions lesson: it doesn't move when the tail does.

What every imputation destroys

before:   NaN,  8.8,  8.8,  NaN
after:    8.8,  8.8,  8.8,  8.8

After filling, a row that was missing is indistinguishable from a row that genuinely sat at the median. The fact of being missing — which we measured as worth 12 percentage points of late rate — is gone, permanently.

add_indicator

SimpleImputer(strategy="median", add_indicator=True)

Fills the gap and appends a 0/1 column recording that it did. The model gets both the filled value and the knowledge that it was filled.

median only              0.8590
median + add_indicator   0.8786

+0.020 AUC from one keyword argument — a bigger gain than most changes of algorithm will ever buy you, and it costs one column. Worth doing whenever the missingness might carry information, which is more often than people assume.

(It also can't really hurt: if the missingness is uninformative, the model gives that column a near-zero coefficient and moves on.)

Beyond SimpleImputer

• KNNImputer — fills from the k most similar rows. Better when columns are correlated; slow on large data.
• IterativeImputer — models each column from all the others, repeatedly. The strongest option and the easiest to over-trust; it's still experimental and needs an explicit import.
• Native support — HistGradientBoostingClassifier handles NaN itself, learning which side of a split missing values belong on. Sometimes the best answer is no imputation step at all.

Start with median plus an indicator. It's usually most of the benefit for a fraction of the complexity, and it's much harder to get subtly wrong.

And the rule from lesson 10 applies

An imputer learns — statistics_ has the underscore. Learn those statistics from the whole dataset and you've leaked the test set's medians into training. Put it in a pipeline. That's the next lesson.