Linear regression

Same data, new question. Instead of "will it be late?" — a category — predict minutes, a number. That's regression, and it's worth noticing that late was minutes > 30 all along: somebody's threshold. Predicting the number keeps everything the threshold discarded, and you can always apply the cut afterwards.

A weighted sum

ŷ = b + w₁x₁ + w₂x₂ + … + wₙxₙ

That's the entire model. Fitting means choosing the weights that make the sum of squared errors as small as possible — and there's a closed-form solution, so nothing is searched or iterated. One matrix operation, which is why it's instant.

from sklearn.linear_model import LinearRegression

model = Pipeline([("pre", pre), ("m", LinearRegression())])
model.fit(X_train, y_train)

model.named_steps["m"].intercept_   # 27.507
model.named_steps["m"].coef_        # one weight per column

The coefficients are sentences

Because the model is a sum, every weight means something you can say out loud:

rain_heavy               +3.884     heavy rain adds ~3.9 minutes
rain_none                −3.057     dry weather saves ~3.1
prep_min                 +2.976     per standard deviation of prep time
distance_km              +2.900     per standard deviation of distance
rider_Asha               −1.556     Asha is quicker than average
rider_Dev                +1.297     Dev is slower
prep_min_missing         +1.164     an unlogged prep time costs ~1.2 minutes
area_Industrial          −0.876

Look at that missingness indicator earning its place — +1.16 minutes when the prep time wasn't logged. That's lesson 12's argument restated in units of time.

Two conditions on reading them

They're only comparable because we scaled. Unscaled, distance_km is "minutes per kilometre" and temp is "minutes per degree" — putting those in a ranked list is meaningless. Scaled, each is "minutes per standard deviation", which is a common currency. This is the interpretability argument from lesson 10.

Each is the effect holding the others fixed. "The effect of distance, for two deliveries alike in every other respect." That is not the same as the overall effect of distance, and the difference matters whenever columns are correlated.

In fact when two columns are correlated the model can split their shared effect between them almost arbitrarily — which is why you sometimes see two large opposing coefficients on related columns. That's instability, not insight, and lesson 15 is partly about fixing it.

What it assumes

• Linearity — the relationship really is a straight line.
• Additivity — effects add; no interactions. (Lesson 16 relaxes this.)
• Constant error spread — misses are about the same size everywhere.
• Independent rows, and no perfect collinearity.

You don't check these against a list — you check them by plotting the residuals, in lesson 17.

Scoring it

R²    0.5948      MAE  3.10 minutes      RMSE  3.86 minutes

DummyRegressor(strategy="mean")
R²   −0.0040      MAE  4.91 minutes

R² is the share of variance explained, and its great virtue is that the baseline is built in: predicting the mean scores exactly 0. Unlike accuracy, 0.59 is immediately readable without knowing anything else about the data.

(The dummy scores −0.004 rather than exactly 0 because it's evaluated on a test set whose mean differs slightly from the training mean it memorised. Close enough to zero to make the point.)

The surprise

LinearRegression              5-fold R² 0.5785 ± 0.061
RandomForestRegressor(200)   5-fold R² 0.5198 ± 0.071

The simple model wins. Not by noise, either — the gap is comparable to the fold spread and consistent in direction.

And the reason is honest: minutes genuinely is a weighted sum of distance, prep time, rain and rider. Linear regression is exactly the right shape for that, while the forest has to approximate a straight line with a staircase of splits and spends its flexibility on noise instead.

So: fit the simple model first

It's instant, it's a real baseline rather than a strawman, and it explains itself. If a forest can't beat it you've learned that your relationship is essentially linear — a genuine finding about your problem, for free. And if the forest wins by a lot, that tells you there are interactions or thresholds worth going and understanding.