Class imbalance

Every classifier in this module has done the same thing:

                      accuracy   recall
logistic regression     0.8044     0.44
random forest           0.8133     0.46
kNN, k=25               0.7956     0.24
tree, depth 3           0.7911     0.08
always "on time"        0.7778     0.00

Every accuracy is within four points of the do-nothing baseline, and every recall is poor. That's not five separate failures — it's one problem, appearing five times.

Why it happens

The loss function counts every row equally, and 78% of rows are "on time". Predicting the common class is nearly always right, so saying "late" and being wrong is punished exactly as hard as saying "on time" and being wrong — and it happens far more often. The training process rewards caution.

Majority voting inside a tree leaf does the same thing at the other end (lesson 20). Both are the model doing precisely what you asked.

class_weight

LogisticRegression(class_weight="balanced")

# weight = n_samples / (n_classes × n_class_samples)
#   on time:  900 / (2 × 700) = 0.64
#   late:     900 / (2 × 200) = 2.25
#   ratio:    3.5×

Each class is weighted by the inverse of its frequency, so one late row now costs 3.5× a punctual one to get wrong — and the model becomes correspondingly less shy about predicting it. One argument, no resampling, no new dependency.

Supported by LogisticRegression, SVC, DecisionTreeClassifier, RandomForestClassifier and more. You can also pass an explicit dict — {0: 1, 1: 10} — when you know the real cost ratio, which is better than "balanced" if you do.

What it buys, and what it costs

                         acc     prec    recall     F1    missed  alarms
plain                   0.8044   0.579    0.44     0.500     28      16
class_weight balanced   0.7733   0.494    0.78     0.605     11      40

Recall 0.44 → 0.78. F1 0.500 → 0.605. Missed late deliveries 28 → 11. For one keyword argument, that is the largest improvement anything in this track has produced on the metric that actually matters.

And the cost: accuracy falls to 0.7733 — below the 0.7778 dummy — while false alarms go from 16 to 40. If you were tracking accuracy you would reject this model. It finds 39 of 50 late deliveries instead of 22, and it "scores worse".

Lesson 5 warned about this in the abstract. Here it is deciding a real choice.

Or just move the threshold

threshold 0.25          0.7733   0.493    0.74     0.592     13      42

Almost exactly the same place. Which makes sense — both are doing the same thing, making the model readier to say "late". One does it during training, the other afterwards.

They are two routes to the same trade, not two improvements. Pick one. Doing both means you've moved twice and will have to re-tune the threshold anyway.

Resampling

• Random oversampling — duplicate minority rows. Crude, and effectively the same as reweighting.
• SMOTE — synthesise new minority rows between neighbours. From imbalanced-learn. Can help on severe imbalance; on 22% it rarely beats class_weight.
• Undersampling — throw away majority rows. Discards data; a last resort.

And a hard rule: resample inside the fold, never before. Oversample the whole dataset and then cross-validate, and copies of the same row land in both train and test — lesson 13's leakage wearing a helpful-looking hat, and it produces spectacular, fake scores.

The honest summary

There's no free lunch here. Every one of these moves the precision/recall trade rather than escaping it. More recall always means more false alarms; that's arithmetic, not a limitation of the technique.

1. Decide the costs. What does a miss cost? A false alarm?
2. Pick the metric from those costs, before you model.
3. Pick one knob — class_weight or threshold.
4. Tune it by cross-validation on the training set, never the test set.
5. Report the confusion matrix, not the accuracy.

For our stall, where a miss costs ₹200 and a text costs ₹5, the balanced model is obviously right. Its accuracy being below the dummy's is not an argument against it — it's an argument against accuracy, which is where this module started.