Capstone: analysis without pandas

You have 270 temperature readings — 90 days across 3 cities — stored the way raw data usually arrives: three flat, parallel arrays, with some readings missing entirely.

city   # 270 city codes: 0, 1 or 2
day    # 270 day numbers: 0 to 89
temp   # 270 readings, some of them nan

This is exactly the shape of data pandas was invented for, and we're going to do the whole analysis without it. Not because that's better — pandas is shorter and clearer for this — but because everything pandas does is built out of the operations you now know, and doing it by hand once makes the next track feel obvious.

1. Inspect before you trust

temp.shape, temp.dtype
np.isnan(temp).sum()             # how many are missing
np.isnan(temp).mean()            # what fraction
np.nanmin(temp), np.nanmax(temp) # a sanity check on the range

Always count the gaps before doing anything else. A reading of -999 or a temperature of 400°C tells you something is wrong with the data, not with the weather.

2. Clean, carefully

The important detail: temp[~np.isnan(temp)] gives you clean temperatures that no longer line up with city and day. Whenever arrays are parallel, mask them all with the same mask:

ok = ~np.isnan(temp)
temp_ok, city_ok, day_ok = temp[ok], city[ok], day[ok]

That's the same lesson as sorting one column of a table, and it's worth being paranoid about — the failure is silent.

3. Group without a groupby

np.bincount counts occurrences of small non-negative integers. Give it weights= and it sums them instead — which is a group-by:

totals = np.bincount(city_ok, weights=temp_ok)
counts = np.bincount(city_ok)
means  = totals / counts

That's the mean per city in three lines and no loops. The same trick handles any grouping whose keys are small integers — and np.unique(..., return_inverse=True) from lesson 18 turns string labels into exactly those integers.

For a mean per city and per month you can group on a combined key — city * 12 + month — then reshape the result. That's what a MultiIndex is doing underneath.

4. Rank and report

order = np.argsort(means)[::-1]
cities[order], means[order].round(1)

And percentiles rather than just averages, since lesson 21 made the case: np.percentile(temp_ok, [5, 50, 95]) tells you far more about a distribution than a mean does.

Work through the exercises

They follow that order — inspect, clean, aggregate, rank — and each one is small. Use the playground freely in between; that's what it's there for.