The index, and alignment

If you take one thing from the pandas track, take this. In NumPy, adding two arrays matches them by position. In pandas, adding two Series matches them by label.

That's called alignment, and it happens automatically, everywhere.

a = pd.Series([120, 80, 150], index=["Delhi", "Mumbai", "Pune"])
b = pd.Series([10, 20, 30],  index=["Mumbai", "Pune", "Jaipur"])

a + b
# Delhi      NaN
# Jaipur     NaN
# Mumbai    90.0
# Pune     170.0

Mumbai found Mumbai even though they sat in different positions. Delhi and Jaipur appear in only one side, so there was nothing to add — and the answer is NaN rather than an error. The result's index is the union of both, sorted.

Notice the dtype came out float64. NaN is a float, so it drags the column with it — the same wart as the last lesson.

Why this is a feature

It means you can add January's numbers to February's without checking that both months have the same shops in the same order. The labels carry the meaning; pandas does the matching. Merge two datasets and the alignment is free.

It's also the source of the classic bug: you expected three rows and got seven, all of them NaN, because two indexes that looked the same weren't — one had trailing spaces, or was string "1" against integer 1. When arithmetic between two frames produces unexpected NaNs, compare the indexes first.

Controlling it

a.add(b, fill_value=0)      # a missing label counts as 0
a.reindex(b.index)          # force a onto b's labels
a.align(b, join="inner")    # keep only labels present in both
a.reset_index(drop=True)    # throw the labels away — positional again
a.values + b.values         # drop to NumPy and add by position

Choosing your index

The default index — 0, 1, 2… — carries no meaning, which is why so much early pandas feels positional. Give it a real one and everything downstream gets easier:

orders.set_index("date")            # a column becomes the index
orders.set_index("date").sort_index()
df.reset_index()                    # the index becomes a column again
df.reset_index(drop=True)           # or is discarded

A good index is meaningful (a date, an id), and ideally unique and sorted — a sorted index makes label lookups a binary search instead of a scan. Check with df.index.is_unique and df.index.is_monotonic_increasing.

And the index is an object in its own right, with methods worth knowing: df.index.difference(other.index) tells you exactly which labels one has and the other doesn't — usually the fastest way to explain a surprising pile of NaNs.