unique and set operations

Real data repeats — categories, user ids, labels, sensor states. np.unique is the workhorse for dealing with that, and it quietly does several jobs at once.

unique sorts, too

x = np.array([3, 1, 3, 7, 1, 1, 9])
np.unique(x)     # [1 3 7 9]

The distinct values, sorted. That's not optional — it falls out of how the function works — and it's worth knowing because the output order won't match the order values first appeared in your data.

Frequency tables

values, counts = np.unique(x, return_counts=True)
# values [1 3 7 9]
# counts [3 2 1 1]

One call and you have a frequency table. This is the NumPy answer to value_counts, and pairing it with argsort from the last lesson gives you "most common first" immediately.

The other two flags

np.unique(x, return_index=True)     # where each value FIRST appeared
np.unique(x, return_inverse=True)  # how to rebuild x from the unique values

return_inverse is more useful than it looks. It gives you an integer code for every element, so uniq[inverse] reconstructs the original array exactly. That is label encoding — turning a categorical column into integers a model can eat — in one line:

cities = np.array(["Delhi", "Pune", "Delhi", "Mumbai"])
uniq, codes = np.unique(cities, return_inverse=True)
codes            # [0 2 0 1]
uniq[codes]      # back to the original

Set operations

Once duplicates are gone, the usual set questions have direct answers:

np.intersect1d(a, b)    # in both
np.union1d(a, b)        # in either
np.setdiff1d(a, b)      # in a but not b
np.setxor1d(a, b)       # in one but not both

All four return sorted, deduplicated arrays, and the 1d in the names is a real warning: they flatten their input, so they answer questions about values rather than about rows.

isin is the one you'll use most

It's the odd one out, and deliberately so. Rather than returning a set, it keeps the shape of the input and returns a boolean mask:

np.isin(x, [1, 9])       # [False True False False True True True]
x[np.isin(x, [1, 9])]    # the matching values

Because it's a mask it plugs straight into everything from lesson 7 — filtering, np.where, counting with .sum(). Think of it as the vectorised in operator, and reach for it instead of chaining a dozen | comparisons.

One performance note: np.isin sorts internally, so it stays fast even when the list of allowed values is long. A chain of == comparisons scans the array once per value, which does not.