Duplicates

Rows 8 and 9 of orders are identical in every column. An export run twice, a form submitted twice, a join gone wrong — they arrive constantly, and they silently inflate every total you compute.

Finding them

orders.duplicated()          # True for rows already seen
orders.duplicated().sum()    # 1 — the number of EXTRA copies

The first occurrence is False and the repeat is True. That's worth holding on to: the count is the number of surplus rows, not the number of rows involved. Two rows that duplicate each other give a count of 1.

keep="first"   # default — mark later copies
keep="last"    # mark earlier copies instead
keep=False     # mark EVERY row that has a twin

keep=False is the one to use when you're looking rather than deleting, because it shows you both sides of each collision:

orders[orders.duplicated(keep=False)].sort_values(["date", "city"])

Removing them

orders.drop_duplicates()                    # 12 of 13 rows — keeps the first of each
orders.drop_duplicates(keep="last")         # keeps the last instead
orders.drop_duplicates(ignore_index=True)   # renumber 0..n afterwards

Like filtering, this keeps the original index labels — row 9 is simply gone and the rest don't renumber. Pass ignore_index=True if you'd rather they did.

subset — the part that matters

By default a duplicate means "every column identical". That's rarely the question you actually have. More often you mean "this key should only appear once":

orders.duplicated(subset=["date", "city"])     # 2 — rows 6 and 9
orders.drop_duplicates(subset=["date", "city"], keep="last")

Now two rows that differ in cups still count as duplicates, because the pair (date, city) is what you've declared should be unique.

Whether they are duplicates is a judgement about your data, not a fact pandas can work out. Two orders from the same stall on the same day might be a data error, or might be a morning and an afternoon delivery. subset= is where you record which.

Look before you delete

drop_duplicates() is one call and it's irreversible. Inspect first, every time:

dupes = orders[orders.duplicated(subset=["date", "city"], keep=False)]
dupes.sort_values(["date", "city"])   # why do these collide?

Very often the answer is that they aren't duplicates at all — they're two real events and the key you picked was too coarse. Deleting them would have quietly lost data.

Related checks

orders["city"].value_counts()      # how often each value appears
orders["city"].nunique()           # 3 distinct cities
orders.index.is_unique             # the index can have duplicates too
orders.value_counts().head()       # which whole rows repeat

A non-unique index causes surprising behaviour later — .loc[3] quietly returns several rows instead of one — so it's worth checking after any concat.