Saving and loading

Two families, and the choice between them is the same trade every time: an exact binary round-trip, or something a human can open.

Binary — .npy and .npz

np.save("a.npy", a)          # one array
back = np.load("a.npy")      # identical dtype, shape and values

np.savez("d.npz", train=X, labels=y)       # several, named
d = np.load("d.npz"); d["train"]           # dict-like access
np.savez_compressed("d.npz", ...)          # zipped — smaller, slower

.npy stores a small header with the dtype and shape, then the raw bytes. There's no parsing, so it's fast, and nothing is lost. If the data is only ever going to be read back by Python, this is the default and you rarely need anything else.

One safety note: np.load refuses pickled object arrays by default, because unpickling runs arbitrary code. Never set allow_pickle=True on a file you didn't create.

Text — CSV and friends

np.savetxt("a.csv", a, delimiter=",", fmt="%.3f")
np.loadtxt("a.csv", delimiter=",")

Readable by a spreadsheet, a colleague, or any other language. And lossy in three specific ways worth knowing before you rely on it:

• dtype is gone. Everything comes back float64, whatever went in.
• shape is limited. 1-D and 2-D only — a text file has no way to express a third axis.
• precision is whatever you asked for. The default fmt rounds, so the values you read back are not quite the ones you wrote.

Fine for sharing a table. Wrong for saving model weights or a checkpoint you intend to resume from.

When the file is messy

np.loadtxt is strict and gives up on anything unexpected. np.genfromtxt is the tolerant one:

np.genfromtxt("data.csv", delimiter=",",
              skip_header=1,        # ignore a header row
              missing_values="NA",
              filling_values=np.nan)

Missing entries become np.nan, which is exactly what lesson 19 prepared you for. That said — if a file has mixed types per column, real headers and irregular missing values, you're describing a DataFrame. That's the next track.

Files bigger than memory

big = np.load("big.npy", mmap_mode="r")
big[1000:2000].mean()    # only these rows are actually read

Memory-mapping hands you something that behaves like an array but pages in from disk on demand. It's how you slice a 50GB file on a 16GB machine, and it costs one keyword.

A note on this lab: the filesystem here is virtual and in memory, so every example runs — the files just vanish when you reload.