Storing and Deduplicating Data
What
A common misconception is that Git stores history as a series of “diffs” or “deltas” (only the lines of code that changed). In reality, Git stores entire copies of every file on a per-commit level. Every commit contains a complete snapshot of your codebase.
How
Despite storing full copies of every file, Git keeps its database small through two major optimization mechanisms:
- Deduplication: Because Git uses content-based hashing (SHA-1), the hash of a file’s blob depends purely on its content. If a file named
titles.mdis modified but a file namedcontents.mdremains unchanged between commits, both commit tree objects will simply point to the exact same pre-existing blob hash on disk. - Delta Compression (Packing): Periodically, Git runs a garbage-collection process (
git gc) that scans the object database, groups similar files together, compresses them, and stores them in single compressed archive files called Packfiles (using.packand.idxextensions). Inside packfiles, files are stored as deltas relative to other versions.
Why
Storing full snapshots instead of chains of diffs makes checkout and history operations extremely fast. If you switch branches or check out a commit from three years ago, Git does not need to compute hundreds of separate diff files sequentially; it simply retrieves the tree and blobs associated with that specific commit, writing them directly to your disk.
Summary
Git stores full copies of every file per commit for maximum retrieval speed, but relies on content-based deduplication and packfile delta compression to prevent the database from growing unnecessarily large.