Skip to content

hash table resize policy: stop pointless and oversized rehashes - #53

Merged
ThomasWaldmann merged 3 commits into
mainfrom
fix-resize-policy
Aug 18, 2026
Merged

hash table resize policy: stop pointless and oversized rehashes#53
ThomasWaldmann merged 3 commits into
mainfrom
fix-resize-policy

Conversation

@ThomasWaldmann

Copy link
Copy Markdown
Member

Three fixes to the HashTable resize policy, found while profiling an
insert-1M / iterate / delete-500k / insert-500k-new / delete-all workload
(32-byte keys, 8-byte values).

Only shrink when the new capacity is actually smaller

__delitem__ shrinks when used < capacity * min_load_factor. Once the capacity is
at the MIN_CAPACITY floor, new_capacity gets clamped back to MIN_CAPACITY, so
every further delete called _resize_table with the capacity the table already had:
a malloc, a full rehash of all buckets and a free, per delete.

  • emptying a 99-entry table: 99 such rehashes, now 0, and 632 -> 70 ns per delete
  • 100k insert/delete cycles on a table with 50 live entries: 99950 rehashes, now 442

Do not grow the table when it is mostly tombstones

The grow check tested used + tombstones against the load budget and then always
doubled the capacity. Tombstones, unlike used entries, do not need more buckets -
_resize_table drops them, because it only copies occupied buckets. A table that had
half of its entries deleted and then refilled therefore doubled although the live
entries alone would have fit at load 0.49.

Now it only grows if the used entries alone occupy more than rehash_threshold (new
keyword argument, default 0.75) of the load budget, else it rehashes at the same
capacity. The threshold keeps that rehash amortized: it leaves at least 25% of the
budget free, i.e. >= 0.125 * capacity insertions of headroom until the next resize,
so a rehash costing O(capacity) buys Omega(capacity) operations.

  • the workload's capacity stays at 2048000 buckets instead of doubling to 4096000
  • physical footprint after the refill: 92 -> 78..83 MiB

Reuse tombstone buckets for new entries

_lookup_index walked past tombstones to the first free bucket, so an insert never
reused a tombstone bucket and tombstones could only ever be dropped by a rehash of
the whole table. It now reports the first tombstone seen on the probe chain and
__setitem__ puts the new entry there. The probe still has to run to the free bucket,
otherwise an entry further down the chain would not be found, so lookups do not get
cheaper - tombstones just stop accumulating. Overwriting a tombstone with a live entry
cannot break a probe chain, only turning it back into a free bucket would.

  • 100k insert/delete cycles on a table with 50 live entries: 442 -> 163 rehashes
  • keeps the probe chains of the deliberately not-grown table short: 0.92 linear steps
    per lookup while refilling 1M entries at load 0.49

Measured effect, main vs this branch

main branch
capacity after delete-500k / insert-500k-new 4,096,000 2,048,000
footprint there 92-93 MiB 78-83 MiB
delete all 1M 113-118 ms 90-95 ms
100k churn cycles, 50 live entries 99950 rehashes, 1203 ns/pair 163 rehashes, 257 ns/pair
empty a 99-entry table 632 ns/delete 70 ns/delete

Insert, iteration and delete-500k timings are unchanged over three repeats each.

Each fix comes with a test that fails without it, plus test_delete_shrinks_table and
test_used_entries_grow_table as guards that real shrinking and real growing still
happen. test_tombstone_is_recycled builds a deterministic probe chain from
key32 % capacity and checks the new entry lands in the tombstone's bucket.

__delitem__ shrinks the table when used < capacity * min_load_factor. Once the
capacity is at the MIN_CAPACITY floor, new_capacity gets clamped back to
MIN_CAPACITY, so every further delete called _resize_table with the capacity the
table already had: a malloc, a full rehash of all buckets and a free, per delete.

Emptying a 99-entry table did 99 such rehashes; it is now 9x faster (632 -> 70 ns
per delete). Insert/delete churn on a table with 50 live entries did 99950
rehashes for 100k cycles, now 442 (941 -> 281 ns per insert+delete pair). Big
tables are barely affected, as the futile rehashes only start below
MIN_CAPACITY * min_load_factor = 100 live entries - but they were still 100 of
the 122 table resizes of an insert-1M / delete-all cycle.

A rehash at unchanged capacity was also the only thing that cleared tombstones at
the floor. They now accumulate until the grow check in __setitem__ rehashes, which
is bounded by capacity * max_load_factor deletes and keeps the capacity at the
floor (verified: the churn test above oscillates between 1000 and 2000 buckets).
The grow check in __setitem__ tested used + tombstones against the load budget and
then always doubled the capacity. But tombstones, unlike used entries, do not need
more buckets - _resize_table drops them, because it only copies occupied buckets.
So a table that had half of its entries deleted and then refilled doubled although
the live entries alone would have fit at load 0.49.

Only grow if the used entries alone occupy more than rehash_threshold (new keyword
argument, default 0.75) of the load budget, else rehash at the same capacity. The
threshold keeps the rehash amortized: it leaves at least 25% of the budget free,
i.e. >= 0.125 * capacity insertions of headroom until the next resize, so a
rehash costing O(capacity) buys Omega(capacity) operations. Do not set it close to
1.0, or alternating delete/insert can rehash every few operations.

For the insert-1M / delete-500k / insert-500k-new workload this keeps the capacity
at 2048000 buckets instead of doubling to 4096000: 83 instead of 92 MiB.
_lookup_index walked past tombstones to the first free bucket, so an insert never
reused a tombstone bucket and tombstones could only ever be dropped by a rehash of
the whole table. Remember the first tombstone seen on the probe chain and put the
new entry there instead of into the free bucket.

The probe still has to run to the free bucket, otherwise an entry further down the
chain would not be found, so this does not make lookups cheaper - it stops
tombstones from accumulating. Overwriting a tombstone with a live entry cannot
break a probe chain (only turning it back into a free bucket would).

100k insert/delete cycles on a table with 50 live entries now need 163 rehashes
instead of 442, and it keeps the probe chains of a table that is deliberately not
grown (see previous commit) short: 0.92 linear steps per lookup while refilling
1M entries at load 0.49.
@ThomasWaldmann
ThomasWaldmann merged commit 0b0247b into main Aug 18, 2026
3 checks passed
@ThomasWaldmann
ThomasWaldmann deleted the fix-resize-policy branch August 18, 2026 05:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant