hash table resize policy: stop pointless and oversized rehashes - #53
Merged
Conversation
__delitem__ shrinks the table when used < capacity * min_load_factor. Once the capacity is at the MIN_CAPACITY floor, new_capacity gets clamped back to MIN_CAPACITY, so every further delete called _resize_table with the capacity the table already had: a malloc, a full rehash of all buckets and a free, per delete. Emptying a 99-entry table did 99 such rehashes; it is now 9x faster (632 -> 70 ns per delete). Insert/delete churn on a table with 50 live entries did 99950 rehashes for 100k cycles, now 442 (941 -> 281 ns per insert+delete pair). Big tables are barely affected, as the futile rehashes only start below MIN_CAPACITY * min_load_factor = 100 live entries - but they were still 100 of the 122 table resizes of an insert-1M / delete-all cycle. A rehash at unchanged capacity was also the only thing that cleared tombstones at the floor. They now accumulate until the grow check in __setitem__ rehashes, which is bounded by capacity * max_load_factor deletes and keeps the capacity at the floor (verified: the churn test above oscillates between 1000 and 2000 buckets).
The grow check in __setitem__ tested used + tombstones against the load budget and then always doubled the capacity. But tombstones, unlike used entries, do not need more buckets - _resize_table drops them, because it only copies occupied buckets. So a table that had half of its entries deleted and then refilled doubled although the live entries alone would have fit at load 0.49. Only grow if the used entries alone occupy more than rehash_threshold (new keyword argument, default 0.75) of the load budget, else rehash at the same capacity. The threshold keeps the rehash amortized: it leaves at least 25% of the budget free, i.e. >= 0.125 * capacity insertions of headroom until the next resize, so a rehash costing O(capacity) buys Omega(capacity) operations. Do not set it close to 1.0, or alternating delete/insert can rehash every few operations. For the insert-1M / delete-500k / insert-500k-new workload this keeps the capacity at 2048000 buckets instead of doubling to 4096000: 83 instead of 92 MiB.
_lookup_index walked past tombstones to the first free bucket, so an insert never reused a tombstone bucket and tombstones could only ever be dropped by a rehash of the whole table. Remember the first tombstone seen on the probe chain and put the new entry there instead of into the free bucket. The probe still has to run to the free bucket, otherwise an entry further down the chain would not be found, so this does not make lookups cheaper - it stops tombstones from accumulating. Overwriting a tombstone with a live entry cannot break a probe chain (only turning it back into a free bucket would). 100k insert/delete cycles on a table with 50 live entries now need 163 rehashes instead of 442, and it keeps the probe chains of a table that is deliberately not grown (see previous commit) short: 0.92 linear steps per lookup while refilling 1M entries at load 0.49.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three fixes to the
HashTableresize policy, found while profiling aninsert-1M / iterate / delete-500k / insert-500k-new / delete-all workload
(32-byte keys, 8-byte values).
Only shrink when the new capacity is actually smaller
__delitem__shrinks whenused < capacity * min_load_factor. Once the capacity isat the
MIN_CAPACITYfloor,new_capacitygets clamped back toMIN_CAPACITY, soevery further delete called
_resize_tablewith the capacity the table already had:a malloc, a full rehash of all buckets and a free, per delete.
Do not grow the table when it is mostly tombstones
The grow check tested
used + tombstonesagainst the load budget and then alwaysdoubled the capacity. Tombstones, unlike used entries, do not need more buckets -
_resize_tabledrops them, because it only copies occupied buckets. A table that hadhalf of its entries deleted and then refilled therefore doubled although the live
entries alone would have fit at load 0.49.
Now it only grows if the used entries alone occupy more than
rehash_threshold(newkeyword argument, default 0.75) of the load budget, else it rehashes at the same
capacity. The threshold keeps that rehash amortized: it leaves at least 25% of the
budget free, i.e. >=
0.125 * capacityinsertions of headroom until the next resize,so a rehash costing O(capacity) buys Omega(capacity) operations.
Reuse tombstone buckets for new entries
_lookup_indexwalked past tombstones to the first free bucket, so an insert neverreused a tombstone bucket and tombstones could only ever be dropped by a rehash of
the whole table. It now reports the first tombstone seen on the probe chain and
__setitem__puts the new entry there. The probe still has to run to the free bucket,otherwise an entry further down the chain would not be found, so lookups do not get
cheaper - tombstones just stop accumulating. Overwriting a tombstone with a live entry
cannot break a probe chain, only turning it back into a free bucket would.
per lookup while refilling 1M entries at load 0.49
Measured effect,
mainvs this branchInsert, iteration and delete-500k timings are unchanged over three repeats each.
Each fix comes with a test that fails without it, plus
test_delete_shrinks_tableandtest_used_entries_grow_tableas guards that real shrinking and real growing stillhappen.
test_tombstone_is_recycledbuilds a deterministic probe chain fromkey32 % capacityand checks the new entry lands in the tombstone's bucket.