Skip to content

Disambiguate hash collisions in internal/hashmap via Hasher.Equal#542

Open
PratikDhanave wants to merge 4 commits into
microsoft:mainfrom
PratikDhanave:fix-hashmap-collision-equality
Open

Disambiguate hash collisions in internal/hashmap via Hasher.Equal#542
PratikDhanave wants to merge 4 commits into
microsoft:mainfrom
PratikDhanave:fix-hashmap-collision-equality

Conversation

@PratikDhanave

Copy link
Copy Markdown
Contributor

Problem

internal/hashmap's Hasher interface requires an Equal(T, T) bool method, and Map stores each entry's key — but Load, Delete and Set keyed purely on the 64-bit hash and never called Equal. The stored key and the Equal method were dead unless used for collision disambiguation.

When two distinct keys hash to the same value:

m.Set("alice", "a")
m.Set("bob", "b")   // same hash → overwrites alice's entry
m.Load("alice")       // → ("b", true)   WRONG value
m.Load("carol")       // → ("b", true)   returns a value for a key never stored
m.Len()                // → 1              silent data loss

Delete similarly removed whatever entry shared the hash, not the one whose key actually matched.

The hashers in use (e.g. the workflow scope/update-key hashers) are maphash-based, so collisions are rare — but the map is a general-purpose keyed container and the Equal contract exists precisely to make it correct regardless.

Fix

Store entries in per-hash buckets (map[uint64][]entry) and use Hasher.Equal to select the matching entry in Load, Delete and Set. This honors the interface contract that the stored keys and Equal method were already there to support. No exported method signature changes. (Also fixes a th//e comment typo.)

Test

Adds the package's first test, TestMap_DistinctKeysWithCollidingHashCoexist, using a hasher that forces all keys to one hash. It asserts distinct colliding keys coexist, a never-stored colliding key resolves to (zero, false), and deleting one colliding key leaves the other. Fails on main (wrong values, Len()==1), passes with the fix. All existing workflow/checkpoint/state callers still pass.

The Hasher interface requires an Equal method and Map stores each entry's
key, but Load, Delete and Set keyed purely on the hash value and never
called Equal. When two distinct keys hashed to the same value:

- Set overwrote the earlier key's entry (silent data loss),
- Load returned another key's value — even for a key never stored,
- Delete removed a different key's entry.

Store entries in per-hash buckets and use Hasher.Equal to select the
matching entry in Load, Delete and Set, honoring the interface contract
the stored keys and Equal method were already there to support. No
exported method signature changes. Adds the package's first test,
covering distinct keys that share a hash.
Copilot AI review requested due to automatic review settings July 18, 2026 02:57
@PratikDhanave
PratikDhanave requested a review from a team as a code owner July 18, 2026 02:57

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes correctness issues in internal/hashmap where Load, Set, and Delete previously keyed only on the 64-bit hash and did not use Hasher.Equal, causing silent data corruption on hash collisions. It changes the internal storage to per-hash buckets and adds a focused regression test that forces collisions to verify correct behavior.

Changes:

  • Store entries as map[uint64][]entry to support multiple keys per hash.
  • Use Hasher.Equal to disambiguate collisions in Load, Set, and Delete (and update iterators/String accordingly).
  • Add TestMap_DistinctKeysWithCollidingHashCoexist to validate collision correctness.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
internal/hashmap/hashmap.go Switches to per-hash buckets and uses Equal to correctly handle hash collisions across map operations.
internal/hashmap/hashmap_test.go Adds a regression test using a forced-collision hasher to ensure distinct colliding keys coexist and delete/load behave correctly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +69 to +73
if len(bucket) == 1 {
delete(m.entries, hash)
} else {
m.entries[hash] = append(bucket[:i], bucket[i+1:]...)
}
Comment thread internal/hashmap/hashmap.go Outdated
Comment on lines +105 to +111
// Len returns the number of map entries.
func (m *Map[K, V]) Len() int {
return len(m.entries)
n := 0
for _, bucket := range m.entries {
n += len(bucket)
}
return n

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done — added an entry count updated in Set/Delete/Clear so Len() is O(1) again, and Delete now shifts the tail down and zeroes the vacated slot to avoid retaining the removed key/value.

Address review feedback:
- Len() is O(1) again via an entry count updated in Set/Delete/Clear,
  instead of summing bucket lengths.
- Delete shifts the tail down and zeroes the vacated slot so the removed
  key/value are not retained by the bucket's backing array.
Adds a test covering Len across insert/update/delete/clear.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants