Skip to content

HDDS-16256. Avoid the per-group list copy in OMKeyRequest.filterOutBlocksStillInUse - #11115

Open
shuan1026 wants to merge 1 commit into
apache:masterfrom
shuan1026:HDDS-16256
Open

HDDS-16256. Avoid the per-group list copy in OMKeyRequest.filterOutBlocksStillInUse#11115
shuan1026 wants to merge 1 commit into
apache:masterfrom
shuan1026:HDDS-16256

Conversation

@shuan1026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

OMKeyRequest.filterOutBlocksStillInUse called OmKeyLocationInfoGroup.createLocationList() on every version group of the reference key just to build a Map<ContainerBlockID, OmKeyLocationInfo> lookup table, allocating a throwaway flattened List per group that is immediately re-streamed and discarded.

This PR replaces the flatMap(createLocationList().stream()) chain with nested stream flattening over getLocationLists(), removing the per-group list allocation while preserving Collectors.toMap duplicate-key semantics.

Additionally, a throwaway (not committed) JUnit micro-benchmark measured per-lookup-map-construction thread allocation for both implementations (JDK21.0.12, 50,000-iteration warmup then 500,000 measured iterations per shape, output maps asserted equal, volatile sink to block JIT elimination):

Key Shape Before After Saved
1 group x 1 block 240 ns / 950 B 107 ns / 704 B 246 B
1 group x 4 blocks 171 ns / 1072 B 169 ns / 872 B 200 B
1 group x 32 blocks 1149 ns / 4736 B 1014 ns / 4200 B 536 B
4 groups x 4 blocks 504 ns / 3648 B 404 ns / 2608 B 1040 B
16 groups x 8 blocks 6973 ns / 21088 B 6449 ns / 16688 B 4400 B

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16256

How was this patch tested?

  • TestOMKeyCommitRequest#testValidateAndUpdateCacheOnOverwrite
  • TestOMKeyCommitRequest#testValidateAndUpdateCacheOnOverwriteWithUncommittedBlocks

@sravani-revuri sravani-revuri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants