Skip to content

HDDS-16255. Avoid the per-group list copy in ContainerMapper.parseOmDB - #11100

Open
shuan1026 wants to merge 1 commit into
apache:masterfrom
shuan1026:HDDS-16255
Open

HDDS-16255. Avoid the per-group list copy in ContainerMapper.parseOmDB#11100
shuan1026 wants to merge 1 commit into
apache:masterfrom
shuan1026:HDDS-16255

Conversation

@shuan1026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

ContainerMapper.parseOmDB called OmKeyLocationInfoGroup.createLocationList() once per version group of every key, only to iterate the result once. createLocationList() is documented as not O(1).

This PR adds one nested loop over the zero-copy getLocationLists() accessor; the inner loop body that builds BlockIdDetails is unchanged.

Additionally, a throwaway (not committed) JUnit micro-benchmark measured per-traversal thread allocation for both implementations (JDK 21.0.12, 50,000-iteration warmup then 500,000 measured iterations per shape, deterministic checksums asserted equal, volatile sink to block JIT elimination). Results, reproduced across two independent JVM processes:

Key Shape Before After Saved
1 group x 1 block 100 ns / 419 B 8 ns / 0 B 419 B
1 group x 4 blocks 87 ns / 408 B 13 ns / 0 B 408 B
1 group x 32 blocks 242 ns / 744 B 61 ns / 0 B 744 B
4 groups x 4 blocks 242 ns / 1632 B 52 ns / 0 B 1632 B
16 groups x 8 blocks 1303 ns / 6528 B 312 ns / 0 B 6528 B

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16255

How was this patch tested?

The following test passed:

  • TestContainerMapper#testContainerMapper

Also ran checkstyle.sh and author.sh successfully.

@sravani-revuri sravani-revuri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @shuan1026 for the patch. few minor comments below.

/**
* Generates Container Id to Blocks and BlockDetails mapping.
* @param configuration @{@link OzoneConfiguration}
* @return {@code Map<Long, List<Map<Long, BlockDetails>>>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: return type here says BlockDetails but the actual type is BlockIdDetails.

innerList = dataMap.get(containerID);
for (List<OmKeyLocationInfo> keyLocationInfo : keyLocationInfoGroup
.getLocationLists()) {
for (OmKeyLocationInfo keyLocation : keyLocationInfo) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this method now has 4 levels of looping which can be hard to read and understand. would it be better to put the logic below into a separate function?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants