aurora: read the MAP relative-velocity direction byte - #218
Open
deisenstein wants to merge 1 commit into
Open
Conversation
Abacus now stores the DIRECTION of a MAP's velocity relative to the mean local DM
alongside the magnitude it already kept, as an Nside=4 NEST healpix pixel index
(192 pixels, one byte). Surface it as a plain uint8 rather than expanding to
three floats, to keep the post-processing footprint at 1 byte/particle; callers
index the lookup table only when they actually want vectors:
from abacusnbody.data.output_particle import rel_vel_healpix_lookup
v3 = t['rel_vel'][:, None] * rel_vel_healpix_lookup[t['rel_vel_healpix']]
output_particle: the MAP flag is now the pid_k band (pid_k & 0xff00) == 0xff00
rather than the single value 0xffff, with the low byte carrying the direction.
LAYOUT is bumped to output_particle_24b accordingly -- an older reader pointed at
a new file would have decoded most MAPs as DM particles with no error at all.
maplog: the multiplicity word is now a count in the low 24 bits with the
direction in the top byte, so mult is masked to 0x00FFFFFF (unconditionally, so
every modality decodes the same way). Adds a LAYOUT of mapnode_24b, which
read_abacus now validates; files written before the split carry no MapLogLayout
key at all, so an absent key is accepted and a mismatched one is not.
Renames maplog's vel_rel to rel_vel. This is a breaking rename, done
deliberately: the two modules had different names for the same physical quantity,
and propagating that into a second field name would have entrenched it.
The 192 unit vectors live in _healpix_nside4.py, generated from the same source
as the C header and re-exported from both readers. Abacus also writes the table
into every ASDF header as the 'rel_vel_healpix_lookup' ndarray, so files remain
self-describing and no healpix library is needed; the module-level copy just
saves opening one.
Adds tests/test_aurora_readers.py, the first tests of any kind for the Aurora
readers. Rows are built by hand rather than from reference files: these formats
are pure bit layouts, so a hand-built row states the layout in a form a reviewer
can check against the C++ by eye. Covers both shared-word splits at their
boundaries (0xfeff is the largest DM pid_k; a full-24-bit count beside a nonzero
direction byte is what a stale reader gets wrong), all four MapNode modalities,
and the invariant that the lookup table inverts the vec2pix_nest64 that Abacus
uses to produce the indices.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We need to update the reader to parse the new velocity direction information in both the maplog and the particle subsample.
From Claude:
Abacus now stores the DIRECTION of a MAP's velocity relative to the mean local DM alongside the magnitude it already kept, as an Nside=4 NEST healpix pixel index (192 pixels, one byte). Surface it as a plain uint8 rather than expanding to three floats, to keep the post-processing footprint at 1 byte/particle; callers index the lookup table only when they actually want vectors:
output_particle: the MAP flag is now the pid_k band (pid_k & 0xff00) == 0xff00 rather than the single value 0xffff, with the low byte carrying the direction. LAYOUT is bumped to output_particle_24b accordingly -- an older reader pointed at a new file would have decoded most MAPs as DM particles with no error at all.
maplog: the multiplicity word is now a count in the low 24 bits with the direction in the top byte, so mult is masked to 0x00FFFFFF (unconditionally, so every modality decodes the same way). Adds a LAYOUT of mapnode_24b, which read_abacus now validates; files written before the split carry no MapLogLayout key at all, so an absent key is accepted and a mismatched one is not.
Renames maplog's vel_rel to rel_vel. This is a breaking rename, done deliberately: the two modules had different names for the same physical quantity, and propagating that into a second field name would have entrenched it.
The 192 unit vectors live in _healpix_nside4.py, generated from the same source as the C header and re-exported from both readers. Abacus also writes the table into every ASDF header as the 'rel_vel_healpix_lookup' ndarray, so files remain self-describing and no healpix library is needed; the module-level copy just saves opening one.
Adds tests/test_aurora_readers.py, the first tests of any kind for the Aurora readers. Rows are built by hand rather than from reference files: these formats are pure bit layouts, so a hand-built row states the layout in a form a reviewer can check against the C++ by eye. Covers both shared-word splits at their boundaries (0xfeff is the largest DM pid_k; a full-24-bit count beside a nonzero direction byte is what a stale reader gets wrong), all four MapNode modalities, and the invariant that the lookup table inverts the vec2pix_nest64 that Abacus uses to produce the indices.