Skip to content

aurora: read the MAP relative-velocity direction byte - #218

Open
deisenstein wants to merge 1 commit into
aurorafrom
aurora-velocity
Open

aurora: read the MAP relative-velocity direction byte#218
deisenstein wants to merge 1 commit into
aurorafrom
aurora-velocity

Conversation

@deisenstein

Copy link
Copy Markdown

We need to update the reader to parse the new velocity direction information in both the maplog and the particle subsample.

From Claude:

Abacus now stores the DIRECTION of a MAP's velocity relative to the mean local DM alongside the magnitude it already kept, as an Nside=4 NEST healpix pixel index (192 pixels, one byte). Surface it as a plain uint8 rather than expanding to three floats, to keep the post-processing footprint at 1 byte/particle; callers index the lookup table only when they actually want vectors:

from abacusnbody.data.output_particle import rel_vel_healpix_lookup
v3 = t['rel_vel'][:, None] * rel_vel_healpix_lookup[t['rel_vel_healpix']]

output_particle: the MAP flag is now the pid_k band (pid_k & 0xff00) == 0xff00 rather than the single value 0xffff, with the low byte carrying the direction. LAYOUT is bumped to output_particle_24b accordingly -- an older reader pointed at a new file would have decoded most MAPs as DM particles with no error at all.

maplog: the multiplicity word is now a count in the low 24 bits with the direction in the top byte, so mult is masked to 0x00FFFFFF (unconditionally, so every modality decodes the same way). Adds a LAYOUT of mapnode_24b, which read_abacus now validates; files written before the split carry no MapLogLayout key at all, so an absent key is accepted and a mismatched one is not.

Renames maplog's vel_rel to rel_vel. This is a breaking rename, done deliberately: the two modules had different names for the same physical quantity, and propagating that into a second field name would have entrenched it.

The 192 unit vectors live in _healpix_nside4.py, generated from the same source as the C header and re-exported from both readers. Abacus also writes the table into every ASDF header as the 'rel_vel_healpix_lookup' ndarray, so files remain self-describing and no healpix library is needed; the module-level copy just saves opening one.

Adds tests/test_aurora_readers.py, the first tests of any kind for the Aurora readers. Rows are built by hand rather than from reference files: these formats are pure bit layouts, so a hand-built row states the layout in a form a reviewer can check against the C++ by eye. Covers both shared-word splits at their boundaries (0xfeff is the largest DM pid_k; a full-24-bit count beside a nonzero direction byte is what a stale reader gets wrong), all four MapNode modalities, and the invariant that the lookup table inverts the vec2pix_nest64 that Abacus uses to produce the indices.

Abacus now stores the DIRECTION of a MAP's velocity relative to the mean local DM
alongside the magnitude it already kept, as an Nside=4 NEST healpix pixel index
(192 pixels, one byte).  Surface it as a plain uint8 rather than expanding to
three floats, to keep the post-processing footprint at 1 byte/particle; callers
index the lookup table only when they actually want vectors:

    from abacusnbody.data.output_particle import rel_vel_healpix_lookup
    v3 = t['rel_vel'][:, None] * rel_vel_healpix_lookup[t['rel_vel_healpix']]

output_particle: the MAP flag is now the pid_k band (pid_k & 0xff00) == 0xff00
rather than the single value 0xffff, with the low byte carrying the direction.
LAYOUT is bumped to output_particle_24b accordingly -- an older reader pointed at
a new file would have decoded most MAPs as DM particles with no error at all.

maplog: the multiplicity word is now a count in the low 24 bits with the
direction in the top byte, so mult is masked to 0x00FFFFFF (unconditionally, so
every modality decodes the same way).  Adds a LAYOUT of mapnode_24b, which
read_abacus now validates; files written before the split carry no MapLogLayout
key at all, so an absent key is accepted and a mismatched one is not.

Renames maplog's vel_rel to rel_vel.  This is a breaking rename, done
deliberately: the two modules had different names for the same physical quantity,
and propagating that into a second field name would have entrenched it.

The 192 unit vectors live in _healpix_nside4.py, generated from the same source
as the C header and re-exported from both readers.  Abacus also writes the table
into every ASDF header as the 'rel_vel_healpix_lookup' ndarray, so files remain
self-describing and no healpix library is needed; the module-level copy just
saves opening one.

Adds tests/test_aurora_readers.py, the first tests of any kind for the Aurora
readers.  Rows are built by hand rather than from reference files: these formats
are pure bit layouts, so a hand-built row states the layout in a form a reviewer
can check against the C++ by eye.  Covers both shared-word splits at their
boundaries (0xfeff is the largest DM pid_k; a full-24-bit count beside a nonzero
direction byte is what a stale reader gets wrong), all four MapNode modalities,
and the invariant that the lookup table inverts the vec2pix_nest64 that Abacus
uses to produce the indices.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@deisenstein
deisenstein requested a review from lgarrison August 4, 2026 00:28
@deisenstein deisenstein self-assigned this Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant