Add support for the alternate screen buffer - #212
Conversation
Private modes 47, 1047, 1048 and 1049 were recorded as inert mode bits, so a full-screen program painted its alternate screen on top of the primary one and never restored it on exit. Reading such a screen back gives a merged, impossible state with nothing reporting a problem -- which is why `less`, `vim` and `htop` could not be emulated (issue selectel#90). Entering saves the primary buffer and starts from an empty alternate one; leaving restores the primary buffer and discards the alternate. Both directions are idempotent, so a program emitting 1049h twice does not lose the primary screen. 1049 also saves and restores the cursor, as it is defined to combine 1047 with 1048. `Screen.alternate_screen` exposes the state, RIS returns to the primary buffer, and `Screen.resize` clips the offscreen buffer so exiting cannot restore an over-wide screen. Lines scrolled off the alternate screen no longer enter HistoryScreen scrollback, matching why paging a file with `less` does not leave the file in the terminal's history. Behaviour was measured against two independent emulators rather than read off the specification, and they settled one point the xterm documentation leaves misleading: the "without clearing" wording for 47/1047 describes clearing, not buffer lifetime. Alternate-screen contents do not survive a round trip in either tmux 3.6b or GNU screen 4.00.03 (with `altscreen on`), so there is nothing for per-mode clear flags to distinguish. Both also keep a single buffer-switch flag, so `ESC [ ? 4 7 l` exits a buffer entered with 1049. The cursor is deliberately not homed on entry, which both confirm.
|
One correction to my own claim above, before anyone spends review time on it. I said the behaviour was measured against two independent emulators. That is true The DECSC/DECRC control run matters: it shows the rig can detect a cursor restore, So Worth noting the same probe shows |
Neither was reachable by the tests as first written, so both get a regression test named after the mechanism it locks. 1048/1049 no longer share the `savepoints` stack with DECSC. Full-screen programs use DECSC *inside* the alternate screen constantly, and an unbalanced `ESC 7` there left the wrong savepoint on top, so exiting restored the program's internal cursor instead of the shell's. Measured on tmux 3.6b: park at row 3 col 7, enter 1049, `ESC [ 1 ; 1 H` then `ESC 7`, exit -- the cursor returns to row 3 col 7, where this code returned it to row 1 col 1. The saved cursor now lives in its own slot, and a stray 1049l with nothing saved leaves the cursor alone rather than falling through to restore_cursor's empty-stack homing. Shrinking via resize() now drops rows from the TOP of the offscreen buffer, matching what the displayed buffer does through delete_lines. Clipping the bottom instead kept the wrong half, so a program that resized while on the alternate screen restored the wrong rows on exit. The existing resize test encoded the bug: its expected value was written by hand from the broken output. It now derives the expectation by resizing a screen that never entered the alternate buffer, so the assertion cannot agree with a defect in the code it checks.
|
Pushed a second commit fixing two defects I found by reviewing my own patch 1049 shared the The 1048/1049 cursor now lives in its own slot. A stray
Worth flagging the reason the second one survived my first pass, since it is the Re-verified after the fixes: 140 passed / 1 xfailed, differential vs real tmux I also probed four more interactions that turned out clean, listed here so they do |
A multi-agent review of the previous two commits found these; each is reproduced in memory and locked by a test named after the mechanism. A nested smcup no longer clobbers the saved cursor. `_switch_screen_buffer` was already idempotent but `_save_alt_cursor` was called unconditionally, so a program emitting `ESC [ ? 1 0 4 9 h` twice lost the shell's cursor and rmcup restored the program's own. Measured on tmux 3.6b: `ESC[3;7H smcup ESC[5;15H smcup rmcup` restores the cursor to row 3 col 7, identical to the single-smcup case. 1048 on its own is an explicit DECSC and still re-saves, including while the alternate screen is up -- suppressing that too would be an over-correction, so both directions are pinned. DECSCNM now reaches both buffers. It is a property of the SCREEN, but pyte realises it by rewriting each cell, and only the displayed buffer was rewritten. So toggling reverse video while on the alternate screen restored a primary screen whose cells contradicted `self.mode`. A new `_all_buffers` helper feeds both set_mode and reset_mode. `HistoryScreen.prev_page` is guarded on the alternate screen. index and reverse_index already were; paging was not, so scrolling back spliced primary scrollback into the running program's display and pushed its rows into history.bottom. next_page needs no guard and does not get one: `before_event` forces the history to the bottom on every non-paging event, switching buffers is one, so by then `position == size` and `history.bottom` is empty. A guard there would be unobservable code, and a test asserting on it would pass either way -- which is how it was caught. The modes.py docstrings for 47 and 1047 claimed neither entering nor leaving clears a buffer. The implementation discards the alternate buffer on exit for all three modes, so the docs contradicted the code and would have shipped that contradiction to the published API reference.
|
Third commit pushed. A multi-agent adversarial review of my own two commits found The claim that was wrongI listed DECSCNM toggled while on the alternate screen among interactions DECSCNM is a property of the SCREEN, but pyte realises it by rewriting each cell, The other threeA nested smcup clobbered the saved cursor.
modes.py contradicted the implementation. The A harness defect worth reporting on its ownWhile re-verifying, the mutation harness itself proved non-deterministic: the same That mattered because the "11/11 mutations caught" figure in my first comment was Current stateThe verification table in the PR body is now stale (it says 137 passed / 20 tests); |
`_alternate_screen` was exactly `_offscreen_buffer is not None` at every reachable state, so it was a second copy of one fact that could drift out of step. A review pass flagged it as a style point; it is more than that — two values that must agree are a defect waiting for the next contributor to set one and not the other. Equivalence was checked before removing it, not assumed: 2197 three-step sequences over smcup/rmcup in all three modes, 1048, RIS, DECCOLM, erase and text, plus a resize after each — 8788 states, zero disagreements. Net effect is one less attribute, one less assignment in the switch, and a property that cannot be wrong. Suite unchanged at 145 passed / 1 xfailed, mypy unchanged, differential against real tmux 19/19 identical, mutation testing 11/11 + 5/5. One knock-on worth noting for anyone editing the mutation harness: removing that assignment made `self.dirty.update(range(self.lines))` non-unique in the file, so the "home cursor on entry" mutation had to anchor on the preceding line as well.
A third review round found that 1048 and 1049 shared `_alt_savepoint`, so a
program issuing `ESC [ ? 1 0 4 8 h` while the alternate screen was up overwrote
the cursor smcup had saved for the shell:
ESC[3;7H smcup ESC[5;15H <inner save> rmcup
nothing saved inside : cursor (2, 6) <- the shell's cursor
ESC 7 inside : cursor (2, 6) <- correct
ESC[?1048h inside : cursor (4, 14) <- the program's cursor
This is the same defect the dedicated slot was introduced to fix for DECSC one
commit earlier — reintroduced by routing 1048 through the same slot when it was
added. 1048 is an independent DECSC that a program may legitimately issue inside
the alternate screen; it needs its own storage.
`_save_alt_cursor`/`_restore_alt_cursor` now take a `cursor_only` flag selecting
between two slots, and set_mode/reset_mode dispatch 1048 and 1049 separately
rather than sharing one branch.
Also closes a blind spot in the new test rather than only adding the case that
failed. The paired sequence (`1048h` then `1048l` inside the alternate screen)
cannot distinguish a suppressed save from a working one — a suppressed save
leaves an empty slot, the restore is inert, and 1049 supplies the same answer
either way. Mutation testing caught that: suppressing 1048 inside the alternate
screen left the test green. The test now moves the cursor between `1048h` and
`1048l`, so the save has to have happened.
146 passed / 1 xfailed, differential against real tmux 19/19 identical, mutation
testing 11/11 + 5/5 with the two slot-sharing reversions both caught, mypy at
its single pre-existing error.
A maintainer-perspective review measured the branch against the rest of the file
and the objections held up:
- `_switch_screen_buffer` was the most prose-heavy method in the branch, and its
longest paragraph was a 10-line note about xterm's "do not clear" wording that
argued a point rather than telling a caller anything. Cut to two short
paragraphs: the cursor is not homed, and all three modes behave identically
because no reference implementation preserves alternate-screen contents.
- Measurement transcripts ("park at row 3 col 7, enter 1049, ...") removed from
docstrings. Master pins no tool version in any docstring and neither should
this.
- The CHANGES entry was the only 7-line bullet in a file whose 90 bullets have a
median of 2. Now 2 lines.
Two deliberate departures from that review. The tool versions stay in the two
INTERNAL comments that record why a branch is written the way it is — a docstring
is for the caller, a comment is for whoever edits the code next, and deleting the
latter loses the evidence. And `HistoryScreen.prev_page` gained a docstring line
rather than losing one: it is exported through `automodule :members:`, so its new
silent no-op on the alternate screen has to be discoverable from the API
reference, where previously only an internal comment mentioned it.
Also replaced the now-unreachable `saved if saved is not None else
self._make_buffer()` fallback with an assert. Since `alternate_screen` became
derived, the early return guarantees the offscreen buffer exists on that path.
146 passed / 1 xfailed, mypy unchanged, differential against real tmux 19/19,
mutation 11/11 + 5/5.
|
Three more commits. A third review round found one real defect, and it was mine — 1048 was clobbering 1049's saved cursor
This is precisely the defect the dedicated slot was added for one commit earlier, cursor_down was missing the same region guard as index and cursor_upNot fixed in this PR — flagging it because it is the same defect class and a
Trimmed to this codebase's registerA maintainer-perspective review measured the branch against the rest of the file,
Two deliberate departures. Tool versions stay in the two internal comments that Also replaced the now-unreachable Two blind spots in my own testsBoth found by mutation testing, not by review, and worth stating because they are The paired 1048 sequence could not see a suppressed save. A row-cap assertion never entered the code it checked. Without a DECSTBM region Current stateVerified after every change, not once at the end. |
1 participant
Closes #90.
Private modes
47,1047,1048and1049were shifted intoScreen.modeasinert bits with no handler, so a full-screen program painted its alternate screen
on top of the primary one and never restored it on exit. Reading such a screen
back gives a merged, impossible state with nothing reporting a problem. As
@shobrook put it on #90: "Most TUIs (e.g.
htop) use the alternate screen buffer.Meaning that
pytecannot emulate most TUIs."What it does
Entering saves the primary buffer and starts from an empty alternate one; leaving
restores the primary buffer and discards the alternate. Both directions are
idempotent, so a program emitting
1049htwice does not lose the primary screen.1049also saves and restores the cursor, since it is defined to combine1047with
1048.Also handled:
Screen.alternate_screenexposes the current state.reset()(RIS) returns to the primary buffer and discards both.resize()clips the offscreen buffer, so exiting cannot restore an over-wideprimary screen.
HistoryScreenscrollback — matching why paging a file with
lessdoes not leave the file inyour terminal's history.
How the behaviour was decided
Measured against two independent emulators rather than read off the spec: tmux
3.6b and GNU screen 4.00.03, feeding identical bytes to a real pane and
diffing the resulting grid cell by cell. Where they agree, that is the target;
they agreed on every case here.
That process settled one point the xterm documentation actively misleads on. The
"without clearing" wording for
47/1047describes the clearing action, notbuffer lifetime, and taken literally it implies alternate-screen contents survive
a round trip. They do not — in either emulator, re-entering shows an empty
alternate screen regardless of which mode was used. I implemented the literal
reading first and the differential test rejected it. Both emulators also keep a
single buffer-switch flag, so
ESC [ ? 4 7 lexits a buffer entered with1049.And the cursor is deliberately not homed on entry, which both confirm.
One caveat worth recording for anyone re-running this: GNU screen's
altscreensetting defaults to off (
man screen), so withoutaltscreen onin thescreenrc it ignores all four modes and the comparison silently measures a
disabled feature.
Verification
less+vimover a PTYmypy(strict = true)The strongest evidence is a control experiment using only behavioural assertions
(no new API), so it runs unchanged against master and against this branch. The
assertion is the one that matters to anyone reading a terminal: after a
full-screen program exits, its body must not still be on screen.
20 tests were added (
tests/test_screen.py,tests/test_history.py), including aparametrised matrix over all nine enter/leave mode combinations. Every one was
mutation-verified — I broke the implementation eight different ways and confirmed
the intended test failed each time.
Unrelated pre-existing note:
mypyreports oneunused-ignoreonpyte/screens.py:40on master as well; newerwcwidthships type information, sothat
# type: ignore[import-untyped]is now redundant. Left alone as out of scope.Prepared with AI assistance and reviewed by me before submitting. The measurement
rig, the two reference emulators and the mutation results are all reproducible; I
am happy to attach the probe scripts or adjust the approach if you would prefer
different semantics for
47/1047.