Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -1070,15 +1070,19 @@ The page the reference design shows (#56), with measurements verified in v1's so
`−`/`+` carry `aria-disabled` and a tooltip naming the limit, never a press that silently
does nothing. `docs/annotations.md` carries the argument.
- **Navigating the picture is one model across every device** (#576). Pan: a two-finger
trackpad scroll, a mouse wheel, a middle- or right-button drag, `Space` held with any
drag, or the hand tool. Zoom: a trackpad pinch, `Ctrl`/`Cmd` with a scroll, the widget's
trackpad scroll, a middle- or right-button drag, `Space` held with any drag, or the hand
tool. Zoom: a mouse wheel, a trackpad pinch, `Ctrl`/`Cmd` with a scroll, the widget's
`−`/`+`, and `mod+0` to fit. On a touchscreen one finger draws — or pans while the hand
is on — and two fingers pinch and drag together. **A bare wheel pans; it used to zoom**,
which is deliberate and is the whole of what made a trackpad workable: a two-finger
scroll is how anybody moves around a canvas, and while it zoomed there was no gesture on
a trackpad that moved the picture at all. Mouse zoom is the modifier and the buttons.
The shortcut sheet's **Navigate** section is the one place this is written for a user,
and it is hand-written rather than derived — a gesture has no chord to be read off.
is on — and two fingers pinch and drag together. **A bare wheel event is answered by
device**: a two-finger scroll pans, a wheel notch zooms. That split is what makes a
trackpad workable without costing the mouse the gesture it has always had — a two-finger
scroll is how anybody moves around a canvas and a notch is how anybody zooms, and a rule
serving one leaves the other with nothing. No browser reports the device, so the test is
a heuristic and **everything uncertain is read as a trackpad**: a mouse it declines still
zooms with the modifier and the buttons, where a trackpad that zoomed by accident would
be the defect over again. The shortcut sheet's **Navigate** section is the one place this
is written for a user, and it is hand-written rather than derived — a gesture has no
chord to be read off.

### The read-only mode

Expand Down
52 changes: 39 additions & 13 deletions docs/annotations.md
Original file line number Diff line number Diff line change
Expand Up @@ -438,19 +438,39 @@ moved the picture at all, while the gesture people actually make, a two-finger s

The whole of that fix is which side of one branch a wheel event falls on:

| Held | What a wheel event means |
| The event | What it means |
| --- | --- |
| `ctrlKey` or `metaKey` | zoom, anchored at the cursor |
| nothing | pan, both axes |
| `ctrlKey` or `metaKey` held | zoom, anchored at the cursor |
| bare, and shaped like a wheel notch | zoom, anchored at the cursor |
| bare, anything else | pan, both axes |

That single branch serves four devices, because **`ctrlKey` on a wheel event is how a browser
reports a trackpad pinch** - on macOS and on a Windows precision touchpad alike, with no gesture
API involved - and `Ctrl`/`Cmd`+wheel is the convention for zooming with a mouse. The two arrive
The first row serves four devices, because **`ctrlKey` on a wheel event is how a browser reports a
trackpad pinch** - on macOS and on a Windows precision touchpad alike, with no gesture API
involved - and `Ctrl`/`Cmd`+wheel is the convention for zooming with a mouse. The two arrive
identically, so they are answered identically.

**A bare wheel now pans where it used to zoom.** That is the deliberate half, and it is what makes
a trackpad workable rather than a nicety on top. Mouse zoom did not become unreachable: it is the
modifier, the widget's `-`/`+`, and `mod+0`.
**A bare event is answered by device, and that is the second question.** The two devices want
opposite things from the same event: a two-finger scroll is how anybody moves around a canvas, and
a wheel notch is how anybody zooms. #576 gave the whole event to the trackpad - which is what made
a trackpad workable rather than a nicety on top, and which took the mouse's zoom away with it. The
device test gives the notch back and leaves the scroll alone.

`isMouseWheel` is that test, and it is the only heuristic in the navigation model, because **no
browser says which device sent a wheel event**. It reads three signals and none of them is
`deltaY`, which the operating system accelerates and which overlaps completely between the two:

- a `deltaMode` other than pixels is a discrete wheel, and nothing else reports lines or pages -
this is Firefox's mouse;
- anything sideways is a scroll, because a wheel has one axis;
- otherwise a whole number of `wheelDelta` notches. Chrome quantises a discrete wheel to units of
120 however much it accelerated `deltaY`, and computes a precise device's as `-3 * deltaY`,
which lands on a multiple of 120 only when the scroll happened to travel an exact multiple of
40 pixels.

**Every uncertain case answers "trackpad".** A trackpad that zooms when it was asked to scroll is
the failure #576 fixed, while a mouse this declines - a Magic Mouse reports as a precise device -
still zooms with the modifier, with the widget's `-`/`+`, and with `mod+0` to refit. The cost of
being wrong is one stray notch inside a hard flick, never a device with no gesture at all.

What `ctrlKey` can no longer do is tell a pinch from a mouse wheel, since it is now set by both.
`wheelZoomFactor` tells them apart by **magnitude** instead: a notch is a large quantised value -
Expand Down Expand Up @@ -481,10 +501,16 @@ somebody navigating a batch they may not edit is who most needs to pan.

All of the arithmetic is in `adapters/viewport.ts` beside the transform, so it is unit-tested
without a browser: `normalizedWheel` folds `deltaMode`, `wheelZoomFactor` carries the softness
split, `pinchBetween` answers the two-finger case, and `zoomAbout` and `panBy` were already there.
`AnnotatorCanvas` holds only the branches. What no unit test can reach - that a browser really
delivers these events - is `e2e/touch.spec.ts` and the wheel scenarios, which drive Chromium's own
input rather than constructing events.
split, `isMouseWheel` answers the device, `pinchBetween` answers the two-finger case, and
`zoomAbout` and `panBy` were already there. `AnnotatorCanvas` holds only the branches. What no unit
test can reach - that a browser really delivers these events - is `e2e/touch.spec.ts` and the wheel
scenarios, which drive Chromium's own input rather than constructing events.

That division is load-bearing for the device test in particular, because a browser suite can only
reach half of it: **CDP's synthetic wheel reports `wheelDeltaY` as ±120 whatever `deltaY` says** -
measured, 7 and 12 and 40 all arrive as -120 - so Playwright can spell a mouse notch and a sideways
scroll, and has no spelling at all for a trackpad's vertical one. The unit tests are what cover the
rest, and `demo.spec.ts` says so where a reader would otherwise go looking.

### Dragging repaints one layer

Expand Down
60 changes: 37 additions & 23 deletions frontend/annotator/src/adapters/react/AnnotatorCanvas.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,11 @@
* gesture that moved the picture. Four things fixed that, and only the first is a
* change to something that already worked:
*
* 1. **The wheel branches on `ctrlKey || metaKey`**: held, it zooms about the
* cursor; bare, it pans both axes. One branch for four devices, because that
* flag is how a browser reports a trackpad pinch *and* how a mouse asks to
* zoom. A bare wheel used to zoom; it pans now, deliberately.
* 1. **The wheel branches on `ctrlKey || metaKey`, then on the device**: held, it
* zooms about the cursor, because that flag is how a browser reports a
* trackpad pinch *and* how a mouse asks to zoom. Bare, a trackpad scroll pans
* both axes and a mouse notch zooms — `isMouseWheel` is the test, and it
* answers "trackpad" whenever it is unsure.
* 2. **`Space` held** is the hand while it is down — a substitution rather than a
* registry row, because a keystroke is a press and this needs a release.
* 3. **`panTool`** is the persistent hand, the host's to own for `suggestion`'s
Expand All @@ -103,8 +104,9 @@
* the second press from a fresh first one.
*
* All four are wiring. The arithmetic — `normalizedWheel`, `wheelZoomFactor`,
* `pinchBetween`, and the `zoomAbout`/`panBy` that were already there — is in
* `adapters/viewport.ts`, where it is unit-tested without a browser.
* `isMouseWheel`, `pinchBetween`, and the `zoomAbout`/`panBy` that were already
* there — is in `adapters/viewport.ts`, where it is unit-tested without a
* browser.
*
* ## Capture is taken after the dispatch, never before
*
Expand Down Expand Up @@ -193,6 +195,7 @@ import {
IDENTITY_VIEWPORT,
fitToViewport,
imageRenderingAt,
isMouseWheel,
normalizedWheel,
panBy,
pinchBetween,
Expand Down Expand Up @@ -665,27 +668,38 @@ export function AnnotatorCanvas({
event.preventDefault();
const [dx, dy] = normalizedWheel(event.deltaX, event.deltaY, event.deltaMode);
/**
* **One branch, and it serves four devices.**
* **Two questions, and between them they serve four devices.**
*
* `ctrlKey` on a wheel event is how a browser reports a trackpad pinch —
* on macOS and on a Windows precision touchpad alike, with no gesture API
* involved — and `ctrl`/`cmd` + wheel is the convention for zooming with
* a mouse. Those are the same flag, so they are the same branch, and
* everything else is a pan.
* First, is a modifier held? `ctrlKey` on a wheel event is how a browser
* reports a trackpad pinch — on macOS and on a Windows precision touchpad
* alike, with no gesture API involved — and `ctrl`/`cmd` + wheel is the
* convention for zooming with a mouse. Those are the same flag, so they
* are the same branch.
*
* The half that changed is what "everything else" now covers. A plain
* wheel used to zoom, which made a two-finger trackpad scroll — the
* ordinary way anyone moves around a canvas — zoom instead of scroll, and
* left a trackpad with no pan at all. Now it pans, `deltaX` included, and
* a mouse wheel pans vertically for the same reason it scrolls a page
* vertically. Zoom did not become unreachable: it is the modifier, the
* pinch, `mod+0` and the two buttons in the corner.
* Second, for a bare event: which device sent it? A two-finger scroll is
* the ordinary way anybody moves around a canvas, and a wheel notch is the
* ordinary way anybody zooms — #576 gave the whole event to the first and
* so took the second away, which is what this restores. `isMouseWheel`
* decides, and it is a heuristic: it answers "trackpad" whenever it is
* unsure, so a mouse it declines still zooms with the modifier while a
* trackpad never zooms when it was asked to scroll.
*
* The sign is inverted because a scroll reports how far the *content*
* should travel against the gesture, and `panBy` moves the content with
* it: scrolling down looks at what is below, so the picture goes up.
* A pan reads both axes; a zoom reads `dy` alone, since a notch has no
* sideways component to spend. The pan's sign is inverted because a scroll
* reports how far the *content* should travel against the gesture, and
* `panBy` moves the content with it: scrolling down looks at what is
* below, so the picture goes up.
*
* `wheelDeltaY` is legacy and TypeScript's DOM library no longer declares
* it, hence the widening — it is read, never required, and a browser
* without it lands on the pan.
*/
if (event.ctrlKey || event.metaKey) {
const { wheelDeltaY = 0 } = event as WheelEvent & { readonly wheelDeltaY?: number };
const zooming =
event.ctrlKey ||
event.metaKey ||
isMouseWheel({ deltaMode: event.deltaMode, deltaX: event.deltaX, wheelDeltaY });
if (zooming) {
const rect = pane.getBoundingClientRect();
applyViewport(
zoomAbout(
Expand Down
32 changes: 32 additions & 0 deletions frontend/annotator/src/adapters/viewport.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ import {
fitToViewport,
imageRenderingAt,
imageToScreen,
isMouseWheel,
normalizedWheel,
panBy,
pinchBetween,
Expand Down Expand Up @@ -300,6 +301,37 @@ describe("a wheel event's travel is read in screen pixels whatever it was report
});
});

describe("a bare wheel event is read as a mouse or as a trackpad", () => {
it("reads a Chrome wheel notch as a mouse", () => {
expect(isMouseWheel({ deltaMode: 0, deltaX: 0, wheelDeltaY: -120 })).toBe(true);
});

it("reads a fast spin, which arrives as several notches at once, as a mouse", () => {
expect(isMouseWheel({ deltaMode: 0, deltaX: 0, wheelDeltaY: 360 })).toBe(true);
});

it("reads Firefox's line deltas as a mouse, since nothing else reports lines", () => {
expect(isMouseWheel({ deltaMode: 1, deltaX: 0, wheelDeltaY: 0 })).toBe(true);
});

it("reads a trackpad's own quantum as a trackpad", () => {
// A precise device's `wheelDeltaY` is `-3 * deltaY`, so 12 pixels of
// two-finger travel arrive as 36 and no notch is a multiple of 36.
expect(isMouseWheel({ deltaMode: 0, deltaX: 0, wheelDeltaY: -36 })).toBe(false);
});

it("reads anything sideways as a trackpad, whatever the vertical looks like", () => {
// The one case a magnitude test alone gets wrong: a scroll that travelled an
// exact multiple of 40 pixels vertically is notch-shaped by accident, and the
// sideways component is what still says it came from two fingers.
expect(isMouseWheel({ deltaMode: 0, deltaX: -4, wheelDeltaY: -120 })).toBe(false);
});

it("reads a browser that fills in no wheel delta as a trackpad, not as a mouse", () => {
expect(isMouseWheel({ deltaMode: 0, deltaX: 0, wheelDeltaY: 0 })).toBe(false);
});
});

describe("the wheel's zoom factor", () => {
it("makes one mouse notch worth one press of a 1.25 step button", () => {
expect(wheelZoomFactor(-120)).toBeCloseTo(1.25, 3);
Expand Down
52 changes: 52 additions & 0 deletions frontend/annotator/src/adapters/viewport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -265,6 +265,58 @@ export function normalizedWheel(
return [deltaX * scale, deltaY * scale];
}

/**
* A notch in the legacy `wheelDelta` unit, which is 120 wherever it is reported.
*
* `wheelDeltaY` is the one field a browser fills differently for the two
* devices. Chrome quantises a discrete wheel to whole notches of 120 however
* much the operating system has accelerated `deltaY`, and computes a precise
* device's as `-3 * deltaY`, which lands on a multiple of 120 only when the
* scroll happened to travel an exact multiple of 40 pixels.
*/
const WHEEL_NOTCH_UNITS = 120;

/** The fields of a wheel event the device test reads. */
export interface WheelShape {
/** 0 pixels, 1 lines, 2 pages — see `DELTA_SCALE`. */
readonly deltaMode: number;
readonly deltaX: number;
/** The legacy `WheelEvent.wheelDeltaY`, or 0 where the browser has none. */
readonly wheelDeltaY: number;
}

/**
* Whether a wheel event came from a mouse wheel rather than a trackpad.
*
* No browser says which device sent a wheel event, so this is a heuristic and
* the only one in the navigation model. It is needed because the two devices
* want opposite things from the same event: a two-finger scroll is how anybody
* moves around a canvas, and a wheel notch is how anybody zooms — and a rule
* that serves one leaves the other with no gesture at all. #576 gave the whole
* event to the trackpad; this gives the notch back.
*
* It reads three signals and none of them is `deltaY`, which is accelerated by
* the operating system and overlaps completely between the devices:
*
* - a `deltaMode` other than pixels is a discrete wheel, and nothing else
* reports lines or pages (this is Firefox's mouse);
* - anything sideways is a scroll, because a wheel has one axis;
* - otherwise a whole number of `wheelDelta` notches, which is what Chrome and
* Safari quantise a wheel to and a precise device rarely lands on.
*
* **Every uncertain case answers `false`.** A trackpad that zooms when it was
* asked to scroll is exactly the failure #576 fixed, and a mouse it declines —
* a Magic Mouse reports as a precise device, deliberately — still zooms with
* `ctrl`/`cmd` held. That modifier is answered before this is consulted, so a
* pinch and a held wheel never reach here.
*/
export function isMouseWheel(wheel: WheelShape): boolean {
if (wheel.deltaMode !== 0) return true;
if (wheel.deltaX !== 0) return false;
if (wheel.wheelDeltaY === 0) return false;
return Math.abs(wheel.wheelDeltaY) % WHEEL_NOTCH_UNITS === 0;
}

/**
* How much wheel travel doubles the zoom. Larger is gentler.
*
Expand Down
2 changes: 2 additions & 0 deletions frontend/annotator/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -280,13 +280,15 @@ export {
fitToViewport,
imageRenderingAt,
imageToScreen,
isMouseWheel,
normalizedWheel,
panBy,
pinchBetween,
screenToImage,
wheelZoomFactor,
zoomAbout,
type Pinch,
type WheelShape,
type Viewport,
} from "./adapters/viewport";

Expand Down
10 changes: 6 additions & 4 deletions frontend/app/e2e/_frame.ts
Original file line number Diff line number Diff line change
Expand Up @@ -301,10 +301,12 @@ export async function openOverflow(page: Page): Promise<void> {
/**
* A zoom notch over `at`, which is a wheel **with the modifier held**.
*
* A bare wheel pans now (#576), and every scenario that used to zoom with one is
* routed through here rather than holding the key inline — a spec that forgot it
* would still pass its "the picture moved" assertions and be measuring the wrong
* gesture entirely.
* A bare wheel is read as a device now (#576): Chromium's synthetic wheel looks
* like a mouse notch, so it would zoom anyway, but the modifier is what makes a
* scenario say which gesture it means. Every scenario that zooms is routed
* through here rather than holding the key inline — a spec that forgot it would
* still pass its "the picture moved" assertions while measuring whatever the
* device test happened to answer.
*
* `Control` and not `Meta`: both work in the product, and Playwright's
* `mouse.wheel` reads the keyboard's live modifier state, so the down/up pair is
Expand Down
Loading
Loading