Skip to content

wb7, wb8: pstore/ramoops panic-log persistence (kernel side)#359

Draft
evgeny-boger wants to merge 16 commits into
feature/v6.18from
feature/wb-pstore-shuttle-v6.18
Draft

wb7, wb8: pstore/ramoops panic-log persistence (kernel side)#359
evgeny-boger wants to merge 16 commits into
feature/v6.18from
feature/wb-pstore-shuttle-v6.18

Conversation

@evgeny-boger

@evgeny-boger evgeny-boger commented Jul 5, 2026

Copy link
Copy Markdown
Member

Persist kernel panic logs across the reboot that follows a crash, on Wiren Board 7 (Allwinner R40/A40i) and Wiren Board 8 (Allwinner H616/T507).

pstore/ramoops keeps a crash log in a reserved DRAM region, but that is only useful if the region address is stable across boots and the reboot preserves DRAM. This adds both, plus a guard against crash loops.

Changes

  • Config — enable pstore/ramoops in the WB7 and WB8 kernel configs.
  • Device tree — ship a fixed ramoops reserved-memory node at 0x43800000 (1 MiB, no-map) in both SoCs' DTBs, so the region is stable across boots and registers without a kernel command line.
  • Driver (drivers/soc/sunxi/wb-ramoops-panic-reset.c):
    • On panic, if ramoops holds unharvested records, arm the SoC watchdog for a ~6 s warm reset that preserves DRAM. The log stays in the region and systemd-pstore harvests it on the next boot.
    • Panic-loop breaker — stamp a breadcrumb in an always-on RTC scratch register (H616 0x0700010c, R40 0x01c20500) that survives a warm reset but is cleared by a full power cycle. On a repeated panic the driver declines to re-arm and lets the board escalate to a power cut, breaking the loop. The breadcrumb is cleared once the system stays up long enough to be considered healthy.
  • wbec MFD — recognise the newer EC power-on-reason codes.

Region address

0x43800000 sits below the kernel decompression buffer and above the relocated kernel and small-DRAM fixup loads, is non-power-of-two (dodging DDR size-scan midpoints), and fits a 128 MiB board.

Scope

This is the kernel side. On these boards a wedged controller is recovered by a full power cycle, not just a warm reset; the U-Boot bootloader parks the region across that cycle and the EC performs it, so a log survives even a hard power-cut. Validated end to end on WB7, WB8/LPDDR4 and WB8/DDR3: a crash log survives a real power cycle and is harvested on the next boot.

evgeny-boger and others added 5 commits July 5, 2026 22:34
Enable CONFIG_PSTORE_RAM and CONFIG_PSTORE_CONSOLE for the wb8 and wb7
flavours. The backing reserved-memory node (compatible = "ramoops",
last 1 MiB of DRAM, record-size 128K, console-size 256K, ecc 32) is
injected at boot by the Wiren Board U-Boot DT fixup, so kernels running
under an older U-Boot simply see no node and nothing changes. With the
node present, panic and console logs survive warm resets and are
harvested by the already-enabled systemd-pstore.service.

Part of the Wiren Board panic-log persistence (pstore shuttle) work:
U-Boot saves/restores this region across full power cycles via the swap
partition.

Targets feature/v6.18 (most recent kernel line) per project branch
policy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT
…sent

A panicked Wiren Board 7 or 8 otherwise parks until the EC watchdog
acts: on pre-2.4.0 EC firmware that is a full power cycle, destroying
DRAM and the just-written ramoops records; 2.4.0+ warm-resets first.
This panic notifier arms the idle SoC watchdog (~6 s, a few MMIO
writes, safe in panic context) so the board takes a DRAM-preserving
warm reset immediately and firmware-independently; U-Boot then parks
the records on eMMC and performs the full recovery power cycle (pstore
shuttle).

The watchdog is resolved from the device tree by compatible and armed
by mirroring the sunxi_wdt.c restart handler for both register layouts
(sun6i-a31-wdt on H616/T507, sun4i-a10-wdt on R40/A40i), but with a
6 s interval (exact hardware encoding) instead of 0.5 s so kmsg_dump
and the serial console can finish before the reset; a configured
panic_timeout below 6 s intentionally wins.

Gated at runtime on an actual ramoops node (injected by the Wiren
Board U-Boot DT fixup): with an older U-Boot there is no node, the
notifier never registers, and panic behaviour is unchanged - a
kernel-only update is a strict no-op.

Targets feature/v6.18 (most recent kernel line) per project branch
policy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT
EC firmware 2.4.x reports two reasons the driver's string table
predates: 8 = watchdog warm reset (warm-watchdog-reset EC series) and
9 = full power cycle request (used by the U-Boot pstore shuttle after
parking panic logs). Add them, plus the firmware's own placeholder 7
(Unknown) so the indices line up, and convert the table to designated
initializers with a source-of-truth note (mirrors LINUX_POWERON_REASON
in wb-embedded-controller src/wbec.c; append-only ABI). Out-of-range
values still print Unknown, so older kernels against even newer
firmware stay harmless.

Targets feature/v6.18 (most recent kernel line) per project branch
policy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT
Add a reserved-memory ramoops node at the fixed address 0x46000000
(1 MiB, record 128K, console 256K, ecc 32, no-map) to the Wiren Board 7
(sun8i-r40) and Wiren Board 8 (sun50i-h616) family device trees. The
address matches U-Boot WB_RAMOOPS_BASE exactly.

Carrying the node in the kernel DTB (rather than relying only on the
U-Boot DT fixup) is what makes a kernel-only update useful: under an old
U-Boot with no shuttle, the node is still present, the region survives a
warm reset, and the same kernel harvests the panic log on the next boot.
A new U-Boot's fixup yields to this node (it already takes precedence),
and its shuttle adds full-power-cycle survival on top.

Placed in the WB family base dtsis (84x/85x for WB8, 72x/74x for WB7;
73x inherits via 72x) so every shipped board gets it exactly once. The
fixed low address is bench-verified to retain across a warm reset on all
DRAM densities.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT
…stamp

A kernel that panics every boot would warm-reset forever. There is no
hardware reset-cause latch on these SoCs (the watchdog status register
is interrupt-mode only and clears on reset), so use the RTC general-
purpose register documented "for storing power-off information": it
survives a warm reset and is wiped by the EC's hard 5V cycle - exactly
the reset-reason signal needed.

The panic handler stamps RTC GP reg 3 (0x0700010c) before arming the
watchdog. A boot that comes up still stamped knows it just warm-reset
from a panic and declines to re-arm, letting the EC watchdog escalate
to a hard power cycle (which wipes the stamp and breaks the loop). A
60 s healthy-uptime delayed work clears the stamp so ordinary panics
always warm-reset. H616 only for now (the R40 RTC's wipe-on-hard-cycle
behaviour is not yet verified; R40 keeps arming unconditionally).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT
@evgeny-boger

Copy link
Copy Markdown
Member Author

Added FIXED-address ramoops DT node + panic-loop-breaker

Two changes on top of the earlier commits:

  1. Fixed reserved-memory ramoops node at 0x46000000 in the WB7 (sun8i-r40) and WB8 (sun50i-h616) family DTs (84x/85x, 72x/74x; 73x inherits via 72x). Matches U-Boot WB_RAMOOPS_BASE. This is what gives a kernel-only update value: the node is present regardless of U-Boot, so panic logs survive a warm reset and the kernel harvests them with no shuttle. Hardware-proven: on a 4 GiB WB8 with stock no-fixup U-Boot, ramoops probes at 0x46000000 from the DTB alone, and a panic → warm reset → kernel self-harvest captured 40 KB dmesg + console.

  2. Panic-loop-breaker in the notifier. There's no hardware reset-cause latch on these SoCs (watchdog status is interrupt-mode, clears on reset), so I use the RTC general-purpose register documented "for storing power-off information" (GP reg 3, 0x0700010c): it survives a warm reset and is wiped by the EC's hard 5V cycle — exactly the reset-reason signal needed. The panic handler stamps it before arming; a boot that comes up still stamped declines to re-arm (letting the EC watchdog hard-cycle to break the loop); a 60 s healthy-uptime delayed-work clears it so ordinary panics always warm-reset. Bench-confirmed: came back from a panic warm reset; holding off re-armhealthy, panic warm-reset re-armed at +60 s. H616 only for now (R40 RTC wipe-on-hard-cycle unverified; R40 keeps arming unconditionally).

Also validated: the full 6.18 kernel boots on a real WB8.4.3 (6.8→6.18 jump), ramoops at 0x46000000, and the full shuttle chain (with new U-Boot) reaches poweron_reason=9 + harvest.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT

evgeny-boger and others added 7 commits July 6, 2026 02:20
…ld off

Review follow-up: the module header still said the ramoops region is at
the top of DRAM and omitted the loop brake; update it for the fixed
0x46000000 placement and mention the decline-to-re-arm behaviour. Also
gate the 'healthy, panic warm-reset re-armed' message on wb_reset_from_panic
so it only prints when a hold-off was actually lifted, not on every boot.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT
…n buffer)

Match the U-Boot change: move the reserved-memory ramoops node from
0x46000000 to 0x43800000 across the WB7/WB8 family DTs, and update the
address in the panic-reset module header. 0x46000000 sat only 32 MiB
above the arm64 kernel decompression buffer at 0x44000000; a kernel
larger than 32 MiB uncompressed would clobber it. 0x43800000 sits in the
gap below the decompression buffer, immune to kernel growth, and is
bench-verified to retain across a warm reset on all densities. Must stay
in sync with U-Boot WB_RAMOOPS_BASE.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT
…7 untested)

The fixed ramoops address was chosen for the arm64/H616 boot map and
reused on 32-bit R40. Document in the R40 dtsi that it clears the used
loads and sits ~4 MiB above the stock (unused) RAMDISK_ADDR_R, that there
is no build-time guard on the R40 side, and that retention on real WB7
hardware is still pending validation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EVPkc2g2yir6SGyrED6uNT
The v2 pad-word SPI protocol is a newer-EC feature; boards with older
EC firmware (e.g. WB74) only ever speak v1, so probe always tries v2
first and falls back. That fallback logged at info level as "WBEC not
found with v2 protocol, trying v1", which reads like a fault and has
repeatedly triggered false bug reports on perfectly healthy v1 boards.

Demote the fallback notice to dev_dbg (still reachable via dynamic
debug) and reword it as an ordinary negotiation step. The eventual
success line ("WBEC found with v1/v2 protocol") already records which
protocol won, so nothing is lost from the default log.

Targets feature/v6.18 (most recent kernel line) per project branch
policy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The RTC reset-reason breadcrumb that brakes panic warm-reset loops was
H616-only: it needs an RTC general-purpose register that survives the SoC
warm reset but is wiped by the EC's hard 5 V cycle, and that behaviour
had not been verified on R40, so R40 armed unconditionally.

Make the breadcrumb register per-SoC by moving it into wb_wdt_variant
(rtc_breadcrumb_phys; 0 = no breadcrumb, arm unconditionally). H616 keeps
its RTC GP data reg 3 (0x0700010c). R40 gets GP data reg 0 (0x01c20500,
RTC base 0x01c20400 + the sun6i-rtc GP_DATA block at +0x100), which
WB7.4.2 bench testing confirmed survives a sun4i-WDT warm reset
(0xCAFE55AA persisted through a panic reset) and is cleared by the EC
full power cycle (reads 0, poweron_reason 9) - the same semantics as the
H616 breadcrumb. GP reg 0 is free: no nvmem-cell in the R40 DT consumes
the RTC nvmem provider.

This drops the of_machine_is_compatible("...h616") special-case; the
per-SoC field now drives it, so both WB7 and WB8 get the crash-loop brake.

Targets feature/v6.18 (most recent kernel line) per project branch policy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ipes RTC GP bank)

The R40 sun6i-rtc GP data registers survive a *bare* sun4i-WDT warm
reset, but E2E bench testing on WB7.4.2 showed the *panic-path* warm
reset wipes the entire GP data bank - all 8 registers, including ones
this driver never writes. A post-panic boot therefore never sees the
"PAN1" stamp, so the panic-loop brake could never engage on R40.

Set rtc_breadcrumb_phys = 0 (arm unconditionally) instead of pointing at
a register that is proven to be cleared, and make the comments honest
about the consequence: a persistent panic on R40 warm-reset loops rather
than escalating to an EC hard cycle. H616/WB8 is unaffected - its GP reg
3 breadcrumb is E2E-confirmed to survive the warm reset and be wiped by
the EC hard 5 V cycle.

Also warn (not silently skip) if the breadcrumb ioremap fails.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uard exists)

The ramoops reserved-memory comment in the R40 board dtsi still said
retention was "pending validation on real WB7 hardware" and that there
was "no build-time guard on the R40 side". Both are now stale: retention
is validated on real WB7.4.2 hardware (panic -> sun4i WDT warm reset ->
kernel harvests dmesg + console from 0x43800000), and the WB7 U-Boot
shuttle carries R40 static_asserts bounding this address against
RAMDISK_ADDR_R, so an env change that shrank the gap trips a build error
there. Comment-only; no functional change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
evgeny-boger and others added 4 commits July 7, 2026 23:43
The preceding commit documented that R40 has no usable loop-brake
breadcrumb: its sun6i-rtc GP data bank is wiped by the panic-path warm
reset (E2E-confirmed on WB7.4.2 - a bare sun4i-WDT reset preserves the
bank, but the panic path wipes all 8 regs), so no GP register can carry
the "PAN1" reset-reason stamp across a panic.

Give R40 a breadcrumb that does survive its panic path: a 4-byte word in
a dedicated no-map DRAM reservation at 0x43900000, immediately above the
ramoops region. Low DRAM survives the panic-path warm reset (the whole
ramoops/pstore premise) and is lost on a genuine 5 V cycle - exactly the
survives-warm-reset / wiped-by-power-cycle semantics the brake needs, the
same as the H616 RTC register. Being no-map, ioremap() maps it uncached,
so the panic-time stamp reaches DRAM without a cache flush (safe in panic
context) and the address does not trip the ARM ioremap-on-RAM check.

Rename the per-SoC field rtc_breadcrumb_phys -> breadcrumb_phys (it is no
longer RTC-only) and log the raw breadcrumb value read at boot. H616 is
unchanged (still its RTC GP data reg 3, 0x0700010c).

Validated on WB7.4.2 (A40i, ALEPBSP3):
- SURVIVE: a sentinel word is overwritten by "PAN1" on panic and read
  back intact after the sun4i-WDT warm reset - the exact reset that wipes
  the RTC bank.
- BRAKE: a second panic while the stamp is held prints "panic again after
  a warm reset; not re-arming" and does not arm the SoC watchdog; a
  healthy 60 s uptime clears the stamp and re-arms.
- WIPE: a genuine EC 5 V cycle (RTC-alarm power-off, poweron_reason "RTC
  alarm") clears the word, so a real power cycle resets the brake.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dcode

The previous commit baked R40's breadcrumb DRAM address (0x43900000)
into the driver. A kernel Image is then only safe on a DTB that reserves
exactly that address: on a DTB without the reservation the driver would
ioremap and stamp 0x43900000 as if it were the breadcrumb, but there it
is live System RAM.

Resolve the address from the DT instead. The panic-breadcrumb reserved-
memory node now carries compatible = "wirenboard,panic-breadcrumb"; the
driver looks it up with of_find_compatible_node() + of_address_to_resource().
If the node is absent - e.g. a kernel Image booted on an older DTB that
predates it - the driver leaves the breadcrumb address 0, skips the
ioremap entirely, and arms unconditionally (no loop brake, no
ioremap-on-RAM). H616 keeps its fixed RTC breadcrumb_phys; only R40 uses
the DT lookup (breadcrumb_compatible).

The reserved-memory node is renamed wb-panic-breadcrumb@43900000 ->
panic-breadcrumb@43900000 (generic node name; the driver matches on the
compatible now, so the name is cosmetic) and keeps its label, reg and
no-map. A compatible with no matching reserved-mem driver is a harmless
static no-map reservation (of_reserved_mem returns -ENOENT, the region
stays reserved).

Re-validated on WB7.4.2 (ALEPBSP3):
- node present: driver resolves "breadcrumb @0x43900000" from DT; SURVIVE
  (PAN1 read back after the sun4i-WDT warm reset), BRAKE ("panic again
  after a warm reset; not re-arming"), and WIPE (a genuine RTC-alarm 5 V
  cycle clears the word) all pass.
- node absent (booted on a DTB without the node): "no
  wirenboard,panic-breadcrumb node in DT; panic-loop brake disabled
  (arms every boot)", armed, and no ioremap-on-RAM warning.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…erfect

Document, at the magic compare, why a plain equality check is used: the
R40 breadcrumb is a DRAM word whose contents survive a warm reset but can
occasionally bit-flip across it (observed on the panic_timeout/declined-
panic reset path in bench testing on WB7.4.2). A non-magic read is the
fail-safe direction - the boot is treated as normal and the notifier
arms, costing at most one extra warm reset before the brake re-engages on
a clean read. A bit-tolerant compare is intentionally avoided: it would
risk a false positive (a random DRAM word within a few bits of MAGIC
wrongly read as post-panic, declining to warm-reset a legitimate panic).

Comment-only; no functional change.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…DRAM detour)

A controlled re-investigation on WB7.4.2 refuted the premise behind the
R40 DRAM-breadcrumb detour: the R40 sun6i-rtc GP data bank DOES survive a
pure panic-path sun4i-WDT warm reset (GP sentinels intact through a
genuine sysrq-c panic; early-boot read, single warm reset, no EC cycle) -
exactly like H616's RTC. The earlier "the R40 panic path wipes the RTC"
finding was an EC-power-cut confound: on the escalation the EC does a
"reset power" - a *brief* 5 V cut that zeroes the RTC but leaves DRAM
intact by remanence, masquerading as a warm reset.

That confound also makes a DRAM word the *wrong* store: it survives the
brief EC power cut byte-perfect (bench-observed), so the "PAN1" stamp never
clears and the loop brake would get stuck holding off. The RTC register
clears on that same cut - the wipe the brake needs.

So revert the whole DRAM detour and give R40 the same simple RTC breadcrumb
as H616:
- R40 breadcrumb_phys = 0x01c20500 (RTC GP data reg 0), a fixed RTC MMIO
  address ioremap()d like H616's 0x0700010c. Drop the per-variant
  breadcrumb_compatible / DT-node lookup (of_find_compatible_node +
  of_address_to_resource), the phys_addr_t local and the extra include.
- Remove the panic-breadcrumb@43900000 no-map reserved-memory node from
  both R40 board dtsi.
- Correct the comments to the re-investigated understanding, and fix the
  loop-brake escalation description: a declined panic reboots via
  panic_timeout -> emergency_restart -> wbec_restart (sys-off priority 192
  > sunxi-wdt 128) -> EC "reset power" -> RTC cleared (or, with
  panic_timeout = 0, the EC watchdog does the cut) - not the previous
  inaccurate "EC watchdog ~60 s" wording.

The Phase-1 logging (warn on ioremap failure, log the raw breadcrumb value)
is kept. H616 is unchanged; ramoops/pstore is untouched.

Re-validated on WB7.4.2 (ALEPBSP3), production panic_timeout=10:
- SURVIVE: panic -> 6 s sun4i-WDT warm reset -> boot reads RTC 0x01c20500
  == 0x50414e31 ("PAN1"), "came back from a panic warm reset".
- BRAKE: 2nd panic while held -> "not re-arming, escalating to an EC power
  cut", no SoC watchdog armed.
- WIPE: the declined panic escalates via wbec_restart; the EC power cut
  clears the RTC (next boot reads 0x00000000) -> arms normally, loop broken
  (the exact case the DRAM word survived byte-perfect).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@evgeny-boger evgeny-boger changed the title wb7, wb8: pstore/ramoops panic-log persistence (shuttle kernel side) wb7, wb8: pstore/ramoops panic-log persistence (kernel side) Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant