Matplotlib math text parsing error with Python 3.13 when using plopp.slicer

[jnianias]

When using plopp.slicer in a conda environment created from the official scipp.yml file and Python 3.13.5, Matplotlib fails to render scientific notation tick labels (e.g., 10^{0}) with a ParseException. I managed to resolve the issue by downgrading Matplotlib to version 3.9.4, originally was running 3.11.1.

Steps to Reproduce:

Using Python 3.13.5, install scipp via conda following the official instructions.

Code:

import plopp as pp
import scipp as sc

%matplotlib widget

filename = 'https://public.esss.dk/groups/scipp/scipp/1/data_dream_diamond.zip'

ds = sc.io.load_csv(
    filename,
    sep='\t',
    header_parser='bracket',
)

table = sc.DataArray(
    sc.ones(sizes=ds.sizes, unit='counts'),
    coords={name:col.data for name, col in ds.items()},
)

binned_xyz = table.bin(z_pos=31, y_pos=31, x_pos=31)  # 31**3 bins
pp.slicer(
    binned_xyz['z_pos', 20:].hist(), keep=['x_pos', 'y_pos'], norm='log', aspect='equal'
)

Full traceback:

---

ValueError Traceback (most recent call last)
Cell In[16], line 3
1 # Multi-dimensional spatial binning
2 binned_xyz = table.bin(z_pos=31, y_pos=31, x_pos=31) # 31**3 bins
----> 3 pp.slicer(
4 binned_xyz['z_pos', 20:].hist(), keep=['x_pos', 'y_pos'], norm='log', aspect='equal'
5 )

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/plopp/plotting/slicer.py:162, in slicer(obj, keep, autoscale, coords, vmin, vmax, cbar, **kwargs)
124 def slicer(
125 obj: PlottableMulti,
126 *,
(...) 133 **kwargs,
134 ) -> FigureLike:
135 """
136 Plot a multi-dimensional object by slicing one or more of the dimensions.
137 This will produce one slider per sliced dimension, below the figure.
(...) 160 See :py:func:plopp.plot for the full list of figure customization arguments.
161 """
--> 162 return Slicer(
163 obj,
164 keep=keep,
165 autoscale=autoscale,
166 vmin=vmin,
167 vmax=vmax,
168 coords=coords,
169 cbar=cbar,
170 **kwargs,
171 ).figure

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/plopp/plotting/slicer.py:119, in Slicer.init(self, obj, keep, coords, vmin, vmax, cbar, **kwargs)
113 else:
114 raise ValueError(
115 f'Slicer plot: the number of dims to be kept must be 1 or 2, '
116 f'but {ndims} were requested.'
117 )
--> 119 self.figure = make_figure(*self.slice_nodes, vmin=vmin, vmax=vmax, **kwargs)
120 require_interactive_figure(self.figure, 'slicer')
121 self.figure.bottom_bar.add(self.slider)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/plopp/graphics/figures.py:42, in imagefigure(*nodes, **kwargs)
34 def imagefigure(*nodes: Node, **kwargs) -> FigureLike:
35 view_maker = partial(
36 GraphicalView,
37 dims={'y': None, 'x': None},
(...) 40 colormapper=True,
41 )
---> 42 return backends.get(group='2d', name='figure')(view_maker, *nodes, **kwargs)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/plopp/backends/matplotlib/figure.py:195, in Figure(*args, **kwargs)
192 from .utils import is_interactive_backend
194 if is_interactive_backend():
--> 195 return InteractiveFigure(*args, **kwargs)
196 else:
197 return StaticFigure(*args, **kwargs)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/plopp/backends/matplotlib/figure.py:143, in InteractiveFigure.init(self, View, *args, **kwargs)
141 self.bottom_bar = HBar()
142 self.top_bar = HBar()
--> 143 super().init(self._make_children())

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/plopp/backends/matplotlib/figure.py:154, in InteractiveFigure._make_children(self)
149 def _make_children(self):
150 from ...widgets import HBar
152 return [
153 self.top_bar,
--> 154 HBar([self.left_bar, self.view.canvas.to_widget(), self.right_bar]),
155 self.bottom_bar,
156 ]

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/plopp/backends/matplotlib/canvas.py:164, in Canvas.to_widget(self)
162 with warnings.catch_warnings():
163 warnings.simplefilter("ignore")
--> 164 self.fig.tight_layout()
165 except RuntimeError:
166 pass

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/figure.py:3665, in Figure.tight_layout(self, pad, h_pad, w_pad, rect)
3663 previous_engine = self.get_layout_engine()
3664 self.set_layout_engine(engine)
-> 3665 engine.execute(self)
3666 if previous_engine is not None and not isinstance(
3667 previous_engine, (TightLayoutEngine, PlaceHolderLayoutEngine)
3668 ):
3669 _api.warn_external('The figure layout has changed to tight')

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/layout_engine.py:188, in TightLayoutEngine.execute(self, fig)
186 renderer = fig._get_renderer()
187 with getattr(renderer, "_draw_disabled", nullcontext)():
--> 188 kwargs = get_tight_layout_figure(
189 fig, fig.axes, get_subplotspec_list(fig.axes), renderer,
190 pad=info['pad'], h_pad=info['h_pad'], w_pad=info['w_pad'],
191 rect=info['rect'])
192 if kwargs:
193 fig.subplots_adjust(**kwargs)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/_tight_layout.py:266, in get_tight_layout_figure(fig, axes_list, subplotspec_list, renderer, pad, h_pad, w_pad, rect)
261 return {}
262 span_pairs.append((
263 slice(ss.rowspan.start * div_row, ss.rowspan.stop * div_row),
264 slice(ss.colspan.start * div_col, ss.colspan.stop * div_col)))
--> 266 kwargs = _auto_adjust_subplotpars(fig, renderer,
267 shape=(max_nrows, max_ncols),
268 span_pairs=span_pairs,
269 subplot_list=subplot_list,
270 ax_bbox_list=ax_bbox_list,
271 pad=pad, h_pad=h_pad, w_pad=w_pad)
273 # kwargs can be none if tight_layout fails...
274 if rect is not None and kwargs is not None:
275 # if rect is given, the whole subplots area (including
276 # labels) will fit into the rect instead of the
(...) 280 # auto_adjust_subplotpars twice, where the second run
281 # with adjusted rect parameters.

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/_tight_layout.py:82, in _auto_adjust_subplotpars(fig, renderer, shape, span_pairs, subplot_list, ax_bbox_list, pad, h_pad, w_pad, rect)
80 for ax in subplots:
81 if ax.get_visible():
---> 82 bb += [martist._get_tightbbox_for_layout_only(ax, renderer)]
84 tight_bbox_raw = Bbox.union(bb)
85 tight_bbox = fig.transFigure.inverted().transform_bbox(tight_bbox_raw)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/artist.py:1485, in _get_tightbbox_for_layout_only(obj, *args, **kwargs)
1479 """
1480 Matplotlib's .Axes.get_tightbbox and .Axis.get_tightbbox support a
1481 for_layout_only kwarg; this helper tries to use the kwarg but skips it
1482 when encountering third-party subclasses that do not support it.
1483 """
1484 try:
-> 1485 return obj.get_tightbbox(*args, **{**kwargs, "for_layout_only": True})
1486 except TypeError:
1487 return obj.get_tightbbox(*args, **kwargs)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/axes/_base.py:4758, in _AxesBase.get_tightbbox(self, renderer, call_axes_locator, bbox_extra_artists, for_layout_only)
4756 if self.axison and axis.get_visible():
4757 if for_layout_only:
-> 4758 ba = martist._get_tightbbox_for_layout_only(axis, renderer)
4759 else:
4760 ba = axis.get_tightbbox(renderer)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/artist.py:1485, in _get_tightbbox_for_layout_only(obj, *args, **kwargs)
1479 """
1480 Matplotlib's .Axes.get_tightbbox and .Axis.get_tightbbox support a
1481 for_layout_only kwarg; this helper tries to use the kwarg but skips it
1482 when encountering third-party subclasses that do not support it.
1483 """
1484 try:
-> 1485 return obj.get_tightbbox(*args, **{**kwargs, "for_layout_only": True})
1486 except TypeError:
1487 return obj.get_tightbbox(*args, **kwargs)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/axis.py:1430, in Axis.get_tightbbox(self, renderer, for_layout_only)
1427 self._update_label_position(renderer)
1429 # go back to just this axis's tick labels
-> 1430 tlb1, tlb2 = self._get_ticklabel_bboxes(ticks_to_draw, renderer)
1432 self._update_offset_text_position(tlb1, tlb2)
1433 self.offsetText.set_text(self.major.formatter.get_offset())

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/axis.py:1407, in Axis._get_ticklabel_bboxes(self, ticks, renderer)
1402 def _get_ticklabel_bboxes(self, ticks, renderer):
1403 """Return lists of bboxes for ticks' label1's and label2's."""
1404 return ([tick.label1.get_window_extent(renderer)
1405 for tick in ticks
1406 if tick.label1.get_visible() and tick.label1.get_in_layout()],
-> 1407 [tick.label2.get_window_extent(renderer)
1408 for tick in ticks
1409 if tick.label2.get_visible() and tick.label2.get_in_layout()])

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/text.py:1076, in Text.get_window_extent(self, renderer, dpi)
1071 raise RuntimeError(
1072 "Cannot get window extent of text w/o renderer. You likely "
1073 "want to call 'figure.draw_without_rendering()' first.")
1075 with cbook._setattr_cm(fig, dpi=dpi):
-> 1076 bbox, _, _ = self._get_layout(self._renderer)
1077 x, y = self.get_unitless_position()
1078 x, y = self.get_transform().transform((x, y))

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/text.py:479, in Text._get_layout(self, renderer)
477 clean_line, ismath = self._preprocess_math(line)
478 if clean_line:
--> 479 w, h, d = _get_text_metrics_with_cache(
480 renderer, clean_line, self._fontproperties,
481 ismath=ismath, dpi=self.get_figure(root=True).dpi)
482 else:
483 w = h = d = 0

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/text.py:59, in _get_text_metrics_with_cache(renderer, text, fontprop, ismath, dpi)
52 get_text_metrics = _get_text_metrics_function(renderer)
53 # call the function to compute the metrics and return
54 #
55 # We pass a copy of the fontprop because FontProperties is both mutable and
56 # has a __hash__ that depends on that mutable state. This is not ideal
57 # as it means the hash of an object is not stable over time which leads to
58 # very confusing behavior when used as keys in dictionaries or hashes.
---> 59 return get_text_metrics(text, fontprop.copy(), ismath, dpi)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/text.py:125, in _get_text_metrics_function.._text_metrics(text, fontprop, ismath, dpi)
120 raise RuntimeError(
121 "Trying to get text metrics for a renderer that no longer exists. "
122 "This should never happen and is evidence of a bug elsewhere."
123 )
124 # do the actual method call we need and return the result
--> 125 return local_renderer.get_text_width_height_descent(text, fontprop, ismath)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/backends/backend_agg.py:259, in RendererAgg.get_text_width_height_descent(self, s, prop, ismath)
256 return super().get_text_width_height_descent(s, prop, ismath)
258 if ismath:
--> 259 parse = self.mathtext_parser.parse(s, self.dpi, prop)
260 return parse.width, parse.height, parse.depth
262 font = self._prepare_font(prop)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/mathtext.py:86, in MathTextParser.parse(self, s, dpi, prop, antialiased)
81 from matplotlib.backends import backend_agg
82 load_glyph_flags = {
83 "vector": LoadFlags.NO_HINTING,
84 "raster": backend_agg.get_hinting_flag(),
85 }[self._output_type]
---> 86 return self._parse_cached(s, dpi, prop, antialiased, load_glyph_flags)

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/mathtext.py:100, in MathTextParser._parse_cached(self, s, dpi, prop, antialiased, load_glyph_flags)
97 if self._parser is None: # Cache the parser globally.
98 self.class._parser = _mathtext.Parser()
--> 100 box = self._parser.parse(s, fontset, fontsize, dpi)
101 output = _mathtext.ship(box)
102 if self._output_type == "vector":

File ~/miniconda3/envs/scipp/lib/python3.13/site-packages/matplotlib/_mathtext.py:2306, in Parser.parse(self, s, fonts_object, fontsize, dpi)
2304 result = self._expression.parse_string(s)
2305 except ParseBaseException as err:
-> 2306 raise ValueError("\n" + err.explain(0)) from None
2307 self._state_stack = []
2308 self._needs_space_after_subsuper = False

ValueError:
$\mathdefault{10^{1}}$
^
ParseException: Expected end of text, found '$' (at char 0), (line:1, col:1)

[SimonHeybrock]

Thanks for the detailed report -- the traceback made this quick to pin down.

This is not specific to Python 3.13, and the Matplotlib downgrade is unfortunately a red herring: the failure is a race, so version changes only shift the timing. We have seen the same error on Matplotlib 3.9.4.

Root cause: JupyterLab >= 4.4 defaults to routing widget messages over kernel subshells, which ipykernel >= 7 services on their own threads. Your figure's canvas is then drawn concurrently with cell execution, and Matplotlib is not thread-safe -- the concurrent access corrupts the global math text parser, which surfaces as the ParseException you saw. It can also show up as blank figures or kernel crashes. ipympl#610 has the full analysis, and previously #499 here.

Two workarounds, both verified:

• In the JupyterLab settings editor, set commsOverSubshells to disabled, then restart JupyterLab. The setting only applies to newly connected kernels, so a plain kernel restart is not enough.
• Or install ipykernel<7.

There is nothing plopp can do to fix this -- the race is between ipympl's message handlers and cell execution, and locking our own calls does not cover ipympl's frame production. #597 at least replaces the cryptic parse error with one that names the cause and the workarounds, and documents them in the installation instructions. That PR closes this issue, but the underlying bug remains with ipympl/ipykernel; matplotlib/ipympl#610 is the place to follow it.

[jnianias]

OK thanks for the explanation -- indeed I found that this doesn't happen every time, only when you run pp.slicer more than once with the same kernel, which makes sense given the nature of the issue. I'll implement the workaround in JupyterLab. Thanks again!