diff --git a/CHANGELOG.md b/CHANGELOG.md index 87f798e8..46a6a015 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,31 @@ All notable changes to PyBNF are documented below. This project adheres to ## [Unreleased] +### Fixed +- **The CMA-ES stagnation restart now fires, because its tolerance is calibrated from the + objective rather than inherited from a step length (#653, ADR-0128).** ADR-0106 gave + `cmaes_tolfun` its own key precisely because it is a range in objective units while + `cmaes_stop_tol` is a step length in sampling space, and then had an unset `cmaes_tolfun` + fall back to it anyway. `cmaes_stop_tol` defaults to 1e-11, so the stagnation range was + 1e-11 in objective units, which on an objective of any ordinary magnitude never fires. + This is #648 in the mirror. There a ratio read as a range was far too loose and stopped + fits early at a wrong answer. Here a step length read as a range is far too strict, and + what it costs is the restart trigger the battery exists for: without it a run polishes a + local basin and never yields to a restart, which is the failure the battery was built to + prevent. The documentation had already told readers the default was "rarely what you + want", which described the defect rather than fixing it. + An unset `cmaes_tolfun` is now 1e-11 times the objective spread measured across the first + generation's population. That spread is a real measure of how much the objective varies + over the search box, it is in the units the tolerance needs, and it is taken before + anything has converged, so it does not drift with the objective the way a fraction of the + current value would. It is calibrated once and every restart reuses it, so a late restart + is not held to a stricter bar than an early one. The fraction is chosen so a problem whose + initial population spans one objective unit gets exactly the 1e-11 this key always + defaulted to, leaving a reference-scaled problem unchanged. The run logs the value it + picked. A population that cannot supply a spread keeps the old fallback rather than + inventing a number, and an explicit `cmaes_tolfun` is never touched. + Only affects `cmaes_restarts > 0`, which is not the default. + ## [v1.8.0] - 2026-08-23 ### Added diff --git a/docs/adr/0128-an-unset-cmaes-tolfun-is-calibrated-from-the-objective-spread-the-first-generation-measures-because-a-sampling-space-step-length-is-not-an-objective-range-and-the-window-under-test-cannot-calibrate-itself.md b/docs/adr/0128-an-unset-cmaes-tolfun-is-calibrated-from-the-objective-spread-the-first-generation-measures-because-a-sampling-space-step-length-is-not-an-objective-range-and-the-window-under-test-cannot-calibrate-itself.md new file mode 100644 index 00000000..b514ac38 --- /dev/null +++ b/docs/adr/0128-an-unset-cmaes-tolfun-is-calibrated-from-the-objective-spread-the-first-generation-measures-because-a-sampling-space-step-length-is-not-an-objective-range-and-the-window-under-test-cannot-calibrate-itself.md @@ -0,0 +1,102 @@ +# An unset `cmaes_tolfun` is calibrated from the objective spread the first generation measures, because a sampling-space step length is not an objective range and the window under test cannot calibrate itself (issue #653) + +## Status + +Accepted. Completes ADR-0106, and is the CMA-ES half of what ADR-0127 did for the +Differential Evolution family. + +## The defect + +ADR-0106 separated `cmaes_tolfun` from `cmaes_stop_tol` and stated the reason plainly: + +> it is a range in objective units, so it gets its own knob ... and falls back to +> `cmaes_stop_tol` when unset + +Those two clauses contradict each other. The code says so even more directly, in the +comment sitting on the line above the fallback: the two "have no common scale and cannot +share one well-set value". `cmaes_stop_tol` defaults to 1e-11, so an unset `cmaes_tolfun` +became a stagnation range of 1e-11 in objective units. + +This is issue #648 in the mirror. There a dimensionless ratio read as an objective range +was far too **loose**, and fits stopped early reporting a wrong answer. Here a +sampling-space step length read as an objective range is far too **strict**, so on an +objective of ordinary magnitude TolFun never fires. + +What that costs is the trigger's whole purpose. Its docstring calls it "the trigger the +reproduction problems need", and the battery exists because otherwise a run "polishes a +local basin forever and never yields to a restart (the IPOP/BIPOP machinery silently +degenerates to one trapped run)". With the threshold at 1e-11 the battery degenerates to +exactly that for the TolFun component. TolX and ConditionCov are unaffected. + +The documentation had already noticed. `docs/config_keys.rst` told the reader the default +"is rarely what you want if you rely on stagnation restarts". The defect was described to +users rather than fixed. + +## Why ADR-0127's remedy does not transfer + +#648 was repaired by restoring a legacy meaning. `stop_tolerance` had always been a ratio, +so reading it as one again returned to known-correct behaviour. There is no such history +here: `cmaes_stop_tol` was never an objective quantity at all, so a fix has to invent a +default rather than restore one. + +## Two candidates rejected + +* **A fraction of the current objective.** This is what ADR-0106 removed, and correctly. + On a likelihood `|f|` grows as the fit improves, so such a threshold rises fastest + exactly where firing it costs the most. +* **A fraction of the window being tested.** Circular, and silently fatal: + `frange <= fraction * frange` is never true for a small fraction, so the trigger is + disabled rather than corrected. This was tried first and caught by ADR-0106's own + regression test for a genuinely flat history, which is the value of keeping that test. + +## The decision + +**An unset `cmaes_tolfun` is `1e-11` times the objective spread across the first scored +generation's population.** + +The anchor is a real measurement of how much this objective varies over the search box. +Three properties make it the right one: + +* It is **in the units TolFun needs**, taken from the objective itself rather than borrowed + across a unit boundary. +* It **does not scale with `|f|`**, because it is fixed at the first generation, before + anything has converged. ADR-0106's objection does not apply to it. +* It is **not the window under test**, so the calibration is not circular. + +It is calibrated **once, on the first run**, and every IPOP/BIPOP restart reuses it. A later +restart starts nearer the optimum and would measure a smaller spread, so recalibrating per +restart would hold the late, large-population restarts to the strictest bar, which is the +shape of failure ADR-0106 fixed. + +The fraction `1e-11` is chosen so a problem whose initial population spans one objective +unit gets precisely the threshold this key has always defaulted to, which is the scale the +reference CMA-ES assumes. The default is therefore unchanged on a reference-scaled problem +and moves in proportion for any other. + +A generation that cannot supply a spread, fewer than two finite scores or every score +identical, leaves `cmaes_stop_tol` in place rather than inventing a number or setting a +threshold of zero. An explicit `cmaes_tolfun` is never touched. + +## Consequences + +* `cmaes_restarts == 0`, the default, is unaffected: the battery is not consulted at all. +* An explicit `cmaes_tolfun` is unaffected. +* A fit on a reference-scaled objective is unaffected, by construction of the fraction. +* A fit whose objective is far from unit scale gets a stagnation threshold in proportion + to it, and the chosen value is written to the log. +* All three of ADR-0106's regression tests pass unchanged. They construct synthetic + distribution state without scoring a generation, so they exercise the fallback, which + still behaves exactly as it did. + +## The wider point + +This defect and #648 are one pattern: an optional key with correct units, defaulted from a +neighbouring key with different units, under a comment explaining that the two units are +incompatible. A sweep of the codebase finds exactly two such fallbacks, `de_tolfun` and +`cmaes_tolfun`, and both are now resolved. `cmaes_run_maxgen` also defaults from unset, but +to `np.inf` rather than to another key, so no unit boundary is crossed. + +#648 was found and fixed without this one being looked for, even though ADR-0115's own +commit message names CMA-ES as its sibling and `de_tolfun` as mirroring `cmaes_tolfun`. The +lesson is to treat a defect that an ADR describes as having a sibling as a defect in a +class, and to check the sibling in the same pass. diff --git a/docs/algorithms.rst b/docs/algorithms.rst index 65272cca..411f5024 100644 --- a/docs/algorithms.rst +++ b/docs/algorithms.rst @@ -792,7 +792,9 @@ tolerance, the smallest range of the best objective across the last counts as progress. Because it measures the objective and ``cmaes_stop_tol`` measures a step in parameter space, the two have no common scale — set ``cmaes_tolfun`` to the objective improvement per window you consider alive, and leave ``cmaes_stop_tol`` at the -convergence step you actually mean. Unset, ``cmaes_tolfun`` follows ``cmaes_stop_tol``. +convergence step you actually mean. Unset, ``cmaes_tolfun`` is derived from the objective spread PyBNF measures across the +first generation's population, so one default means the same thing on problems whose +objectives differ by many decades, and the run logs the value it chose (#653). .. _alg-gradient: diff --git a/docs/config_keys.rst b/docs/config_keys.rst index db76dda7..098fe3cf 100644 --- a/docs/config_keys.rst +++ b/docs/config_keys.rst @@ -1856,7 +1856,7 @@ These settings for the :ref:`CMA-ES ` optimizer apply both to ``job_t * ``cmaes_sigma0 = 0.5`` **cmaes_stop_tol** - Stop when the largest principal standard deviation of the search distribution falls below this value. This is a step length in the parameter sampling space, and in restart mode (``cmaes_restarts > 0``) it is also the threshold below which every individual coordinate step counts as collapsed. The stagnation threshold on the *objective* is ``cmaes_tolfun``. + Stop when the largest principal standard deviation of the search distribution falls below this value. This is a step length in the parameter sampling space, and in restart mode (``cmaes_restarts > 0``) it is also the threshold below which every individual coordinate step counts as collapsed. The stagnation threshold on the *objective* is ``cmaes_tolfun``, which is a different quantity in different units and is no longer defaulted from this key (#653). Default: 1e-11 @@ -1865,7 +1865,9 @@ These settings for the :ref:`CMA-ES ` optimizer apply both to ``job_t * ``cmaes_stop_tol = 1e-8`` **cmaes_tolfun** - Stagnation tolerance on the objective, used only in restart mode (``cmaes_restarts > 0``): a run is declared finished — and yields to the next restart — when the range of its best objective over the last ``10 + ceil(30 × (number of parameters) / population_size)`` generations falls to this value or below. It is an absolute range in the units of your objective function, unlike ``cmaes_stop_tol``, which is a step length in the parameter sampling space; set it to the smallest objective improvement per window that you still consider progress. Unset, it follows ``cmaes_stop_tol``, which is rarely what you want if you rely on stagnation restarts: a value loose enough to detect a stalled run is far looser than a converged search distribution. + Stagnation tolerance on the objective, used only in restart mode (``cmaes_restarts > 0``): a run is declared finished — and yields to the next restart — when the range of its best objective over the last ``10 + ceil(30 × (number of parameters) / population_size)`` generations falls to this value or below. It is an absolute range in the units of your objective function, unlike ``cmaes_stop_tol``, which is a step length in the parameter sampling space. Set it to the smallest objective improvement per window that you still consider progress. + + Unset, it is derived from your problem's own objective scale: PyBNF measures the spread of the objective across the first generation's population and takes 1e-11 of it. That fraction is chosen so a problem whose initial population spans one objective unit gets exactly 1e-11, the value this key defaulted to historically, and a problem on any other scale gets a threshold in proportion. The run logs the number it chose. Previously an unset value followed ``cmaes_stop_tol``, which is a step length rather than an objective range, and at its 1e-11 default was far too strict for an objective of ordinary magnitude, so the stagnation restart never fired (#653). Default: unset (follows ``cmaes_stop_tol``) diff --git a/pybnf/algorithms/optimizers/cmaes.py b/pybnf/algorithms/optimizers/cmaes.py index e855472d..19797237 100644 --- a/pybnf/algorithms/optimizers/cmaes.py +++ b/pybnf/algorithms/optimizers/cmaes.py @@ -165,8 +165,15 @@ def __init__(self, config, refine=False): # back to stop_tol -- the single knob TolFun used before -- so an existing # config keeps the threshold magnitude it had, without the |f| scaling. configured_tolfun = config.config['cmaes_tolfun'] + self.tolfun_is_explicit = configured_tolfun is not None self.tolfun = (self.stop_tol if configured_tolfun is None else float(configured_tolfun)) + # ...but falling back to a step length was still borrowing a number across a unit + # boundary the line above says cannot be crossed (#653). An UNSET cmaes_tolfun is + # now calibrated from the objective scale the run measures for itself, in + # _calibrate_tolfun, and stop_tol is only the last resort when that measurement is + # unavailable. None until the first generation has been scored. + self._objective_scale = None self.max_generations = config.config['max_iterations'] # Restart schedule (#498, ADR-0070). cmaes_restarts == 0 (the default) is a @@ -333,6 +340,7 @@ def got_result(self, res): def _update_distribution(self): order = sorted(range(self.lam), key=lambda i: self.gen_score[i]) self._run_best_history.append(float(self.gen_score[order[0]])) # TolFun window (#506) + self._calibrate_tolfun() # first scored generation only (#653) x_sorted = np.array([self.gen_x[i] for i in order[:self.mu]]) # (mu, n) m_old = self.mean @@ -470,6 +478,65 @@ def _battery_stop_reason(self): % (cond, self._COND_COV_MAX)) return None + _TOLFUN_SCALE_FRACTION = 1e-11 + + def _calibrate_tolfun(self): + """Set the TolFun threshold from the objective scale this problem actually has + (#653), once, from the first scored generation of the fit. + + TolFun is a range in objective units. ``cmaes_stop_tol`` is a step length in + sampling space. ADR-0106 says in as many words that the two "have no common scale + and cannot share one well-set value", and then had an unset ``cmaes_tolfun`` fall + back to it anyway, for want of anything better to default to. The consequence is + the mirror image of #648: there a ratio's magnitude read as a range was far too + LOOSE and stopped fits early, here a sampling-space step of 1e-11 read as an + objective range is far too STRICT, so on an objective of any ordinary magnitude + TolFun never fires and the restart battery loses the trigger its own docstring + calls "the trigger the reproduction problems need". + + The anchor is the **objective spread across the first generation's population**. + That is a real measurement of how much this objective varies over the search box, + it is in the units TolFun needs, and it is taken before the run has converged + anything, so it is fixed for the fit. Three properties matter: + + * It does **not** scale with ``|f|``, which is what ADR-0106 forbade and why the + obvious "fraction of the current objective" is wrong: on a likelihood ``|f|`` + grows as the fit improves, so such a threshold rises fastest where firing it + costs most. + * It is **not** the window TolFun tests. Calibrating from the window under test is + circular -- ``frange <= fraction * frange`` is never true for a small fraction -- + and would silently disable the trigger. + * It is calibrated **once, on the first run**, and every IPOP/BIPOP restart reuses + it. A later restart starts nearer the optimum and would measure a smaller spread, + so recalibrating per restart would hold exactly the late, large-population + restarts to the strictest bar, which is the shape of failure ADR-0106 fixed. + + The fraction is ``1e-11``, chosen so that a problem whose initial population spans + one objective unit gets precisely the threshold this key has always defaulted to. + The default is therefore unchanged on a problem scaled the way the reference CMA-ES + assumes, and adapts away from it in proportion to how far the real objective sits + from that assumption. + + A population that cannot supply a spread -- fewer than two finite scores, or every + score identical -- leaves ``stop_tol`` in place rather than inventing a number, and + an explicit ``cmaes_tolfun`` is never touched. + """ + if self.tolfun_is_explicit or self._objective_scale is not None: + return + finite = [float(s) for s in self.gen_score + if s is not None and np.isfinite(s)] + if len(finite) < 2: + return + spread = max(finite) - min(finite) + if not np.isfinite(spread) or spread <= 0.0: + return + self._objective_scale = spread + self.tolfun = self._TOLFUN_SCALE_FRACTION * spread + logger.info( + 'CMA-ES TolFun tolerance set to %g, from an initial objective spread of %g ' + '(cmaes_tolfun was not set; set it to state the stagnation range directly).' + % (self.tolfun, spread)) + def _tolfun_window(self): """The TolFun stagnation window in generations: Hansen's ``10 + ceil(30 N / lambda)``. It scales with the population, so a larger restart diff --git a/tests/test_optimizer_integration.py b/tests/test_optimizer_integration.py index 89abb6b1..9e45c2b9 100644 --- a/tests/test_optimizer_integration.py +++ b/tests/test_optimizer_integration.py @@ -582,6 +582,92 @@ def test_cmaes_tolfun_still_fires_on_a_genuinely_flat_history(tmp_path): assert reason is not None and 'stagnated' in reason and 'tolerance' in reason +def _score_first_generation(alg, scores): + """Hand ``alg`` a first scored generation, then run the calibration hook that + ``_update_distribution`` calls, without simulating anything.""" + alg.gen_score = list(scores) + alg._calibrate_tolfun() + + +def test_an_unset_tolfun_is_calibrated_from_the_problems_own_objective_spread(tmp_path): + """The #653 defect at its decision point. + + An unset ``cmaes_tolfun`` fell back to ``cmaes_stop_tol``, a step length in sampling + space, whose 1e-11 default read as an objective range is far too strict for any + objective of ordinary magnitude. TolFun then never fires and the restart battery + loses the trigger it exists for. It is now calibrated from the objective spread the + first generation actually measures. + """ + alg = _battery_alg(tmp_path, cmaes_restarts=3, cmaes_stop_tol=1e-11) + assert alg.tolfun == alg.stop_tol # before any generation is scored + _score_first_generation(alg, [1.0e6, 1.4e6, 2.0e6, 1.1e6, 1.9e6, 1.2e6, 1.5e6, 1.3e6]) + # Spread 1e6, so the stagnation range is 1e-11 * 1e6 = 1e-5 rather than 1e-11. + assert alg._objective_scale == pytest.approx(1.0e6) + assert alg.tolfun == pytest.approx(1.0e-5) + # And that is the difference between firing and not. A run flat to within 1e-6 on a + # problem of this scale HAS stagnated; the old threshold could not say so. + _tolfun_state(alg, np.linspace(1.0e6, 1.0e6 + 1.0e-6, alg._tolfun_window())) + reason = alg._battery_stop_reason() + assert reason is not None and 'stagnated' in reason + alg.tolfun = alg.stop_tol # what the fallback would have given + assert alg._battery_stop_reason() is None # ...and it stays silent on the same run + + +def test_a_unit_spread_problem_keeps_the_threshold_this_key_always_had(tmp_path): + """The calibration is anchored, not invented. The fraction is chosen so that a + problem whose initial population spans one objective unit gets exactly the 1e-11 this + key has always defaulted to, which is the scale the reference CMA-ES assumes. The + default only moves for a problem that is not on that scale.""" + alg = _battery_alg(tmp_path, cmaes_restarts=3, cmaes_stop_tol=1e-11) + _score_first_generation(alg, [0.0, 0.25, 0.5, 0.75, 1.0, 0.4, 0.6, 0.8]) + assert alg._objective_scale == pytest.approx(1.0) + assert alg.tolfun == pytest.approx(1e-11) + + +def test_the_calibrated_threshold_tracks_the_objective_scale(tmp_path): + """Two problems six decades apart get thresholds six decades apart, which is the + whole point: one default that means the same thing on both.""" + got = {} + for scale in (1.0e-3, 1.0e3): + alg = _battery_alg(tmp_path, cmaes_restarts=3, cmaes_stop_tol=1e-11) + _score_first_generation(alg, [0.0, scale * 0.5, scale, scale * 0.25]) + got[scale] = alg.tolfun + assert got[1.0e3] / got[1.0e-3] == pytest.approx(1.0e6) + + +def test_an_explicit_tolfun_is_never_calibrated(tmp_path): + """An explicit ``cmaes_tolfun`` is a range its author chose in their own objective's + units. Scoring a generation must not move it.""" + alg = _battery_alg(tmp_path, cmaes_restarts=3, cmaes_stop_tol=1e-11, cmaes_tolfun=1e-3) + _score_first_generation(alg, [1.0e6, 2.0e6, 1.5e6, 1.2e6]) + assert alg.tolfun == 1e-3 and alg._objective_scale is None + + +def test_a_population_with_no_spread_keeps_the_old_fallback(tmp_path): + """The calibration measures or it declines. A generation that cannot supply a spread + -- every score identical, or fewer than two finite ones -- leaves ``stop_tol`` in + place rather than inventing a number or setting a threshold of zero.""" + for scores in ([5.0, 5.0, 5.0, 5.0], # no spread + [np.inf, np.inf, 7.0, np.inf], # one finite score + [np.inf, np.inf, np.inf, np.inf]): # none + alg = _battery_alg(tmp_path, cmaes_restarts=3, cmaes_stop_tol=1e-11) + _score_first_generation(alg, scores) + assert alg._objective_scale is None + assert alg.tolfun == alg.stop_tol + + +def test_a_restart_does_not_recalibrate_the_threshold(tmp_path): + """Calibrated once, on the first run, and reused by every restart. A later restart + starts nearer the optimum and would measure a smaller spread, so recalibrating would + hold exactly the late, large-population restarts to the strictest bar -- the shape of + failure ADR-0106 fixed.""" + alg = _battery_alg(tmp_path, cmaes_restarts=3, cmaes_stop_tol=1e-11) + _score_first_generation(alg, [0.0, 1.0e6]) + first = alg.tolfun + _score_first_generation(alg, [0.0, 1.0e-6]) # a much tighter later restart + assert alg.tolfun == first + + def test_cmaes_tolfun_is_a_knob_of_its_own(tmp_path): """cmaes_stop_tol is a step length in sampling space u; TolFun's is a range in objective units. Sharing one key forced a fit that wanted a meaningful stagnation