Calibrate the CMA-ES stagnation tolerance from the objective, not from a step length - #654
Merged
Merged
Conversation
…m a step length (#653) This is the sibling of #648, in the optimizer #648's own ADR says it mirrors. ADR-0106 gave cmaes_tolfun its own key because it is a range in objective units while cmaes_stop_tol is a step length in sampling space, and then had an unset cmaes_tolfun fall back to it anyway. The comment on the line above the fallback says the two have no common scale and cannot share one well-set value. The line below it makes them share one. cmaes_stop_tol defaults to 1e-11, so the stagnation range was 1e-11 in objective units. That is #648 in the mirror. There a dimensionless ratio read as an objective range was far too loose, and fits stopped early reporting a wrong answer. Here a step length read as an objective range is far too strict, so on an objective of ordinary magnitude the trigger never fires. What that costs is the trigger the restart battery exists for: without it a run polishes a local basin and never yields to a restart, which is the failure the battery was built to prevent. The documentation already told readers the default was rarely what they wanted, which described the defect rather than fixing it. The #648 remedy does not transfer. There, stop_tolerance had always been a ratio, so reading it as one again returned to known-correct behaviour. cmaes_stop_tol was never an objective quantity, so a default has to be invented rather than restored. Two candidates were rejected first. A fraction of the current objective is what ADR-0106 removed, because a likelihood's magnitude grows as the fit improves. A fraction of the window being tested is circular, and silently disables the trigger rather than fixing it; ADR-0106's own flat-history test caught that attempt, which is the value of having kept it. An unset cmaes_tolfun is now 1e-11 times the objective spread across the first scored generation's population. That spread measures how much this objective varies over the search box, it is in the units the tolerance needs, and it is taken before anything has converged so it does not drift with the objective. It is calibrated once and every restart reuses it, so a late restart is not held to a stricter bar than an early one. The fraction is chosen so a problem whose initial population spans one objective unit gets exactly the 1e-11 this key always defaulted to, leaving a reference-scaled problem unchanged. A generation that cannot supply a spread keeps the old fallback rather than inventing a number, and an explicit cmaes_tolfun is never touched. All three of ADR-0106's regression tests pass unchanged: they build synthetic distribution state without scoring a generation, so they still exercise the fallback. Six new tests cover the calibration, the anchor value, the scaling across decades, the explicit key, the degenerate populations and the no-recalibration-on-restart rule. A sweep for the same pattern finds exactly two fallbacks in the codebase that cross a unit boundary this way, de_tolfun and cmaes_tolfun. Both are now resolved. cmaes_run_maxgen also defaults from unset, but to infinity rather than to another key, so no boundary is crossed. Only affects cmaes_restarts > 0, which is not the default. Full suite including the slow and recovery tiers: 4778 passed, 13 skipped, none failed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #653. This is the sibling of #648, in the optimizer #648's own ADR says it mirrors,
and it should have been found and fixed in the same pass.
The defect
ADR-0106 gave
cmaes_tolfunits own key because it is a range in objective units whilecmaes_stop_tolis a step length in sampling space. Then it had an unsetcmaes_tolfunfall back to it anyway. The comment sitting on the line above the fallback says the two
"have no common scale and cannot share one well-set value". The line below makes them
share one.
cmaes_stop_toldefaults to 1e-11, so the stagnation range was 1e-11 inobjective units.
This is #648 in the mirror.
stop_tolerance= 0.002, a dimensionless ratiocmaes_stop_tol= 1e-11, a sampling-space stepWhat the strict direction costs is the trigger the restart battery exists for. Its own
docstring calls it "the trigger the reproduction problems need", and the battery is there
because otherwise a run "polishes a local basin forever and never yields to a restart (the
IPOP/BIPOP machinery silently degenerates to one trapped run)". TolX and ConditionCov are
unaffected.
docs/config_keys.rstalready told readers the default was "rarely what you want if yourely on stagnation restarts". The defect was documented rather than fixed.
Why #648's remedy does not transfer
#648 was repaired by restoring a legacy meaning:
stop_tolerancehad always been a ratio,so reading it as one again returned to known-correct behaviour.
cmaes_stop_tolwas neveran objective quantity, so a default has to be invented rather than restored.
Two candidates were rejected before the third, and the second is worth recording:
likelihood
|f|grows as the fit improves, so the threshold rises fastest where firingit costs most.
frange <= fraction * frangeis never true for a small fraction, so the trigger isdisabled rather than corrected. I tried this first and ADR-0106's own
test_cmaes_tolfun_still_fires_on_a_genuinely_flat_historycaught it immediately. Thatis the value of having kept that test.
The fix
An unset
cmaes_tolfunis1e-11times the objective spread across the first scoredgeneration's population.
|f|, because it is fixed at the first generation, beforeanything has converged. ADR-0106's objection does not reach it.
the optimum and would measure a smaller spread, so recalibrating would hold the late,
large-population restarts to the strictest bar, which is the shape of failure ADR-0106
fixed.
The fraction is chosen so a problem whose initial population spans one objective unit gets
exactly the 1e-11 this key always defaulted to. A reference-scaled problem is unchanged
by construction; everything else scales in proportion. The run logs the value it picked.
A generation that cannot supply a spread (fewer than two finite scores, or all identical)
keeps the old fallback rather than inventing a number or setting zero. An explicit
cmaes_tolfunis never touched.Evidence
All three of ADR-0106's regression tests pass unchanged. They construct synthetic
distribution state without scoring a generation, so they still exercise the fallback and
still assert
alg.tolfun == alg.stop_tol.Six new tests: the defect at its decision point (including that the old threshold stays
silent on a run the new one correctly stops), the anchor value, scaling across six decades,
the explicit key, three degenerate populations, and the no-recalibration-on-restart rule.
Full suite including the slow and recovery tiers CI skips: 4778 passed, 13 skipped, none
failed.
Scope
Only reachable with
cmaes_restarts > 0, which is not the default, and nothing in theshipped corpus sets it. It cannot produce a wrong answer; it weakens the search.
The class, swept
A grep for the same pattern finds exactly two fallbacks in the codebase that cross a unit
boundary this way:
de_tolfunandcmaes_tolfun. Both are now resolved.cmaes_run_maxgenalso defaults from unset, but to infinity rather than to another key, so no boundary is
crossed. ADR-0128 records the lesson: treat a defect whose ADR names a sibling as a defect
in a class, and check the sibling in the same pass.