[BUG] raise an informative error when CLA gets a singular covariance matrix#749
Closed
jaydevkalivarapu wants to merge 1 commit into
Closed
[BUG] raise an informative error when CLA gets a singular covariance matrix#749jaydevkalivarapu wants to merge 1 commit into
jaydevkalivarapu wants to merge 1 commit into
Conversation
…matrix Closes PyPortfolio#737 The critical line algorithm inverts the covariance submatrix of the free assets on each iteration. When the covariance matrix is singular — e.g. perfectly correlated or duplicated assets, or fewer observations than assets — that inversion can fail, and users saw a bare `numpy.linalg.LinAlgError: Singular matrix` raised from a private method, with no indication of what was wrong or how to fix it. Route the four inversion sites through a helper that translates the LinAlgError into a ValueError explaining the likely cause and pointing at concrete remedies. Note that validating positive-definiteness up front (as the issue suggests) would be a regression: a singular covariance matrix only breaks CLA if a *submatrix* of the free assets happens to be singular. In a randomised check over 40 rank-deficient covariance matrices, 29 solved fine and only 4 raised LinAlgError, so an up-front check would reject inputs that work today. Translating the error where it actually occurs keeps those working and only improves the failure path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #737
Summary
CLAinverts the covariance submatrix of the free assets on every iteration. When the covariance matrix is singular — perfectly correlated or duplicated assets, or fewer observations than assets — that inversion can fail, and the user gets a barenumpy.linalg.LinAlgError: Singular matrixthrown from a private method, with no hint as to the cause or the fix.This routes the four inversion sites through a small helper that translates
LinAlgErrorinto aValueErrornaming the likely cause and pointing at concrete remedies.Before:
After:
Why not validate up front?
The issue suggests raising a validation error before optimisation begins. I tried that first and backed it out — it is a regression. A singular covariance matrix only breaks CLA if a submatrix of the currently free assets happens to be singular, which often never occurs. Over 40 randomly generated rank-deficient covariance matrices, 29 solved fine and only 4 raised
LinAlgError, so an up-front positive-definiteness check would reject a majority of inputs that work today.Worth noting for anyone reaching for the existing helper:
risk_models._is_positive_semidefinitereturnsTruefor these singular matrices (it adds1e-16jitter before the Cholesky), so it is not a usable guard here — CLA needs positive definiteness, not semidefiniteness.Translating the error where it actually occurs keeps the working cases working and only improves the failing path. Verified: the same 29 still succeed, and the 4
LinAlgErrors become clearValueErrors.Tests
Two tests added:
test_cla_singular_covariance_raises— the exact reproducer from the issue, acrossmax_sharpe/min_volatility/efficient_frontier. Note it asserts the error is not aLinAlgError:np.linalg.LinAlgErrorsubclassesValueError, so a plainpytest.raises(ValueError)would pass against the old code and prove nothing.test_cla_singular_covariance_fixed_by_shrinkage— verifies the remedy the error message recommends actually works, i.e. thatCovarianceShrinkagereturns a positive definite matrix CLA can solve. An error message giving advice should have that advice tested.Both fail on
mainand pass here. Full suite: 281 passed. The 5 failures intest_hrp.pyare pre-existing on a cleanmain(ascipy.clusterincompatibility) and unrelated to this change.🤖 Generated with Claude Code