Skip to content

[BUG] raise an informative error when CLA gets a singular covariance matrix#749

Closed
jaydevkalivarapu wants to merge 1 commit into
PyPortfolio:mainfrom
jaydevkalivarapu:fix/cla-singular-covariance
Closed

[BUG] raise an informative error when CLA gets a singular covariance matrix#749
jaydevkalivarapu wants to merge 1 commit into
PyPortfolio:mainfrom
jaydevkalivarapu:fix/cla-singular-covariance

Conversation

@jaydevkalivarapu

Copy link
Copy Markdown
Contributor

Closes #737

Summary

CLA inverts the covariance submatrix of the free assets on every iteration. When the covariance matrix is singular — perfectly correlated or duplicated assets, or fewer observations than assets — that inversion can fail, and the user gets a bare numpy.linalg.LinAlgError: Singular matrix thrown from a private method, with no hint as to the cause or the fix.

This routes the four inversion sites through a small helper that translates LinAlgError into a ValueError naming the likely cause and pointing at concrete remedies.

Before:

numpy.linalg.LinAlgError: Singular matrix

After:

ValueError: The covariance matrix is singular, so the critical line algorithm cannot
invert it. This usually means some assets are perfectly correlated or duplicated, or
that the covariance matrix was estimated from fewer observations than there are assets.
Consider removing redundant assets, or using a shrinkage estimator such as
risk_models.CovarianceShrinkage, which returns a positive definite matrix.

Why not validate up front?

The issue suggests raising a validation error before optimisation begins. I tried that first and backed it out — it is a regression. A singular covariance matrix only breaks CLA if a submatrix of the currently free assets happens to be singular, which often never occurs. Over 40 randomly generated rank-deficient covariance matrices, 29 solved fine and only 4 raised LinAlgError, so an up-front positive-definiteness check would reject a majority of inputs that work today.

Worth noting for anyone reaching for the existing helper: risk_models._is_positive_semidefinite returns True for these singular matrices (it adds 1e-16 jitter before the Cholesky), so it is not a usable guard here — CLA needs positive definiteness, not semidefiniteness.

Translating the error where it actually occurs keeps the working cases working and only improves the failing path. Verified: the same 29 still succeed, and the 4 LinAlgErrors become clear ValueErrors.

Tests

Two tests added:

  • test_cla_singular_covariance_raises — the exact reproducer from the issue, across max_sharpe / min_volatility / efficient_frontier. Note it asserts the error is not a LinAlgError: np.linalg.LinAlgError subclasses ValueError, so a plain pytest.raises(ValueError) would pass against the old code and prove nothing.
  • test_cla_singular_covariance_fixed_by_shrinkage — verifies the remedy the error message recommends actually works, i.e. that CovarianceShrinkage returns a positive definite matrix CLA can solve. An error message giving advice should have that advice tested.

Both fail on main and pass here. Full suite: 281 passed. The 5 failures in test_hrp.py are pre-existing on a clean main (a scipy.cluster incompatibility) and unrelated to this change.

🤖 Generated with Claude Code

…matrix

Closes PyPortfolio#737

The critical line algorithm inverts the covariance submatrix of the free
assets on each iteration. When the covariance matrix is singular — e.g.
perfectly correlated or duplicated assets, or fewer observations than
assets — that inversion can fail, and users saw a bare
`numpy.linalg.LinAlgError: Singular matrix` raised from a private method,
with no indication of what was wrong or how to fix it.

Route the four inversion sites through a helper that translates the
LinAlgError into a ValueError explaining the likely cause and pointing at
concrete remedies.

Note that validating positive-definiteness up front (as the issue
suggests) would be a regression: a singular covariance matrix only breaks
CLA if a *submatrix* of the free assets happens to be singular. In a
randomised check over 40 rank-deficient covariance matrices, 29 solved
fine and only 4 raised LinAlgError, so an up-front check would reject
inputs that work today. Translating the error where it actually occurs
keeps those working and only improves the failure path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@jaydevkalivarapu

Copy link
Copy Markdown
Contributor Author

Closing this — withdrawing the change. The error-translation approach here isn't the right shape for this code, and the underlying CLA singularity behaviour needs more thought (and likely coordination with #616 / #700) before a patch is worth landing.

@jaydevkalivarapu
jaydevkalivarapu deleted the fix/cla-singular-covariance branch July 17, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] CLA crashes with LinAlgError on singular covariance matrices instead of raising a user-facing validation error

1 participant