Add DeregisterAll and automatic indicator deregistration on security removal - #9673
Draft
jhonabreul wants to merge 2 commits into
Draft
Add DeregisterAll and automatic indicator deregistration on security removal#9673jhonabreul wants to merge 2 commits into
jhonabreul wants to merge 2 commits into
Conversation
…removal - QCAlgorithm.DeregisterAll(symbol) disposes every indicator and consolidator created for the symbol through the algorithm helper methods in one call - Settings.AutomaticIndicatorDeregistration (default false) runs the same cleanup automatically when a security is completely removed from the algorithm, e.g. when it leaves the universe - Fix SubscriptionManager.RemoveConsolidator leaking the consolidator scan wrapper when the symbol's subscription configs were already removed: the wrapper stayed in the scan queue being re-enqueued forever
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Universe churn leaks helper-created indicators and consolidators: when a security leaves the universe, consolidators created by the indicator helpers are never disposed and keep being re-scanned forever. Algorithms with per-symbol state and add/remove churn accumulate them until an
OutOfMemoryExceptionhours into a run.The change:
QCAlgorithm.DeregisterAll(Symbol)— one-line cleanup of everything the helpers created for a symbol.RegisterConsolidator(the choke point all helpers funnel through) now tracks indicators/consolidators per symbol;DeregisterAlldisposes them via the existing paths.B(target, reference)) are fully deregistered through any of their symbols.SubscriptionManager.AddConsolidatorare intentionally not tracked, so they survive universe removals.Settings.AutomaticIndicatorDeregistration(defaultfalse) — opt-in automaticDeregisterAllwhen a security is removed from the algorithm.SubscriptionManager.RemoveConsolidator: the scan wrapper was only disposed while the symbol still had subscription configs, so post-universe-removal cleanup left it in the scan queue forever. It is now disposed unconditionally.Also audited: a user-state slot on
Securityalready exists (security["my_state"] = objviaDynamicObject) — documentation gap, no engine work. Split-safe re-warm of selection indicators is deferred: there is no engine-side seam to user filter functions, and feeding adjusted prices avoids the problem.Related Issue
N/A
Motivation and Context
Per-symbol cleanup on universe removal takes a hand-rolled loop today, and getting it wrong is invisible until an OOM. A one-line (or automatic) cleanup makes the correct pattern the easy one.
Requires Documentation Change
Yes:
DeregisterAllandSettings.AutomaticIndicatorDeregistration, next toDeregisterIndicatorand the universe cleanup examples.How Has This Been Tested?
AlgorithmDeregisterAllTests(7 tests): target-symbol-only disposal, multi-symbol indicators, no-op after manual deregistration, rawAddConsolidatorconsolidators kept, automatic deregistration on removal iff the setting is on.SubscriptionManagerTestscase reproducing the wrapper leak: red before the fix, green after.DeregisterAllRegressionAlgorithm(C# + Python) andAutomaticIndicatorDeregistrationRegressionAlgorithm(C#): add/remove churn, cleanup asserted, fresh indicators work after re-adding.Tests.Algorithmnamespace with CI filters.Types of changes
Checklist:
bug-<issue#>-<description>orfeature-<issue#>-<description>