Skip to content

Improve pandas history ergonomics: self-describing KeyErrors, DataFrame.get(symbol), lazy DataFrame on typed history - #9668

Draft
jhonabreul wants to merge 2 commits into
QuantConnect:masterfrom
jhonabreul:feature-history-pandas-ergonomics
Draft

Improve pandas history ergonomics: self-describing KeyErrors, DataFrame.get(symbol), lazy DataFrame on typed history#9668
jhonabreul wants to merge 2 commits into
QuantConnect:masterfrom
jhonabreul:feature-history-pandas-ergonomics

Conversation

@jhonabreul

@jhonabreul jhonabreul commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Description

Improves history/pandas ergonomics in Python, targeting the most common interop failure modes:

  • Self-describing PandasMapper KeyErrors. The old error was opaque:

    KeyError: "No key found for either mapped or original key. Mapped Key: []; Original Key: []"
    

    The error now names the requested keys, lists the frame's columns and index levels, and adds a targeted hint for the three recurring mistakes: an index level requested as a column (→ get_level_values/reset_index), a symbol that lives in the index (→ df.loc/df.xs), and a symbol with no data (→ df.get(key)). Any failure while building the description falls back to the legacy message.

  • DataFrame.get(symbol) now returns the symbol's sub-frame, or None when the symbol has no data, instead of raising. Column lookups keep pandas semantics.

  • Typed history results expose .data_frame. The typed History<T> overloads now return DataHistory<T> instead of a plain IEnumerable, so self.history[TradeBar](symbol, 10).data_frame works. DataHistory<T> still implements IEnumerable<T>, the conversion is lazy, and this mirrors the existing OptionHistory/FutureHistory pattern.

Deferred: opt-in wide/unstacked frame shapes (breaking default change, needs its own design), and pandas-only attribute hints on typed results (needs an interop-layer extension point).

Related Issue

N/A

Motivation and Context

MultiIndex/typed-result confusion is a dominant Python crash class. Each opaque KeyError costs a full backtest iteration to decode. These changes make the errors self-explanatory and add safe accessors, without changing any default frame shape.

Requires Documentation Change

The typed history return type (DataHistory<T>) and history.get(symbol)/.data_frame accessors could be mentioned in the history docs.

How Has This Been Tested?

  • New PandasIndexingTests cases: each KeyError variant (missing column, index-level key, symbol in index, missing symbol) asserts the new message; DataFrame.get tested for symbol, missing symbol, and column keys. All red before the fix.
  • New AlgorithmHistoryTests.TypedHistoryResultsExposeADataFrame (C# and Python): typed results expose the expected frame and remain enumerable.
  • Full AlgorithmHistoryTests, PandasConverterTests, PandasIndexingTests, PythonUtilTests fixtures pass.
  • History regression backtests (C# + Python) pass unchanged.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant