Skip to content

fix(autocast): add missing guard for use_standalone_type_inference#1743

Open
agkphysics wants to merge 1 commit into
NVIDIA:mainfrom
agkphysics:fix-autocast-shapes
Open

fix(autocast): add missing guard for use_standalone_type_inference#1743
agkphysics wants to merge 1 commit into
NVIDIA:mainfrom
agkphysics:fix-autocast-shapes

Conversation

@agkphysics

@agkphysics agkphysics commented Jun 15, 2026

Copy link
Copy Markdown

What does this PR do?

Type of change: Bug fix

Adds a missing check for use_standalone_type_inference in AutoCast which can cause shape issues because some dims are replaces with unk.

Testing

I have tested this with a model that previously failed after AutoCast conversion with the following error, but succeeds with this patch:

onnxruntime.capi.onnxruntime_pybind11_state.RuntimeException: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running LayerNormalization node. Name:'node_layer_norm' Status Message: /onnxruntime_src/onnxruntime/core/framework/op_kernel.cc:83 virtual OrtValue* onnxruntime::OpKernelContext::OutputMLValue(int, const onnxruntime::TensorShape&) status.IsOK() was false. Shape mismatch attempting to re-use buffer. {1,1,768} != {1,261,768}. Validate usage of dim_value (values should be > 0) and dim_param (all values with the same string should equate to the same size) in shapes in the model.

Before your PR is "Ready for review"

  • Is this change backward compatible?: N/A
  • If you copied code from any other sources or added a new PIP dependency, did you follow guidance in CONTRIBUTING.md: N/A
  • Did you write any new necessary tests?: N/A
  • Did you update Changelog?: N/A
  • Did you get Claude approval on this PR?: ❌

Summary by CodeRabbit

  • Bug Fixes
    • Improved type inference accuracy when standalone type inference is enabled, ensuring proper handling of tensor dimensions in ONNX models.

Signed-off-by: Aaron Keesing <aaron.keesing@zozo.com>
@agkphysics agkphysics requested a review from a team as a code owner June 15, 2026 23:58
@agkphysics agkphysics requested a review from ajrasane June 15, 2026 23:58
@copy-pr-bot

copy-pr-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

In PrecisionConverter._clear_types_and_shapes_recursive, the dim_param reset to "unk" for value_info tensors in the main graph is now conditional on not self.use_standalone_type_inference. When standalone type inference is active, only elem_type clearing occurs; dimension parameter rewriting is skipped.

Changes

Conditional dim_param rewrite in PrecisionConverter

Layer / File(s) Summary
Guard dim_param reset on standalone type inference flag
modelopt/onnx/autocast/precisionconverter.py
Wraps the dim_param = "unk" assignment inside _clear_types_and_shapes_recursive with not self.use_standalone_type_inference, so that block is skipped when standalone type inference is enabled.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (1 error)

Check name Status Explanation Resolution
Security Anti-Patterns ❌ Error PR adds 14 # nosec comments across 8 new modelopt files, explicitly violating SECURITY.md rule #5 which prohibits nosec comments to bypass security checks. Remove all # nosec comments from added code. Per SECURITY.md, if security patterns are genuinely necessary, add inline justification comments instead and request explicit review from @NVIDIA/modelopt-setup-codeowners with justification...
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding a missing guard condition for use_standalone_type_inference in the autocast functionality.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant