Skip to content

NEW @W-22462042@ fix: flow engine fails on Windows for any flow containing a non-ASCII character - #494

Merged
nikhil-mittal-165 merged 2 commits into
devfrom
fix/issue-2074-flow-utf8-encoding
Aug 11, 2026
Merged

NEW @W-22462042@ fix: flow engine fails on Windows for any flow containing a non-ASCII character#494
nikhil-mittal-165 merged 2 commits into
devfrom
fix/issue-2074-flow-utf8-encoding

Conversation

@nikhil-mittal-165

@nikhil-mittal-165 nikhil-mittal-165 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Root Cause

quick_validate() in FlowScanner/public/parse_utils.py:1418 opens flow files with open(flow_path,'r') and no encoding argument, so Python uses the platform locale encoding. On Windows (cp1252) any flow XML containing a non-cp1252 UTF-8 byte (e.g. 0x9D from U+201D smart quote) raises UnicodeDecodeError. The bare except logs CRITICAL and returns False, dropping the flow; when all flows are dropped the scanner prints 'No flow files found to scan. Exiting...' and never writes flowScannerResultsFile.json, so the Node wrapper FlowScannerCommandWrapper.js fails to read it and surfaces a misleading ENOENT UnexpectedEngineError. Flow XML is authored/declared UTF-8, so locale-based decoding is simply wrong. Confirmed by PYTHONUTF8=1 fixing it, and by the fact that main.py:104 and custom_parser.get_root already read/parse as UTF-8 correctly.

Fix

Read flow files as UTF-8 (with cp1252 fallback) in quick_validate so UTF-8 flows with non-cp1252 bytes are not dropped on Windows locales; added Python unittest coverage and bumped package to 0.40.1-SNAPSHOT

Testing

  • ✅ Unit tests added/updated
  • ✅ All tests passing
  • ✅ Lint checks passing

Functional Testing Evidence

Tested on Dreamhouse project:

  • Test 1: Unit tests - UTF-8 flow validation with cp1252 locale - PASS (3/3 tests passed)
  • Test 2: CLI scan on UTF-8 flow with smart quotes - PASS (flow at 100% completion, no UnicodeDecodeError)
  • Test 3: Fix verification - PASS (commit c3e3ffa correctly reads flows as UTF-8 with cp1252 fallback)

Overall Status: PASS ✅

Files Changed

  • packages/code-analyzer-flow-engine/FlowScanner/public/parse_utils.py
  • packages/code-analyzer-flow-engine/FlowScanner/tests/test_parse_utils.py
  • packages/code-analyzer-flow-engine/package.json

@git2gus

git2gus Bot commented Aug 4, 2026

Copy link
Copy Markdown

Git2Gus App is installed but the .git2gus/config.json doesn't have right values. You should add the required configuration.

@nikhil-mittal-165 nikhil-mittal-165 changed the title fix: flow engine fails on Windows for any flow containing a non-ASCII character (cp1252 UnicodeDecodeError surfaced as ENOENT UnexpectedEngineError) (#2074) NEW @W-22462042@ fix: flow engine fails on Windows for any flow containing a non-ASCII character Aug 4, 2026
@nikhil-mittal-165
nikhil-mittal-165 marked this pull request as ready for review August 4, 2026 12:35

@aruntyagiTutu aruntyagiTutu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed against the PR review guide. Diff and CI checked.

Root cause is well-identified: quick_validate() opened flow XML with the platform default encoding, which is cp1252 on Windows — any UTF-8 byte sequence undefined in cp1252 (e.g. the smart-quote example used in the new test) raised UnicodeDecodeError and silently dropped a valid flow. The fix reads UTF-8 first and falls back to cp1252 for legacy files, matching the existing pattern in flow_scanner/__main__.py.

New test file test_parse_utils.py proves the regression directly: it monkeypatches builtins.open to emulate a cp1252-locale default, confirms the fix handles it, and also covers the plain UTF-8 locale and missing-file cases. Version bump to 0.40.1-SNAPSHOT is appropriate for a patch fix. CI green across macOS/Ubuntu/Windows.

No concerns. Approving.

@nikhil-mittal-165
nikhil-mittal-165 merged commit 0e4b017 into dev Aug 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants