NEW @W-22462042@ fix: flow engine fails on Windows for any flow containing a non-ASCII character - #494
Conversation
|
Git2Gus App is installed but the |
aruntyagiTutu
left a comment
There was a problem hiding this comment.
Reviewed against the PR review guide. Diff and CI checked.
Root cause is well-identified: quick_validate() opened flow XML with the platform default encoding, which is cp1252 on Windows — any UTF-8 byte sequence undefined in cp1252 (e.g. the smart-quote example used in the new test) raised UnicodeDecodeError and silently dropped a valid flow. The fix reads UTF-8 first and falls back to cp1252 for legacy files, matching the existing pattern in flow_scanner/__main__.py.
New test file test_parse_utils.py proves the regression directly: it monkeypatches builtins.open to emulate a cp1252-locale default, confirms the fix handles it, and also covers the plain UTF-8 locale and missing-file cases. Version bump to 0.40.1-SNAPSHOT is appropriate for a patch fix. CI green across macOS/Ubuntu/Windows.
No concerns. Approving.
Summary
Root Cause
quick_validate() in FlowScanner/public/parse_utils.py:1418 opens flow files with open(flow_path,'r') and no encoding argument, so Python uses the platform locale encoding. On Windows (cp1252) any flow XML containing a non-cp1252 UTF-8 byte (e.g. 0x9D from U+201D smart quote) raises UnicodeDecodeError. The bare except logs CRITICAL and returns False, dropping the flow; when all flows are dropped the scanner prints 'No flow files found to scan. Exiting...' and never writes flowScannerResultsFile.json, so the Node wrapper FlowScannerCommandWrapper.js fails to read it and surfaces a misleading ENOENT UnexpectedEngineError. Flow XML is authored/declared UTF-8, so locale-based decoding is simply wrong. Confirmed by PYTHONUTF8=1 fixing it, and by the fact that main.py:104 and custom_parser.get_root already read/parse as UTF-8 correctly.
Fix
Read flow files as UTF-8 (with cp1252 fallback) in quick_validate so UTF-8 flows with non-cp1252 bytes are not dropped on Windows locales; added Python unittest coverage and bumped package to 0.40.1-SNAPSHOT
Testing
Functional Testing Evidence
Tested on Dreamhouse project:
Overall Status: PASS ✅
Files Changed