Improve failed pipeline handling, fix config and fix logger backup - #15
Open
jlegrand62 wants to merge 4 commits into
Open
Improve failed pipeline handling, fix config and fix logger backup#15jlegrand62 wants to merge 4 commits into
jlegrand62 wants to merge 4 commits into
Conversation
- Update `romi_run_task.py` to filter out parameters with `None` (`if param is not None`) instead of all falsy values when building `compat_cfg`. This fix the replacement by their default value of any falsy parameter.
…task execution - Introduce `logfile_path` to hold the temporary log file path and pass it to `get_logging_config`. - Update `get_logging_config` call to use the new `logfile_path` variable. - After a successful subprocess run, copy `logging.cfg` and the backup configuration (`scan.toml` or `pipeline.toml`) into the created dataset directory, overwriting any existing files and logging errors if moves fail. - Fix indentation on the `local_config` update line for consistency.
- Change `subprocess.run(..., check=True)` to `check=False` and add comment to manage failures manually. - Remove the previous block that moved temporary logging config and backup TOML files to the dataset after a successful run. - On non‑zero return code, copy the temporary working directory (`tmpd`) to a new `*_failed` directory (`failed_tmp_workdir`) and log the action.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
subprocess.run(..., check=True)tocheck=False, handle non‑zero return codes manually, and archive the temporary working directory to a*_failedfolder when a task fails.scan.tomlorpipeline.toml) into the created dataset directory, overwriting existing files and logging any errors.Nonevalues (if param is not None) instead of all falsy values, preserving valid falsy parameters.local_configupdate line for consistency.