Skip to content

Nextflow translator, Nextflow/Streamflow/RO-Crate log parser rewrites/re-orgs#162

Merged
henricasanova merged 38 commits into
mainfrom
nextflow_translator_ro_crate_log_parsers_rewrite
Jun 30, 2026
Merged

Nextflow translator, Nextflow/Streamflow/RO-Crate log parser rewrites/re-orgs#162
henricasanova merged 38 commits into
mainfrom
nextflow_translator_ro_crate_log_parsers_rewrite

Conversation

@henricasanova

@henricasanova henricasanova commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

This PR includes:

  • A streamlined implementation of the Nextflow translator that: (i) always uses sub-workflows for scalability; (ii) produces a workflow that, when run with the nf-prov plugin, will produce a useful RO-Crate
  • A re-write the the RO-Crate log parser
  • An implementation of a Streamflow log parser, which uses the RO-Crate log parser
  • An re-implementation of the Nextflow log parser, which uses the RO-Crate log parser, and always runs workflow as sub-workflows for scalability purposes
  • Testing of the above in unit tests
  • Minor bug fixes

@henricasanova
Translator / Logger updates (RO-Crate, Nextflow)
a7153c6
@henricasanova
Re-establishing all tests as before
8cad889
@henricasanova
Removing unneeded import
823973a
@henricasanova
Temporary Fix revert in the RO-Crate log parser
41593a1
@henricasanova
Updated matplotlib colormap call
9d99ef8
@henricasanova
colormap (real) fix
87b5ded
@henricasanova
Re-establishing all tests
800ea04
@henricasanova
RO-Crate parsing fix for a Nextflow workflow
2cb8795
@henricasanova
Added a way to get task times from Nextflow executions (because RO-Crate
386e4d7 
doesn't have it!)
@henricasanova
Improved RO-Crate parsing for Nextflow
b6fc1f8
@henricasanova
Re-enabled all tests
939beca
@henricasanova
Made the RO-Crate log parser handle file sizes correctly for
bd26c58 
Nextflow-generated RO-Crate
@henricasanova
Added task id sanitization to the RO-Crate log parser
e2d2b1d
@henricasanova
Re-establishing all tests
2fcfb71
@henricasanova
For Nextflow/RO-Crate parsing, makde is so that if multiple trace
026af2d 
files are present, the latest one is used
@henricasanova
Drafted a new implementation of the nextflow translator
8921bc5
@henricasanova
More robust way of finding nextflow execution trace file
97a83bd
@henricasanova
Nextflow execution update
e415bac
@henricasanova
Re-established all "nextflow sub-workflow" code
f8c7b33
@henricasanova
tweaks
8cf5d50
@henricasanova
tmp commit
3bfe63f
@henricasanova
Added trace creation to the (translated) nextflow execution
d98d69c
@henricasanova
Merge branch 'nextflow_log_parser' into nextflow_log_parser_refactored
48043fd
@henricasanova
Streamflow/Nextflow/RO-Crate code refactoring
2d8285f
@henricasanova
More code re-factoring
97d6e58
@henricasanova
debug
ba44621
@henricasanova
More code re-factoring
c3d89fc
@henricasanova
Merge branch 'nexflow_translator' into tmp_branch
2a2663c
@henricasanova
Added RO-Crate detectable workflow input files for nextflow translator
59c7580
@henricasanova
RE-established all tests
081f528
@henricasanova henricasanova merged commit c054c33 into main Jun 30, 2026
5 of 9 checks passed
@henricasanova henricasanova deleted the nextflow_translator_ro_crate_log_parsers_rewrite branch July 1, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@henricasanova