.. _cwl-parallel-workflow: ################################# Parallel CWL Workflow :new:`flag` ################################# This tutorial runs the BioExcel `CWL Protein MD Setup tutorial with mutations `_ from WorkflowHub on one compute node. Before starting, it is recommended to read :ref:`container_workflow`. The workflow first prepares a protein structure. It then uses the CWL ``scatter`` feature to create three independent molecular-dynamics setup branches, one for each mutation. Starting ``cwltool`` with ``--parallel`` lets these branches run concurrently on the cores of one node. For a multi-node CWL example, see :ref:`cwl-distributed-wrf`. Download the WorkflowHub Package ================================ Open `workflow on WorkflowHub `_, select *Download RO-Crate* and save the file as ``workflowhub-289-v3.crate.zip``. Create ``cwl-mutations-input``, which will become your input dataset, and unzip the package there. .. code-block:: bash mkdir cwl-mutations-input unzip workflowhub-289-v3.crate.zip -d cwl-mutations-input Check that the main workflow, job inputs and molecular structures are at the root of ``cwl-mutations-input``: .. code-block:: bash test -f cwl-mutations-input/workflow.cwl test -f cwl-mutations-input/workflow_input_descriptions.yml test -f cwl-mutations-input/structure.pdb test -f cwl-mutations-input/ions.pdb All four commands should finish without an error. The package also contains ``biobb_adapters/`` and two subworkflows referenced by ``workflow.cwl``. Review the Parallel Inputs ========================== Open ``cwl-mutations-input/workflow_input_descriptions.yml``. Its ``mutations_list`` input contains three mutations: .. code-block:: yaml mutations_list: - '{"properties": {"mutation_list":"A:Gly4Lys"}}' - '{"properties": {"mutation_list":"A:Leu8Met"}}' - '{"properties": {"mutation_list":"A:Tyr20Gln"}}' ``workflow.cwl`` scatters its ``subworkflow_mutate`` step over this list. The common structure preparation runs once, after which one complete branch runs for each list item. Leave the supplied values unchanged for the first run. Build the Runner Image ====================== WorkflowHub also publishes an `official companion container workflow `_. Outside your input folder, create ``biobb-cwl-runner.def`` with the following definition. It extends that BioExcel image with ``cwltool`` and Node.js: .. code-block:: singularity Bootstrap: docker From: ghcr.io/bioexcel/biobb_wf_md_setup_mutations:latest %post /opt/conda/bin/conda install --yes \ --name biobb_wf_md_setup_mutations \ --channel conda-forge \ cwltool nodejs /opt/conda/bin/conda clean --all --yes mkdir -p /mnt/input /mnt/output %environment export PATH="/opt/conda/envs/biobb_wf_md_setup_mutations/bin:${PATH}" export OMP_NUM_THREADS="${OMP_NUM_THREADS:-4}" %runscript set -eu cd /mnt/input exec cwltool \ --parallel \ --no-container \ --debug \ --relax-path-checks \ --outdir /mnt/output \ workflow.cwl \ workflow_input_descriptions.yml Build the image (with container.sif as filename). .. code-block:: bash apptainer build container.sif biobb-cwl-runner.def Depending on the local Apptainer installation, building may require ``--fakeroot`` or administrator privileges. ``--parallel`` enables concurrent ready jobs. ``--no-container`` tells ``cwltool`` to use the BioBB applications already installed in this runner instead of trying to start nested containers on the compute node. ``--relax-path-checks`` permits the colon in output directory names such as ``A:Gly4Lys``. Prepare the Input Dataset ========================= Create an archive whose root contains ``workflow.cwl`` rather than an extra ``cwl-mutations-input`` directory: .. code-block:: bash cd cwl-mutations-input zip -r ../cwl-mutations-input.zip . cd .. Open *Data Management/Datasets* and upload ``cwl-mutations-input.zip``. Enable unpacking and name the dataset ``CWL Mutations Input``. See :ref:`user-dataset-creation` for the complete upload procedure. Upload the Runner Image ======================= Open *Data Management/Containers* and create a container from ``container.sif``. Name it ``CWL Mutations Runner``, select the project and choose the required access level. See :ref:`uploading-container` for the complete container upload procedure. Create the Container Workflow ============================= Open *Workflows*, select *Container* and create a workflow from ``CWL Mutations Runner``. Configure it as follows: 1. Name the workflow ``Parallel CWL Workflow``. 2. Select one CPU cluster, partition and computation resource. 3. Enable input staging and select ``CWL Mutations Input``. 4. Enable output staging and use ``CWL Mutations Output`` as the dataset title. Leave the source path empty to collect everything written to ``output``. 5. Leave the container parameters empty. 6. Start with a walltime of ``14400`` seconds and adjust it for the selected resource if necessary. 7. Set *Max cores* to ``12``. The requested cores must fit on one node. 8. Review the summary and create the workflow. Run and Monitor the Workflow ============================ Open the workflow details and select *Create Workflow Execution*. Name the execution ``Run 1``, keep the configured dataset and resource values, and create it. In the execution's *Progress* view, open the container task and select *View HPC Job Logs*. With ``--debug`` enabled, ``cwltool`` records each job as it starts and finishes. The preparation steps run first. Then the log should show multiple scattered ``subworkflow_mutate`` jobs active before the previous mutation branch has completed. Their exact ordering can differ between runs. Check the Results ================= After the execution succeeds, open ``CWL Mutations Output`` from the execution details. It should contain these mutation directories: .. code-block:: text A:Gly4Lys/ A:Leu8Met/ A:Tyr20Gln/ Each directory contains the outputs of one branch, including files such as ``md.gro``, ``md.cpt``, ``md.imaged.rot.xtc``, analysis ``.xvg`` files and the prepared production ``gppmdsim.tpr`` file.