Workflows

Guidelines and tools for building reproducible, efficient research workflows on Anunna.

In this section

• Workflow Migration from Laptop to HPC — moving your work to the cluster
• Reproducibility Guidelines — keeping your work reproducible
• Workflow Engines (Snakemake, Nextflow) — managing multi-step pipelines
• Debugging Jobs — working out why a job failed
• Checkpointing — saving and restarting long jobs
• Scheduled tasks (cron) — running recurring tasks with scrontab

Parallel Workflows

• Types of Parallel Workflows
• Embarrassingly Parallel - Array jobs
• Multi-threaded Workflows
• Multi-Process Workflows
• Hybrid Workflows

• Performance Optimization/Multiple CPUs — using several cores in one job
• Performance Optimization/Multiple nodes (MPI) — scaling across nodes with MPI
• Performance Optimization/Multiple nodes (arrayjobs) — scaling with job arrays