Workflows: Difference between revisions
Jump to navigation
Jump to search
No edit summary |
mNo edit summary |
||
| (6 intermediate revisions by the same user not shown) | |||
| Line 12: | Line 12: | ||
=== Parallel Workflows === | === Parallel Workflows === | ||
* | * [[Workflows/Parallel-Computing|Parallel Computing]] | ||
* Embarrassingly Parallel | * [[Workflows/Serial|Serial]] | ||
* Multi-threaded | * [[Workflows/Embarassinly-Parallel|Embarrassingly Parallel]] | ||
* Multi-Process | * [[Workflows/Multi-threaded|Multi-threaded]] | ||
* Hybrid Workflows | * [[Workflows/Multi-Process|Multi-Process]] | ||
* Hybrid Workflows | |||
* GPU workflows | |||
* <s>[[Performance Optimization/Multiple CPUs]] — using several cores in one job</s> | * <s>[[Performance Optimization/Multiple CPUs]] — using several cores in one job</s> | ||
* <s>[[Performance Optimization/Multiple nodes (MPI)]] — scaling across nodes with MPI</s> | * <s>[[Performance Optimization/Multiple nodes (MPI)]] — scaling across nodes with MPI</s> | ||
* <s>[[Performance Optimization/Multiple nodes (arrayjobs)]] — scaling with job arrays</s> | * <s>[[Performance Optimization/Multiple nodes (arrayjobs)]] — scaling with job arrays</s> | ||
Latest revision as of 09:55, 5 August 2026
Guidelines and tools for building reproducible, efficient research workflows on Anunna.
In this section
- Workflow Migration from Laptop to HPC — moving your work to the cluster
- Reproducibility Guidelines — keeping your work reproducible
- Workflow Engines (Snakemake, Nextflow) — managing multi-step pipelines
- Debugging Jobs — working out why a job failed
- Checkpointing — saving and restarting long jobs
- Scheduled tasks (cron) — running recurring tasks with scrontab
Parallel Workflows
- Parallel Computing
- Serial
- Embarrassingly Parallel
- Multi-threaded
- Multi-Process
- Hybrid Workflows
- GPU workflows
Performance Optimization/Multiple CPUs — using several cores in one jobPerformance Optimization/Multiple nodes (MPI) — scaling across nodes with MPIPerformance Optimization/Multiple nodes (arrayjobs) — scaling with job arrays