Jump to content

Main Page: Difference between revisions

From HPCwiki
← Older edit
Honfi001 (talk | contribs)
Bureaucrats, Administrators
250 edits
VisualWikitext
Dawes001 (talk | contribs)
231 edits
Reorder
mNo edit summary
 
(134 intermediate revisions by 14 users not shown)
Line 1: Line 1:
The Agrogenomics cluster is a [http://en.wikipedia.org/wiki/High-performance_computing High Performance Compute] (HPC) infrastructure hosted by [http://www.wageningenur.nl/nl/activiteit/Opening-High-Performance-Computing-cluster-HPC.htm Wageningen University & Research Centre]. It is open for use for all WUR research groups as well as other organizations, including companies, that have collaborative projects with WUR.  
Anunna is a [http://en.wikipedia.org/wiki/High-performance_computing High Performance Computing] (HPC) cluster hosted by [https://www.wur.nl/ Wageningen University & Research]. It is open to all WUR research groups, and to other organisations and companies running collaborative projects with WUR.


The Agrogenomics HPC was an initiative of the [http://www.breed4food.com/en/breed4food.htm Breed4Food] (B4F) consortium, consisting of the [[About_ABGC | Animal Breeding and Genomics Centre]] (WU-Animal Breeding and Genomics and Wageningen Livestock Research) and four major breeding companies: [http://www.cobb-vantress.com Cobb-Vantress], [https://www.crv4all.nl CRV], [http://www.hendrix-genetics.com Hendrix Genetics], and [http://www.topigs.com TOPIGS]. Currently, in addition to the original partners, the HPC (HPC-Ag) is used by other groups from Wageningen UR (Bioinformatics, Centre for Crop Systems Analysis, Environmental Sciences Group, and Plant Research International) and plant breeding industry (Rijk Zwaan).  
You reach the cluster over [[SSH Access|SSH]] at <code>login.anunna.wur.nl</code>, or through your browser at the [[Apps Portal]] (https://apps.anunna.wur.nl/).


== Access Policy ==
== [[About]] ==
[[Access_Policy | Main Article: Access Policy]]
* <h3>[[Mission and Governance]]</h3> What Anunna is for and who runs it
* <h3>[[Roadmap]]</h3> How we decide what to work on
* <h3>[[Cluster Architecture Overview]]</h3> How the cluster is put together
* <h3>[[Compute_Hardware_Overview]]</h3> The node types and their hardware
* <h3>[[Storage Systems Overview]]</h3> The storage tiers at a glance
* <h3>[[Tariffs]]</h3> Costs of using Anunna
* <h3>[[Network & Security]]</h3> Interconnect and data-security posture
* <h3>[[History of the Cluster]]</h3> How Anunna came to be
* <h3>[[Sustainability]]</h3> Green-HPC policy and measures
* <h3>[[FAQ]]</h3> Frequently Asked Questions
* <h3>[[Policies and Terms of Use]]</h3> Who may use Anunna and on what terms
*


Access needs to be granted actively (by creation of an account on the cluster by FB-IT). Use of resources is limited by the scheduler. Depending on availability of queues ('partitions') granted to a user, priority to the system's resources is regulated. Note that the use of the HPC-Ag is not free of charge. List price of CPU time and storage, and possible discounts on that list price for your organisation, can be retrieved from CAT-AGRO or FB-ICT.
== [[Get Started]] ==
* <h3>[[Who Can Access?]]</h3> Eligibility
* <h3>[[Account Application Process]]</h3> How to request an account
* <h3>[[User Responsibilities]]</h3> What is expected of you as a user
* <h3>[[SSH Access]]</h3> Logging in over SSH to the cluster
* <h3>[[Apps Portal]]</h3> Using applications on Anunna through your web browser
* <h3>[[Workflow Migration from Laptop to HPC]]</h3> Moving your work to the cluster from your computer
* <h3>[[Grant_Support_%26_Acknowledgement|Grant support & Acknowledgement]]</h3> Guidance and standard text that can be used in grant proposals, research papers, theses and other publications.


== Using the HPC-Ag ==
== [[System Access]] ==
=== Gaining access to the HPC-Ag ===
* <h3>[[Login Nodes]]</h3> The entry point you connect to
Access to the cluster and file transfer are done by [http://en.wikipedia.org/wiki/Secure_Shell ssh-based protocols].
* <h3>[[Compute Nodes]]</h3> Where your jobs run, and how to reach them
* [[log_in_to_B4F_cluster | Logging into cluster using ssh and file transfer]]
* <h3>[[Filesystems]]</h3> The storage tiers
* <h3>[[Quotas]]</h3> Storage limits and how to check them
* <h3>[[Remote Access (VPN, Gateway)]]</h3> Reaching Anunna from off campus
* <h3>[[Data Transfer Methods]]</h3> Moving data to and from Anunna


=== Cluster Management Software and Scheduler ===
== [[Scheduling|Job Scheduling and Resource Management]] ==
The HPC-Ag uses Bright Cluster Manager software for overall cluster management, and Slurm as job scheduler.
* <h3>[[Scheduler Overview (Slurm)]]</h3> How scheduling works
* [[BCM_on_B4F_cluster | Monitor cluster status with BCM]]
* <h3>[[Partitions / Queues]]</h3> The available partitions
* [[SLURM_on_B4F_cluster | Submit jobs with Slurm]]
* <h3>[[Choosing a node (constraints)]]</h3> Targeting particular hardware
* [[node_usage_graph | Be aware of how much work the cluster is under right now with 'node_usage_graph']]
* <h3>[[Batch Jobs]]</h3> Submitting a batch script
* [[SLURM_Compare | Rosetta Stone of Workload Managers]]
* <h3>[[Interactive Jobs]]</h3> An interactive shell on a compute node
* <h3>[[Array Jobs]]</h3> Running many similar jobs at once
* <h3>[[Monitoring Jobs]]</h3> Checking on your jobs
* <h3>[[Cancelling Jobs]]</h3> Stopping a job
* <h3>[[Reservations]]</h3> Reserving nodes
* <h3>[[Fair Use Policy]]</h3> Using shared resources considerately


=== Installation of software by users ===
== [[Software]] ==
* <h3>[[Software Overview]]</h3> The software landscape
* <h3>[[Environment Modules]]</h3> Loading software with modules and buckets
* <h3>[[Installing Personal Software]]</h3> Installing into your own space
* <h3>[[Licensed Software]]</h3> Software that needs a licence


* [[Domain_specific_software_on_B4Fcluster_installation_by_users | Installing domain specific software: installation by users]]
* <h3>Scripting languages</h3>
* [[Setting local variables]]
** <h4>[[Python]]</h4> Python modules and environment managers
* [[Installing_R_packages_locally | Installing R packages locally]]
** <h4>[[R]]</h4> R modules, packages, and parallel R jobs
* [[Setting_up_Python_virtualenv | Setting up and using a virtual environment for Python3 ]]
** <h4>[[Julia]]</h4> Loading Julia, packages, and running it in a job
* [[Virtual_environment_Python_3.4_or_higher | Setting up and using a virtual environment for Python3.4 or higher ]]


=== Installed software ===
* <h3>Containers</h3>
** <h4>[[Apptainer]]</h4> Portable, reproducible containers for HPC


* [[Globally_installed_software | Globally installed software]]
* <h3>MPI implementations</h3>
* [[ABGC_modules | ABGC specific modules]]
** <h4>[[OpenMPI]]</h4> The MPI library Anunna is built around
** <h4>[[IntelMPI]]</h4> Intel MPI, with the Intel compilers and MKL


=== Being in control of Environment parameters ===
== [[Storage]] ==
* <h3>[[Storage Systems Overview]]</h3> The storage tiers at a glance
* <h3>[[Home Directory]]</h3> Your personal space
* <h3>[[Compute Storage]]</h3> The fast Lustre filesystem for active work
* <h3>[[Shared Storage]]</h3> Sharing data within a group
* <h3>[[Backup Policy]]</h3> What is backed up, and what is not
* <h3>[[Archival Storage]]</h3> Long-term storage on tape
* <h3>[[Quotas]]</h3> Storage limits
* <h3>[[Data Lifecycle Policy]]</h3> How data moves from active to archived to removed
* <h3>[[Data storage best practices|Data Storage Best Practices]]</h3> Keeping data safe, tidy, and cheap
* <h3>[[Data Transfer Best Practices]]</h3> Moving data efficiently and reliably
* <h3>[[Handling Sensitive Data (GDPR, etc.)]]</h3> Confidential and personal data


* [[Using_environment_modules | Using environment modules]]
== Graphical Interface Applications ==
* [[Setting local variables]]
* <h3>[[Portal Overview]]</h3> What the portal is and how the dashboard is laid out
* [[Setting_TMPDIR | Set a custom temporary directory location]]
* <h3>[[Running GUI Applications]]</h3> The general way to launch and connect to a graphical application
* [[Installing_R_packages_locally | Installing R packages locally]]
* <h3>[[How to Launch a Desktop Session]]</h3> Start a full Linux desktop in your browser
* [[Setting_up_Python_virtualenv | Setting up and using a virtual environment for Python3 ]]
* <h3>[[Jupyter]]</h3> Jupyter notebooks
* <h3>[[RStudio]]</h3> The RStudio IDE for R and Python
* <h3>[[File Browser]]</h3> Manage your files in the browser
* <h3>[[Shell Access]]</h3> A terminal in the browser
* <h3>[[Jobs Queue Overview]]</h3> See your jobs in the browser


=== Controlling costs ===
== [[Training]] ==
* <h3>[[Training Materials]]</h3> Course slides and self-study resources
* <h3>[[Tutorials]]</h3> Hands-on tutorials
* <h3>[[Workshops]]</h3> Instructor-led courses and their dates


* [[SACCT | using SACCT to see your costs]]
== [[Support]] ==
* [[get_my_bill | using the "get_my_bill" script to estimate costs]]
* <h3>[[How to Get Help]]</h3> Who to contact and how
* <h3>[[Support Ticket System]]</h3> The WUR support portal
* <h3>[[Reporting Incidents]]</h3> Writing a good problem report
* <h3>[[Known Issues]]</h3> Current problems and workarounds
* <h3>[[Our User community]]</h3> The Anunna user community
* <h3>[[Maintenance Schedule]]</h3> Planned downtimes


== Management ==
== [[For PIs]] ==
[[HPC_management | Main Article: HPC management]]
* <h3>[[Dos and Don'ts]]</h3> Good and bad practice at a glance
* <h3>[[Managing Group Members]]</h3> Working with groups
* <h3>[[Storage Requests]]</h3> Requesting more storage
* <h3>[[Resource Allocation Requests]]</h3> Requesting larger or dedicated allocations
* <h3>[[External Collaborator Access]]</h3> Giving access to collaborators outside WUR
* <h3>[[Reporting Usage]]</h3> Tracking your group's usage and costs
* <h3>[[Grant Support & Acknowledgement]]</h3> Facility descriptions and support for funding applications


Project Leader of the HPC is Stephen Janssen (Wageningen UR,FB-IT, Service Management). [[User:pollm001 | Koen Pollmann (Wageningen UR,FB-IT, Infrastructure)]] and [[User:dawes001 | Gwen Dawes (Wageningen UR, FB-IT, Infrastructure)]] are responsible for [[Maintenance_and_Management | Maintenance and Management]].
== [[Workflows]] ==
* <h3>[[Workflow Migration from Laptop to HPC]]</h3> Moving your work to the cluster
* <h3>[[Reproducibility Guidelines]]</h3> Keeping your work reproducible
* <h3>[[Workflow Engines (Snakemake, Nextflow)]]</h3> Managing multi-step pipelines
* <h3>[[Debugging Jobs]]</h3> Working out why a job failed
* <h3>[[Checkpointing]]</h3> Saving and restarting long jobs
* <h3>[[Scheduled tasks (cron)|Scheduled Tasks (cron)]]</h3> Running recurring tasks with scrontab


== Users ==
* <h3>Parallel Workflows</h3>
** <h4>[[Workflows/Parallel-Computing|Parallel Computing]]</h4> What parallel computing means on Anunna, and which type fits your work
** <h4>[[Workflows/Serial|Serial]]</h4> One program on a single core — the baseline
** <h4>[[Workflows/Embarassinly-Parallel|Embarrassingly Parallel]]</h4> Running the same program many times over
** <h4>[[Workflows/Multi-threaded|Multi-threaded]]</h4> Many cores on one machine, within a single process
** <h4>[[Workflows/Multi-Process|Multi-Process]]</h4> One calculation spread across several machines


* [[List_of_users | List of users (alphabetical order)]]
== Quick links ==
* [[Mailinglist | Electronic mail discussion lists]]


== Miscellaneous ==
* [[How to Get Help]] — contact the HPC team
* [[History_of_the_Cluster | Historical information on the startup of the HPC]]
* [[Maintenance Schedule]] — planned downtimes
* [[Bioinformatics_tips_tricks_workflows | Bioinformatics tips, tricks, and workflows]]
* [[Glossary of Terms]] — common HPC and Anunna terms explained
* [[Convert_between_MediaWiki_and_other_formats | Convert between MediaWiki format and other formats]]
* [[Manual GitLab | GitLab: Create projects and add scripts]]
 
== See also ==
* [[Maintenance_and_Management | Maintenance and Management]]
* [[BCData | BCData]]
* [[Mailinglist | Electronic mail discussion lists]]
* [[About_ABGC | About ABGC]]
* [[Computer_cluster | High Performance Computing @ABGC]]
* [[Lustre_PFS_layout | Lustre Parallel File System layout]]


== External links ==
== External links ==
{| width="90%"
|- valign="top"
| width="30%" |
* [http://www.breed4food.com/en/show/Breed4Food-initiative-reinforces-the-Netherlands-position-as-an-innovative-country-in-animal-breeding-and-genomics.htm Breed4Food programme]
* [http://www.wageningenur.nl/en/Expertise-Services/Facilities/CATAgroFood-3/CATAgroFood-3/Our-facilities/Show/High-Performance-Computing-Cluster-HPC.htm CATAgroFood offers a HPC facilty]
* [http://www.cobb-vantress.com Cobb-Vantress homepage]


| width="30%" |
* [https://www.wur.nl/en/Value-Creation-Cooperation/Facilities/Wageningen-Shared-Research-Facilities/Our-facilities/Show/High-Performance-Computing-Cluster-HPC-Anunna.htm Wageningen Shared Research Facilities — HPC]
* [https://www.crv4all.nl CRV homepage]
* [http://www.hendrix-genetics.com Hendrix Genetics homepage]
* [http://www.topigs.com TOPIGS homepage]
| width="30%" |
* [http://en.wikipedia.org/wiki/Scientific_Linux Scientific Linux]
* [http://en.wikipedia.org/wiki/Help:Cheatsheet Help with editing Wiki pages]
|}

Latest revision as of 06:49, 31 August 2026

Anunna is a High Performance Computing (HPC) cluster hosted by Wageningen University & Research. It is open to all WUR research groups, and to other organisations and companies running collaborative projects with WUR.

You reach the cluster over SSH at login.anunna.wur.nl, or through your browser at the Apps Portal (https://apps.anunna.wur.nl/).

About

Get Started

System Access

Job Scheduling and Resource Management

Software

  • Scripting languages

    • Python

      Python modules and environment managers

    • R

      R modules, packages, and parallel R jobs

    • Julia

      Loading Julia, packages, and running it in a job

  • Containers

    • Apptainer

      Portable, reproducible containers for HPC

  • MPI implementations

    • OpenMPI

      The MPI library Anunna is built around

    • IntelMPI

      Intel MPI, with the Intel compilers and MKL

Storage

Graphical Interface Applications

Training

Support

For PIs

Workflows

Retrieved from "https://wiki.anunna.wur.nl/index.php?title=Main_Page&oldid=3181"