Main Page: Difference between revisions

From HPCwiki
Jump to navigation Jump to search
(Reorder)
 
(47 intermediate revisions by 13 users not shown)
Line 1: Line 1:
The Agrogenomics cluster is a [http://en.wikipedia.org/wiki/High-performance_computing High Performance Compute] (HPC) infrastructure hosted by [http://www.wageningenur.nl/nl/activiteit/Opening-High-Performance-Computing-cluster-HPC.htm Wageningen University & Research Centre]. It is open for use for all WUR research groups as well as other organizations, including companies, that have collaborative projects with WUR.
Anunna is a [http://en.wikipedia.org/wiki/High-performance_computing High Performance Computer] (HPC) infrastructure hosted by [https://www.wur.nl/en/show/supercomputer-anunna-opens-up-more-opportunities-for-data-storage-and-artificial-intelligence-applications.htm Wageningen University & Research Centre]. It is open for use for all WUR research groups as well as other organizations, including companies, that have collaborative projects with WUR.  
 
The Agrogenomics HPC was an initiative of the [http://www.breed4food.com/en/breed4food.htm Breed4Food] (B4F) consortium, consisting of the [[About_ABGC | Animal Breeding and Genomics Centre]] (WU-Animal Breeding and Genomics and Wageningen Livestock Research) and four major breeding companies: [http://www.cobb-vantress.com Cobb-Vantress], [https://www.crv4all.nl CRV], [http://www.hendrix-genetics.com Hendrix Genetics], and [http://www.topigs.com TOPIGS]. Currently, in addition to the original partners, the HPC (HPC-Ag) is used by other groups from Wageningen UR (Bioinformatics, Centre for Crop Systems Analysis, Environmental Sciences Group, and Plant Research International) and plant breeding industry (Rijk Zwaan).  


== Access Policy ==
== Access Policy ==
[[Access_Policy | Main Article: Access Policy]]
[[Access_Policy | Main Article: Access Policy]]


Access needs to be granted actively (by creation of an account on the cluster by FB-IT). Use of resources is limited by the scheduler. Depending on availability of queues ('partitions') granted to a user, priority to the system's resources is regulated. Note that the use of the HPC-Ag is not free of charge. List price of CPU time and storage, and possible discounts on that list price for your organisation, can be retrieved from CAT-AGRO or FB-ICT.
Access needs to be granted actively (by creation of an account on the cluster by FB-IT). Use of resources is limited by the scheduler. Note that the use of Anunna is not free of charge.  
 
= Using Anunna =
* [[Tariffs | Costs associated with resource usage]]
 
== Gaining access to Anunna==
Access to the cluster and file transfer are traditionally done via [http://en.wikipedia.org/wiki/Secure_Shell SSH and SFTP].
* [[log_in_to_B4F_cluster | Logging into cluster using ssh]]
* [[file_transfer | File transfer options]]
* [[Services | Alternative access methods, and extra features and services on Anunna]]
* [[Filesystems | Data storage methods on Anunna]]


== Using the HPC-Ag ==
== Using Anunna for courses (mainly jupyter notebooks) ==
=== Gaining access to the HPC-Ag ===
* [[steps_for_courses | Steps involved to run a course on Anunna]]
Access to the cluster and file transfer are done by [http://en.wikipedia.org/wiki/Secure_Shell ssh-based protocols].
* [[log_in_to_B4F_cluster | Logging into cluster using ssh and file transfer]]
= Events =


=== Cluster Management Software and Scheduler ===
* [[Courses]] that have happened and are happening
The HPC-Ag uses Bright Cluster Manager software for overall cluster management, and Slurm as job scheduler.
* [[Downtime]] that will affect all users
* [[BCM_on_B4F_cluster | Monitor cluster status with BCM]]
* [[Meetings]] that may affect the policies of Anunna
* [[SLURM_on_B4F_cluster | Submit jobs with Slurm]]
 
= Software =
* [[Modules]]
* [[Apptainer]]
* [[Python]]
* [[R]]
* [[Julia]]
 
=Web Apps=
 
*[[Jupyter]]
 
= Other Software =
 
== Cluster Scheduler ==
Anunna uses Slurm as job scheduler.
* [[Using_Slurm | Submit jobs with Slurm]]
* [[node_usage_graph | Be aware of how much work the cluster is under right now with 'node_usage_graph']]
* [[node_usage_graph | Be aware of how much work the cluster is under right now with 'node_usage_graph']]
* [[SLURM_Compare | Rosetta Stone of Workload Managers]]
* [[SLURM_Compare | Rosetta Stone of Workload Managers]]


=== Installation of software by users ===
== Installation of software by users ==


* [[Domain_specific_software_on_B4Fcluster_installation_by_users | Installing domain specific software: installation by users]]
* [[Domain_specific_software_on_B4Fcluster_installation_by_users | Installing domain specific software: installation by users]]
Line 27: Line 51:
* [[Setting_up_Python_virtualenv | Setting up and using a virtual environment for Python3 ]]
* [[Setting_up_Python_virtualenv | Setting up and using a virtual environment for Python3 ]]
* [[Virtual_environment_Python_3.4_or_higher | Setting up and using a virtual environment for Python3.4 or higher ]]
* [[Virtual_environment_Python_3.4_or_higher | Setting up and using a virtual environment for Python3.4 or higher ]]
* [[Installing WRF and WPS]]
* [[Running scripts on a fixed timeschedule (cron)]]


=== Installed software ===
== Installed software ==


* [[Globally_installed_software | Globally installed software]]
* [[Globally_installed_software | Globally installed software]]
* [[ABGC_modules | ABGC specific modules]]
* [[ABGC_modules | ABGC specific modules]]


=== Being in control of Environment parameters ===
= Useful Notes =
 
== Being in control of Environment parameters ==


* [[Using_environment_modules | Using environment modules]]
* [[Using_environment_modules | Using environment modules]]
* [[Aliases and local variables]]
* [[Setting local variables]]
* [[Setting local variables]]
* [[Setting_TMPDIR | Set a custom temporary directory location]]
* [[Setting_TMPDIR | Set a custom temporary directory location]]
Line 41: Line 70:
* [[Setting_up_Python_virtualenv | Setting up and using a virtual environment for Python3 ]]
* [[Setting_up_Python_virtualenv | Setting up and using a virtual environment for Python3 ]]


=== Controlling costs ===
== Controlling costs ==


* [[SACCT | using SACCT to see your costs]]
* [[SACCT | using SACCT to see your costs]]
Line 47: Line 76:


== Management ==
== Management ==
[[HPC_management | Main Article: HPC management]]
Product Owner of Anunna is Alexander van Ittersum (Wageningen UR,FB-IT, C&PS). [[User: prins089 | Fons Prinsen (Wageningen UR, FB-IT, C&PS)]] is responsible for [[Maintenance_and_Management | Maintenance and Management]] of the cluster.
 
Project Leader of the HPC is Stephen Janssen (Wageningen UR,FB-IT, Service Management). [[User:pollm001 | Koen Pollmann (Wageningen UR,FB-IT, Infrastructure)]] and [[User:dawes001 | Gwen Dawes (Wageningen UR, FB-IT, Infrastructure)]] are responsible for [[Maintenance_and_Management | Maintenance and Management]].


== Users ==
* [[Roadmap | Ambitions regarding innovation, support and administration of Anunna ]]


* [[List_of_users | List of users (alphabetical order)]]
= Miscellaneous =
* [[Mailinglist | Electronic mail discussion lists]]
* [[Mailinglist | Electronic mail discussion lists]]
 
* [[History_of_the_Cluster | Historical information on the startup of Anunna]]
== Miscellaneous ==
* [[History_of_the_Cluster | Historical information on the startup of the HPC]]
* [[Bioinformatics_tips_tricks_workflows | Bioinformatics tips, tricks, and workflows]]
* [[Bioinformatics_tips_tricks_workflows | Bioinformatics tips, tricks, and workflows]]
* [[Parallel_R_code_on_SLURM | Running parallel R code on SLURM]]
* [[Convert_between_MediaWiki_and_other_formats | Convert between MediaWiki format and other formats]]
* [[Convert_between_MediaWiki_and_other_formats | Convert between MediaWiki format and other formats]]
* [[Manual GitLab | GitLab: Create projects and add scripts]]
* [[Manual GitLab | GitLab: Create projects and add scripts]]
* [[Monitoring_executions | Monitoring job execution]]
* [[Shared_folders | Working with shared folders in the Lustre file system]]
* [[Old_binaries | Running older binaries on the updated OS]]


== See also ==
= See also =
* [[Maintenance_and_Management | Maintenance and Management]]
* [[Maintenance_and_Management | Maintenance and Management]]
* [[BCData | BCData]]
* [[BCData | BCData]]
Line 70: Line 99:
* [[Lustre_PFS_layout | Lustre Parallel File System layout]]
* [[Lustre_PFS_layout | Lustre Parallel File System layout]]


== External links ==
= External links =
{| width="90%"
{| width="90%"
|- valign="top"
|- valign="top"
| width="30%" |
| width="30%" |
* [http://www.breed4food.com/en/show/Breed4Food-initiative-reinforces-the-Netherlands-position-as-an-innovative-country-in-animal-breeding-and-genomics.htm Breed4Food programme]
* [https://www.wur.nl/en/Value-Creation-Cooperation/Facilities/Wageningen-Shared-Research-Facilities/Our-facilities/Show/High-Performance-Computing-Cluster-HPC-Anunna.htm SRF offers a HPC facilty]
* [http://www.wageningenur.nl/en/Expertise-Services/Facilities/CATAgroFood-3/CATAgroFood-3/Our-facilities/Show/High-Performance-Computing-Cluster-HPC.htm CATAgroFood offers a HPC facilty]
* [http://www.cobb-vantress.com Cobb-Vantress homepage]
 
| width="30%" |
* [https://www.crv4all.nl CRV homepage]
* [http://www.hendrix-genetics.com Hendrix Genetics homepage]
* [http://www.topigs.com TOPIGS homepage]
| width="30%" |
| width="30%" |
* [http://en.wikipedia.org/wiki/Scientific_Linux Scientific Linux]
* [http://en.wikipedia.org/wiki/Scientific_Linux Scientific Linux]
* [http://en.wikipedia.org/wiki/Help:Cheatsheet Help with editing Wiki pages]
* [http://en.wikipedia.org/wiki/Help:Cheatsheet Help with editing Wiki pages]
|}
|}

Latest revision as of 16:19, 2 December 2024

Anunna is a High Performance Computer (HPC) infrastructure hosted by Wageningen University & Research Centre. It is open for use for all WUR research groups as well as other organizations, including companies, that have collaborative projects with WUR.

Access Policy

Main Article: Access Policy

Access needs to be granted actively (by creation of an account on the cluster by FB-IT). Use of resources is limited by the scheduler. Note that the use of Anunna is not free of charge.

Using Anunna

Gaining access to Anunna

Access to the cluster and file transfer are traditionally done via SSH and SFTP.

Using Anunna for courses (mainly jupyter notebooks)

Events

  • Courses that have happened and are happening
  • Downtime that will affect all users
  • Meetings that may affect the policies of Anunna

Software

Web Apps

Other Software

Cluster Scheduler

Anunna uses Slurm as job scheduler.

Installation of software by users

Installed software

Useful Notes

Being in control of Environment parameters

Controlling costs

Management

Product Owner of Anunna is Alexander van Ittersum (Wageningen UR,FB-IT, C&PS). Fons Prinsen (Wageningen UR, FB-IT, C&PS) is responsible for Maintenance and Management of the cluster.

Miscellaneous

See also

External links

Retrieved from "https://wiki.anunna.wur.nl/index.php?title=Main_Page&oldid=2413"