HPCwiki - User contributions [en]

Reservations

2017-08-25T12:52:05Z

Lith010: /* Cost */

SLURM is capable of assuring resources be available at a specified time for a specified set of users via the use of reservations. A reservation can be made far in advance of the time when the resources are needed, and SLURM will automatically prevent jobs from being scheduled that do not belong to that reservation during that time. In fact, the earlier the better - this allows long jobs to terminate properly.

This may be required in extreme circumstances where resources must be available for demonstration or teaching purposes at the moment when the event is happening.

== Usage ==
In order to use a reservation, simply add the following line into your sbatch script:
<nowiki>#SBATCH --reservation="MyReservation"</nowiki>
Replacing 'MyReservation' with the name of the reservation you wish to use. To see current reservations:
<nowiki>scontrol show reservations</nowiki>
The currently valid reservations are marked as State=ACTIVE; When you submit your job using a reservation definition, SLURM is then asked if you can use these resources. If yes, then your job will proceed as soon as there are resources available inside the reserved allocation. If no, then your job will fail to submit.

== Requesting a Reservation ==
There is no way to create a reservation yourself - an admin must create it. To request a reservation, please email your system administrator and define the list of users (not groups!) who may use this reservation. Given that this will dedicate the resources away from general use, there will be a financial cost involved in this, typically equivalent to the cost of running jobs for the entire period defined.

In addition, it's not currently possible to reserve less than entire nodes at a time, as there's no means to reserve memory and cores separately (And there'd be no guarantee that they'd be on the same machine if so). Thus, the minimum reservation possible is one normal node, and the maximum should be no more than three in order to not disrupt the main workflow.

== Costs ==
The costs will be 50 euro per node per day + the additional costs of the job itself.

Reservations

2017-08-25T12:51:54Z

Lith010:

SLURM is capable of assuring resources be available at a specified time for a specified set of users via the use of reservations. A reservation can be made far in advance of the time when the resources are needed, and SLURM will automatically prevent jobs from being scheduled that do not belong to that reservation during that time. In fact, the earlier the better - this allows long jobs to terminate properly.

This may be required in extreme circumstances where resources must be available for demonstration or teaching purposes at the moment when the event is happening.

== Usage ==
In order to use a reservation, simply add the following line into your sbatch script:
<nowiki>#SBATCH --reservation="MyReservation"</nowiki>
Replacing 'MyReservation' with the name of the reservation you wish to use. To see current reservations:
<nowiki>scontrol show reservations</nowiki>
The currently valid reservations are marked as State=ACTIVE; When you submit your job using a reservation definition, SLURM is then asked if you can use these resources. If yes, then your job will proceed as soon as there are resources available inside the reserved allocation. If no, then your job will fail to submit.

== Requesting a Reservation ==
There is no way to create a reservation yourself - an admin must create it. To request a reservation, please email your system administrator and define the list of users (not groups!) who may use this reservation. Given that this will dedicate the resources away from general use, there will be a financial cost involved in this, typically equivalent to the cost of running jobs for the entire period defined.

In addition, it's not currently possible to reserve less than entire nodes at a time, as there's no means to reserve memory and cores separately (And there'd be no guarantee that they'd be on the same machine if so). Thus, the minimum reservation possible is one normal node, and the maximum should be no more than three in order to not disrupt the main workflow.

== Cost ==
The cost will be 50 euro per node per day + the additional costs of the job itself.

User:Lith010

2017-07-03T11:54:20Z

Lith010: Created page with "[https://www.vcard.wur.nl/Views/Profile/View.aspx?id=56890 vCard information]"

[https://www.vcard.wur.nl/Views/Profile/View.aspx?id=56890 vCard information]

Main Page

2017-07-03T11:06:29Z

Lith010:

BCData

2016-07-19T07:46:01Z

Lith010: Created page with "This page contains information about BCData"

This page contains information about BCData

Main Page

2016-07-19T07:45:19Z

Lith010:

The Agrogenomics cluster is a [http://en.wikipedia.org/wiki/High-performance_computing High Performance Compute] (HPC) infrastructure hosted by [http://www.wageningenur.nl/nl/activiteit/Opening-High-Performance-Computing-cluster-HPC.htm Wageningen University & Research Centre]. It is open for use for all WUR research groups as well as other organizations, including companies, that have collaborative projects with WUR.

The Agrogenomics HPC was an initiative of the [http://www.breed4food.com/en/breed4food.htm Breed4Food] (B4F) consortium, consisting of the [[About_ABGC | Animal Breeding and Genomics Centre]] (WU-Animal Breeding and Genomics and Wageningen Livestock Research) and four major breeding companies: [http://www.cobb-vantress.com Cobb-Vantress], [https://www.crv4all.nl CRV], [http://www.hendrix-genetics.com Hendrix Genetics], and [http://www.topigs.com TOPIGS]. Currently, in addition to the original partners, the HPC (HPC-Ag) is used by other groups from Wageningen UR (Bioinformatics, Centre for Crop Systems Analysis, Environmental Sciences Group, and Plant Research International) and plant breeding industry (Rijk Zwaan).

== Rationale and Requirements for a new cluster ==
[[File:Breed4food-logo.jpg|thumb|right|200px|The Breed4Food logo]]
The Agrogenomics Cluster was originally conceived as being the 7th pillar of the [http://www.breed4food.com/en/show/Breed4Food-initiative-reinforces-the-Netherlands-position-as-an-innovative-country-in-animal-breeding-and-genomics.htm Breed4Food programme]. While the other six pillars revolve around specific research themes, the Cluster represents a joint infrastructure. The rationale behind the cluster is to enable the increasing computational needs in the field of genetics and genomics research, by creating a joint facility that will generate benefits of scale, thereby reducing cost. In addition, the joint infrastructure is intended to facilitate cross-organisational knowledge transfer. In that capacity, the HPC-Ag acts as a joint (virtual) laboratory where researchers - academic and applied - can benefit from each other's know-how. Lastly, the joint cluster, housed at Wageningen University campus, allows retaining vital and often confidential data sources in a controlled environment, something that cloud services such as Amazon Cloud or others usually can not guarantee.
{{-}}

== Process of acquisition and financing ==

[[File:Signing_CatAgro.png|thumb|left|300px|Petra Caessens, manager operations of CAT-AgroFood, signs the contract of the supplier on August 1st, 2013. Next to her Johan van Arendonk on behalf of Breed4Food.]]
The Agrogenomics cluster was financed by [http://www.wageningenur.nl/en/Expertise-Services/Facilities/CATAgroFood-3/CATAgroFood-3/News-and-agenda/Show/CATAgroFood-invests-in-a-High-Performance-Computing-cluster.htm CATAgroFood]. The [[B4F_cluster#IT_Workgroup | IT-Workgroup]] formulated a set of requirements that in the end were best met by an offer from [http://www.dell.com/learn/nl/nl/rc1078544/hpcc Dell]. [http://www.clustervision.com ClusterVision] was responsible for installing the cluster at the Theia server centre of FB-ICT.
{{-}}

== Architecture of the cluster ==
[[Architecture_of_the_HPC | Main Article: Architecture of the Agrogenomics HPC]]
[[File:Cluster_scheme.png|thumb|right|600px|Schematic overview of the cluster.]]
The new Agrogenomics HPC has a classic cluster architecture: state of the art Parallel File System (PSF), headnodes, compute nodes (of varying 'size'), all connected by superfast network connections (Infiniband). Implementation of the cluster will be done in stages. The initial stage includes a 600TB PFS, 48 slim nodes of 16 cores and 64GB RAM each, and 2 fat nodes of 64 cores and 1TB RAM each. The overall architecture, that include two head nodes in fall-over configuration and an infiniband network backbone, can be easily expanded by adding nodes and expanding the PFS. The cluster management software is designed to facilitate a heterogenous and evolving cluster.
{{-}}

== Housing at Theia ==
[[File:Map_Theia.png|thumb|left|200px|Location of Theia, just outside of Wageningen campus]]
The Agrogenomics Cluster is housed at one of two main server centres of WUR-FB-IT, near Wageningen Campus. The building (Theia) may not look like much from the outside (used to function as potato storage) but inside is a modern server centre that includes, a.o., emergency power backup systems and automated fire extinguishers. Many of the server facilities provided by FB-ICT that are used on a daily basis by WUR personnel and students are located there, as is the Agrogenomics Cluster. Access to Theia is evidently highly restricted and can only be granted in the presence of a representative of FB-IT.
{{-}}
{| width="90%"
|- valign="top"
| width="10%" |

| width="30%" |
[[File:Cluster2_pic.png|thumb|left|220px|Some components of the cluster after unpacking.]]
| width="70%" |
[[File:Cluster_pic.png|thumb|right|400px|The final configuration after installation.]]
|}
{{-}}

== Management ==
[[HPC_management | Main Article: HPC management]]

Project Leader of the HPC is Stephen Janssen (Wageningen UR,FB-IT, Service Management). [[User:pollm001 | Koen Pollmann (Wageningen UR,FB-IT, Infrastructure)]] and [[User:dawes001 | Gwen Dawes (Wageningen UR, FB-IT, Infrastructure)]] are responsible for [[Maintenance_and_Management | Maintenance and Management]].

== Access Policy ==
[[Access_Policy | Main Article: Access Policy]]

Access needs to be granted actively (by creation of an account on the cluster by FB-IT). Use of resources is limited by the scheduler. Depending on availability of queues ('partitions') granted to a user, priority to the system's resources is regulated. Note that the use of the HPC-Ag is not free of charge. List price of CPU time and storage, and possible discounts on that list price for your organisation, can be retrieved from CAT-AGRO or FB-ICT.

== Users ==

* [[List_of_users | List of users (alphabetical order)]]
* [[Mailinglist | Electronic mail discussion lists]]

== Using the HPC-Ag ==
=== Gaining access to the HPC-Ag ===
Access to the cluster and file transfer are done by [http://en.wikipedia.org/wiki/Secure_Shell ssh-based protocols].
* [[log_in_to_B4F_cluster | Logging into cluster using ssh and file transfer]]

=== Cluster Management Software and Scheduler ===
The HPC-Ag uses Bright Cluster Manager software for overall cluster management, and Slurm as job scheduler.
* [[BCM_on_B4F_cluster | Monitor cluster status with BCM]]
* [[SLURM_on_B4F_cluster | Submit jobs with Slurm]]
* [[node_usage_graph | Be aware of how much work the cluster is under right now with 'node_usage_graph']]
* [[SLURM_Compare | Rosetta Stone of Workload Managers]]

=== Installation of software by users ===

* [[Domain_specific_software_on_B4Fcluster_installation_by_users | Installing domain specific software: installation by users]]
* [[Setting local variables]]
* [[Installing_R_packages_locally | Installing R packages locally]]
* [[Setting_up_Python_virtualenv | Setting up and using a virtual environment for Python3 ]]
* [[Virtual_environment_Python_3.4_or_higher | Setting up and using a virtual environment for Python3.4 or higher ]]

=== Installed software ===

* [[Globally_installed_software | Globally installed software]]
* [[ABGC_modules | ABGC specific modules]]

=== Being in control of Environment parameters ===

* [[Using_environment_modules | Using environment modules]]
* [[Setting local variables]]
* [[Setting_TMPDIR | Set a custom temporary directory location]]
* [[Installing_R_packages_locally | Installing R packages locally]]
* [[Setting_up_Python_virtualenv | Setting up and using a virtual environment for Python3 ]]

=== Controlling costs ===

* [[SACCT | using SACCT to see your costs]]
* [[get_my_bill | using the "get_my_bill" script to estimate costs]]

== Miscellaneous ==
* [[Bioinformatics_tips_tricks_workflows | Bioinformatics tips, tricks, and workflows]]
* [[Convert_between_MediaWiki_and_other_formats | Convert between MediaWiki format and other formats]]
* [[Manual GitLab | GitLab: Create projects and add scripts]]

== See also ==
* [[Maintenance_and_Management | Maintenance and Management]]
* [[BCData | BCData]]
* [[Mailinglist | Electronic mail discussion lists]]
* [[About_ABGC | About ABGC]]
* [[Computer_cluster | High Performance Computing @ABGC]]
* [[Lustre_PFS_layout | Lustre Parallel File System layout]]

== External links ==
{| width="90%"
|- valign="top"
| width="30%" |
* [http://www.breed4food.com/en/show/Breed4Food-initiative-reinforces-the-Netherlands-position-as-an-innovative-country-in-animal-breeding-and-genomics.htm Breed4Food programme]
* [http://www.wageningenur.nl/en/Expertise-Services/Facilities/CATAgroFood-3/CATAgroFood-3/Our-facilities/Show/High-Performance-Computing-Cluster-HPC.htm CATAgroFood offers a HPC facilty]
* [http://www.cobb-vantress.com Cobb-Vantress homepage]

| width="30%" |
* [https://www.crv4all.nl CRV homepage]
* [http://www.hendrix-genetics.com Hendrix Genetics homepage]
* [http://www.topigs.com TOPIGS homepage]
| width="30%" |
* [http://en.wikipedia.org/wiki/Scientific_Linux Scientific Linux]
* [http://en.wikipedia.org/wiki/Help:Cheatsheet Help with editing Wiki pages]
|}