Checkpointing - Revision history

Dawes001 at 13:42, 19 March 2019

2019-03-19T13:42:54Z

← Older revision		Revision as of 15:42, 19 March 2019
Line 1:		Line 1:
	Some recipes and tips for checkpointing on ~~the B4F HPC~~		Some recipes and tips for checkpointing on Anunna

	==Definition==		==Definition==
Line 27:		Line 27:
	Many software we used are checkpointable:<br />		Many software we used are checkpointable:<br />
	* If it is the case, the (SLURM) script should be adapt to use this advantage.		* If it is the case, the (SLURM) script should be adapt to use this advantage.
	* If the software is really not checkpointable, some software are available to checkpoint existing programs. One of them is DMTCP (Distributed MultiThreaded CheckPointing; [http://dmtcp.sourceforge.net/ DMTCP]). The SLURM script must be modified to use it efficiently on ~~the B4F HPC~~.		* If the software is really not checkpointable, some software are available to checkpoint existing programs. One of them is DMTCP (Distributed MultiThreaded CheckPointing; [http://dmtcp.sourceforge.net/ DMTCP]). The SLURM script must be modified to use it efficiently on Anunna.

Vande018: /* Recipe */

2015-11-12T13:40:03Z

Recipe

← Older revision		Revision as of 15:40, 12 November 2015
Line 19:		Line 19:

	This recipe is applicable to several languages (e.g., R, Python, Matlab, Fortran, C, shell,...). Also, the checkpointing of parallel programs is easier after a global synchronization.<br />		This recipe is applicable to several languages (e.g., R, Python, Matlab, Fortran, C, shell,...). Also, the checkpointing of parallel programs is easier after a global synchronization.<br />
	The checkpointing requires ~~some efforts~~:<br />		The checkpointing requires:<br />
	* to write some additional code;		* some efforts to write some additional code;
	* some memory (e.g. for a state file). One must be careful to what is saved;		* some memory (e.g. for a state file). One must be careful to what is saved;
	* some time (e.g., to write the state file). Therefore, checkpointing too often must be avoided. Also, SLURM and other software features can be used to checkpoint at some specific points.		* some time (e.g., to write the state file). Therefore, checkpointing too often must be avoided. Also, SLURM and other software features can be used to checkpoint at some specific points.

Vande018: /* Recipe */

2015-11-12T13:39:23Z

Recipe

← Older revision		Revision as of 15:39, 12 November 2015
Line 11:		Line 11:
	Modify the code to implement the following recipe:		Modify the code to implement the following recipe:

	1. Look for a state file that includes all imformation required to restore the state when the program was stopped.~~<br />~~		<source lang=text>
	2. If it exists, read it and restore the state. Else, create an intial state.~~<br />~~		1. Look for a state file that includes all imformation required to restore the state when the program
			was stopped.
			2. If it exists, read it and restore the state. Else, create an intial state.
	3. Periodically save the state to the state file.		3. Periodically save the state to the state file.
			</source>

	This recipe is applicable to several languages (e.g., R, Python, Matlab, Fortran, C, shell,...). Also, the checkpointing of parallel programs is easier after a global synchronization.<br />		This recipe is applicable to several languages (e.g., R, Python, Matlab, Fortran, C, shell,...). Also, the checkpointing of parallel programs is easier after a global synchronization.<br />
Line 20:		Line 23:
	* some memory (e.g. for a state file). One must be careful to what is saved;		* some memory (e.g. for a state file). One must be careful to what is saved;
	* some time (e.g., to write the state file). Therefore, checkpointing too often must be avoided. Also, SLURM and other software features can be used to checkpoint at some specific points.		* some time (e.g., to write the state file). Therefore, checkpointing too often must be avoided. Also, SLURM and other software features can be used to checkpoint at some specific points.


	===The code is not accessible===		===The code is not accessible===

Vande018: /* The code is not accessible */

2015-11-12T13:35:28Z

The code is not accessible

← Older revision		Revision as of 15:35, 12 November 2015
Line 23:		Line 23:

	===The code is not accessible===		===The code is not accessible===
	Many software we used are checkpointable. If it is the case, the (SLURM) script should be adapt to use this advantage. If the software is really not checkpointable, some software are available to checkpoint existing programs. One of them is DMTCP (Distributed MultiThreaded CheckPointing; [http://dmtcp.sourceforge.net/ DMTCP]). The SLURM script must be modified to use it ~~effeciently~~ on the B4F HPC.		Many software we used are checkpointable:<br />
			* If it is the case, the (SLURM) script should be adapt to use this advantage.
			* If the software is really not checkpointable, some software are available to checkpoint existing programs. One of them is DMTCP (Distributed MultiThreaded CheckPointing; [http://dmtcp.sourceforge.net/ DMTCP]). The SLURM script must be modified to use it efficiently on the B4F HPC.

Vande018 at 13:34, 12 November 2015

2015-11-12T13:34:19Z

← Older revision		Revision as of 15:34, 12 November 2015
Line 23:		Line 23:

	===The code is not accessible===		===The code is not accessible===
			Many software we used are checkpointable. If it is the case, the (SLURM) script should be adapt to use this advantage. If the software is really not checkpointable, some software are available to checkpoint existing programs. One of them is DMTCP (Distributed MultiThreaded CheckPointing; [http://dmtcp.sourceforge.net/ DMTCP]). The SLURM script must be modified to use it effeciently on the B4F HPC.

Vande018 at 13:26, 12 November 2015

2015-11-12T13:26:51Z

← Older revision		Revision as of 15:26, 12 November 2015
Line 14:		Line 14:
	2. If it exists, read it and restore the state. Else, create an intial state.<br />		2. If it exists, read it and restore the state. Else, create an intial state.<br />
	3. Periodically save the state to the state file.		3. Periodically save the state to the state file.

			This recipe is applicable to several languages (e.g., R, Python, Matlab, Fortran, C, shell,...). Also, the checkpointing of parallel programs is easier after a global synchronization.<br />
			The checkpointing requires some efforts:<br />
			* to write some additional code;
			* some memory (e.g. for a state file). One must be careful to what is saved;
			* some time (e.g., to write the state file). Therefore, checkpointing too often must be avoided. Also, SLURM and other software features can be used to checkpoint at some specific points.


	===The code is not accessible===		===The code is not accessible===

Vande018 at 13:21, 12 November 2015

2015-11-12T13:21:22Z

← Older revision		Revision as of 15:21, 12 November 2015
Line 1:		Line 1:
	Some recipes and tips for checkpointing on the B4F HPC: ~~[[Media~~:~~checkwiki~~.~~pdf]]~~		Some recipes and tips for checkpointing on the B4F HPC

			==Definition==
			'''Checkpointing''': Saving the program's state at a checkpoint with the aim to restart it from that point in case of (un)planned stop of failure.

			It is interesting for, e.g., long jobs that could be (un)voluntary killed, or jobs running on unstable computing systems.

			==Two types of program==
			===The code is accessible===
			====Recipe====
			Modify the code to implement the following recipe:

			1. Look for a state file that includes all imformation required to restore the state when the program was stopped.<br />
			2. If it exists, read it and restore the state. Else, create an intial state.<br />
			3. Periodically save the state to the state file.

			===The code is not accessible===

Vande018 at 13:09, 12 November 2015

2015-11-12T13:09:56Z

← Older revision		Revision as of 15:09, 12 November 2015
Line 1:		Line 1:
	Some recipes and tips for checkpointing on the B4F HPC		Some recipes and tips for checkpointing on the B4F HPC: [[Media:checkwiki.pdf]]

Vande018: Vande018 moved page Checkpoint to Checkpointing

2015-11-12T13:02:03Z

Vande018 moved page Checkpoint to Checkpointing

← Older revision	Revision as of 15:02, 12 November 2015
(No difference)

Vande018: Created page with "Some recipes and tips for checkpointing on the B4F HPC"

2015-11-12T13:00:39Z

Created page with "Some recipes and tips for checkpointing on the B4F HPC"

New page

Some recipes and tips for checkpointing on the B4F HPC