Setting TMPDIR - Revision history

Prins0891: to

 to fix code blocks

2023-06-15T14:45:48Z

<source> to <pre> to fix code blocks

← Older revision		Revision as of 16:45, 15 June 2023
Line 8:		Line 8:

	First, create a temporary directory on user designated space on on the 'scratch' partition:		First, create a temporary directory on user designated space on on the 'scratch' partition:
	<~~source lang='bash'~~>		<pre>
	mkdir /lustre/scratch/WUR/ABGC/[user]/tmp		mkdir /lustre/scratch/WUR/ABGC/[user]/tmp
	</~~source~~>		</pre>

	Set the <code>TMPDIR</code> variable to point at the custom directory in either <code>~/.bash_profile</code> or <code>~/.bashrc</code> using a command-line editor of choice (vi, emacs, nano, etc), adding the following line of code:		Set the <code>TMPDIR</code> variable to point at the custom directory in either <code>~/.bash_profile</code> or <code>~/.bashrc</code> using a command-line editor of choice (vi, emacs, nano, etc), adding the following line of code:
	<~~source lang='bash'~~>		<pre>
	export TMPDIR=/lustre/scratch/WUR/ABGC/[user]/tmp		export TMPDIR=/lustre/scratch/WUR/ABGC/[user]/tmp
	</~~source~~>		</pre>

	The temporary directory will be set to the custom location once you log in. For the current open terminal to set the new location immediately, without need to log out and in again:		The temporary directory will be set to the custom location once you log in. For the current open terminal to set the new location immediately, without need to log out and in again:
	<~~source lang='bash'~~>		<pre>
	source ~/.bash_profile		source ~/.bash_profile
	</~~source~~>		</pre>

	Note that certain (compiled) apps may in fact ignore the <code>TMPDIR</code> environment variable. These can include Java-apps and c++/g++ compiled apps.		Note that certain (compiled) apps may in fact ignore the <code>TMPDIR</code> environment variable. These can include Java-apps and c++/g++ compiled apps.
Line 27:		Line 27:

	Java applications often do not respond properly to re-setting of the system default by manipulating the <code>TMPDIR</code> environment variable. Java behaviour can be modified by setting the <code>_JAVA_OPTIONS</code> environment variable:		Java applications often do not respond properly to re-setting of the system default by manipulating the <code>TMPDIR</code> environment variable. Java behaviour can be modified by setting the <code>_JAVA_OPTIONS</code> environment variable:
	<~~source lang='bash'~~>		<pre>
	export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/[user]/tmp		export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/[user]/tmp
	</~~source~~>		</pre>
	Note that this does not - strictly speaking - manipulates a temporary directory specific environment variable; rather it sets a Java specific environment variable that allows to pass options to Java on runtime.		Note that this does not - strictly speaking - manipulates a temporary directory specific environment variable; rather it sets a Java specific environment variable that allows to pass options to Java on runtime.

Bohme001 at 11:03, 29 December 2013

2013-12-29T11:03:24Z

← Older revision		Revision as of 13:03, 29 December 2013
Line 1:		Line 1:
	'''Note: all nodes are partioned with a large /tmp (+- 400G)~~, /local is much smaller!~~'''		'''Note: all nodes are partioned with a large /tmp (+- 400G).'''

	Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command <code>sort</code>, part of the <code>bash</code> toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually <code>/tmp</code> on Linux systems. The <code>/tmp</code> partition is however often too limited in size and can get filled up. When this happens all users that require write access to <code>/tmp</code> will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output.		Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command <code>sort</code>, part of the <code>bash</code> toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually <code>/tmp</code> on Linux systems. The <code>/tmp</code> partition is however often too limited in size and can get filled up. When this happens all users that require write access to <code>/tmp</code> will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output.

Bohme001 at 11:03, 29 December 2013

2013-12-29T11:03:01Z

← Older revision		Revision as of 13:03, 29 December 2013
Line 1:		Line 1:
	'''Note: all nodes are partioned with a large /~~local~~ (+- 400G), /~~tmp~~ is much smaller!'''		'''Note: all nodes are partioned with a large /tmp (+- 400G), /local is much smaller!'''

	Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command <code>sort</code>, part of the <code>bash</code> toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually <code>/tmp</code> on Linux systems. The <code>/tmp</code> partition is however often too limited in size and can get filled up. When this happens all users that require write access to <code>/tmp</code> will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output.		Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command <code>sort</code>, part of the <code>bash</code> toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually <code>/tmp</code> on Linux systems. The <code>/tmp</code> partition is however often too limited in size and can get filled up. When this happens all users that require write access to <code>/tmp</code> will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output.

Megen002: /* Overriding the system default - works for many applications (but not all) */

2013-12-28T20:18:28Z

Overriding the system default - works for many applications (but not all)

← Older revision		Revision as of 22:18, 28 December 2013
Line 21:		Line 21:
	source ~/.bash_profile		source ~/.bash_profile
	</source>		</source>

			Note that certain (compiled) apps may in fact ignore the <code>TMPDIR</code> environment variable. These can include Java-apps and c++/g++ compiled apps.

	== Setting custom temporary directory for Java applications ==		== Setting custom temporary directory for Java applications ==

Megen002: /* Setting custom temporary directory for Java applications */

2013-12-28T19:43:22Z

Setting custom temporary directory for Java applications

← Older revision		Revision as of 21:43, 28 December 2013
Line 28:		Line 28:
	export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/[user]/tmp		export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/[user]/tmp
	</source>		</source>
			Note that this does not - strictly speaking - manipulates a temporary directory specific environment variable; rather it sets a Java specific environment variable that allows to pass options to Java on runtime.

	== See also ==		== See also ==

Megen002: /* See also */

2013-12-28T19:39:51Z

Megen002 at 19:39, 28 December 2013

2013-12-28T19:39:22Z

← Older revision		Revision as of 21:39, 28 December 2013
Line 30:		Line 30:

	== See also ==		== See also ==
			[[https://master.hpcagrogenomics.wur.nl/mediawiki/index.php/Main_Page#Being_in_control_of_Environment_parameters \| Controling environment on the Agrogenomics cluster]]
	== External links ==		== External links ==

Megen002: /* Setting custom temporary directory for Java applications */

2013-12-28T19:38:01Z

Setting custom temporary directory for Java applications

← Older revision		Revision as of 21:38, 28 December 2013
Line 26:		Line 26:
	Java applications often do not respond properly to re-setting of the system default by manipulating the <code>TMPDIR</code> environment variable. Java behaviour can be modified by setting the <code>_JAVA_OPTIONS</code> environment variable:		Java applications often do not respond properly to re-setting of the system default by manipulating the <code>TMPDIR</code> environment variable. Java behaviour can be modified by setting the <code>_JAVA_OPTIONS</code> environment variable:
	<source lang='bash'>		<source lang='bash'>
	export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/~~megen002~~/tmp		export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/[user]/tmp
	</source>		</source>

Megen002 at 19:34, 28 December 2013

2013-12-28T19:34:23Z

← Older revision		Revision as of 21:34, 28 December 2013
Line 28:		Line 28:
	export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/megen002/tmp		export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/megen002/tmp
	</source>		</source>

			== See also ==

			== External links ==

Megen002 at 19:33, 28 December 2013

2013-12-28T19:33:59Z

← Older revision		Revision as of 21:33, 28 December 2013
Line 2:		Line 2:

	Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command <code>sort</code>, part of the <code>bash</code> toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually <code>/tmp</code> on Linux systems. The <code>/tmp</code> partition is however often too limited in size and can get filled up. When this happens all users that require write access to <code>/tmp</code> will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output.		Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command <code>sort</code>, part of the <code>bash</code> toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually <code>/tmp</code> on Linux systems. The <code>/tmp</code> partition is however often too limited in size and can get filled up. When this happens all users that require write access to <code>/tmp</code> will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output.

			== Overriding the system default - works for many applications (but not all) ==

	Users can set a custom temporary folder location by setting the <code>TMPDIR</code> environment variable. A custom temporary folder location is best placed in the 'scratch' space on the /lustre filesystem, because this will ensure periodic tidying up of the custom temporary directory thereby reducing the opportunity for very large temporary files that have gone unnoticed to remain on the filesystem. In addition, this is in line with protocols for the default system temporary directory <code>/tmp</code> as this directory is also periodically purged of older files.		Users can set a custom temporary folder location by setting the <code>TMPDIR</code> environment variable. A custom temporary folder location is best placed in the 'scratch' space on the /lustre filesystem, because this will ensure periodic tidying up of the custom temporary directory thereby reducing the opportunity for very large temporary files that have gone unnoticed to remain on the filesystem. In addition, this is in line with protocols for the default system temporary directory <code>/tmp</code> as this directory is also periodically purged of older files.
Line 18:		Line 20:
	<source lang='bash'>		<source lang='bash'>
	source ~/.bash_profile		source ~/.bash_profile
			</source>

			== Setting custom temporary directory for Java applications ==

			Java applications often do not respond properly to re-setting of the system default by manipulating the <code>TMPDIR</code> environment variable. Java behaviour can be modified by setting the <code>_JAVA_OPTIONS</code> environment variable:
			<source lang='bash'>
			export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/megen002/tmp
	</source>		</source>