Setting TMPDIR: Difference between revisions
No edit summary |
No edit summary |
||
| Line 2: | Line 2: | ||
Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command <code>sort</code>, part of the <code>bash</code> toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually <code>/tmp</code> on Linux systems. The <code>/tmp</code> partition is however often too limited in size and can get filled up. When this happens all users that require write access to <code>/tmp</code> will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output. | Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command <code>sort</code>, part of the <code>bash</code> toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually <code>/tmp</code> on Linux systems. The <code>/tmp</code> partition is however often too limited in size and can get filled up. When this happens all users that require write access to <code>/tmp</code> will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output. | ||
== Overriding the system default - works for many applications (but not all) == | |||
Users can set a custom temporary folder location by setting the <code>TMPDIR</code> environment variable. A custom temporary folder location is best placed in the 'scratch' space on the /lustre filesystem, because this will ensure periodic tidying up of the custom temporary directory thereby reducing the opportunity for very large temporary files that have gone unnoticed to remain on the filesystem. In addition, this is in line with protocols for the default system temporary directory <code>/tmp</code> as this directory is also periodically purged of older files. | Users can set a custom temporary folder location by setting the <code>TMPDIR</code> environment variable. A custom temporary folder location is best placed in the 'scratch' space on the /lustre filesystem, because this will ensure periodic tidying up of the custom temporary directory thereby reducing the opportunity for very large temporary files that have gone unnoticed to remain on the filesystem. In addition, this is in line with protocols for the default system temporary directory <code>/tmp</code> as this directory is also periodically purged of older files. | ||
| Line 18: | Line 20: | ||
<source lang='bash'> | <source lang='bash'> | ||
source ~/.bash_profile | source ~/.bash_profile | ||
</source> | |||
== Setting custom temporary directory for Java applications == | |||
Java applications often do not respond properly to re-setting of the system default by manipulating the <code>TMPDIR</code> environment variable. Java behaviour can be modified by setting the <code>_JAVA_OPTIONS</code> environment variable: | |||
<source lang='bash'> | |||
export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/megen002/tmp | |||
</source> | </source> | ||
Revision as of 20:33, 28 December 2013
Note: all nodes are partioned with a large /local (+- 400G), /tmp is much smaller!
Many programs require writing intermediary or temporary information. A process may even require writing to a temporary location without a user knowing it. For instance, the command sort, part of the bash toolkit, requires a lot of temporary file space when sorting large volumes of data. Often programs take the system default, which is usually /tmp on Linux systems. The /tmp partition is however often too limited in size and can get filled up. When this happens all users that require write access to /tmp will experience problems in running jobs, which can range from unexpected quitting of processes, to erroneous output.
Overriding the system default - works for many applications (but not all)
Users can set a custom temporary folder location by setting the TMPDIR environment variable. A custom temporary folder location is best placed in the 'scratch' space on the /lustre filesystem, because this will ensure periodic tidying up of the custom temporary directory thereby reducing the opportunity for very large temporary files that have gone unnoticed to remain on the filesystem. In addition, this is in line with protocols for the default system temporary directory /tmp as this directory is also periodically purged of older files.
First, create a temporary directory on user designated space on on the 'scratch' partition: <source lang='bash'> mkdir /lustre/scratch/WUR/ABGC/[user]/tmp </source>
Set the TMPDIR variable to point at the custom directory in either ~/.bash_profile or ~/.bashrc using a command-line editor of choice (vi, emacs, nano, etc), adding the following line of code:
<source lang='bash'>
export TMPDIR=/lustre/scratch/WUR/ABGC/[user]/tmp
</source>
The temporary directory will be set to the custom location once you log in. For the current open terminal to set the new location immediately, without need to log out and in again: <source lang='bash'> source ~/.bash_profile </source>
Setting custom temporary directory for Java applications
Java applications often do not respond properly to re-setting of the system default by manipulating the TMPDIR environment variable. Java behaviour can be modified by setting the _JAVA_OPTIONS environment variable:
<source lang='bash'>
export _JAVA_OPTIONS=-Djava.io.tmpdir=/lustre/scratch/WUR/ABGC/megen002/tmp
</source>