Old binaries - Revision history

Dawes0011 at 09:27, 19 June 2023

2023-06-19T09:27:57Z

← Older revision		Revision as of 11:27, 19 June 2023
Line 8:		Line 8:

	There is a special environment variable: <code>LD_LIBRARY_PATH</code> which tells the linker (ld) where extra to go find libraries. We use this extensively in the underlying module system on Anunna to control which libraries can be used. One such module, <code>sl7_libs</code> automatically adds the older libraries in to older binaries, and has been added as a prerequisite for most older installs.		There is a special environment variable: <code>LD_LIBRARY_PATH</code> which tells the linker (ld) where extra to go find libraries. We use this extensively in the underlying module system on Anunna to control which libraries can be used. One such module, <code>sl7_libs</code> automatically adds the older libraries in to older binaries, and has been added as a prerequisite for most older installs.


	If you've compiled something yourself that isn't a module, you may find yourself needing this module. Simply:		If you've compiled something yourself that isn't a module, you may find yourself needing this module. Simply:

	<code>		<code>
	module load sl7-libs/main		module load sl7-libs/main
Line 38:		Line 40:

	When editing JSON, please be mindful that all elements in an object must be separated by commas, and that only doublequotes may be used.		When editing JSON, please be mindful that all elements in an object must be separated by commas, and that only doublequotes may be used.


	To test your JSON, try the following:		To test your JSON, try the following:

Dawes0011 at 09:27, 19 June 2023

2023-06-19T09:27:26Z

← Older revision		Revision as of 11:27, 19 June 2023
Line 40:		Line 40:

	To test your JSON, try the following:		To test your JSON, try the following:

	<code>		<code>
	cat ~/.local.share/jupyter/kernels/mykernel/kernel.json \| jq		cat ~/.local.share/jupyter/kernels/mykernel/kernel.json \| jq

Dawes0011: Created page with "On Linux, most binaries rely on shared libraries that are loaded by the OS before the binary is executed. This allows them to share common runtimes, and avoid having to redo the same work multiple times. This is useful for optimisation and security, as debugging only has to be done in one place for multiple applications. However, a significant number of these libraries are automatically inserted by the OS, and as the cluster upgrades and changes some of these will no l..."

2023-06-19T09:27:09Z

Created page with "On Linux, most binaries rely on shared libraries that are loaded by the OS before the binary is executed. This allows them to share common runtimes, and avoid having to redo the same work multiple times. This is useful for optimisation and security, as debugging only has to be done in one place for multiple applications. However, a significant number of these libraries are automatically inserted by the OS, and as the cluster upgrades and changes some of these will no l..."

New page

On Linux, most binaries rely on shared libraries that are loaded by the OS before the binary is executed.

This allows them to share common runtimes, and avoid having to redo the same work multiple times. This is useful for optimisation and security, as debugging only has to be done in one place for multiple applications.

However, a significant number of these libraries are automatically inserted by the OS, and as the cluster upgrades and changes some of these will no longer be automatically available. This will make older installs fail to execute, giving, for instance, errors related to glibc, or missing symbols.

Thankfully, the entire library from previous OS builds is made available on the shared filesystem, granting older binaries the ability to access them whilst also running on a younger OS.

There is a special environment variable: <code>LD_LIBRARY_PATH</code> which tells the linker (ld) where extra to go find libraries. We use this extensively in the underlying module system on Anunna to control which libraries can be used. One such module, <code>sl7_libs</code> automatically adds the older libraries in to older binaries, and has been added as a prerequisite for most older installs.

If you've compiled something yourself that isn't a module, you may find yourself needing this module. Simply:
<code>
module load sl7-libs/main
</code>

To add this path.

== Jupyter ==

This may also extend to custom-built kernels on Jupyter. Here, however, you can't add this module, as instantiating a kernel is done though <code>kernel.json</code> instead of bash.

To add the needed libraries, you need to modify your notebook to look like the following:
<code>
{
"language": "python",
"argv": [
"/path/to/my/venv/bin/python",
"-m",
"ipykernel",
"-f",
"{connection_file}"
],
"display_name": "myvenv",
"env" : {"LD_LIBRARY_PATH": "/usr/lib:/usr/lib64:/usr/lib/x86_64-linux-gnu:/shared/legacyapps/sl7-libs/lib:/shared/legacyapps/sl7-libs/lib64" }
}
</code>

When editing JSON, please be mindful that all elements in an object must be separated by commas, and that only doublequotes may be used.

To test your JSON, try the following:
<code>
cat ~/.local.share/jupyter/kernels/mykernel/kernel.json | jq
</code>

If <code>jq</code> repeats your JSON, then it can parse it and you are good to go.