Conda for teaching: Difference between revisions
No edit summary |
|||
(7 intermediate revisions by 2 users not shown) | |||
Line 2: | Line 2: | ||
== Setup == | == Setup == | ||
First - find a good location that everyone can read (and not write). I'd suggest somewhere under /cm/shared/apps/SHARED/ as a starting point - this allows everyone to access this location. It's important not to put anything secret there - it's a public resource, so please bear that in mind. | First - find a good location that everyone can read (and not write). I'd suggest somewhere under <code>/cm/shared/apps/SHARED/</code> as a starting point - this allows everyone to access this location. It's important not to put anything secret there - it's a public resource, so please bear that in mind. | ||
Next - create a folder for your environment: | Next - create a folder for your environment: | ||
< | <pre> | ||
mkdir /cm/shared/apps/SHARED/my_conda_env | mkdir /cm/shared/apps/SHARED/my_conda_env | ||
chmod +r /cm/shared/apps/SHARED/my_conda_env | chmod +r /cm/shared/apps/SHARED/my_conda_env | ||
</ | </pre> | ||
You may want to manipulate the permissions for this folder if someone is going to set this up with you. Consider the commands in [[Shared | You may want to manipulate the permissions for this folder if someone is going to set this up with you. Consider the commands in [[Shared folders]]. | ||
Then, install Anaconda into it (choose the year and month yourself!): | |||
Also, note that you need to make the sh script executable before you can run it. | |||
< | <pre> | ||
wget https://repo.anaconda.com/archive/Anaconda3-YEAR.MONTH-Linux-x86_64.sh | wget https://repo.anaconda.com/archive/Anaconda3-YEAR.MONTH-Linux-x86_64.sh | ||
./Anaconda3-YEAR.MONTH-Linux-x86_64.sh -s -b -p /cm/shared/apps/SHARED/my_conda_env | ./Anaconda3-YEAR.MONTH-Linux-x86_64.sh -s -b -p /cm/shared/apps/SHARED/my_conda_env | ||
</ | </pre> | ||
Now you have a working conda environment in this folder. You can manipulate this here by running /cm/shared/apps/SHARED/my_conda_env/bin/conda , or, I would recommend creating a modulefile so that you can use it as default. | Now you have a working conda environment in this folder. You can manipulate this here by running <code>/cm/shared/apps/SHARED/my_conda_env/bin/conda</code>, or, I would recommend creating a modulefile so that you can use it as default. | ||
Create the following example modulefile in a matching /cm/shared/modulefiles/SHARED/my_conda_env : | Create the following example modulefile in a matching <code>/cm/shared/modulefiles/SHARED/my_conda_env</code>: | ||
< | <pre> | ||
#%Module -*- tcl -*- | #%Module -*- tcl -*- | ||
## | ## | ||
Line 54: | Line 56: | ||
prepend-path CPATH $root/include | prepend-path CPATH $root/include | ||
prepend-path MANPATH $root/share/man | prepend-path MANPATH $root/share/man | ||
</ | </pre> | ||
This will allow you to | This will allow you to <code>module load SHARED/my_conda_env</code> and thus have <code>conda</code> pathed to the currently active environment. | ||
== Jupyter Kernel == | == Jupyter Kernel == | ||
Line 62: | Line 64: | ||
In order for students to be able to use this environment in jupyter, they will need a kernel definition. | In order for students to be able to use this environment in jupyter, they will need a kernel definition. | ||
Kernel definitions are usually a separate folder containing, in particular, a file called <code>kernel.json</code>, plus an icon that is displayed that represents this kernel, and other helper code. | |||
The setup for this is that you should create the following folder for their access: | |||
<pre> | |||
mkdir -p /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env | |||
chmod +r /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env | |||
</pre> | |||
This folder is something they will need to copy in place to their home directory, specifically <code>$HOME/.local/share/jupyter/kernels/</code> | |||
Inside this folder, create the following <code>kernel.json</code> file. Watch out that the paths will need to match the current environment path if you're using a different location! | |||
=== Python Kernel === | |||
<pre> | |||
vim /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env/kernel.json | |||
</source> | |||
<source lang='bash'> | |||
{ | { | ||
"env": { | "env": { | ||
Line 79: | Line 99: | ||
"display_name": "my_conda_env" | "display_name": "my_conda_env" | ||
} | } | ||
</pre> | |||
For Python, that's it. Ipykernel is installed automatically on conda initialisation. | |||
=== R Kernel === | |||
For an R kernel, you need to make sure that the IRkernel package is installed. This is the package that is used to communicate from Jupyter to your running R kernel. | |||
<pre> | |||
#MISSING EXAMPLE CODE | |||
#PROBABLY /cm/shared/apps/SHARED/my_conda_env/bin/conda install R_irkernel | |||
</pre> | |||
You'll also need to create two files: <code>kernel.json</code> and <code>kernel.js</code>. the <code>kernel.js</code> is a helper script to allow jupyter to communicate to R effectively: | |||
<pre> | |||
vim /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env/kernel.json | |||
</pre> | |||
<pre> | |||
{ | |||
"env": { | |||
"LD_LIBRARY_PATH": | |||
"/cm/shared/apps/SHARED/my_conda_env/lib:/cm/shared/apps/SHARED/my_conda_env/lib64" | |||
}, | |||
"argv": ["/cm/shared/apps/SHARED/my_conda_env/bin/R", "--slave", "-e", "IRkernel::main()", "--args", "{connection_file}"], | |||
"display_name": "MAE50806-AdvMolEcol/Sandbox_R", | |||
"language": "R" | |||
} | |||
</pre> | |||
<pre> | |||
vim /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env/kernel.js | |||
</pre> | |||
<pre> | |||
const cmd_key = /Mac/.test(navigator.platform) ? 'Cmd' : 'Ctrl' | |||
const edit_actions = [ | |||
{ | |||
name: 'R Assign', | |||
shortcut: 'Alt--', | |||
icon: 'fa-long-arrow-left', | |||
help: 'R: Inserts the left-assign operator (<-)', | |||
handler(cm) { | |||
cm.replaceSelection(' <- ') | |||
}, | |||
}, | |||
{ | |||
name: 'R Pipe', | |||
shortcut: `Shift-${cmd_key}-M`, | |||
icon: 'fa-angle-right', | |||
help: 'R: Inserts the magrittr pipe operator (%>%)', | |||
handler(cm) { | |||
cm.replaceSelection(' %>% ') | |||
}, | |||
}, | |||
{ | |||
name: 'R Help', | |||
shortcut: 'F1', | |||
icon: 'fa-book', | |||
help: 'R: Shows the manpage for the item under the cursor', | |||
handler(cm, cell) { | |||
const {anchor, head} = cm.findWordAt(cm.getCursor()) | |||
const word = cm.getRange(anchor, head) | |||
const callbacks = cell.get_callbacks() | |||
const options = {silent: false, store_history: false, stop_on_error: true} | |||
cell.last_msg_id = cell.notebook.kernel.execute(`help(\`${word}\`)`, callbacks, options) | |||
}, | |||
}, | |||
] | |||
const prefix = 'irkernel' | |||
function add_edit_shortcut(notebook, actions, keyboard_manager, edit_action) { | |||
const {name, shortcut, icon, help, handler} = edit_action | |||
const action = { | |||
icon, help, | |||
help_index : 'zz', | |||
handler: () => { | |||
const cell = notebook.get_selected_cell() | |||
handler(cell.code_mirror, cell) | |||
}, | |||
} | |||
const full_name = actions.register(action, name, prefix) | |||
Jupyter.keyboard_manager.edit_shortcuts.add_shortcut(shortcut, full_name) | |||
} | |||
function render_math(pager, html) { | |||
if (!html) return | |||
const $container = pager.pager_element.find('#pager-container') | |||
$container.find('p[style="text-align: center;"]').map((i, e) => | |||
e.outerHTML = `\\[${e.querySelector('i').innerHTML}\\]`) | |||
$container.find('i').map((i, e) => | |||
e.outerHTML = `\\(${e.innerHTML}\\)`) | |||
MathJax.Hub.Queue(['Typeset', MathJax.Hub, $container[0]]) | |||
} | |||
define(['base/js/namespace'], ({ | |||
notebook, | |||
actions, | |||
keyboard_manager, | |||
pager, | |||
}) => ({ | |||
onload() { | |||
edit_actions.forEach(a => add_edit_shortcut(notebook, actions, keyboard_manager, a)) | |||
pager.events.on('open_with_text.Pager', (event, {data: {'text/html': html}}) => | |||
render_math(pager, html)) | |||
}, | |||
})) | |||
</pre> | |||
== Last Steps == | |||
In order to help your students get their kernel definitions into <code>$HOME/.local/share/jupyter/kernels/</code>, it's probably a good idea to write a small and simple notebook to do this when executed, or else instruct them to do: | |||
<pre> | |||
cp -rv /cm/shared/apps/SHARED/my_conda_env/kernel/* .local/share/jupyter/kernels/ | |||
</pre> |
Latest revision as of 11:43, 25 January 2024
You are going to give a teaching course, and you need a specific code environment.
Setup
First - find a good location that everyone can read (and not write). I'd suggest somewhere under /cm/shared/apps/SHARED/
as a starting point - this allows everyone to access this location. It's important not to put anything secret there - it's a public resource, so please bear that in mind.
Next - create a folder for your environment:
mkdir /cm/shared/apps/SHARED/my_conda_env chmod +r /cm/shared/apps/SHARED/my_conda_env
You may want to manipulate the permissions for this folder if someone is going to set this up with you. Consider the commands in Shared folders.
Then, install Anaconda into it (choose the year and month yourself!):
Also, note that you need to make the sh script executable before you can run it.
wget https://repo.anaconda.com/archive/Anaconda3-YEAR.MONTH-Linux-x86_64.sh ./Anaconda3-YEAR.MONTH-Linux-x86_64.sh -s -b -p /cm/shared/apps/SHARED/my_conda_env
Now you have a working conda environment in this folder. You can manipulate this here by running /cm/shared/apps/SHARED/my_conda_env/bin/conda
, or, I would recommend creating a modulefile so that you can use it as default.
Create the following example modulefile in a matching /cm/shared/modulefiles/SHARED/my_conda_env
:
#%Module -*- tcl -*- ## ## conda environment modulefile ## set loadedmodules [split $::env(LOADEDMODULES) ":"] set modulepath [split $ModulesCurrentModulefile "/"] set envpath [lrange $modulepath 4 end] set root /cm/shared/apps/[join $envpath "/"] proc ModulesHelp { } { global version puts stderr "\tThis module provides the conda environment at $envpath" } if { [module-info mode] != "whatis" } { puts stderr "[module-info mode] environent $envpath ." } module-whatis "Provides environment $envpath" prepend-path PATH $root/bin prepend-path LD_LIBRARY_PATH $root/lib prepend-path LIBRARY_PATH $root/lib prepend-path CPATH $root/include prepend-path MANPATH $root/share/man
This will allow you to module load SHARED/my_conda_env
and thus have conda
pathed to the currently active environment.
Jupyter Kernel
In order for students to be able to use this environment in jupyter, they will need a kernel definition.
Kernel definitions are usually a separate folder containing, in particular, a file called kernel.json
, plus an icon that is displayed that represents this kernel, and other helper code.
The setup for this is that you should create the following folder for their access:
mkdir -p /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env chmod +r /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env
This folder is something they will need to copy in place to their home directory, specifically $HOME/.local/share/jupyter/kernels/
Inside this folder, create the following kernel.json
file. Watch out that the paths will need to match the current environment path if you're using a different location!
Python Kernel
vim /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env/kernel.json </source> <source lang='bash'> { "env": { "PATH": "/cm/shared/apps/SHARED/my_conda_env/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin" }, "language": "python", "argv": [ "/cm/shared/apps/SHARED/my_conda_env/bin/python", "-m", "ipykernel", "-f", "{connection_file}" ], "display_name": "my_conda_env" }
For Python, that's it. Ipykernel is installed automatically on conda initialisation.
R Kernel
For an R kernel, you need to make sure that the IRkernel package is installed. This is the package that is used to communicate from Jupyter to your running R kernel.
#MISSING EXAMPLE CODE #PROBABLY /cm/shared/apps/SHARED/my_conda_env/bin/conda install R_irkernel
You'll also need to create two files: kernel.json
and kernel.js
. the kernel.js
is a helper script to allow jupyter to communicate to R effectively:
vim /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env/kernel.json
{ "env": { "LD_LIBRARY_PATH": "/cm/shared/apps/SHARED/my_conda_env/lib:/cm/shared/apps/SHARED/my_conda_env/lib64" }, "argv": ["/cm/shared/apps/SHARED/my_conda_env/bin/R", "--slave", "-e", "IRkernel::main()", "--args", "{connection_file}"], "display_name": "MAE50806-AdvMolEcol/Sandbox_R", "language": "R" }
vim /cm/shared/apps/SHARED/my_conda_env/kernel/my_conda_env/kernel.js
const cmd_key = /Mac/.test(navigator.platform) ? 'Cmd' : 'Ctrl' const edit_actions = [ { name: 'R Assign', shortcut: 'Alt--', icon: 'fa-long-arrow-left', help: 'R: Inserts the left-assign operator (<-)', handler(cm) { cm.replaceSelection(' <- ') }, }, { name: 'R Pipe', shortcut: `Shift-${cmd_key}-M`, icon: 'fa-angle-right', help: 'R: Inserts the magrittr pipe operator (%>%)', handler(cm) { cm.replaceSelection(' %>% ') }, }, { name: 'R Help', shortcut: 'F1', icon: 'fa-book', help: 'R: Shows the manpage for the item under the cursor', handler(cm, cell) { const {anchor, head} = cm.findWordAt(cm.getCursor()) const word = cm.getRange(anchor, head) const callbacks = cell.get_callbacks() const options = {silent: false, store_history: false, stop_on_error: true} cell.last_msg_id = cell.notebook.kernel.execute(`help(\`${word}\`)`, callbacks, options) }, }, ] const prefix = 'irkernel' function add_edit_shortcut(notebook, actions, keyboard_manager, edit_action) { const {name, shortcut, icon, help, handler} = edit_action const action = { icon, help, help_index : 'zz', handler: () => { const cell = notebook.get_selected_cell() handler(cell.code_mirror, cell) }, } const full_name = actions.register(action, name, prefix) Jupyter.keyboard_manager.edit_shortcuts.add_shortcut(shortcut, full_name) } function render_math(pager, html) { if (!html) return const $container = pager.pager_element.find('#pager-container') $container.find('p[style="text-align: center;"]').map((i, e) => e.outerHTML = `\\[${e.querySelector('i').innerHTML}\\]`) $container.find('i').map((i, e) => e.outerHTML = `\\(${e.innerHTML}\\)`) MathJax.Hub.Queue(['Typeset', MathJax.Hub, $container[0]]) } define(['base/js/namespace'], ({ notebook, actions, keyboard_manager, pager, }) => ({ onload() { edit_actions.forEach(a => add_edit_shortcut(notebook, actions, keyboard_manager, a)) pager.events.on('open_with_text.Pager', (event, {data: {'text/html': html}}) => render_math(pager, html)) }, }))
Last Steps
In order to help your students get their kernel definitions into $HOME/.local/share/jupyter/kernels/
, it's probably a good idea to write a small and simple notebook to do this when executed, or else instruct them to do:
cp -rv /cm/shared/apps/SHARED/my_conda_env/kernel/* .local/share/jupyter/kernels/