|
|
| (10 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| With many Python packages available, which are often in conflict or requiring different versions depending on application, installing and controlling packages and versions is not always easy. In addition, so many packages are often used only occasionally, that it is questionable whether a system administrator of a centralized server system or a High Performance Compute (HPC) infrastructure can be expected to resolve all issues posed by users of the infrastructure. Even on a local system with full administrative rights managing versions, dependencies, and package collisions is often very difficult. The solution is to use a virtual environment, in which a specific set of packages can then be installed. As many different virtual environments can be created, and used side-by-side, as is necessary.
| | #REDIRECT [[Python#Virtual environments]] |
| | |
| | |
| == creating a new virtual environment ==
| |
| | |
| If you do not already have a directory in your <code>)$HOME</code>) dir where your virtual environments live, first make one (it is assumed that you will over the course of time create several virtual environments for different projects and different versions of Python side-by-side, best to organise them a bit).
| |
| | |
| <source lang='bash'>
| |
| mkdir ~/my_envs
| |
| </source>
| |
| | |
| Then, load either Python 3.4 or 3.5 module (Python 3.3.3 should also work):
| |
| | |
| <source lang='bash'>
| |
| module load python/3.5.0
| |
| </source>
| |
| And then simply create an environment with a reasonably descriptive name (remember, you may accumulate as many as you desire), in this example <code>)p35_myproj</code>).
| |
| | |
| <source lang='bash'>
| |
| pyvenv install ~/my_envs/p35_myproj
| |
| </source>
| |
| | |
| | |
| == activating a virtual environment ==
| |
| Once the environment is created, each time the environment needs to be activated, the following command needs to be issued:
| |
| <source lang='bash'>
| |
| source ~/my_envs/p35_myproj/bin/activate
| |
| </source>
| |
| | |
| This assumes that the folder that contains the virtual environment documents (in this case called <code>newenv</code>), is in the present working directory.
| |
| When working on the virtual environment, the virtual environment name will be between brackets in front of the <code>user-host-prompt</code> string.
| |
| <code>
| |
| (p35_myproj)user@host:~$
| |
| </code>
| |
| | |
| == installing modules on the virtual environment ==
| |
| Installing modules is the same as usual. The difference is that modules are in <code>/path/to/virtenv/lib</code>, which may be living somewhere on your home directory. An easy way of installing modules is using <code>)pip</code>).
| |
| | |
| Before you start installing modules, first update pip itself:
| |
| <source lang='bash'>
| |
| pip install --upgrade pip
| |
| </source>
| |
| | |
| you can then install other modules as you like, for instance numpy:
| |
| | |
| <source lang='bash'>
| |
| pip install numpy
| |
| </source>
| |
| | |
| <code>
| |
| (p35_myproj) [user@nfs01 ~]$ pip install numpy
| |
| Collecting numpy
| |
| Using cached numpy-1.10.4.tar.gz
| |
| Installing collected packages: numpy
| |
| Running setup.py install for numpy ... done
| |
| Successfully installed numpy-1.10.4
| |
| </code>
| |
| | |
| Similarly, installing packages from source works exactly the same as usual (note: only relevant for modules that can't be pulled through <code>pip</code>).
| |
| <source lang='bash'>
| |
| python setup.py install
| |
| </source>
| |
| | |
| == deactivating a virtual environment ==
| |
| Quitting a virtual environment can be done by using the command <code>deactivate</code>, which was loaded using the <code>source</code> command upon activating the virtual environment.
| |
| <source lang='bash'>
| |
| deactivate
| |
| </source>
| |
| == Make IPython work under virtualenv ==
| |
| IPython may not work initially under a virtual environment. It may produce an error message like below:
| |
| | |
| <code>
| |
| File "/usr/bin/ipython", line 11
| |
| print "Could not start qtconsole. Please install ipython-qtconsole"
| |
| ^
| |
| </code>
| |
| | |
| This can be resolved by adding a soft link with the name <code>ipython</code> to the <code>bin</code> directory in the virtual environment folder.
| |
| <source lang='bash'>
| |
| ln -s /path/to/virtenv/bin/ipython3 /path/to/virtenv/bin/ipython
| |
| </source>
| |
| | |
| == See also ==
| |
| * [[B4F_cluster | B4F Cluster]]
| |
| | |
| == External links ==
| |
| * [https://pypi.python.org/pypi/virtualenv Python3 documentation for virtualenv]
| |
| * [http://cemcfarland.wordpress.com/2013/03/09/getting-ipython3-working-inside-your-virtualenv/ Solving the IPython hickup under virtual environment]
| |