Using conda to manage your own Python environments

Background

Conda is an open source system for managing Python environments.  A python environment is a version of Python and some associated Python packages.  We highly recommend that researchers create Python environments for projects because it will give them a stable and reproducible place to run their code.   With conda, you can do things like

• install the packages you need without relying on an administrator
• let conda worry about pulling in other packages that a package you want depends upon
• avoid changes to packages because someone else on the system requested them
• maintain different environments for different needs
• clone an environment to experiment with before risking changes to your production environment
• export an environment file that can be use to recreate the environment on a different system

Common conda commands

We are only listing a few of the more useful commands here.  You can add “--help” to the end of any conda command to get some help on how to use it.  See https://conda.io/projects/conda/en/latest/ for complete documentation.

• conda create – make a new environment 
• conda env remove – remove an existing environment
• conda info --envs – show environments
• conda install – add a package to an environment
• conda remove – remove a package from an environment
• conda list – show packages installed in an environment
• conda search – show available packages
• conda activate – activate (use) an environment
• conda deactivate – deactivate an environment

Creating your first conda environment

First enable the conda command.  We recommend putting this into your .bashrc file so it happens automatically every time you login.

There is a bug in the conda command requiring a one time work-around.  If you already have a .conda directory in your home this is unnecessary but it won't hurt anything if you run it again.

When creating a Python environment, you'll definitely want your own Python in it so specify a version at creation time.  That is done with the conda create command and a python=version argument.  version can be

• nothing - you'll get the latest version of Python available in the conda respository
• the major version (i.e. 2 or 3) - you will get the latest available release of that major version
• a specific version (e.g. 3.4.3) - you will get exactly that version

Other packages work the same way.  You may specify multiple packages when you create the environment and of course can also add packages later.  Here is an example which creates an environment named mytest that has python 3.7.2 and the latest version of the numpy package.

Be sure to activate your new environment before trying to use it!

Anaconda environments

Anaconda is a special "meta" package which includes hundreds of other packages.  Anaconda environments are quite common in the research world.  Creating one is no different than creating any other environment.  Pick a version of Python and also include the anaconda meta package.  Like this

Be sure to activate your new environment before using it

Installing conda packages into an existing environment

You use the "conda install" command to do this.

Important: You must activate an environment before installing packages in it (or specify the environment as part of the install command with --name env).

Package specifications work the same way as for "conda create".  You can request multiple packages on the command line and specify (or not) a version for each one.  Here is an example of installing a specific version of the numpy package and whatever is current for the ldap3 package into the "myenv" environment.  Note: part of what conda does is to figure out dependencies so you would also be installing a couple dozen packages required by numpy and ldap3 if you run this command.

How do I see what packages are available?

Use the "conda search" command for this.

To see every version of the package, provide only the package name.  This often gives a very long list. Here is an example searching for the ldap3 package.

$ conda search ldap3
# Name                       Version           Build  Channel             
ldap3                        0.9.8.4          py26_0  pkgs/free           
ldap3                        0.9.8.4          py27_0  pkgs/free           
ldap3                        0.9.8.4          py33_0  pkgs/free
[...]

You can tell conda to return only packages which will work with a particular Python version. Here is an example searching for the ldap3 package but only if it works with Python 3.7.  Note: the '*' in the example command is a wildcard for the ldap3 package version.  You could also specify a particular version of the ldap3 package  though that isn't usually what people want.

$ conda search ldap3=*=py37_0
# Name                       Version           Build  Channel             
ldap3                            2.5          py37_0  pkgs/main           
ldap3                          2.5.1          py37_0  pkgs/main

You can also use wildcards in the package name if you aren't sure what it is called.  This example shows all packages that have "ldap" in the name.

$ conda search *ldap*
# Name                       Version           Build  Channel             
flask-ldap-login               0.3.0          py27_0  pkgs/free           
flask-ldap-login               0.3.0          py27_1  pkgs/free           
ldap3                        0.9.8.4          py26_0  pkgs/free
[...]

What if the package I want isn't available through conda?

First of all, it may be.  There are more conda package repositories (called channels) than the ones we have configured for you by default.  For example, if you wanted the "ads" package, it's available in the conda-forge channel.  You could install it like this.

If you cannot find the package distributed via conda then it is possible to use other kinds of Python packages like "pip", or "easy_install", or "python setup.py" but conda will not know their dependencies so if you update the conda packages in an environment that mixes conda packages with other packaging styles there is a risk of breaking those others (until you manually update them).  In the case of pip, make sure you add the "--user" option so it installs to your home directory.