Python Working Environment for Beginners: Part 2

Part 2 of Python Working Environment. For ways to set Python Working Environment, check out Part 1 here.

Creating Virtual Environments

Understanding Virtual Environments

As we saw in the Installing Packages section , when we install third-party dependencies using pip (or easy_install), they are installed in the <python-base-dir>/lib/python<major>.<minor>/site-packages directory. But, what if two different applications (hosted on the same machine) need two different versions of the same package? Virtual Environments solve this problem by isolating one application's environment from another. These environments are based on a Python installation and can be considered as light-weight containers for holding application-specific dependencies. Virtual environments also come in handy when we do not have write permissions to the Python installation's site-packages directory.

Python virtual environments were originally provided by “virtualenv” package. Starting from Python 3.3, virtual environments have been added to Python language in the module “venv”. In this post, we will see how to create virtual environments using “virtualenv” as well as Python’s out-of-the-box virtual environments provided using “venv”.

Virtual Environments using "virtualenv"

“virtualenv” creates an isolated Python environment in a directory structure which contains the “site-packages” directory. When we activate the virtual environment and install packages, the packages are placed in the virtual environment's “site-packages” directory instead of Python Installation's site-packages directory.

Let us install virtualenv and try to create a virtual environment using it.

# Update PATH to include Python Installation's bin directory
$ export PATH= <python-base-dir>/bin:$PATH
# Install virtualenv using pip.
$ pip install virtualenv

After the installation of “virtualenv”, we can create a new virtual environment in an empty / non-existent directory using “virtualenv”.

$ virtualenv my-project-env

The command virtualenv does the below important things:

• Creates “python”, “easy_install” and “pip” executables for the virtual environment in “bin” directory
• Creates a “lib” directory to hold all the libraries and a lib/python<major>.<minor>/site-packages directory to hold the third-party dependencies.
• Creates an activate script in the “bin” directory which is used to bring the virtual environment into effect.

Let us now proceed to activate the environment and install a package.

# Activate the Python Environment. This also changes the shell prompt to indicate the environment.
$ cd my-project-env
$ source bin/activate

Observe that the shell prompt is changed to (my-project-env) indicating that the virtual environment is activated.

Let us now try to install “requests” package in the virtual environment and notice that it is installed in the “site-packages” of the virtual environment and isolated from the Python installation. 

# Install “requests” package in the virtual environment.
(my-project-env) $ pip install requests
# Verify that “requests” is installed in virtual environment using “pip show” command.
(my-project-env) $ pip show requests

Once you are done working with the virtual environment, you can deactivate the virtual environment using the function call 'deactivate'.

(my-project-env) $ deactivate

Virtual Environments using "venv"

Virtual environments have been added to Python language from Python 3.3 in the module “venv”. The “pyvenv” executable helps you create the virtual environments. The detailed documentation can be found here. One important difference in Python 3.3 virtual environments from those created by “virtualenv” package is that the binaries “pip” and “easy_install” are not copied over to “bin” directory, because of which, we should install these packages additionally in the virtual environment. However, with Python 3.4.1, this issue has been fixed and "pip" executable is made available in the bin directory of the virtual environment for our ready use.

Let us try to create a virtual environment using "pyvenv" executable, install "requests" package and verify that the packages are installed in the virtual environment.

# Create a virtual environment in the directory my-project-env.
$ pyvenv my-project-venv
# Change to the virtual environment directory.
$ cd my-project-venv
# Activate the virtual environment.
$ source bin/activate
# Install “requests” package.
(my-project-venv) $ pip install requests
# Observe that “requests” is installed in the virtual environment using “pip show” command.
(my-project-venv) $ pip show requests

Once you are done working with the virtual environment, you can deactivate the virtual environment using the function call 'deactivate'.

(my-project-env) $ deactivate

Summary

In a nutshell, setting up basic Python environments involves installing the required version of Python, creating virtual environments for your project based on the Python installation and installing the required third-party dependencies using “pip”.

I hope this post helps you understand how to set up basic Python environments and gives an overview of the ecosystem around packages and their management using virtual environments.

References

• Installing Python from Sources
• Setuptools documentation
• pip documentation
• Why use pip over easy_install - Stack Overflow