For those starting in Python, setting up Python Environments in Projects could prove to be tricky as there are a lot of tools and terms in the Python ecosystem to be familiar with, in order to understand the process. This post aims to make it easy for beginners to create basic Python environments by giving step-by-step guidance.
Setting up basic Python Environments involves the below three steps.
- Python Installation
- Installing Packages
- Creating Virtual Environments
We will see how to go about the above in detail in the sections below.
Python Installation
The first step in building Python Working Environments is to install Python itself. Python can be installed either using an already available installer or by building from sources.
Using an installer
Python releases are available on Python's Official website - Downloads. You can select the required version ( For this post, I am using the latest version available at this time, Python 3.4.1) and proceed to the Download page which has links to installers for supported operating systems. Download the installer corresponding to your OS and run it.
On successful installation,
- Installed files will be placed in OS-specific default directories.
- On Windows, the installation directory’s location can be specified during installation.
- On Mac OS, the installation directory’s location is /Library/Frameworks/Python.framework/Versions/<major>.<minor> where major is the major release number and minor is the minor release number. In the case of installation of 3.4.1, the installation location would be /Library/Frameworks/Python.framework/Versions/3.4.
- Inside the Python installation directory,
- On Windows, the python executable (python.exe) is available directly in the installation directory and the Lib directory holds the site-packages sub-directory which is the default destination for third-party dependency installs (which we are going to see in the next section).
- On Mac OS, the bin directory contains the python executables (python3 / python3.4) and the lib directory holds the site-packages sub-directory which is the default destination for third-party dependency installs.
Note: We will refer to the Python installation directory as <python-base-dir> in the commands.
After the installation, you can verify if the installation is successful by running Python executable.
On Mac OS
### Testing Python installation
# Navigate to <python-base-dir>/bin
$ cd /Library/Frameworks/Python.framework/Versions/<major>.<minor>/bin
$ python3 -V
# Current installed version should be displayed
Python 3.4.1
On Mac OS, you can create a symbolic link for the Python executable in /usr/local/bin pointing to the python3 / python3.4 executable in the bin directory so that python executable is available in common path.
### Creating Symbolic link for python executable $ ln -s /Library/Frameworks/Python.framework/Versions/3.4/bin/python3 /usr/local/bin/python
On Windows
### Testing Python installation # Navigate to <python-base-dir> $ cd <python-base-dir> $ python.exe -V # Current installed version should be displayed Python 3.4.1
Installing From Sources
You may want to install Python from sources for two reasons.
- There is no supported installer for your operating system (like any flavors of Unix / Linux).
- You want to have more control on the Python installation by fine tuning the installation directories for different kinds of files like documentation, user executables.
- On Mac OS, you do not want to install Python in the System Libraries as forced by the installer.
The version release page (For 3.41, it is Python 3.4.1 Release Page) also contains links to the Python source distribution under the "Files" section. Click on “Gzipped source tarball” link and download and extract the tarball into a directory.
Customizing the Installation
After unzipping the source tarball, we can use the “configure” script in the extracted directory to customize the installation (On Windows, you can use cygwin / any other terminal emulator software to work with the shell scripts). “configure” script lets you configure the installation as per your needs. It primarily gives you fine-grained control over the different directories into which different types of files (like user executables, html documentation) will be installed. It also lets you change the different language features that you want to install. For the list of different options, please see "./configure --help".
### Configuring Python installation’s base directory $ ./configure --help
If we do not want to change the different installation directories, rather just want to provide a root directory for the Python installation (with respect to which the different directories would be installed), we can use the “--prefix” option. If no options are used with “configure” script, Python installs all packages and files in the /usr/local/bin/python<major.minor> and /usr/local/lib/python<major.minor> directories. (This location is specified when you run “configure” script with --help option)
### Configuring Python installation’s base directory # Run `configure` script with --prefix option pointing to a # location of your choice for Python installation's base # directory $ ./configure --prefix=<python-base-dir>
Now that we have configured our installation, the next step is to install Python using “make install”. You may have to run this command using “sudo” on Mac OS / Unix / Linux if you do not have sufficient permissions.
$ make install
This completes the installation of Python in the directory <python-base-dir>. There will be four sub-directories created under <python-base-dir>, "bin", "lib", "share" and "include". “python<major>” executable will reside in the <python-base-dir>/bin directory. The “site-packages” directory for third-party dependencies can be located at <python-base-dir>/lib/python<major>.<minor>/site-packages.
As in the case of installation using installer, you can verify the installation by running the python executable.
### Testing the Python installation $ cd <python-base-dir>/bin $ ./python3 -V # Current installed version should be displayed Python 3.4.1
You can create a symbolic link “python” for “python<major>” executable in the <python-base-dir>.
### Creating Symbolic link for python executable $ ln -s <python-base-dir>/bin/python3 <python-base-dir>/bin/python
Installing Packages
Understanding Modules and Packages
Modules and Packages are Python's way of maintaining reusable code. You can think of them as jars in Java or gems in Ruby. A module is the basic unit of code re-usability in Python.
There are three types of modules in Python.
- pure Python modules - modules contained in a single .py file
- extension modules - modules written in low-level languages like C and have extensions such as .so, .pyd
- packages - a directory in file system having a special file __init__.py (that makes it recognizable as a package by Python interpreter) containing Python modules or packages
The third-party dependencies required by your project are usually packages and will mostly be available at Python Package Index (PyPI) which is an official repository for third-party libraries in Python.
Understanding Packaging Libraries
There are some libraries that help in packaging Python modules.The primary task of these libraries is to define a contract to the developer to specify the package metadata, and provide means of packaging the project like creating a source distribution.
There are three popular packaging libraries for Python, namely,
- distutils
- setuptools
- distribute
distutils is the native packager that comes with Python while setuptools and distribute are enhancements over it. The history of these libraries is an interesting read.
For all practical purposes today, setuptools is the preferred tool for packaging.
Installing Packages from source distributions
You can install most of the Python packages from their source distributions using distutils' basic packaging commands. After downloading the source distribution and unzipping it, you will find a setup.py file in the root directory. You should run "python setup.py install" in the root directory to install the package. The package will be installed in the Python installation's site-packages directory.
Installing Packages from PyPI
Python Package Index (PyPI) is an official repository for Python packages and hosts most of the packages you may require for your project. To install packages from PYPI you could use one of the two package managers, pip or easy_install. These two libraries are helpful for downloading, installing, and managing packages and their dependencies. “pip” is the more widely used between the two. Here is an excerpt from pip documentation about it's advantages over easy_install. ( For more information, please see pip documentation )
- Native support for other version control systems (Git, Mercurial and Bazaar)
- Uninstallation of packages.
- Simple to define fixed sets of requirements and reliably reproduce a set of packages.
Before we install packages using “pip”, we should check if “pip” is already installed in <python-base-dir>/bin as “pip<major>” (“pip” is being installed out-of-the-box with the recent versions of Python). If so, we can create a symbolic link “pip” in the bin directory pointing to “pip<major>” and go ahead with installing packages. If “pip” is not already installed, the installation can be done with the help of “get-pip.py”. The below steps show how to install “pip”.
### Install pip # Update PATH Variable to include <python-base-dir>/bin while searching for commands $ export PATH=<python-base-dir>/bin:$PATH $ curl -O https://bootstrap.pypa.io/get-pip.py $ python get-pip.py
You can create a symbolic link “pip” pointing to “pip<major” executable, if it is not already available in <python-base-dir>/bin.
### Create symbolic link “pip” $ ln -s <python-base-dir>/bin/pip<major> <python-base-dir>/bin/pip
After “pip” is installed, we can proceed to use it to install Python packages.
### Install requests package using “pip”. # pip looks for the most recent version of “requests” on PYPI that is compatible with the current version of Python. $ pip install requests --verbose # The below command prints the version of installed “requests” package. $ pip list | grep requests requests (2.4.1) # Now, let us try to uninstall “requests” and try to install a different version of it. $ pip uninstall requests --verbose # Now install a different version of requests $ pip install requests==2.2.1 # Look up the installed version $ pip list | grep requests requests (2.2.1)
Note that the “requests” package is installed into the "site-packages" directory of <python-base-dir>.
Projects generally use a file to keep track of all the third-party dependencies. All the third-party dependencies for the project can be installed with “pip install -r <requirements-file>”. For more information, please see pip documentation about requirements
What's next?
Read Python Working Environment in part 2 of this article.