Creating and Using Python Virtual Environments on Clipper

Python virtual environments are the recommended way to manage Python projects on the Clipper HPC cluster.

Virtual environments are isolated Python installations that allow for easy management of project-specific dependencies. Virtual environments allow individuals to create a tailored environment without affecting the global Python installation available to all Clipper users.

This document describes how to create a Python virtual environment on Clipper.

Creating Virtual Environments

Private virtual environments can be stored either in your /home folder or project space allocated on /active.

To create a virtual environment, use the built-in venv Python module. This example installs a virtual environment into a hypothetical /home/hpcuser1 directory.

Navigate to the directory in which you wish to create the virtual environment:

[hpcuser1@clipper ~]$ cd /home/hpcuser1

Create the virtual environment with the venv module:

[hpcuser1@clipper ~]$ python3 -m venv myproject

This will create a directory named myproject containing the virtual environment’s files inside of /home/hpcuser1.

Using Virtual Environments

Activation and Deactivation

Virtual environments must be activated. Activating a virtual environment makes the isolated Python interpreter and packages available in your shell.

To activate the earlier-created environment, run:

[hpcuser1@clipper ~]$ source myproject/bin/activate

Once activated, your shell prompt will change to indicate you are in the virtual environment:

(myproject) [hpcuser1@clipper ~]$

To deactivate the environment and return to the system default Python environment, type:

deactivate

Installing Packages

Use the pip package manager to install packages inside the virtual environment. Once you have activated the environment, run:

pip install <package>

For example, to install the BeautifulSoup library, run:

pip install beautifulsoup4

pip will handle the installation of Beautiful Soup and all necessary dependencies automatically.

Packages can also be removed from the virtual environment:

pip uninstall <package>

Best Practices

• Create a separate virtual environment for each project. It is important to isolate each project and prevent conflicts between packages.

• If you are using Git for managing your project’s assets, include the virtual environment’s directory in the .gitignore file. This stops the virtual environment from being committed to your repository.

Python Version Note

Clipper inherits the Python version of its underlying operating system, Red Hat Enterprise Linux 9. As of November 6, 2023, the Python version available on Clipper is 3.9.16.

Should you need a newer version of Python, please contact Academic Research Computing support with your request and justification.