Set up venv in Python

venv

A Python virtual environment (venv) is an isolated workspace where you can install Python packages without affecting your system-wide Python installation or other projects. Similar to R’s renv, it can be used to captures the specific package versions used in a project, ensuring that code runs consistently over time and across systems

Create a virtual environment

To create a Python virtual environment:

• Open Positron/VS Code/RStudio
  • If using RStudio, ensure Powershell is your designated terminal in Global Options
• Select the Terminal tab
• Navigate to the project’s root directory
• Execute the following command replacing [virtual-environment-name] with the name of the environment.

python -m venv [virtual-environment-name]

# Alternative command for HPC18
py -m venv [virtual-environment-name]

In Positron, creating a virtual environment may result in a message to install ipykernel. Select yes if prompted or install it if not prompted. Also, depending on the installation, the python -m venv command may not work, but the py -m venv will. This could be due to different configurations of how Python was installed on a machine or whether or not its set in the path.

Activate a virtual environment

After a virtual environment is created it must be activated. In R, an renv enabled project will always load the renv in the root working directory. In contrast, a Python virtual environment may need to be activated manually. To activate a Python virtual environment, use the Windows PowerShell to navigate to the root project directory. From there run the Activateps1 file. When the python environment is active, the name of command prompt will change to show the virtual environment name in parentheses¹.

.\py_env\Scripts\Activate.ps1

spawn cmd noent error

Upon restarting Positron, I’ve encountered a spawn cmd noent error and the application fails to activate the virtual environment. To overcome this error, set PATH environment variable to “C:\Windows\System32\” per this post from baibalorenzo.

Package management

Install modules

When a virtual environment is activated all subsequently installed packages will only be available in that environment. To install packages use pip, the package installer for Python in the Terminal tab. This marks as an important difference between R and Python. In R, packages can be installed from the console. In Python, packages may need to be installed from the shell/terminal of an activated environment.

pip install [module-name]

Save modules in a virtual environment to requirements.txt

After installing all desired packages, one can create a “requirements.txt” file. This file records each package and its version which can later be used to install the same set of packages in a different environment. The requirements.txt file can then be modified as need after each additional package is installed.

pip freeze > requirements.txt

Install modules from requirements.txt

If a requirements.txt file is available from a different project using the same packages, that file can be copied and pasted into the virtual environment directory and used to automatically install packages.

pip install -r requirements.txt

Configure venv for Quarto (optional)

When rendering a Quarto document, the default is to use the global system Python library. It may be more desirable to render the Quarto document with a specific virtual environment. To do this, the Quarto installation must be checked and the virtual environment must be registered.

Check Quarto

Our first step is to check the Quarto installation. This will ensure the all packages and dependencies are properly installed for quarto to work. During this check I frequently encounter the need to install PyYAML, nbformat, and nbclient.

quarto check

pip install PyYAML    # Example module that may need to be installed

pip install nbformat  # Example module that may need to be installed

pip install nbclient  # Example module that may need to be installed

Register the virtual environment

To ensure that rendering a Quarto document uses the desired virtual environment instead of the global library, the virtual environment needs to be registered. In a PowerShell terminal, use the following command to see the available registered evironments.

jupyter kernelspec list

A list of “Available kernels:” should appear. If the virtual environment is not listed, then register it with the following command.

python -m ipykernel install --user --name=[venv] --display-name=["venv"]

Configure the YAML header of Quarto document

Lastly, a virtual environment needed to render a Quarto document can be specified with the following command in the YAML header.

---
jupyter: [venv]
---

Footnotes

1. It is possible that the desired virtual environment is activated, but does not appear so because the command prompt does not display the name in parentheses. Positron will display a message in the lower right hand corner of the application when this occurs.