Skip to content

Project Makefile using plain Pip

Michael Goerz edited this page Nov 18, 2023 · 3 revisions
.PHONY: help init jupyter-notebook jupyter-lab clean clean-venv distclean

PYTHON = .venv/bin/python

help:   ## Show this help
	@grep -E '^([a-zA-Z_-]+):.*## ' $(MAKEFILE_LIST) | awk -F ':.*## ' '{printf "%-20s %s\n", $$1, $$2}'

$(PYTHON): requirements.txt
	python3 -m venv .venv
	$@ -m pip install --upgrade pip
	$@ -m pip install -r requirements.txt
	$@ -m pip freeze > requirements-frozen.tex
	@touch $@  # mark updated

init: $(PYTHON)  ## Create the virtual project environment

jupyter-notebook: $(PYTHON)  ## Run a Jupyter notebook server
	jupyter notebook --debug --no-browser

jupyter-lab: $(PYTHON)  ## Run a Jupyter lab server
	jupyter lab --debug --no-browser

clean: ## Remove generated files

clean-venv: ## Remove environment
	rm -rf .venv

distclean: clean  clean-venv  ## Restore clean repository state
	rm -rf *.ipynb  # if using jupytext
	rm -f requirements-frozen.txt
	rm -rf .ipynb_checkpoints

Notes:

  • The help target could also be implemented with a Python script
  • The requirements.txt file must include ipykernel (the "project kernel")
  • The requirements-frozen.txt file is for informational purposes only, to keep track of the actually installed package versions for both direct and indirect dependencies.
  • The jupyter executable is not (and should not be) part of the .venv. Instead, there should be a single "site" installation of Jupyter.
  • The Jupyter installation should have the Jupytext extension installed. This allows to commit notebooks as small (and diffable) .py files.
  • To make the project kernel available to the site Jupyter installation, make sure that python-localvenv-kernel is installed into the same environment as Jupyter. All notebooks should use this kernel.
  • The clean target should be used to remove, e.g., __pycache__ files, if applicable.
  • The distclean target should remove *.ipynb files only if the Jupytext extension is used. Note that this removes the output of all notebooks. If your notebooks take a long time to run, take steps as to not accidentally delete the notebooks.

With an installable kernel

If it is not possible to install python-localvenv-kernel, the project kernel must be installed into the user's Jupyter data directory. To this end, we can add install and uninstall targets:

.PHONY: help init install uninstall jupyter-notebook jupyter-lab clean clean-venv distclean

PYTHON = .venv/bin/python

help:   ## Show this help
	@grep -E '^([a-zA-Z_-]+):.*## ' $(MAKEFILE_LIST) | awk -F ':.*## ' '{printf "%-20s %s\n", $$1, $$2}'

$(PYTHON): requirements.txt
	python3 -m venv .venv
	$@ -m pip install --upgrade pip
	$@ -m pip install -r requirements.txt
	$@ -m pip freeze > requirements-frozen.txt
	@touch $@  # mark updated

init: $(PYTHON)  ## Create the virtual project environment

install: $(PYTHON) ## Install Jupyter kernel into user environment
	$< -m ipykernel install --user --name my-project-name --display-name "My Project Name"

uninstall: ## Remove Jupyter kernel from user environment
	jupyter kernelspec remove -f my-project-name || true

jupyter-notebook: install  ## Run a Jupyter notebook server
	jupyter notebook --debug --no-browser

jupyter-lab: install  ## Run a Jupyter lab server
	jupyter lab --debug --no-browser

clean: ## Remove generated files

clean-venv: uninstall  ## Remove environment
	rm -rf .venv

distclean: clean clean-venv  ## Restore clean repository state
	rm -f *.ipynb  # if using jupytext
	rm -f requirements-frozen.txt
	rm -rf .ipynb_checkpoints

Note that install because a prerequisite for make jupyter-notebook and make jupyter-lab, and uninstall becomes a prerequisite for make clean-venv.

Clone this wiki locally