pip broken in debian package

[paulmueller]

Whenever I install a new version of the CKAN package, I have to reinstall pip, because it is somehow broken.
Here are the steps to reproduce (with a clean vagrant image for your convenience):

Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/focal64"
end

Commands:

vagrant up
vagrant ssh
sudo -s
apt update
apt install nginx-full libpq5
wget https://packaging-ckan-org.s3-eu-west-1.amazonaws.com/python-ckan_2.9-py3-focal_amd64.deb
dpkg -i python-ckan_2.9-py3-focal_amd64.deb
source /usr/lib/ckan/default/bin/activate
# This will fail:
pip install --upgrade pip

Error message:

Traceback (most recent call last):
  File "/usr/lib/ckan/default/bin/pip", line 5, in <module>
    from pip._internal.cli.main import main
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 10, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 9, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 7, in <module>
    from pip._internal.cli import cmdoptions
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pip/_internal/cli/cmdoptions.py", line 25, in <module>
    from pip._internal.locations import USER_CACHE_DIR, get_src_prefix
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pip/_internal/locations.py", line 19, in <module>
    from pip._internal.utils import appdirs
  File "/usr/lib/ckan/default/lib/python3.8/site-packages/pip/_internal/utils/appdirs.py", line 13, in <module>
    from pip._vendor import appdirs as _appdirs
ImportError: cannot import name 'appdirs' from 'pip._vendor' (/usr/lib/ckan/default/lib/python3.8/site-packages/pip/_vendor/__init__.py)

Workaround: To repair the environment, re-install pip via get-pip.py.

[EDIT: Please ignore the package version issue below]

I also realized that at least one of the packages installed in that environment do not have the correct version:

pip freeze | grep rq

Outputs rq==1.0, but should be rq==1.10.0.

This is due to a wrong requirements file in /usr/lib/ckan/default/src/ckan/requirements.txt which I believe is caused by ckan/ckan#6528.

[aucuparia]

Another workaround is to install pipenv (apt install pipenv). Seems to work on Vagrant box ubuntu/focal64 (virtualbox, 20220215.1.0), which installs Ubuntu 20.04.4 LTS, with python-ckan_2.9-py3-focal_amd64.deb.