pip dependency resolver issue - singularity container - cuda and torch

I want to build a singularity container for a python library using the following def file:

Bootstrap: docker
From: nvcr.io/nvidia/pytorch:21.06-py3

%post
        export DEBIAN_FRONTEND=noninteractive

        apt-get update
        apt-get -y install pip wget git vim
        pip install opencv-python-headless

        pip install 'git+https://github.com/facebookresearch/fvcore'

        cd /opt
        git clone https://github.com/facebookresearch/detectron2.git detectron2_repo
        cd detectron2_repo
        pip install .

While building the container, after the line pip install ., I get the following error:

Successfully built detectron2 fvcore antlr4-python3-runtime pycocotools fairscale
Installing collected packages: zipp, numpy, antlr4-python3-runtime, tomli, platformdirs, pathspec, omegaconf, mypy-extensions, iopath, importlib-resources, huggingface-hub, click, timm, pycocotools, hydra-core, fvcore, fairscale, cloudpickle, black, detectron2
  Attempting uninstall: numpy
    Found existing installation: numpy 1.20.3
    Uninstalling numpy-1.20.3:
      Successfully uninstalled numpy-1.20.3
  Attempting uninstall: iopath
    Found existing installation: iopath 0.1.10
    Uninstalling iopath-0.1.10:
      Successfully uninstalled iopath-0.1.10
  Attempting uninstall: click
    Found existing installation: click 7.1.2
    Uninstalling click-7.1.2:
      Successfully uninstalled click-7.1.2
  Attempting uninstall: pycocotools
    Found existing installation: pycocotools 2.0+nv0.5.1
    Uninstalling pycocotools-2.0+nv0.5.1:
      Successfully uninstalled pycocotools-2.0+nv0.5.1
  Attempting uninstall: fvcore
    Found existing installation: fvcore 0.1.6
    Uninstalling fvcore-0.1.6:
      Successfully uninstalled fvcore-0.1.6
ERROR: pips dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
typer 0.3.2 requires click<7.2.0,>=7.1.1, but you have click 8.1.3 which is incompatible.
scipy 1.6.3 requires numpy<1.23.0,>=1.16.5, but you have numpy 1.24.2 which is incompatible.

So, it first uninstalls click 7.1.2 and numpy 1.20.3 which are the compatible versions and then installs other versions of them which are not compatible, and complains that they are incompatible.
Why does this happen? and how to fix it?

what I tried:

1. add extra lines to the setup.py file in order to specify the correct versions for installation. But it still does the same thing.
2. start from another release of pytorch from nvidia ngc. It produces the same error for other packages.
3. start the container from plain ubuntu without installing cuda. I do not get any error but I need cuda for the gpu.
4. build the container without the last line in the def file, then run the container and from inside the container run pip install --user e . . This resolves the errors but I still need to install the package using the container.

I solved the problem in two ways:

1. either use a virtual environment inside the container. However in this case, one needs to re-install torch inside the virtual environment which is nonsense.

Bootstrap: docker
From: nvcr.io/nvidia/pytorch:21.06-py3
%post
        export DEBIAN_FRONTEND=noninteractive
        apt-get update
        apt-get -y install pip wget git vim  python3-venv
        python3 -m venv /opt/myenv
        . /opt/myenv/bin/activate # use . instead of source command in container
        pip install tensorboard
        pip install opencv-python-headless
        pip install 'git+https://github.com/facebookresearch/fvcore'
        pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113
        cd /opt
        git clone https://github.com/facebookresearch/detectron2.git
        cd detectron2_repo
        pip install .

2. or build the container based on a cuda docker image instead of a torch docker image, and then install torch.

Bootstrap: docker
From: nvcr.io/nvidia/cuda:11.3.1-cudnn8-devel-ubuntu20.04
%post
        export DEBIAN_FRONTEND=noninteractive
        apt-get update
        apt-get -y install pip wget git vim  python3-venv
        pip install opencv-python-headless
        pip install tensorboard
        pip install torch==1.11.0+cu113 torchvision==0.12.0+cu113 torchaudio==0.11.0 --extra-index-url https://download.pytorch.org/whl/cu113
        pip install 'git+https://github.com/facebookresearch/fvcore'
        cd /opt
        git clone https://github.com/facebookresearch/detectron2.git
        cd detectron2_repo
        pip install .

Obviously, the 2nd way is more reasonable.