RAPIDS pip installation issue

I've been trying to install RAPIDS in my Docker environment, which initially went smoothly. However, over the past one or two weeks, I've been encountering an error.

The issue seems to be that pip is attempting to fetch from the default PyPi registry, where it encounters a placeholder project. I'm unsure who placed it there or why, as it appears to serve no practical purpose.

 => ERROR [12/19] RUN pip3 install cudf-cu11 cuml-cu11 cugraph-cu11 cucim --extra-index-url=https://pypi.nvidia.com                                             2.1s 
------                                                                                                                                                               
 > [12/19] RUN pip3 install cudf-cu11 cuml-cu11 cugraph-cu11 cucim --extra-index-url=https://pypi.nvidia.com:                                                        
#0 1.038 Looking in indexes: https://pypi.org/simple, https://pypi.nvidia.com                                                                                        
#0 1.466 Collecting cudf-cu11                                                                                                                                        
#0 1.542   Downloading cudf-cu11-23.6.0.tar.gz (6.8 kB)                                                                                                              
#0 1.567   Preparing metadata (setup.py): started
#0 1.972   Preparing metadata (setup.py): finished with status 'error'
#0 1.980   error: subprocess-exited-with-error
#0 1.980   
#0 1.980   × python setup.py egg_info did not run successfully.
#0 1.980   │ exit code: 1
#0 1.980   ╰─> [16 lines of output]
#0 1.980       Traceback (most recent call last):
#0 1.980         File "<string>", line 2, in <module>
#0 1.980         File "<pip-setuptools-caller>", line 34, in <module>
#0 1.980         File "/tmp/pip-install-8463q674/cudf-cu11_9d3e1a792dae4026962cdff29926ce8d/setup.py", line 137, in <module>
#0 1.980           raise RuntimeError(open("ERROR.txt", "r").read())
#0 1.980       RuntimeError:
#0 1.980       ###########################################################################################
#0 1.980       The package you are trying to install is only a placeholder project on PyPI.org repository.
#0 1.980       This package is hosted on NVIDIA Python Package Index.
#0 1.980       
#0 1.980       This package can be installed as:
#0 1.980       ```
#0 1.980       $ pip install --extra-index-url https://pypi.nvidia.com cudf-cu11
#0 1.980       ```
#0 1.980       ###########################################################################################
#0 1.980       
#0 1.980       [end of output]
#0 1.980   
#0 1.980   note: This error originates from a subprocess, and is likely not a problem with pip.
#0 1.983 error: metadata-generation-failed
#0 1.983 
#0 1.983 × Encountered error while generating package metadata.
#0 1.983 ╰─> See above for output.
#0 1.983 
#0 1.983 note: This is an issue with the package mentioned above, not pip.
#0 1.983 hint: See above for details.

I attempted to explicitly set the --index-url to pypi.nvidia.com, but this approach wasn't feasible either, as the dependencies for the RAPIDS packages appear to be hosted on the default PyPi.

The issue has already been reported in the cudf repository and it seems to have existed since version 23.06.

I will try the suggested workaround of installing version 23.04 and will report back if this temporarily resolves the problem.

Original Issue:

https://github.com/rapidsai/cudf/issues/13642

Feedback:

Using strict versioning to 23.04 worked perfectly.
I simply replaced the unversioned pip install with a versioned one:

# Before
pip3 install cudf-cu11 cuml-cu11 cugraph-cu11 cucim --extra-index-url=https://pypi.nvidia.com

# After
pip3 install cudf-cu11==23.04 cuml-cu11==23.04 cugraph-cu11==23.04 cucim --extra-index-url=https://pypi.nvidia.com

One thing to note for anyone else encountering this issue while building in an Ubuntu 20.04 container: you need to add RUN apt remove python3-psutil before installing rapids, to allow pip to install psutil in the correct version.