How to use cpp_extension with an existing library built on top of libtorch?

I've successfully completed the c++/CUDA extension tutorial. I wrote and bound a different utility than what they used.

Then, I wrote and compiled a CMake project to create a static library out of the same utility I used in the extensions tutorial.

Next, I want to know how I can take the library (.lib) I built and bind it to Python. I want to sort of blend the two tutorials I linked above.

What I tried:

In my original project, where the library exists, I added a new directory called pybind:

├───external
│   └───libtorch
│       ├───bin
│       ├───cmake
│       ├───include
│
├───libray
│   │   CMakeLists.txt
│   │
│   ├───include    *** Public API of libray ***
│   │       CMakeLists.txt
│   │       raycast.cpp
│   │       raycast.h
│   │
│   └───src        *** Helpers, PRIVATE in cmake ***
│           raycast_cuda.cu
│           raycast_cuda.cuh
│           torch_checks.cpp
│           torch_checks.h
│
├───pybind
            extension.cpp
            setup.py

Inside that directory, I added the extension.cpp file that would include the header for the utilities I want to bind, and then bind the functions contained by those headers:

#include <torch/extension.h>
#include "raycast.h"

PYBIND11_MODULE(TORCH_EXTENSION_NAME, m) {
  m.def("find_distance", &find_distances, "Get minimum distance from ray origin to surface.");
  m.def("find_intersections", &find_intersections, "Get ray's intersection with surface.");
}

along with the setup.py described in the first link in my question:

from setuptools import setup
from torch.utils import cpp_extension

setup(name='raycast',
      ext_modules=
      [
          Extension(
              name="raycast", 
              sources=['extension.cpp'],
              include_dirs=[
                  '../external/libtorch/include', 
                  "../external/libtorch/include/torch/csrc/api/include",
                  "../libray/include"],
              library_dirs=[
                  "../build/libray/include/Debug/", 
                  "../external/libtorch/lib"],
              libraries=["libray", "torch", "torch_cpu", "torch_cuda", "c10"]
      ],
      cmdclass={'build_ext': cpp_extension.BuildExtension})

Note: I also added the private header files to the ext_modules list. This defeats the purpose of the question and also didn't work.

What happens

1. When I run python setup.py install, I get an error saying there are 17 unresolved externals.

extension.obj : error LNK2001: unresolved external symbol "class at::Tensor __cdecl find_distances(class at::Tensor,class at::Tensor,class at::Tensor,class at::Tensor)" (?find_distances@@YA?AVTensor@at@@V12@000@Z)
extension.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __cdecl pybind11::detail::type_caster<class at::Tensor,void>::type_caster<class at::Tensor,void>(void)" (__imp_??0?$type_caster@VTensor@at@@X@detail@pybind11@@QEAA@XZ)
extension.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __cdecl pybind11::detail::type_caster<class at::Tensor,void>::~type_caster<class at::Tensor,void>(void)" (__imp_??1?$type_caster@VTensor@at@@X@detail@pybind11@@QEAA@XZ)
extension.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) public: __cdecl pybind11::detail::type_caster<class at::Tensor,void>::operator class at::Tensor &&(void)&& " (__imp_??B?$type_caster@VTensor@at@@X@detail@pybind11@@QEHAA$$QEAVTensor@at@@XZ)
libray.lib(raycast.obj) : error LNK2001: unresolved external symbol __imp__invalid_parameter
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __imp__invalid_parameter
libray.lib(torch_checks.obj) : error LNK2001: unresolved external symbol __imp__invalid_parameter
libray.lib(raycast.obj) : error LNK2001: unresolved external symbol __imp__calloc_dbg
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __imp__calloc_dbg
libray.lib(torch_checks.obj) : error LNK2001: unresolved external symbol __imp__calloc_dbg
libray.lib(raycast.obj) : error LNK2001: unresolved external symbol __imp__CrtDbgReport
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __imp__CrtDbgReport
libray.lib(torch_checks.obj) : error LNK2001: unresolved external symbol __imp__CrtDbgReport
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol cudaLaunchKernel
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __cudaPushCallConfiguration
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __cudaPopCallConfiguration
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __cudaRegisterFatBinary
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __cudaRegisterFatBinaryEnd
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __cudaUnregisterFatBinary
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __cudaRegisterVar
libray.lib(raycast_cuda.obj) : error LNK2001: unresolved external symbol __cudaRegisterFunction
build\lib.win-amd64-3.8\raycast.cp38-win_amd64.pyd : fatal error LNK1120: 17 unresolved externals

2. I get these errors as well, but many less than the other one.

libray.lib(raycast.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '2' doesn't match value '0' in extension.obj
libray.lib(raycast.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MDd_DynamicDebug' doesn't match value 'MD_DynamicRelease' in extension.obj
libray.lib(raycast_cuda.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '2' doesn't match value '0' in extension.obj
libray.lib(raycast_cuda.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MDd_DynamicDebug' doesn't match value 'MD_DynamicRelease' in extension.obj
libray.lib(torch_checks.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '2' doesn't match value '0' in extension.obj
libray.lib(torch_checks.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MDd_DynamicDebug' doesn't match value 'MD_DynamicRelease' in extension.obj
   Creating library build\temp.win-amd64-3.8\Release\raycast.cp38-win_amd64.lib and object build\temp.win-amd64-3.8\Release\raycast.cp38-win_amd64.exp
LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other libs; use /NODEFAULTLIB:library

I have looked up the meaning of LNK2038 and LNK2001 errors. I was able to get down from 158 to 17 LNK2001 errors by including more .lib files from libtorch/lib/. However, I still appear to be missing something.

My questions

What am I missing, whether it be including paths or extra libraries to resolve these issues?

In order to solve this problem, I ended up taking a look at the contents of the
setuptools.Extension object created by calling cpp_extension.CUDAExtension.

from setuptools import setup, Extension
from torch.utils import cpp_extension

cuda_extension = cpp_extension.CUDAExtension('raycast', ['raycast.cpp', 'raycast_cuda.cu', 'torch_checks.cpp'])

for attribute in dir(cuda_extension):
    attr = getattr(cuda_extension, attribute)
    print("*********************************")
    print(attribute)
    print(attr)
    print("*********************************")

This ended up yielding the libraries and their locations that I was missing.

*********************************
include_dirs
['C:\\Users\\wesle\\anaconda3\\lib\\site-packages\\torch\\include', 'C:\\Users\\wesle\\anaconda3\\lib\\site-packages\\torch\\include\\torch\\csrc\\api\\include', 'C:\\Users\\wesle\\anaconda3\\lib\\site-packages\\torch\\include\\TH', 'C:\\Users\\wesle\\anaconda3\\lib\\site-packages\\torch\\include\\THC', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\include']
*********************************

*********************************
libraries
['c10', 'torch', 'torch_cpu', 'torch_python', 'cudart', 'c10_cuda', 'torch_cuda_cu', 'torch_cuda_cpp']
*********************************

*********************************
library_dirs
['C:\\Users\\wesle\\anaconda3\\lib\\site-packages\\torch\\lib', 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\lib/x64'] 
*********************************

Strangely this still failed, as the library torch_python.lib could not be found. So, I searched for it in my conda env and added its location to the library_dirs list. And it worked!

There may be some redundancies in the search paths, as I included libtorch directories (stored in my project) and torch directories (from my conda env pytorch install). However, here is the final working setup.py:

from setuptools import setup, Extension
from torch.utils import cpp_extension

setup(name='raycast',
      ext_modules=
      [
          Extension(
              name="raycast", 
              sources=['extension.cpp'],
              
              include_dirs= 
                  cpp_extension.include_paths() + 
                  [
                        '../external/libtorch/include', 
                        "../external/libtorch/include/torch/csrc/api/include",
                        "../libray/include",
                        'C:\\Users\\wesle\\anaconda3\\lib\\site-packages\\torch\\include',
                        'C:\\Users\\wesle\\anaconda3\\lib\\site-packages\\torch\\include\\torch\\csrc\\api\\include', 
                        'C:\\Users\\wesle\\anaconda3\\lib\\site-packages\\torch\\include\\TH',
                        'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\include'
                  ],

              library_dirs=[
                  "../build/libray/include/Release/", 
                  "../external/libtorch/lib",
                  "C:\\Users\\wesle\\anaconda3\\Lib\\site-packages\\torch\\lib",
                  'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.6\\lib/x64'],

              libraries=["libray", 'c10', 'torch', 'torch_cpu', 
                         'torch_python', 'cudart', 'c10_cuda', 
                         'torch_cuda_cu', 'torch_cuda_cpp']) 
      ],
      cmdclass={'build_ext': cpp_extension.BuildExtension})