CMake failing to build nested subdirectories

Take the following fairly standard looking directory structure for a firmware project (an ESP32 project):

Folder Struct

- myProject/
         - CMakeLists.txt
         - sdkconfig
         - components/ - component1/ - CMakeLists.txt
		 
                                     - component1a/ - CMakeLists.txt
                                                    - src/monkey_test.c
													- include/monkey_test.h
													 
									 - component1b/	- CMakeLists.txt
													- src/src1b.c
													- include/src1b.h
													 
                       - component2/ - CMakeLists.txt
                                     - src/src2.c
                                     - include/ - src2.h
									 
         - main/       - src/app_main.c
                       - src/other_stuff.c

         - build/

Now let's start with the simplest thing possible. If I create monkey_test.h & .h as:

Source

#include <monkey_test.h>

int monkey_test(void) {
    return 1;
}

Include

#ifndef MONKEY_WIFI_H__
#define MONKEY_WIFI_H__

int monkey_test(void);

#endif

Main

#include <monkey_test.h>
main() {
  
    sprintf("test value %s", monkey_test());
	
}

CMake Files

Top Level CMakeLists.txt

cmake_minimum_required(VERSION 3.16)
set(EXTRA_COMPONENT_DIRS ${CMAKE_CURRENT_LIST_DIR}/components)
list(APPEND EXTRA_COMPONENT_DIRS ${CMAKE_CURRENT_LIST_DIR}/../common)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(proj_name)

component1 CMakeLists.txt

list(APPEND EXTRA_COMPONENT_DIRS ${CMAKE_CURRENT_LIST_DIR}/component1a)

Component CMakeLists.txt (for all components)

idf_component_register(  
    SRC_DIRS     "src"     
    INCLUDE_DIRS "include" 
)

When I build this, it doesn't work.

../main/src/app_main.c:54:10: fatal error: monkey_test.h: No such file or directory
#include <monkey_test.h>

Things I've tried

I tried rewriting the component1 CMakeLists.txt as include(wifi/CMakeLists.txt) and idf_build_process(wifi). Neither work, and I don't really know why I thought it might.

Also, I've tried just removing the component CMakLists.txt and replacing the component1 CMakeLists.txt to directly bring in the component there but even that didn't work???

idf_component_register(  
     SRC_DIRS  "component1a/src"            
     INCLUDE_DIRS "component1a/include"     
)

I'm sure it's staring me in the face, and I'm being very blind indeed, but anyone know what I'm doing wring here?

You can't really as of the time of this answer.

The problem is the following:

The idea of the idf superset of CMake commands (e.g. all commands prefixed with idf*, such as for example idf_components_register) is to automatically resolve dependencies across a multitude of components that change rapidly. So what the authors did was to have a script walk through all folders in components and EXTRA_COMPONENTS_DIR and pick up any component that registers itself as an idf component and remember its dependencies. The project will then automatically create CMake targets and link each dependent component against each other.

So what if you just include the subcomponenent into the generator tree using add_subdirectory? You can do that, but only AFTER script mode has processed idf_component_register, if you put the command before CMake will complain that add_subdirectory is not a scriptable command. And if you put it after, the subcomponents will not be taken up in the dependency graph as the script completes after executing idf_components_register, so it will not build.

You could manually add CMakeTargets for each individual subcomponent, and link it back to the root component such as done by esp_mbedls. If you include components from external repos this is the way to go, but it only works if your subcomponent doesn't depend on other esp_idf components, as the link targets won't be added automatically for manual components (as the aforementioned script has already been executed by that time).

I hope espressif will add support for nested components in the future, until then we're stuck with the solution that are provided to us unfortunately.

The way I handle it right now is to just have one metacomponent that "manages" the subcomponent, i.e. it adds headers and sources of each subcomponent into its own sources and includes.