Amount of Threads with subtasks

An optimum of threads in a pool is something that is case specific, though there is a rule of thumb which says #threads = #CPU +1.
However, how does this work with threads spanning other threads and waiting (i.e. blocked until thread.join() is successful) for these 'subthreads'?

Assume that I have code that requires the execution of list of tasks (2), which has subtasks(2), which has subsubtasks(3) and so on. The total number of tasks is 223 = 12, though 18 threads will be created (because a threads will 'spawn' more subtasks (threads), where the thread spawning more threads will be blocked untill all is over. See below for pseudo code.

I am assuming that for a CPU with N cores there is a rule of thumb that everything can be parallelized if the highest number of active threads (12) is #CPU + 1. Is this correct?

PseudoCode

outputOfTask = []
for subtask in SubTaskList
   outputOfTask --> append(subtask.doCompute())
// wait untill all output is finished.

in subtask.java:
Each subtask, for example, implements the same interface, but can be different.

   outputOfSubtask = []
    for task in subsubTaskList
        // do some magic depending on the type of subtask
        outputOfSubtask -> append( task.doCompute())
    return outputOfSubtask

in subsubtask.java:

 outputOfSubsubtask = []
    for task in subsubsubtask
        // do some magic depending on the type of subsubtask
        outputOfSubsubtask -> append( task.doCompute())
    return outputOfSubsubtask

EDIT:
Dummy code Java code. I used this in my original question to check how many threads were active, but I assume that the pseudocode is more clear. Please note: I used the Eclipse Collection, this introduces the asParallel function which allows for a shorter notation of the code.

@Test
public void testasParallelthreads() {
	// // ExecutorService executor = Executors.newWorkStealingPool();
	ExecutorService executor = Executors.newCachedThreadPool();
	
	MutableList<Double> myMainTask = Lists.mutable.with(1.0, 2.0);
	MutableList<Double> mySubTask = Lists.mutable.with(1.0, 2.0);
	MutableList<Double> mySubSubTask = Lists.mutable.with(1.0, 2.0);
	MutableList<Double> mySubSubSubTask = Lists.mutable.with(1.0, 2.0, 2.0);
	
	MutableList<Double> a = myMainTask.asParallel(executor, 1)
			.flatCollect(task -> mySubTask.asParallel(executor,1)
			.flatCollect(subTask -> mySubSubTask.asParallel(executor, 1)
			.flatCollect(subsubTask   -> mySubSubSubTask.asParallel(executor, 1)
			.flatCollect(subsubTask -> dummyFunction(task, subTask, subsubTask, subsubTask,executor))
			.toList()).toList()).toList()).toList();		

	System.out.println("pool size: " + ((ThreadPoolExecutor) executor).getPoolSize());
	executor.shutdownNow();
}

private MutableList<Double> dummyFunction(double a, double b, double c, double d, ExecutorService ex) {
	System.out.println("ThreadId: " + Thread.currentThread().getId());
	System.out.println("Active threads size: " + ((ThreadPoolExecutor) ex).getActiveCount());
	return Lists.mutable.with(a,b,c,d);
}

>However, how does this work with threads spanning other threads and waiting (i.e. blocked until thread.join() is successful) for these 'subthreads'?

Threads will block, and it is up to the os/jvm to schedule another one if possible. If you have a single thread pool executor and call join from one of your tasks, the other task won't even get started. With executors that use more threads, then the blocking task will block a single thread and the os/jvm is free to scheduled other threads.

>These blocked threads should not consume CPU time, because they are blocked. So I am assuming that for a CPU with N cores there is a rule of thumb that everything can be parallelized if the highest number of active threads (24) is #CPU + 1. Is this correct?

Active threads can be blocking. I think you're mixing terms here, #CPU, the number of cores, and the number of virtual cores. If you have N physical cores, then you can run N cpu bound tasks in parallel. When you have other types of blocking or very short lived tasks, then you can have more parallel tasks.

> I am assuming that for a CPU with N cores there is a rule of thumb that everything can be parallelized if the highest number of active threads (12) is #CPU + 1. Is this correct?

This topic is extremely hard to generalize about. Even with the actual code, the performance of your application is going to be very difficult to determine. Even if you could come up an estimation, the actual performance may vary wildly between runs – especially considering that the threads are interacting with each other. The only time we can take the #CPU + 1 number is if the jobs that are submitted into the thread-pool are independent and completely CPU bound.

I'd recommend trying a number of different thread-pool size values under simulated load to find the optimal values for your application. Examining the overall throughput numbers or system load stats should give you the feedback you need.