Why does thread pool takes tasks independenty and not concurrently?

I am trying to get the basics very strong about thread pool. I learnt that it internally uses blocking queue to 'steal' tasks and run them into given threads in pool. Meaning that if I had 10 tasks and 5 threads, it could run only 5 tasks at the same time, until 1 finishes ENTIRELY.

Question is: Why not concurrently? Why not just time-slice those 10 tasks?
What is the reason of this implementation?

> Why not concurrently? Why not just time-slice those 10 tasks?

You can have a thread pool that is able to perform ten concurrent tasks. You just need to configure it to have at least ten worker threads. "Time-slicing" tasks is what threads do. What thread pools do is:

• Allow your program to control the number of threads that it uses to perform "background" tasks, and
• Allow your program to re-use threads, which can be much more efficient than creating a new thread for each new task, and then destroying the thread when the task is complete.

In order to "time-slice 10 tasks", those tasks need to be in 10 separate threads that run concurrently.

The time-slicing scheduling algorithm is implemented by the operating system, not by Java. Time slicing applies to threads in Java because Java threads are implemented as native operating system threads: every Java thread has a native thread of its own, and these threads are scheduled by the operating system as it sees fit.

There is no difference between "thread pool threads" and "raw threads" here. If you give an instance of Runnable to a thread (whether it's part of a thread pool or not) it will run from beginning to end, subject to the time slicing scheduling algorithm of the operating system.

So why not use thousands of threads, why even bother with thread pools? It turns out that operating system threads are a relatively expensive and scarce resource, and therefore so are Java threads.

Since operating system threads are so expensive, Project Loom is investigating adding lightweight user space threads to Java. Some of the details in this answer may change when/if loom gets merged into main stream Java.

Some good answers but I thought I'd respond to your questions specifically.

> I learnt that it internally uses blocking queue to 'steal' tasks and run them into given threads in pool. Meaning that if I had 10 tasks and 5 threads, it could run only 5 tasks at the same time, until 1 finishes ENTIRELY.

If you configure your thread pool to have 5 threads (Executors.newFixedThreadPool(5)) then it will start 5 threads to run your jobs. Initially 5 jobs are given to the 5 threads to run concurrently (if your server has 5 CPUs available). Once one of the 5 jobs finishes, a 6th job will be immediately started on the idle thread. This continues until all 10 jobs have been run.

> Question is: Why not concurrently? Why not just time-slice those 10 tasks? What is the reason of this implementation?

You can instead use a cached thread pool (Executors.newCachedThreadPool()) if you want which will start a thread for each of the 10 jobs that you submit concurrently. This might work fine for 10 jobs but won't work well with 100,000 jobs – you would not want to start 100,000 threads. We use a fixed thread pool when we want to limit the number of jobs that run concurrently. Even though it seems like running 5 jobs concurrently would always run slower than running 10 jobs at once, but this isn't necessarily the case. There is a cost when the OS time slices between the jobs and the overall job throughput may be faster with 5 threads than 10 depending on how many processors your hardware has. Limiting the number of concurrent jobs also does not stress your server as much and should make your application work better with other running applications.

See my answer here about scaling threads: https://stackoverflow.com/questions/17840397/concept-behind-putting-wait-notify-methods-in-object-class/17841450#17841450