I need multiple threadpools, but that's unadvised. How should I substitute that?

In my code, I've got a queue of things I've got to finish, but they're not important to be done as soon as possible, hence I'd like to allocate at most N threads to finish them—everything else I'd like to use for the main software itself & its processing. Please don't get attached to a specific number of threads, they're just examples.

So, for example - I can use up to 100 threads in the global thread pool. I'd like to just a minimum of 1 and a maximum of 20 threads to do the not-important "side job" and the remaining for the main processing. Sometimes main processing uses only 20 threads, sometimes it uses 90 threads.

I've seen at multiple places that having separate thread pools is impossible and I shouldn't even think about it anyways. If it's the case, then how should I view my problem differently?

I'm not necessarily strict about having 100 threads (for the example above), so a solution can be to fire up 20 fixed threads to work on the queue and then let the main application just use the thread pool. The total of 120 threads instead of 100 is something I don't care about, os will probably handle it, but inside it feels terrible. What if I need more of this behaviour? Am I going to do this fixed-thread hack five times? Sounds pretty lame at first sight.

Ps.: Under no circumstances do I want to stop processing the "not-that-important" queue, thus having at least N amount of workers for that is a necessity.

> I've seen at multiple places that having separate thread pools is impossible.

That appears to be a limitation of the System.Threading.ThreadPool class, which only has static methods. It is not a limitations of thread pools in general.

I do not know C#, so I don't know whether or not there are other freely available thread pool implementations that you could use. If it came to the worst, you could create your own thread pools. (It's not hard.)

> I'd like to [reserve some threads for] the not-important "side job" and the remaining for the main processing.

One option: If you can find or create a thread pool implementation that lets you use a priority queue as the task queue, would be to enqueue the "side job" tasks with lower priority than the "main processing" tasks.

Another option: (probably you would have to create your own thread pool for this) would be to have a single pool of threads with two queues, and some strategy for picking tasks in such a way that;

1. "main processing" tasks are generally prioritized over "side job" tasks, but

2. with some machinery to prevent side job tasks from being completely starved in the event that there is a continuous supply of main processing tasks.

TBH I don't know the current "official dogma" about this, but let me tell you a little story:

Once I was programming in Java SE and my duties involved using dedicated ThreadPools a lot.

Then I switched to another company and returned to coding in C# and boy oh boy: Since the last time I coded in it, there were Tasks and TPL introduced... I struggled.

But then, when I got a little more familiar and confident, it dawned on me: I had let go of Threads. I didn't cling to the concept of "my thread" anymore.

I came to the enlightenment, that I can use DataFlow and Parllel.ForEach while telling them how much to parallelize tops. And if I managed to max out the threadpool, I'd have bigger problems than this piece of code. And I didn't have to provide idle threads. Because I actually didn't need to and I would have to do sophisticated acrobatics to do it. So I didn't. Never looked back.

Now to your specific situation: you want to _de_prioritize some task, while still guarantee that it is processed. Your approach is to have two differently sized ThreadPools.

Let's say we let go of the Threadpool idea. Now, how I would solve the problem, is to have two "pipelines":

If I have a fixed sized set of tasks, I'd probably use 2 Parallel.ForEach with different maxParallelism settings.

If I don't know the count of tasks or if it is rather a constant influx of tasks ("open end" so to speak), I'd probably have two DataFlow pipelines (or one that splits into two, but that's details...), in which you can even set parallelism in each Block.

Another different approach (already mentioned in Solomon's answer) would be to construct a priority system, that ensures deprioritization of specific items, while ensuring they won't be starved. That could be as easy as having three queues: The prio queue, the low-prio queue and the actual processing queue. Then you could take and enqueue in a specific ratio. For example: 10 prio, then 2 low-prio if any or maybe 5 to 1 (which would about reflect your desired 100:20 ratio but without idle threads). Now you can scale how many of those would be processed in parallel while keeping the overall ratio of prio to low-prio.

How each of those utilize threads (or not): I don't care, really, until it becomes a problem. And if it does, I solve that.