Concurrent IO on single core Java 11

After using languages like Erlang and others, that have lightweight concurrent processes, I find it hard to understand how this translates into Java. Given I use a single core machine, is there a way to perform multiple concurrent IO bound operations (http)?

What I have found is the following ExecutorService and CompletableFuture. The issue I have is that they are based on a threadpool. The default threadpool uses core# - 1, which, on a single core machine, that I am using, has NO concurrency. Would the solution be to just provide a custom Executor with a higher number of threads? or is there a more idiomatic way for IO bound concurrency on single core machines in Java?

I am running this code on a AWS Lambda with a single core.

"The default threadpool uses core# - 1, which, on a single core machine, that I am using, has NO concurrency." - Why? A concurrent program can very well run on a single core machine. It has nothing to do with parallelism.

When a Java thread is waiting for I/O, the kernel's scheduler will move it to the wait queue, and some other thread that requires CPU time will be run. So you can create a thread pool with as many threads as you want, and the scheduler will take care of the concurrency. And this will work fine even on a single core machine.

The only limit here is the number of threads you will create. The default stack size of a thread varies b/w 512K to 1M. So this does not scale very well, and at some point, you'll run out of threads. On my system, I could create around 5k of them. Languages like Go manage this by multiplexing multiple goroutines on a limited number of kernel threads. This requires scheduling by the Go runtime.

If you want to alleviate this, you should look into NIO. I wrote a quick program that you can use to find out how many concurrent connections you can actually support this way. This should run as-is after the imports:

public class ConcurrentBlockingServer {

  private ExecutorService pool = Executors.newCachedThreadPool();

  public static void main(String[] args) {
    ConcurrentBlockingServer bs = new ConcurrentBlockingServer();
    try {
      bs.listen();
    } catch (IOException e) {
      e.printStackTrace();
    }
  }

  private void listen() throws IOException {
    int connectionId = 0;
    ServerSocket ss = new ServerSocket(8080);
    while (true) {
      Socket s = ss.accept(); // blocking call, never null
      System.out.println("Connection: " + (++connectionId));
      process(s);
    }
  }

  private void process(Socket s) {
    Runnable task =
        () -> {
          try (InputStream is = s.getInputStream();
              OutputStream os = s.getOutputStream()) {
            int data;
            // -1 is EOF, .read() is blocking
            while (-1 != (data = is.read())) {
              os.write(flipCase(data));
              os.flush();
            }
          } catch (IOException e) {
            e.printStackTrace();
          }
        };
    pool.submit(task);
  }

  private int flipCase(int input) {
    if (input >= 65 && input <= 90) {
      return input + 32;
    } else if (input >= 97 && input <= 122) {
      return input - 32;
    } else {
      return input;
    }
  }
}

Run this program and see how many connections you could make.

public class RogueClient {

  private static long noClients = 9000;

  public static void main(String[] args) {
    for (int i = 0; i < noClients; i++) {
      try {
        new Socket("localhost", 8080);
        System.out.println("Connection No: " + i);
      } catch (IOException e) {
        System.err.println("Exception: " + e.getMessage() + ", for connection: " + i);
      }
    }
    try {
      Thread.sleep(Integer.MAX_VALUE);
    } catch (InterruptedException e) {
      e.printStackTrace();
    }
  }
}

Edit: The pool size should depend upon the nature of your program. If it's an I/O bound task, you could go ahead and create many threads. But for CPU bound programs, the number of threads should be equal to the number of cores.

Threads are the basic (and only) concurrency mechanism in Java. ExecutorService has methods for creating thread pools with an arbitrary number of threads; also CompletableFutures are not limited to ForkJoinPool.commonPool. The ones provided with the number of threads = number of cores are just convenience methods for CPU-bound tasks.

For IO bound work you could allocate more threads as you've mentioned, while they're doing the IO they are blocked, so others can still utilize the CPU.

You could also have a look at Project Reactor which can come handy if you don't want to manually take care of threading with this particular scheduler: https://projectreactor.io/docs/core/release/api/reactor/core/scheduler/Schedulers.html#boundedElastic--