Multiple Spark Executors on single GPU

We are trying to improve the Spark Job processing performance by introducing GPUs to the nodes. But after enabling Spark3 with GPUs we are seeing downtrend in spark job performance, due to limited number of spark executors creation with GPU enabled.

i.e
with just CPU cores(without GPU) we are able to create hundreds of executors as we have got hundreds of CPU cores.

with GPU enabled, we are able to create only 6 executors as we have got only 6 GPU hardware.

So, is there anyway to run multiple executors with single GPU

If you are using Spark resource scheduling to assign executors to GPUs, I do not believe there is a way to assign multiple executors to the same GPU in Spark scheduling. The resource amount config is an integer, so there isn't a way to specify a fractional amount of a GPU to assign to each executor.

If you are bypassing Spark's GPU scheduling feature and are assigning executors to GPUs via some other mechanism, there might be a way to have executors share the GPU. However this depends on the software within the executor that is using the GPU and whether that software can be configured to not assume it can use the entire GPU. It may need to artificially use less GPU memory to make room for other executors, and this may cause the software to perform suboptimally or encounter out-of-memory errors. There is also process context switching overhead between processes that share a GPU which can impact performance relative to each process exclusively using its own GPU.