Understanding the Performance Advantages of C++ over Other Languages

Why does C++ generally exhibit better execution speed than Java and Python? What factors contribute to this performance disparity? I conducted a series of tests to compare the execution speeds of these languages and seek a deeper understanding of the underlying reasons.

Context: As a computer science student, I have been exploring various programming languages to comprehend their performance characteristics. Through my experiments, I have consistently observed that C++ tends to outperform Java and Python in terms of execution speed. However, I desire a comprehensive understanding of the factors contributing to this performance difference.

Hardware and Compilation Details: To ensure a fair comparison, I executed the same algorithm using identical logic and datasets across all three languages. The experiments were conducted on a system equipped with an Intel Core i7 processor (8 cores) and 16 GB of RAM.

For the C++ code, I utilized GCC 10.2.0 with the following compilation flags:

g++ -O3 -march=native -mtune=native -std=c++17 -o program program.cpp

Java was executed using OpenJDK 11.0.1 with the following command:

java -Xmx8G -Xms8G Program

Python code was executed using Python 3.9.0 as follows:

python3 program.py
C++ code:

#include <iostream>
#include <chrono>
#include <vector>
#include <random>

// Function to generate a random matrix of size m x n
std::vector<std::vector<int>> generateRandomMatrix(int m, int n) {
    std::vector<std::vector<int>> matrix(m, std::vector<int>(n));
    std::random_device rd;
    std::mt19937 gen(rd());
    std::uniform_int_distribution<> dis(1, 100);

    for (int i = 0; i < m; ++i) {
        for (int j = 0; j < n; ++j) {
            matrix[i][j] = dis(gen);
        }
    }

    return matrix;
}

// Matrix multiplication function
std::vector<std::vector<int>> matrixMultiplication(const std::vector<std::vector<int>>& A, const std::vector<std::vector<int>>& B) {
    int m = A.size();
    int n = B[0].size();
    int k = B.size();

    std::vector<std::vector<int>> result(m, std::vector<int>(n, 0));

    for (int i = 0; i < m; ++i) {
        for (int j = 0; j < n; ++j) {
            for (int x = 0; x < k; ++x) {
                result[i][j] += A[i][x] * B[x][j];
            }
        }
    }

    return result;
}

int main() {
    // Generate random matrices A and B of size 3 x 3
    std::vector<std::vector<int>> A = generateRandomMatrix(3, 3);
    std::vector<std::vector<int>> B = generateRandomMatrix(3, 3);

    // Measure execution time
    auto start = std::chrono::steady_clock::now();

    // Perform matrix multiplication
    std::vector<std::vector<int>> result = matrixMultiplication(A, B);

    auto end = std::chrono::steady_clock::now();
    auto duration = std::chrono::duration_cast<std::chrono::microseconds>(end - start).count();

    std::cout << "Execution time (C++): " << duration << " microseconds" << std::endl;

    return 0;
}

Java code:

import java.util.Arrays;
import java.util.Random;

public class Program {
    // Function to generate a random matrix of size m x n
    public static int[][] generateRandomMatrix(int m, int n) {
        int[][] matrix = new int[m][n];
        Random random = new Random();

        for (int i = 0; i < m; ++i) {
            for (int j = 0; j < n; ++j) {
                matrix[i][j] = random.nextInt(100) + 1;
            }
        }

        return matrix;
    }

    // Matrix multiplication function
    public static int[][] matrixMultiplication(int[][] A, int[][] B) {
        int m = A.length;
        int n = B[0].length;
        int k = B.length;

        int[][] result = new int[m][n];

        for (int i = 0; i < m; ++i) {
            for (int j = 0; j < n; ++j) {
                for (int x = 0; x < k; ++x) {
                    result[i][j] += A[i][x] * B[x][j];
                }
            }
        }

        return result;
    }
    public static void main(String[] args) {
        // Generate random matrices A and B of size 3 x 3
        int[][] A = generateRandomMatrix(3, 3);
        int[][] B = generateRandomMatrix(3, 3);

        // Measure execution time
        long start = System.nanoTime();

        // Perform matrix multiplication
        int[][] result = matrixMultiplication(A, B);

        long end = System.nanoTime();
        long duration = end - start;

        System.out.println("Execution time (Java): " + duration + " nanoseconds");
    }
}

Python code:

import time
import numpy as np
import random

# Function to generate a random matrix of size m x n
def generateRandomMatrix(m, n):
    return [[random.randint(1, 100) for _ in range(n)] for _ in range(m)]

# Matrix multiplication function
def matrixMultiplication(A, B):
    A = np.array(A)
    B = np.array(B)
    result = np.dot(A, B)
    return result.tolist()

if __name__ == "__main__":
    # Generate random matrices A and B of size 3 x 3
    A = generateRandomMatrix(3, 3)
    B = generateRandomMatrix(3, 3)

    # Measure execution time
    start = time.time()

    # Perform matrix multiplication
    result = matrixMultiplication(A, B)

    end = time.time()
    duration = (end - start) * 1e6

    print("Execution time (Python): {} microseconds".format(duration))

I noticed a substantial performance difference in favor of C++. The execution times demonstrate C++'s superiority over Java and Python.

I understand that C++ is typically compiled, Java employs virtual machine emulation, and Python is interpreted. Consequently, I acknowledge that differences in execution approaches and compiler optimizations may significantly contribute to these performance disparities. Nonetheless, I would appreciate a more detailed explanation of the specific reasons underlying the observed performance differences.

Furthermore, I have taken the recommendation into account and conducted longer tests, running the algorithms on larger datasets to minimize the impact of initial startup costs for Java and Python. Nevertheless, the performance gap between C++ and the other languages remains substantial.

Could someone shed light on why C++ consistently outperforms Java and Python in this scenario? Are there specific compiler flags or runtime settings that could be adjusted in Java and Python to enhance their performance in similar computational tasks?

Thank you for sharing your insights and expertise!

Let me start (and maybe finish--not sure how ambitious I'll be) with a digression.

Benchmarking is Hard

Based on the times you're getting, it appears likely to me that you're timing execution of the entire program. Unfortunately, this is almost certainly dominated by the time taken to write the data to standard output, so if (for example) you redirected output to a file, chances are pretty good it would run a lot faster. Depending on testing environment and such, it would be fairly easy for this to produce fairly misleading results.

Just for example, I'd guess you're timing this under Windows, which has fairly slow console output. When I run your Python code under Linux, I get a time of 0.108 seconds--Python running faster than you got for C++. But that's almost certainly due primarily to Linux having much faster console output than Windows, not anything related to Python or C++ themselves at all.

Let's time just the computation part of the C++ and see what we get.

// preceding code as you wrote it

int main() {
    int n = 1000;
    using namespace std::chrono; // added

    auto start = high_resolution_clock::now(); // added

    std::vector<int> primeNumbers = generatePrimes(n);

    auto stop = high_resolution_clock::now(); //added

    std::cout << "First " << n << " prime numbers: ";
    for (int prime : primeNumbers) {
        std::cout << prime << " ";
    }    
    std::cout << std::endl;

    // added:
    std::cout << "Time: " << duration_cast<microseconds>(stop-start).count() << "us\n";
    return 0;
}

On my machine (which isn't particularly fast) I get:

Time: 394us

So even though the program took much longer to run, essentially all of that was consumed with displaying the output. The actual computation took only 0.394 milliseconds--a tiny fraction of the overall time.

It is still important to write out that result though--if you don't write out the result, the compiler can often detect that you're not using, it, and simply not compute it at all.

Even with the additions I made, the timing may easily be less than perfect, to put to mildly. In particular, modern CPUs can execute instructions out of order, so it's possible that the computation started before I grabbed the start time and/or finished after I grabbed the stop time. This can also be prevented, but it's more difficult (and in this case, doing a quick calculation of how much time it should take, I'd guess what the clock produced is at least reasonably accurate). But you often have to start with at least a decent guess at what you've expecting, and if your result is off by a significant factor, you may need to look at things more carefully.

C++ vs. Python

The big difference between C++ and Python hinges around dynamic vs. static typing. In Python, a variable can be attached to a value of any type, and that type can change over its lifetime. For example, we can perfectly legally have some variable a that holds a number at one time, and string at a different time.

a = 3
print(a, type(a))
a = "three"
print(a, type(a))

Result:

3 <class 'int'>
three <class 'str'>

Because of this, when you do almost any operation in Python, the code has to look up (at runtime) the type of each variable, to figure out what actual operation is being done, because the operation it carries out depends on their types.

In C++ types are mostly static (though you can use std::any<> and std::variant<> to get at least a limited imitation of dynamic types). Certainly the types in your code are entirely static. That means the compiler has done that lookup of what operation to apply to the operands at compile time, so it doesn't affect execution time at all.

In this case, you're doing a large number of simple operations. In the Python code, it's quite likely that most of the time is really spent looking up operand types, and figuring out what actions are appropriate for those operands, and only a small fraction of the time to actually doing the operation itself.

Now, it is true that a sufficiently smart implementation of Python could often (probably even usually) infer types ahead of time. When I execute something like a = 7, a is obviously an int. Based on the operations applied to a afterwards, it could often track what type a is going to have throughout its lifetime, and generate code specifically for that type.

So, much of the speed penalty isn't necessarily inherent to Python as a language, only to the most common implementation(s) of it. At least theoretically, an implementation could be written that would probably provide performance much closer to C++ (especially for code like you've written where nothing depends on user inputs).

My guess, however, is that the average Pythonista's reaction to doing this would generally be: "if you want C++, you know where to find it." At least for the vast majority of them, there are more important ways to spend that time and effort (and it would be a lot of time and effort).