Fastest modern C++ way to generate a 1GB file with arbitrary binary data

So, I've tried this:

uint64_t size = 1 * 1000 * 1000 * 1000;
std::vector<char> data(size, 'a');

std::ofstream myfile;
myfile.open("test.dat");
for (auto it = data.begin(); it != data.end(); ++it)
{
	myfile << *it;
}

myfile.close();

Which works but takes > 15 minutes in release. I'm running with a 1TB NVME ssd and 64GBs of RAM, I feel like I should be able to do this in seconds, or less than a second.

I do have access to C++20. I prefer something modern for its readability so long as its reasonably performant. I feel like anything over 10 seconds is too slow.

To be clear, pushing back into the array all of the data is nearly instant. But writing it is what's taking minutes.

Writing one byte at a time leads to a high overhead, especially if the file is unbuffered and you're making a system call for each byte. Likewise, operator<< writes formatted data, which does the right thing for a char but is more error-prone, since many inputs will get formatted during writing.

Instead, you already have access to a buffer full of char - the simplest solution is to just write the whole buffer with ofstream::write:

#include <vector>
#include <iostream>
#include <fstream>

int main()
{
	uint64_t size = 2 * 1000 *1000 * 1000;
	std::vector<char> data;
	data.reserve(size);

	for (int i = 0; i < size; ++i)
	{
		data.push_back('a');
	}

	std::ofstream myfile;
	myfile.open("test.dat");
	myfile.write(data.data(), size);

	myfile.close();
}

When straced, the example above only uses one syscall to write:

writev(3, [{iov_base=NULL, iov_len=0}, {iov_base="aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"..., iov_len=2000000000}], 2) = 2000000000

Better yet, write (e.g.) a 1 MB block 1000 times, to avoid allocating a whole gigabyte of RAM:

#include <vector>
#include <iostream>
#include <fstream>

int main()
{
	uint64_t size = 1000 * 1000;
	std::vector<char> data;
	data.reserve(size);

	for (int i = 0; i < size; ++i)
	{
		data.push_back('a');
	}

	std::ofstream myfile;
	myfile.open("test.dat");
    for (int i = 0; i < 1000; i++) {
	    myfile.write(data.data(), size);
    }
	myfile.close();
}

Finally, pay attention to what you're putting into the buffer. \ba is is a multibyte character - \b is an escape for 0x08 and a is a. On my machine, this gets truncated to just a. If you want more interesting data in your file, instead change the loop that fills the buffer.

Don't make a 2GB array in memory.

Take a chunk size, something like 65536 bytes. The exact optimal size depends on your OS and filesystem.

Fill a buffer of that chunk size and write it with iostream::write.

Something like this:

const size_t sz = 65536;
uint8_t data[sz];
for ( /* blah */) {
    fill_random_data(data, sz);
    outfile.write(data, sz);
}