strategies when memory limits matrix multiplication in matlab

I need to calculate sum(foo(x(:)*y),2)), where x is a 1-D vector (1x1000) and y is a 1-D vector(1x1e8), both not sparse, and foo is composed of several trig functions that are not separable or have a summation rule (see edit at the end of the question). For efficiency I made the vectors to be Single-precision arrays. Another info that may or may not be important is that x is just a monotonically increasing vector such as x=linspace(0,1,1000).

Trying to to x(:)*y I got into a memory problem. So I thought I'll do a for loop.

for i=1:length(y)
    A=A+sum(foo(x(:)*y(i)),2);
end

This is inefficient and takes forever, so I thought of breaking down the sum into several chunks of similar size (beside the last one which may be slightly different size) :

        nn = 10 ;% # of chunks 
        idN=length(y);
        splitn = floor(idN/nn);
        split = [1:splitn:idN];

        for i=1:nn-2
            A=A+sum(foo(x(:)*y(split(ni): split(ni)+splitn-1)),2)   
        end

        A = A +sum(foo(x(:)*y(split(nn-1):end)),2);

This takes a lot of time but is not forever (~45 min). What else can I do to make this faster besides buying a new computer? I dont think that parfor will help as there is only a certain amount of memory anyway. Does the # of chunks matter at all as long as it is all contained in memory per iteration?
Any strategy or suggestion in improving this multiplication will be great.

Edit:

I was a little careless in my original minimal example. the real question is not about simply sum(x(:)*y,2) but more like a sum(foo(x(:)*y),2). there's no simple way to avoid the matrix multiplication. Sorry again for not making this clear.

Which version of MATLAB are you using and what on what machine? I construct vectors of the same size as yours

x = linspace(0,1,1000);
y = rand(1,1e8);

Sure enough, if I try a direct computation I get a memory error

>> A=sum(x(:)*y,2)
Error using  * 
Requested 1000x100000000 (745.1GB) array exceeds maximum array size preference (31.7GB). This might cause MATLAB to
become unresponsive.

Related documentation

So I try your loop

tic
Asplit=0;
for i=1:length(y)
    Asplit=Asplit+sum(x(:)*y(i),2);
end
toc

This completes in 74 seconds. Rather faster than the 45 minutes you quote.
I am using the MATLAB 2023a beta on an 11th Gen Intel. Core(TM) i7-11700 @ 2.50GHz.

I can also get that time down more by using a thread based parpool and a parfor loop

parpool('threads');

tic
Asplit=0;
parfor i=1:length(y)
    Asplit=Asplit+sum(x(:)*y(i),2);
end
toc

Down to 15.5 seconds on my 8 core system. 5x speedup with 8 CPUs. Not linear scaling but I'll take it.

Also note that I am using double precision here. You say that you are using single precision. Let's see what that looks like

%try again using single precision
x = single(linspace(0,1,1000));
y = rand(1,1e8,'single');

parpool('threads');

tic
Asplit=0;
parfor i=1:length(y)
    Asplit=Asplit+sum(x(:)*y(i),2);
end
toc

12.8 seconds. Faster but is it worth the loss in precision? I guess that depends on your application.

Finally, let's use the trick A=x(:)*sum(y) discussed in the comments by @beaker and others. Compare the result with the split method that we've been looking at so far.

x = linspace(0,1,1000);
y = rand(1,1e8);

parpool('threads');

tic
Asplit=0;
parfor i=1:length(y)
    Asplit=Asplit+sum(x(:)*y(i),2);
end
toc

tic
Aalt = x(:)*sum(y);
toc

max(abs(Aalt-Asplit))

x(:)*sum(y) took 0.026 seconds and the largest absolute difference between the two methods was of the order of 1e-6.

1.- memory bottleneck problem

No matter whatever you do with the complete forms x and y : if you push such data and operations as shown in the question, 45 minutes wait or out of memory warning sure shows up.

2.- Option 1 : Increase RAM

One can go as far as the deep is pocket or full is the purse. Good luck with it.

3.- Option 2 : Parallel Pool

Good luck with this one too.

4.- My Suggestion : Break down large variables into processable chunks

So this is:

• Break down the variables that jam memory

• Operate/process each chunk at a time

• Store results in disk

One way to do this with one of my favourite MATLAB commands : the all-weather, mighty and many-battles-hardened evalin (trumpets here)

I tried to break down y until I found that 100 pieces of it works. May be larger pieces may work too, but just 4 or 10 may not even start.

close all;clear all;clc

x=randi([-1e3 1e3],1,1e3);
y=randi([-1e3 1e3],1,1e8);

N=1e2

A1=zeros(numel(x),numel(y)/N);
A2=zeros(numel(x),numel(y)/N);
A3=zeros(numel(x),numel(y)/N);
A4=zeros(numel(x),numel(y)/N);


k1=2;
L1=['A' num2str(k1) '=zeros(numel(x),numel(y)/N);'];
evalin('base',L1)

tic

for k1=1:N
    
    for k2=1:length(x)/N
        a1=x(k2);
        a2=a1*y(k2);
        L1=['A' num2str(k1) '=zeros(numel(x),numel(y)/N);'];
        evalin('base',L1);
        L2=['A' num2str(k1) '(k2,:)=sum(a2);'];
        evalin('base',L2);
        
        % WRITE Ak1 to FILE
    end

end

Replace the line % WRITE Ak1 to FILE with something like

L3=['']   % build command line into string
evalin('base',L3);  % invoke evalin

MATLAB has a diverse array of commands to write and read files, in many formats, for instance .txt and Excel or directly to binary file with command fwrite .

To read from file you also have to build strings and invoke them with evalin but as shown above, building such concise strings is easy, because all that is done is building variables with code and an varying index has to be embedded in the declaration of such variables, the pieces of y that it has been broken into.

In a nutshell : use variables that can reasonably move in and out of RAM without ending up like that recent Suez canal traffic jam of a vessel blocking every one else.