Trying to build a jacobian matrix using the multiprocessing library in python – how to share a matrix variable across multiple processes?

huangapple go评论65阅读模式
英文:

Trying to build a jacobian matrix using the multiprocessing library in python - how to share a matrix variable across multiple processes?

问题

以下是您提供的内容的翻译:

我试图在Python中使用中心差分法构建雅可比矩阵。我想要利用我可以访问的多个核心来构建这个矩阵。我尝试使用Process库。我不确定这个库还是pool库能让我利用更多核心来完成这个任务。关于哪个更好的建议将不胜感激!

总之,我的思路是我会定义一个空的NxN矩阵,然后让进程使用多进程填充这个矩阵的每一列。在我看来,输出是零矩阵(我最初实例化它为零矩阵),但当我查看每个进程的执行时,矩阵的列是正确的。

雅可比矩阵在进程1之后
[[2. 0.]
 [4. 0.]]
雅可比矩阵在进程2之后
[[0. 1.]
 [0. 4.]]
最终雅可比矩阵
[[0. 0.]
 [0. 0.]]
p = Process(target=jac_part, args=(i, x, h, Jac))

对我来说,似乎传递的"Jac"没有被视为一个正在被编辑的对象。有人有任何想法,我如何能够在不同的进程工作在矩阵上时保持状态?以下是一个简化的代码 - 任何建议将不胜感激。

谢谢

import numpy as np
import multiprocessing
from multiprocessing import Process

def Jacobian(f, h=1e-6):

    return lambda x: jacobian_pool(f, x, h=h)

def jacobian_pool(f, x, h):
    # pool = Pool(multiprocessing.cpu_count()-1)
    Jac = np.zeros((len(x), len(x)))
    processes = []
    for i in range(len(x)):
        p = Process(target=jac_part, args=(i, x, h, Jac))
        processes.append(p)

    for p in processes:
        p.start()
        p.join()

    return Jac

def jac_part(i, x, h, Jac):
    x_plus = x.copy()
    x_minus = x.copy()
    x_plus[i] += h
    x_minus[i] -= h
    f_plus = f(x_plus)
    f_minus = f(x_minus)
    Jac[:, i] = (f_plus - f_minus) / (2 * h)
    print(Jac)

def f(x):
    return np.array([x[0] ** 2 + x[1], 2 * x[0] ** 2 + x[1] ** 2])

x = np.array([1., 2.])
jf = Jacobian(f)
print(jf(x))

希望这有助于您理解代码的内容和问题。

英文:

I was trying to construct the jacobian matrix in python using central differences. I want to use multiple cores that I have access to to build this matrix. I tried to use the Process library. I wasn't sure if this or the pool library would let me utilize more cores to complete this task. Suggestions on what would be better would be greatly appreciated!

Anyhow, my thought process was that I would define an empty NxN matrix, and then have the process fill in every column of this matrix using multiprocessing. It looks to me the output is the zero matrix (what I instatiated it to be orginally), but when I look at every process execution, the column of the matrix is correct.

jacobian after process 1
[[2. 0.]
 [4. 0.]]
jacobian after process 2
[[0. 1.]
 [0. 4.]]
final jacobian
[[0. 0.]
 [0. 0.]]


p=Process(target=jac_part, args=(i,x,h,Jac))

It seems to me that the "Jac" I am passing is not being treated as an object that is being edited. Anyone have any ideas how I can maintain the state at the different process work on the matrix? Below is a bare bones code - any suggestions would greatly be appreciated.

Thank you

import numpy as np
import multiprocessing
from multiprocessing import Process

def Jacobian(f, h = 1e-6):
	
	return lambda x: jacobian_pool(f, x, h = h)

def jacobian_pool(f, x, h):
	#pool = Pool(multiprocessing.cpu_count()-1)
	Jac = np.zeros((len(x),len(x)))
	processes=[]
	for i in range(len(x)):
		p=Process(target=jac_part, args=(i,x,h,Jac))
		processes.append(p)

	for p in processes:
		p.start()
		p.join()
	
	return Jac


def jac_part(i,x,h,Jac):
    x_plus = x.copy()
    x_minus = x.copy()
    x_plus[i] += h
    x_minus[i] -= h
    f_plus = f(x_plus)
    f_minus = f(x_minus)
    Jac[:, i] = (f_plus - f_minus) / (2 * h)
    print(Jac)



def f(x):
		return np.array([x[0] ** 2 + x[1], 2 * x[0] ** 2 + x[1] ** 2])
x = np.array([1., 2.])
jf = Jacobian(f)
print(jf(x))

答案1

得分: 1

I was succesfull using pool. I use pool to calculate each column of the matrix, then in post-processing filled those columns into the actual matrix I am returning. Thanks for the suggestions all.

import numpy as np
import multiprocessing
from multiprocessing import Process
from multiprocessing import Pool

def Jacobian(f, h=1e-6):

    return lambda x: jacobian_pool(f, x, h=h)

def jacobian_pool(f, x, h):
    pool = Pool(multiprocessing.cpu_count() - 1)
    Jac = np.zeros((len(x), len(x)))
    async_results = [pool.apply_async(jac_part, args=(i, x, h)) for i in range(len(x))]
    results = [ar.get() for ar in async_results]
    for r in results:
        jac_pie = r[0]
        i = r[1]
        Jac[:, i] = jac_pie
    return Jac

def jac_part(i, x, h):
    x_plus = x.copy()
    x_minus = x copy()
    x_plus[i] += h
    x_minus[i] -= h
    f_plus = f(x_plus)
    f_minus = f(x_minus)
    # Jac[:, i] = (f_plus - f_minus) / (2 * h)
    return ((f_plus - f_minus) / (2 * h), i)

def f(x):
    return np.array([x[0] ** 2 + x[1], 2 * x[0] ** 2 + x[1] ** 2])

x = np.array([1., 2.])
jf = Jacobian(f)
print(jf(x))
英文:

I was succesfull using pool. I use pool to calculate each column of the matrix, then in post-processing filled those columns into the actual matrix I am returning. Thanks for the suggestions all.

import numpy as np
import multiprocessing
from multiprocessing import Process
from multiprocessing import Pool

def Jacobian(f, h = 1e-6):
	
	return lambda x: jacobian_pool(f, x, h = h)

def jacobian_pool(f, x, h):
    pool = Pool(multiprocessing.cpu_count()-1)
    Jac = np.zeros((len(x),len(x)))
    async_results = [pool.apply_async(jac_part, args=(i, x, h)) for i in range(len(x))]
    results = [ar.get() for ar in async_results]
    for r in results:
        jac_pie = r[0]
        i= r[1]
        Jac[:,i] = jac_pie
    return Jac
	# processes=[]
	# for i in range(len(x)):
	# 	p=Process(target=jac_part, args=(i,x,h,Jac))
	# 	processes.append(p)

	# for p in processes:
	# 	p.start()
	# 	p.join()
	
	# return Jac


def jac_part(i,x,h):
    x_plus = x.copy()
    x_minus = x.copy()
    x_plus[i] += h
    x_minus[i] -= h
    f_plus = f(x_plus)
    f_minus = f(x_minus)
    # Jac[:, i] = (f_plus - f_minus) / (2 * h)
    return ((f_plus - f_minus) / (2 * h),i)
    



def f(x):
		return np.array([x[0] ** 2 + x[1], 2 * x[0] ** 2 + x[1] ** 2])
x = np.array([1., 2.])
jf = Jacobian(f)
print(jf(x))

huangapple
  • 本文由 发表于 2023年5月7日 12:52:39
  • 转载请务必保留本文链接:https://go.coder-hub.com/76192245.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定