英文:
Calculating Expected Value With Matrix Values
问题
以下是翻译好的部分：
我有以下的输入数据
class_p = [0.0234375, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1748046875, 0.0439453125, 0.0, 0.35302734375, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.3828125]
league_p = [0.4765625, 0.0, 0.00634765625, 0.4658203125, 0.0, 0.0, 0.046875, 0.0, 0.0, 0.0029296875, 0.0, 0.0, 0.0, 0.0, 0.0]
a2_p = [0.1171875, 0.0, 0.0, 0.1171875, 0.0, 0.0078125, 0.30322265625, 0.31103515625, 0.0, 0.0, 0.0, 0.1435546875, 0.0, 0.0, 0.0]
p1_p = [0.0, 0.03125, 0.375, 0.09375, 0.0234375, 0.0, 0.46875, 0.0078125, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
p2_p = [0.3984375, 0.0, 0.0, 0.3828125, 0.08935546875, 0.08935546875, 0.023345947265625, 0.007720947265625, 0.0, 0.0, 0.0087890625, 0.00018310546875, 0.0, 0.0, 0.0]
class_v = [55, 75, 55, 75, 500, 10000, 55, 55, 55, 75, 75, 55, 55, 500, 55, 55, 75, 75, 55, 55, 55]
league_v = [0, 0, 0, 0, 0, 0, 0, 0, 40, 40, 40, 40, 1500, 1500, 3000]
a2_v= [0, 0, 0, 0, 0, 0, 0, 0, 40, 40, 40, 40, 1500, 1500, 3000]
p1_v = [0, 0, 0, 0, 0, 0, 0, 40, 40, 40, 40, 40, 1500, 1500, 3000]
p2_v = [0, 0, 0, 0, 0, 0, 0, 0, 40, 40, 40, 40, 1500, 1500, 3000]
使用这些数据，我正在生成每种组合发生的概率。
例如，为了生成给定组合的概率
class_p[0]
league_p[6]
a2_p[11]
p1_p[7]
p2_p[3]
我将它们的值相乘
`0.0234375x0.046875x0.1435546875x0.0078125x0.3828125`
这将给我 `4.716785042546689510345458984375 × 10^-7`
由于给定组合具有 `class_p[0], league_p[6], a2_p[11], p1_p[7], p2_p[3]`，我将在“values”数组中获取以下值。
我将计算
`class_v[0] + league_v[6] + a2_v[11] + p1_v[7] + p2_v[3]`
这将给我 `55+0+40+40+0 = 135`
为了完成这个过程，我会执行
`(0.0234375*0.046875*0.1435546875*0.0078125*0.3828125)*(55+0+40+40+0) =
0.00006367659807`
完整的最终计算是
`(0.0234375×0.046875×0.1435546875×0.0078125×0.3828125) (55 + 0 + 40 + 40 + 0)`
`(combination_chance)*(combination_value)`
我需要对所有可能的`combination_chance`执行这个过程
这将给我一列值（1xN）。如果我对该列值求和，我将得到整体的期望值，通过对各个组合的期望值进行求和。
计算`combination_chance` 工作正常。我的问题是如何将给定的组合与其对应的值求和（`combination_value`）对齐。目前，我在`*_p`数组上附加了额外的标识符，然后使用字符串比较来确定要使用哪个`combination`值。这对于数十亿次比较来说非常慢，因此我正在探索更好的方法。
我正在使用Python 3.8 和NumPy 1.24。
<details>
<summary>英文:</summary>
I have the following input data
class_p = [0.0234375, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1748046875, 0.0439453125, 0.0, 0.35302734375, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.3828125]
league_p = [0.4765625, 0.0, 0.00634765625, 0.4658203125, 0.0, 0.0, 0.046875, 0.0, 0.0, 0.0029296875, 0.0, 0.0, 0.0, 0.0, 0.0]
a2_p = [0.1171875, 0.0, 0.0, 0.1171875, 0.0, 0.0078125, 0.30322265625, 0.31103515625, 0.0, 0.0, 0.0, 0.1435546875, 0.0, 0.0, 0.0]
p1_p = [0.0, 0.03125, 0.375, 0.09375, 0.0234375, 0.0, 0.46875, 0.0078125, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
p2_p = [0.3984375, 0.0, 0.0, 0.3828125, 0.08935546875, 0.08935546875, 0.023345947265625, 0.007720947265625, 0.0, 0.0, 0.0087890625, 0.00018310546875, 0.0, 0.0, 0.0]
class_v = [55, 75, 55, 75, 500, 10000, 55, 55, 55, 75, 75, 55, 55, 500, 55, 55, 75, 75, 55, 55, 55]
league_v = [0, 0, 0, 0, 0, 0, 0, 0, 40, 40, 40, 40, 1500, 1500, 3000]
a2_v= [0, 0, 0, 0, 0, 0, 0, 0, 40, 40, 40, 40, 1500, 1500, 3000]
p1_v = [0, 0, 0, 0, 0, 0, 0, 40, 40, 40, 40, 40, 1500, 1500, 3000]
p2_v = [0, 0, 0, 0, 0, 0, 0, 0, 40, 40, 40, 40, 1500, 1500, 3000]
With that data, I am generating the odds of each combination occurring.
As an example to generate the chance of a given combination
class_p[0]
league_p[6]
a2_p[11]
p1_p[7]
p2_p[3]
I would multiply their values with each other
`0.0234375x0.046875x0.1435546875x0.0078125x0.3828125`
That would give me `4.716785042546689510345458984375 × 10^-7`
Since the given combination had `class_p[0], league_p[6], a2_p[11], p1_p[7], p2_p[3]`, I would take the following values in the "values" arrays.
I would sum
`class_v[0] + league_v[6] + a2_v[11] + p1_v[7] + p2_v[3]`
That would give me `55+0+40+40+0 = 135`
To finalize the process I would do
`(0.0234375*0.046875*0.1435546875*0.0078125*0.3828125)*(55+0+40+40+0) =
0.00006367659807`
The full final calc is
`(0.0234375×0.046875×0.1435546875×0.0078125×0.3828125) (55 + 0 + 40 + 40 + 0)`
`(combintation_chance)*(combination_value)`
I need to do this process for all possible combinations of `combintation_chance`
This should give me a column of values(1xN). If I sum the values of that column I reach the EV overall, by summing the EV of individual combinations.
Calculating `combintation_chance` is working just fine. My issue is how to line up the given combination with its corresponding value sum (`combination_value`). At the moment, I have additional identifiers attached to the `*_p` arrays and I then do a string comparison with them to determine which `combination` value to use. This is very slow for billions of comparisons, therefore I am exploring a better approach.
I am using python 3.8 & numpy 1.24
**Edit**
The question has been adjusted to include much more detail
</details>
# 答案1
**得分**: 1
# Broadcasting（广播）
好的，看起来这是一个简单的广播问题。
你想要一个概率的5D数组，乘以一个5D值的数组。当然，你希望这个过程不需要使用任何循环。
在NumPy中，让NumPy为你执行嵌套循环的经典方式（事实上，这比手动循环要快得多）是使用广播。
让我们从一个2D示例开始（这是你最初的意图。这是一个不错的主意。问题是有点模糊，但将问题限制在2D并不差）。
你有：
```python
class_p = np.array([0.0234375, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1748046875, 0.0439453125, 0.0, 0.35302734375, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.3828125])
league_p = np.array([0.4765625, 0.0, 0.00634765625, 0.4658203125, 0.0, 0.0, 0.046875, 0.0, 0.0, 0.0029296875, 0.0, 0.0, 0.0, 0.0, 0.0])
一种方法（不是唯一的方法，但可能是最容易适应类似问题的方法）是使用广播。
如果你将 class_p 转换成一个列，也就是一个 21x1 的2D数组，将 league_p 转换成一行，即一个1x15的2D数组，那么如果你将它们相乘，结果将是一个 21x15 的2D数组，其中包含了所有可能的组合。
因为
np.array([[1],[2],[3]]) * np.array([[4,5]])
是
[[4,5],
[8,10],
[12,15]]
这就是广播的工作原理。
有几种方法可以将1D数组转换为2D数组的行或列。例如，你可以使用 .reshape，像 class_p.reshape(-1,1) 和 league_p.reshape(1,-1)。但最快的方法是添加一个新的轴，像 class_p[:,None] 和 league_p[None,:]。注意，第二种方式实际上并没有创建一个新的数组，它只是同一个数组的不同视图。这就是为什么它更快的原因。
所以，我们的2D概率图是：
class_p[:,None]*league_p[None,:]
同样，为了得到所有值的和的21x15组合，你可以依赖相同的广播来执行加法：
class_v[:,None]+league_v[None,:]
广播解决方案
所以，在2D中，使用广播的解决方案是：
class_p[:,None]*league_p[None,:] * (class_v[:,None] + league_v[None,:])
在5D中，使用所有你的变量，它仍然是可管理的（但不要添加太多维度！结果会很庞大。而且我怀疑你最终真正感兴趣的是所有这些的总和），这次不是一行（尽管不是不可能以这种方式完成，但那将是一行很长的代码...）：
pr = class_p[:,None,None,None,None]*league_p[None,:,None,None,None]*a2_p[None,None,:,None,None]*p1_p[None,None,None,:,None]*p2_p[None,None,None,None,:]
vl = class_v[:,None,None,None,None]+league_v[None,:,None,None,None]+a2_v[None,None,:,None,None]+p1_v[None,None,None,:,None]+p2_v[None,None,None,None,:]
pr*vl
add.outer 和 multiply.outer
正如你所看到的，在5D中，这有点繁琐。但我想在介绍另一种方式之前向你展示广播的原理，这种方式并不会更短，但会稍微少一些麻烦。Reinderien已经提供过这种方式，但由于在你澄清问题之前，这不是正确的结果，但原则是相同的。
在2D中：
np.multiply.outer(class_p, league_p) * np.add.outer(class_v, league_v)
不幸的是，这些函数只接受2个参数。因此，在5D中，你必须将它们链接在一起：
pr = np.multiply.outer(class_p, np.multiply.outer(league_p, np.multiply.outer(a2_p, np.multiply.outer(p1_p, p2_p))))
vl = np.add.outer(class_v, np.add.outer(league_v, np.add.outer(a2_v, np.add.outer(p1_v, p2_v)))
pr * vl
期望值
请注意，如果所有这些的目的是计算期望的“值”（不管这个值是什么），即 Σ p(i,j,k,l,m)×v(i,j,k,l,m)，对于所有可能的结果，那么以这种方式做可能不是一个好主意。
对于你的示例，这是可以管理的。你计算了“仅仅”100万种可能的结果，即100万个概率（每个概率都有4个乘法运算）和100万个相关的值（每个值都有4个加法运算）。然后进行了100万次这两组100万个概率和值的乘法运算。然后对结果进行求和，这是额外的100万次加法运算。总共，这只是1000万次基本的算术操作。对于现代计算机来说并不多，而且
英文:
Broadcasting
Ok, so it seems that this is a simple broadcasting problem.
You want a 5D-array of probabilities, times a 5d-array of values. And, of course, you want it without any for loop.
In numpy the classical way to have numpy do nested loops for you (which is, indeed, way faster than doing them yourself. First rule of numpy is "avoid at all cost to iterate over elements. No for loop"), is to use broadcasting.
Let's start with 2d example (as was your first intention. And that was a good idea. Problem was it was ambiguous, but restraining your question to 2d was not bad).
You have
class_p = np.array([0.0234375, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.1748046875, 0.0439453125, 0.0, 0.35302734375, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.3828125])
league_p = np.array([0.4765625, 0.0, 0.00634765625, 0.4658203125, 0.0, 0.0, 0.046875, 0.0, 0.0, 0.0029296875, 0.0, 0.0, 0.0, 0.0, 0.0])
One way (not the only one, but probably the one easier to adapt to any similar question) is to use broadcasting.
If you indeed convert class_p in a column, that is a 21×1 2D array, and league_p into a line, that is a 1×15 2D array, then, if you multiply both, result will be a 21x15 2D array, containing all combinations.
Because
np.array([[1],[2],[3]]) * np.array([[4,5]])
is
[[4,5],
[8,10],
[12,15]]
That's how broadcasting works.
There are several way to convert a 1D-array so a row or a column of a 2D-array. For example you could use .reshape. Like class_p.reshape(-1,1) and league_p.reshape(1,-1). But the fastest is to add a new axis. Like class_p[:,None] and league_p[None,:]. Note that the second way doesn't really create a new array. It is just a different view of the same array. This is way it is faster.
So, our 2D probability map is
class_p[:,None]*league_p[None,:]
Likewise, to have all 21×15 combination of sum of values, you can rely on the same broadcasting to perform additon
class_v[:,None]+league_v[None,:]
Broadcasting solution
So solution, in 2D, using broadcasting, is
class_p[:,None]*league_p[None,:] * (class_v[:,None] + league_v[None,:])
In 5D, with all your variables, it is still manageable (but don't add too much dimensions! it would soon become a huge result. And I suspect what you are really interested at the end is just the sum of all that), this time, not in one line (not that it couldn't be done that way, but, that would be a big line...)
pr = class_p[:,None,None,None,None]*league_p[None,:,None,None,None]*a2_p[None,None,:,None,None]*p1_p[None,None,None,:,None]*p2_p[None,None,None,None,:]
vl = class_v[:,None,None,None,None]+league_v[None,:,None,None,None]+a2_v[None,None,:,None,None]+p1_v[None,None,None,:,None]+p2_v[None,None,None,None,:]
pr*vl
add.outer and multiply.outer
As you can see, in 5D, it is a little bit tedious. But I wanted to show you the principle of broadcasting, before introducing another (not really shorter, but a bit less tedious) way. Way that was already given by Reinderien. But since it was before you clarified the question, it was not the good result, but principle is the same
In 2D
np.multiply.outer(class_p, league_p) * np.add.outer(class_v, league_v)
Unfortunately, those function take only 2 args. So in 5D, you have to chain them
pr = np.multiply.outer(class_p, np.multiply.outer(league_p, np.multiply.outer(a2_p, np.multiply.outer(p1_p, p2_p))))
vl = np.add.outer(class_v, np.add.outer(league_v, np.add.outer(a2_v, np.add.outer(p1_v, p2_v))))
pr * vl
Expected value
Note that if the aim of all this is to compute the expected "value" (whatever that value is), that is Σ p(i,j,k,l,m)×v(i,j,k,l,m), for all possible outcomes, then, doing it that way is probably not a good idea.
For your example, it is manageable. You are computing "only" 1 million possible outcomes that is 1 million probabilities (each of them being 4 multiplications) and 1 million associated values (4 additions each). And the performing 1 million multiplication between those 2 sets of 1 million probabilities and values. And then summing the result, that is one extra million addition. Altogether, that is only 10 millions elementary arithmetic operation. Not much for a modern computer, and response still feels instantaneous. But, yet, it is O(Nᵏ) is both cpu and memory. N being the typical length of an array, and k the number of variables.
But if you intend to add more dimensions (more variables, associated with more set of probabilities and set of values), then that is unnecessary explosive, in both CPU time, and memory (those 5D arrays of probabilities and values are stored), or simply if you intend to perform this computation more than once, that expected value can be computed way faster, using just O(Nk) operations.
I spare you the development (but it is just a matter of expanding sum Σᵢⱼₖₗₘ pᵢpⱼpₖpₗpₘ (vᵢ+vⱼ+vₖ+vₗ+vₘ)), you can compute it faster like this
P1 = class_p.sum()
PV1 = (class_p*class_v).sum()
P2 = league_p.sum()
PV2 = (league_p*league_v).sum()
P3 = a2_p.sum()
PV3 = (a2_p*a2_v).sum()
P4 = p1_p.sum()
PV4 = (p1_p*p1_v).sum()
P5 = p2_p.sum()
PV5 = (p2_p*p2_v).sum()
expectedValue = P1*P2*P3*P4*PV5 + P1*P2*P3*PV4*P5 + P1*P2*PV3*P4*P5 + P1*PV2*P3*P4*P5 + PV1*P2*P3*P4*P5
sameAs = (pr*vl).sum()
It appears more complicated because there are more lines. But each line is along 1 dimension only. So it is replacing an order of magnitude of n₁n₂n₃n₄n₅ operations by an order of magnitude of n₁+n₂+n₃+n₄+n₅ operations, where n₁,...,n₅ are the size of arrays of each of the 5 variables.
So, again, if your objective is to compute expected value, then, trying to compute the 5D arrays (as your question is), is a really costly way.
答案2
得分: 0
这不会尝试缓存中间结果等。
import numpy as np
class_percentages = (0.0, 0.0, 0.0, 0.3, 0.50)
league_percentages = (0.1, 0.0, 0.2, 0.1, 0.05)
class_values = (50, 50, 50, 75, 100)
league_values = (0, 10, 10, 25, 75)
combined = np.add.outer(class_percentages, league_percentages)*np.add.outer(class_values, league_values)
print(combined)
输出：
[[ 5. 0. 12. 7.5 6.25]
[ 5. 0. 12. 7.5 6.25]
[ 5. 0. 12. 7.5 6.25]
[30. 25.5 42.5 40. 52.5 ]
[60. 55. 77. 75. 96.25]]
英文:
This doesn't make any attempt to cache intermediate results, etc.
import numpy as np
class_percentages = (0.0, 0.0, 0.0, 0.3, 0.50)
league_percentages = (0.1, 0.0, 0.2, 0.1, 0.05)
class_values = (50, 50, 50, 75, 100)
league_values = (0, 10, 10, 25, 75)
combined = np.add.outer(class_percentages, league_percentages)*np.add.outer(class_values, league_values)
print(combined)
Output:
[[ 5. 0. 12. 7.5 6.25]
[ 5. 0. 12. 7.5 6.25]
[ 5. 0. 12. 7.5 6.25]
[30. 25.5 42.5 40. 52.5 ]
[60. 55. 77. 75. 96.25]]
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论