浮点数在时间关键的C++循环中出现舍入误差,寻找高效的解决方案。

huangapple go评论70阅读模式
英文:

Float rounding error in time-critical C++ loop, looking for an efficient solution

问题

作为前提,我知道这个问题已经被解决了,但是从我找到的信息来看,从未在这个特定的情况下解决过。

在一个时间关键的代码片段中,我有一个循环,其中一个浮点值x必须线性增长,从精确地0增长到包括精确地1在内,共'z'步。

未经优化的解决方案,但是不会有舍入误差,是:

const int z =(某个数字);
int c;
float x;

for(c=0; c<z; c++)
{
   x = (float)c/(float)(z-1);
   // 在这里对x进行操作
}

显然,我可以避免浮点数转换,并使用两个循环变量和缓存(float)(z-1):

const int z =(某个数字);
int c;
float xi,x;
const float fzm1 =float)(z-1;

for(c=0,xi=0.f; c<z; c++, xi+=1.f)
{
   x=xi/fzm1;
   // 在这里对x进行操作
}

但是,谁会在每次循环中都重复除以一个常数呢?显然,任何人都会将它转换为一个乘法:

const int z =(某个数字);
int c;
float xi,x;
const float invzm1 = 1.f/(float)(z-1);

for(c=0,xi=0.f; c<z; c++, xi+=1.f)
{
   x=xi * invzm1;
   // 在这里对x进行操作
}

这里显然可能会出现明显的舍入问题。对于一些整数值的z,(z-1)*(1.f/(float)(z-1))将不会精确地给出1,而是0.999999...,所以在最后一个循环周期中,x所假定的值将不会是精确的1。

如果使用加法器,即

const int z =(某个数字);
int c;
float x;
const float x_adder = 1.f/(float)(z-1);

for(c=0,x=0.f; c<z; c++, x+=x_adder)
{
   // 在这里对x进行操作
}

情况会变得更糟,因为x_adder中的误差会累积。

所以,我唯一能想到的解决方案是在某个地方使用条件语句,例如:

const int z =(某个数字);
int c;
float xi,x;
const float invzm1 = 1.f/(float)(z-1);

for(c=0,xi=0.f; c<z; c++, xi+=1.f)
{
   x = (c==z-1) ? 1.f : xi * invzm1;
   // 在这里对x进行操作
}

但在一个时间关键的循环中,应该尽量避免使用分支!

哦,而且我甚至不能将循环分割并进行如下操作:

for(c=0,xi=0.f; c<z-1; c++, xi+=1.f) // 注意:现在循环运行到z-2
{
   x=xi * invzm1;
   // 在这里对x进行操作
}

x=1.f;
// 在这里对x进行操作

因为我将不得不复制整个代码块'do something with x',而它既不简短也不简单,我无法将它变成一个函数调用(效率低,要传递太多局部变量),也不想使用#defines(非常糟糕、不优雅和不实用)。你能想出任何高效或巧妙的解决方案吗?

英文:

As a premise, I am aware that this problem has been addressed already, but never in this specific scenario, from what I could find searching.

In a time-critical piece of code, I have a loop where a float value x must grow linearly from exactly 0 to-and-including exactly 1 in 'z' steps.

The un-optimized solution, but which would work without rounding errors, is:

const int z = (some number);
int c;
float x;

for(c=0; c&lt;z; c++)
{
   x = (float)c/(float)(z-1);
   // do something with x here
}

obviously I can avoid the float conversions and use two loop variables and caching (float)(z-1):

const int z = (some number);
int c;
float xi,x;
const float fzm1 = (float)(z-1);

for(c=0,xi=0.f; c&lt;z; c++, xi+=1.f)
{
   x=xi/fzm1;
   // do something with x
}

But who would ever repeat a division by a constant for every loop pass ? Obviously anyone would turn it into a multiplication:

const int z = (some number);
int c;
float xi,x;
const float invzm1 = 1.f/(float)(z-1);

for(c=0,xi=0.f; c&lt;z; c++, xi+=1.f)
{
   x=xi * invzm1;
   // do something with x
}

Here is where obvious rounding issues may start to manifest.
For some integer values of z, (z-1)*(1.f/(float)(z-1)) won't give exactly one but 0.999999..., so the value assumed by x in the last loop cycle won't be exactly one.

If using an adder instead, i.e

const int z = (some number);
int c;
float x;
const float x_adder = 1.f/(float)(z-1);

for(c=0,x=0.f; c&lt;z; c++, x+=x_adder)
{
   // do something with x
}

the situation is even worse, because the error in x_adder will build up.

So the only solution I can see is using a conditional somewhere, like:

const int z = (some number);
int c;
float xi,x;
const float invzm1 = 1.f/(float)(z-1);

for(c=0,xi=0.f; c&lt;z; c++, xi+=1.f)
{
   x = (c==z-1) ? 1.f : xi * invzm1;
   // do something with x
}

but in a time-critical loop a branch should be avoided if possible !

Oh, and I can't even split the loop and do


for(c=0,xi=0.f; c&lt;z-1; c++, xi+=1.f) // note: loop runs now up to and including z-2
{
   x=xi * invzm1;
   // do something with x
}

x=1.f;
// do something with x

because I would have to replicate the whole block of code 'do something with x' which is not short or simple either, I cannot make of it a function call (would be inefficient, too many local variables to pass) nor I want to use #defines (would be very poor and inelegant and impractical).

Can you figure out any efficient or smart solution to this problem ?

答案1

得分: 4

首先,一个一般性的考虑:xi += 1.f 引入了一个循环延迟依赖链,该链需要你的CPU进行浮点加法所需的周期数(可能是3或4)。它还会破坏任何尝试进行矢量化的努力,除非你使用 -ffast-math 编译。如果在现代超标量桌面CPU上运行,我建议使用整数计数器,并在每次迭代中进行浮点转换。

在我看来,避免整数到浮点的转换是过时的建议,来自x87 FPU时代。当然,你必须考虑整个循环才能得出最终的结论,但吞吐量通常与浮点加法相当。

对于实际的问题,我们可以看看其他人已经做过的事情,例如 Eigen 在他们的 LinSpaced 操作的 实现 中。还在他们的错误跟踪器中有一份相当广泛的讨论

他们的最终解决方案非常简单,我认为可以在这里进行改写,针对你的特定情况进行简化:

float step = 1.f / (n - 1);
for(int i = 0; i &lt; n; ++i)
    float x = (i + 1 == n) ? 1.f : i * step;

编译器可能会选择展开最后一次迭代以消除分支,但总体来说也不算太糟糕。在标量代码中,分支预测效果良好。在矢量化代码中,它涉及到一个打包的比较和混合指令。

我们还可以通过适当重组代码来强制决定展开最后一次迭代。Lambda 表达式在这方面非常有帮助,因为它们既方便使用又能够非常强大地进行内联。

auto loop_body = [&amp;](int i, float x) mutable {
    ...;
};
for(int i = 0; i &lt; n - 1; ++i)
    loop_body(i, i * step);
if(n &gt; 0)
    loop_body(n - 1, 1.f);

通过使用 Godbolt(使用简单的数组初始化进行循环体),GCC 只对第二个版本进行矢量化。Clang 对两者都进行了矢量化,但在第二个版本中做得更好。

英文:

First, a general consideration: xi += 1.f introduces a loop-carried dependency chain of however many cycles your CPU needs for floating point addition (probably 3 or 4). It also kills any attempt at vectorization unless you compile with -ffast-math. If you run on a modern super-scalar desktop CPU, I recommend using an integer counter and converting to float in each iteration.

In my opinion, avoiding int->float conversions is outdated advise from the era of x87 FPUs. Of course you have to consider the entire loop for the final verdict but the throughput is generally comparable to floating point addition.

For the actual problem, we may look at what others have done, for example Eigen in the implementation of their LinSpaced operation. There is also a rather extensive discussion in their bug tracker.

Their final solution is so simple that I think it is okay to paraphrase it here, simplified for your specific case:

float step = 1.f / (n - 1);
for(int i = 0; i &lt; n; ++i)
    float x = (i + 1 == n) ? 1.f : i * step;

The compiler may choose to peel off the last iteration to get rid of the branch but in general it is not too bad anyway. In scalar code branch prediction will work well. In vectorized code it's a packed compare and a blend instruction.

We may also force the decision to peel off the last iteration by restructuring the code appropriately. Lambdas are very helpful for this since they are a) convenient to use and b) very strongly inlined.

auto loop_body = [&amp;](int i, float x) mutable {
    ...;
};
for(int i = 0; i &lt; n - 1; ++i)
    loop_body(i, i * step);
if(n &gt; 0)
    loop_body(n - 1, 1.f);

Checking with Godbolt (using a simple array initialization for the loop body), GCC only vectorizes the second version. Clang vectorizes both but does a better job with the second.

答案2

得分: 1

你需要的是Bresenham的直线算法

它可以让你避免乘法和除法,只使用加法和减法。只需调整你的范围,使其可以用整数表示,并在最后阶段四舍五入,以确保精确地分割成部分在数学上(或“代表性上”)不可行。

英文:

What you need is Bresenham's line algorithm.

It would allow you to avoid multiplication and divisions and use add/sub only. Just scale your range so that it could be represented by integer numbers and round up at final stage if precise split to parts is mathematically (or "representatively") impossible.

答案3

得分: 0

以下是翻译好的部分:

考虑使用以下代码:

const int z = (某个数字 > 0);
const int step = 1000000/z;
for(int c=0; c<z-1; ++c)
{
   x += step; // 只有在真正需要转换时才使用,需要时除以1000000
   // 处理 x
}
x = 1.f;
// 用 x 完成最后一步

如果不真正需要转换,就不进行转换,第一个和最后一个值如预期,乘法变成了累积。通过更改1000000,您可以手动控制精度。
英文:

Consider using this:

const int z = (some number &gt; 0);
const int step = 1000000/z;
for(int c=0; c&lt;z-1; ++c)
{
   x += step; //just if you really need the conversion, divide it by 1000000 when required
   // do something with x
}
x = 1.f;
//do the last step with x

No conversions if you don't really need it, first and last values are as expected, multiplication is reduced to accumulation.
By changing of 1000000 you can manually control the precision.

答案4

得分: 0

我建议你从你展示的最后一个选择开始,并使用 lambda 表达式来避免传递局部变量:

auto do_something_with_x = [&](float x){/*...*/}
    
for(c=0,xi=0.f; c<z-1; c++, xi+=1.f) // 注意:现在循环运行直到 z-2
{
    x=xi * invzm1;
    do_something_with_x(x);
}
    
do_something_with_x(1.f);

如果还有其他需要翻译的部分,请继续提问。

英文:

I suggest that you start with the last alternative you have shown and use lambda to avoid passing local variables:

auto do_something_with_x = [&amp;](float x){/*...*/}

for(c=0,xi=0.f; c&lt;z-1; c++, xi+=1.f) // note: loop runs now up to and including z-2
{
   x=xi * invzm1;
   do_something_with_x(x);
}

do_something_with_x(1.f);

huangapple
  • 本文由 发表于 2023年2月7日 00:41:33
  • 转载请务必保留本文链接:https://go.coder-hub.com/75364164.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定