不同的算法用于50-50 A/B测试

huangapple go评论42阅读模式
英文:

Different Algorithms for 50-50 A/B Testing

问题

我们正在对Web应用的客户进行A/B测试,根据customerId。每个客户将看到不同的次要UX更改。
试图防止使用功能标志,因为目前还没有在我们的系统中设置。

最初,我们尝试了基于CustomerId数字的奇偶比例50-50%来测试Feature 1。例如,UserId 4是偶数,7是奇数。
然而,当测试另一个Feature 2时,进行奇偶50-50%的操作会使Feature 1的组与Feature 2具有匹配的组,因为它们都共享相同的算法。

有没有另一种数学算法方法,可以运行散列或50-50%的算法,以便我可以区分开来?
我们可能会有10个要测试的功能,因此需要一种在FeatureFlag算法中添加参数的方法,并将其跟踪在文档表中。

顺便说一下:组应该稳定而不是随机的,例如,奇偶数将产生一致的结果。

英文:

We are running A/B tests on web app customers, given a customerId. Each customer will see different minor UX changes.
Trying to prevent usage of Feature Flags as its not currently setup yet in our system.

Initially we tried Even-Odd on CustomerId number, 50-50% ratio to test Feature 1. Example UserId 4 is even, 7 is odd.
However, when testing another Feature 2, doing Even-Odd 50-50% would make , Feature 1 Groups to have a matching group with Feature 2, as they both share Same algorithm.

What is another mathematical algorithm method, to run a hash or 50-50% algorithm , so I can differentiate?
We will have probably 10 Features to test, so need a way to add a parameter in the FeatureFlag Algorithm, and will track in a Document Table.

We are assigning groups with Javascript/Typescript btw.

Note: Groups should be steady and not random , eg Even-odd will give a consistent result.

答案1

得分: 1

以下是翻译好的部分:

为了获得每个客户ID和特征编号的一致且非随机的分组分配,您可以考虑使用以下方法:

  • 哈希 + 取模:

    我们可以使用一个简单的哈希函数来均匀分布客户ID,然后应用取模运算将它们分成组。

    • 我们将客户ID与特征编号一起进行哈希。
    • 然后,将结果模除一个大质数,以确保均匀分布。
    • 之后,我们使用模2将其分成两组。
    function hashCode(customerId, featureNum) {
        const str = customerId.toString() + featureNum.toString();
        let hash = 0;
        for (let i = 0; i < str.length; i++) {
            const char = str.charCodeAt(i);
            hash = ((hash << 5) - hash) + char;
            hash |= 0; // 转换为32位整数
        }
        return hash;
    }
    
    function assignGroup(customerId, featureNum) {
        const largePrime = 15485863; // 仅为示例,选择任何大质数
        const hash = hashCode(customerId, featureNum);
        return ((hash % largePrime) % 2) === 0 ? 'A' : 'B';
    }
    
  • 哈希,以更均匀且一致的方式分配ID,避免重叠。

    function hashFunc(customerId, featureNum) {
      let str = customerId.toString() + featureNum.toString();
      let hash = 0;
      for (let i = 0; i < str.length; i++) {
        let char = str.charCodeAt(i);
        hash = ((hash << 5) - hash) + char; // 这是一个简单而常用的哈希函数
        hash = hash & hash; // 转换为32位整数
      }
      return hash;
    }
    
    function assignGroup(customerId, featureNum) {
      return (hashFunc(customerId, featureNum) % 2) === 0 ? 'A' : 'B';
    }
    
  • 数字之和,使用客户ID和特征编号的数字之和,并使用结果将客户分成组:

    function sumOfDigits(number) {
      let sum = 0;
      while (number) {
        sum += number % 10;
        number = Math.floor(number / 10);
      }
      return sum;
    }
    
    function assignGroup(customerId, featureNum) {
      return (sumOfDigits(customerId + featureNum) % 2) === 0 ? 'A' : 'B';
    }
    
英文:

In order to get consistent and non-random group assignments for each customer ID and feature number, you can consider groups using:

  • Hashing + Modulus:

    We can make use of a simple hash function to distribute the customer IDs uniformly, then apply the modulus operation to divide them into groups.

    • we'll hash the customer ID together with the feature number.
    • The result is then modded by a large prime number to help ensure a uniform distribution.
    • After that, we use mod 2 to divide into two groups.
    function hashCode(customerId, featureNum) {
        const str = customerId.toString() + featureNum.toString();
        let hash = 0;
        for (let i = 0; i &lt; str.length; i++) {
            const char = str.charCodeAt(i);
            hash = ((hash &lt;&lt; 5) - hash) + char;
            hash |= 0; // Convert to 32bit integer
        }
        return hash;
    }
    
    function assignGroup(customerId, featureNum) {
        const largePrime = 15485863; // Just an example, choose any large prime number
        const hash = hashCode(customerId, featureNum);
        return ((hash % largePrime) % 2) === 0 ? &#39;A&#39; : &#39;B&#39;;
    }
    
  • Hashing, to distribute IDs more uniformly and consistently without such overlaps.

    function hashFunc(customerId, featureNum) {
      let str = customerId.toString() + featureNum.toString();
      let hash = 0;
      for (let i = 0; i &lt; str.length; i++) {
        let char = str.charCodeAt(i);
        hash = ((hash &lt;&lt; 5) - hash) + char; // This is a simple, commonly used hash function
        hash = hash &amp; hash; // Convert to a 32-bit integer
      }
      return hash;
    }
    
    function assignGroup(customerId, featureNum) {
      return (hashFunc(customerId, featureNum) % 2) === 0 ? &#39;A&#39; : &#39;B&#39;;
    }
    
  • Sum of Digits, with the sum of digits of the customer ID and feature number, and using the result to divide customers into groups:

    function sumOfDigits(number) {
      let sum = 0;
      while (number) {
        sum += number % 10;
        number = Math.floor(number / 10);
      }
      return sum;
    }
    
    function assignGroup(customerId, featureNum) {
      return (sumOfDigits(customerId + featureNum) % 2) === 0 ? &#39;A&#39; : &#39;B&#39;;
    }
    

答案2

得分: 1

以下是代码的中文翻译部分:

这是一个简单的解决方案:

const getGroup = (user_id, feature_id) => {
  const power = Math.pow(2, feature_id + 1)
  const groupLimit = power / 2
  return (user_id % power) < groupLimit ? 'A' : 'B'
}

const userCount = 13
const userIds = Array.from(Array(userCount).keys())

console.log(`在 ${userCount} 位用户上测试特性 0`)
userIds.forEach(user_id => console.log(`用户 ${user_id}: ${getGroup(user_id, 0)}`))

console.log(`在 ${userCount} 位用户上测试特性 1`)
userIds.forEach(user_id => console.log(`用户 ${user_id}: ${getGroup(user_id, 1)}`))

console.log(`在 ${userCount} 位用户上测试特性 2`)
userIds.forEach(user_id => console.log(`用户 ${user_id}: ${getGroup(user_id, 2)}`))

这将得到:
不同的算法用于50-50 A/B测试

英文:

Here is a simple solution:

  const getGroup = (user_id, feature_id) =&gt; {
    const power = Math.pow(2, feature_id + 1)
    const groupLimit = power / 2
    return (user_id % power) &lt; groupLimit ? &#39;A&#39; : &#39;B&#39;
  }


  const userCount = 13
  const userIds = Array.from(Array(userCount).keys())

  console.log(`TEST FEATURE 0 ON ${userCount} USERS`)
  userIds.forEach(user_id =&gt; console.log(`user ${user_id}: ${getGroup(user_id, 0)}`))

  console.log(`TEST FEATURE 1 ON ${userCount} USERS`)
  userIds.forEach(user_id =&gt; console.log(`user ${user_id}: ${getGroup(user_id, 1)}`))

  console.log(`TEST FEATURE 2 ON ${userCount} USERS`)
  userIds.forEach(user_id =&gt; console.log(`user ${user_id}: ${getGroup(user_id, 2)}`))

Which gives:
不同的算法用于50-50 A/B测试

huangapple
  • 本文由 发表于 2023年5月11日 05:40:58
  • 转载请务必保留本文链接:https://go.coder-hub.com/76222729.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定