问题

I am trying to implement custom itertools.product because I don't to create the whole cartesian product but I want last couple of elements to be fixed. I just copied the code from here https://docs.python.org/3/library/itertools.html#itertools.product

    def product(*args, repeat=1):
        # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy
        # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111
        pools = [tuple(pool) for pool in args] * repeat
        result = [[]]
        for pool in pools:
            result = [x+[y] for x in result for y in pool]
        for prod in result:
            yield tuple(prod)

and tried to customize. However, even without any change it has different behaviour than

for el in itertools.product(range(2), 30)
    print(el)

Cases 1: the code snippet (function product) I provide from docs. First (as you can see the code) create the whole result and then yield one by one. But

Case 2.

for el in itertools.product(range(2), 30)
    print(el)

Yields the elements immediately which is something I would expect from generator because sometimes the product doesn't fit into my memory.

Does anybody know the real implementation of itertools.product?

    def product(*args, repeat=1):
        # product('ABCD', 'xy') --> Ax Ay Bx By Cx Cy Dx Dy
        # product(range(2), repeat=3) --> 000 001 010 011 100 101 110 111
        pools = [tuple(pool) for pool in args] * repeat
        result = [[]]
        for pool in pools:
            result = [x+[y] for x in result for y in pool]
        for prod in result:
            yield tuple(prod)

for el in itertools.product(range(2), 30)
    print(el)

for el in itertools.product(range(2), 30)
    print(el)

答案1

得分: 1

以下是代码部分的中文翻译：

这相当接近[实际实现][1]：

    def product(*args, repeat=1):
        pools = [tuple(pool) for pool in args] * repeat
        npools = len(pools)
        try:
            result = [pool[0] for pool in pools]
        except IndexError:
            return
        yield result
        indices = [0] * npools
        p = npools - 1
        while p >= 0:
            pool = pools

            indices
 += 1
            try:
                result
 = pool[indices
]
            except IndexError:
                indices
 = 0
                result
 = pool[indices
]
                p -= 1
                continue
            else:
                p = npools - 1
                yield tuple(result)

这是应用于您原始问题时的效果：

    >>> big = product(range(2), repeat=30)
    >>> next(big)
    [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    >>> next(big)
    (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
    >>> next(big)
    (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0)
    >>> next(big)
    (0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1)

[1]: https://github.com/python/cpython/blob/42f54d1f9244784fec99e0610aa05a5051e594bb/Modules/itertoolsmodule.c#L2192

def product(*args, repeat=1):
pools = [tuple(pool) for pool in args] * repeat
npools = len(pools)
try:
result = [pool[0] for pool in pools]
except IndexError:
return
yield result
indices = [0] * npools
p = npools - 1
while p >= 0:
pool = pools

indices
 += 1
try:
result
 = pool[indices
]
except IndexError:
indices
 = 0
result
 = pool[indices
]
p -= 1
continue
else:
p = npools - 1
yield tuple(result)

>>> big = product(range(2), repeat=30)
>>> next(big)
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
>>> next(big)
(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1)
>>> next(big)
(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0)
>>> next(big)
(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1)