Why are those algorithm so different in term of their execution time ? Is the walrus operator less time consuming in Python?

huangapple go评论59阅读模式
英文:

Why are those algorithm so different in term of their execution time ? Is the walrus operator less time consuming in Python?

问题

以下是您提供的内容的翻译部分:

在制作一个用来操作typing模块中的TypeVar的函数时,我遇到了一个问题。我的函数虽然能够工作,但我想让它更快,所以我使用了timeit模块来测试一些不同的版本。然而,结果并不如我所期望的那样。


澄清一些可能的疑问:

check_type(object, type)是一个用来测试对象是否与某种类型相符的函数(目前非常耗时)。

isTypeVar(type)用于测试类型是否为TypeVar。

NullType是一种不能描述对象的类型,因此当它与check_type进行测试时,它总是返回False。

restrict_TypeVar_for_object(typeVar, object)是我创建的函数,用于移除不符合对象的类型变量约束的所有约束。

我如何测量我的函数:

from timeit import timeit

func_version = """..."""
task="""..."""
timeit(task, setups=func_version, number=10_000_000)

这是我计时的任务(正如您上面所看到的,我重复了它10000000次):

Number = int | float
T = TypeVar("T", int, str, Number)

restrict_TypeVar_for_object(T, 5)
#   --> TypeVar("T", int, Number)

我的测试:

  • 第一版本:
def restrict_TypeVar_for_object(ty, obj):
    if not isTypeVar(ty): #只是为了确保ty是TypeVar
        raise ValueError ('Expected a TypeVar as an argument')

    if not ty.__constraints__:
        return type(obj)

    lCons=[cons for cons in ty.__constraints__ if check_type(obj, cons)]
    return (TypeVar(ty.__name__, *lCons)
            if len(lCons)>1 else
            lCons[0]
            if lCons else
            NullType)

耗时:378.5483768秒

  • 第二版本:
def restrict_TypeVar_for_object(ty, obj):
    if not isTypeVar(ty): #只是为了确保ty是TypeVar
        raise ValueError ('Expected a TypeVar as an argument')

    if not ty.__constraints__:
        return type(obj)

    return (TypeVar(ty.__name__, *lCons)
            if len(lCons:=[cons for cons in ty.__constraints__ if check_type(obj, cons)])>1 else
            lCons[0]
            if lCons else
            NullType)

耗时:376.9706139秒

  • 第三版本:
def restrict_TypeVar_for_object(ty, obj):
    if not isTypeVar(ty): #只是为了确保ty是TypeVar
        raise ValueError ('Expected a TypeVar as an argument')

    if not ty.__constraints__:
        return type(obj)

    return (TypeVar(ty.__name__, *lCons[:-1])
            if len(lCons:=[c for c in ty.__constraints__ if check_type(obj,c)]+[NullType]) > 2
            else
            lCons[0])

耗时:391.5145658000001秒


正如您所看到的,第二个版本是最快的。然而,我没有预料到第三个版本会与其他版本相比如此之慢。坦率地说,我甚至期望它会是最快的,因为在这个版本中我使用了更少的if语句。

我有两种假设可以解释这种情况:

  • 首先,是海象运算符,但如果是这样,为什么第二个版本会最快。

  • 其次,是我在lCons的末尾添加了[NullType],但我不知道为什么这会耗费这么多时间。

英文:

While making a function to manipulate the TypeVar from the typing module, I came across a problem.
My function do work but I wanted to make it faster so I used the timeit module to test some different versions.
However the result aren't what I expected.


Clearing some potential interrogation :

check_type(object , type) is a function meant to test if an object correspond to a type (quite time consuming currently).

isTypeVar(type) test if type is a TypeVar.

NullType is a type that cannot describe an object so when it's tested with check_type it always return False.

restrict_TypeVar_for_object(typeVar , object) is the function that I have made in order to remove all of the constraints of my typeVar that doesn't correspond to object.

How I time my functions :

from timeit import timeit

func_version = """..."""
task="""..."""
timeit(task , setups=func_version , number=10_000_000)

Here is the task that I have timed (as you can see above I have repeated it 10 000 000 times) :

Number = int | float
T = TypeVar("T" , int , str , Number)

restrict_TypeVar_for_object(T , 5)
#   --> TypeVar("T" , int , Number)

My test :

  • 1st version :
def restrict_TypeVar_for_object(ty , obj):
    if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
        raise ValueError ('Expected a TypeVar as an argument')

    if not ty.__constraints__ :
        return type(obj)

    lCons=[cons for cons in ty.__constraints__ if check_type(obj , cons)]
    return (TypeVar(ty.__name__ , *lCons)
            if len(lCons)>1 else
            lCons[0]
            if lCons else
            NullType)

Takes : 378.5483768 s

  • 2nd version :
def restrict_TypeVar_for_object(ty , obj):
    if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
        raise ValueError ('Expected a TypeVar as an argument')

    if not ty.__constraints__ :
        return type(obj)

    return (TypeVar(ty.__name__ , *lCons)
            if len(lCons:=[cons for cons in ty.__constraints__ if check_type(obj , cons)])>1 else
            lCons[0]
            if lCons else
            NullType)

Takes : 376.9706139 s

  • 3rd version :
def restrict_TypeVar_for_object(ty , obj):
    if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
        raise ValueError ('Expected a TypeVar as an argument')

    if not ty.__constraints__ :
        return type(obj)

    return (TypeVar(ty.__name__ , *lCons[:-1])
            if len(lCons:=[c for c in ty.__constraints__ if check_type(obj,c)]+[NullType]) > 2
            else
            lCons[0])

Takes : 391.5145658000001 s


As you can see the 2nd one is the quickest.
However I wasn't expecting the 3rd one to be that slow compared to the other.
To be frank I was even expecting it to be the quickest one because I did less if statement in this one.

I have 2 hypothesis for why this is the case :

-First, the walrus operator but if it is, why is the 2nd version the quickest.

-Second, the fact that I add [NullType] at the end of lCons but I don't know why it would be this time consuming.

答案1

得分: 1

以下是您要翻译的内容:

首先,正如 tdelaney 指出的,它们执行时间上的差异来自于第三个版本中创建许多列表

  • [c for c in ty.__constraints__ if check_type(obj,c)] 这对所有版本都是通用的。
  • [NullType],由 lCons + [NullType]lCons[:-1] 连接而成的列表,这些是第三个版本独有的。

现在是我问题的第二部分,关于 吃符号操作符的时间消耗

吃符号操作符比简单赋值操作花费更多的时间,如下面的测试所示:

from timeit import timeit

# 赋值语句:
tsk1 = ""
a = 5
""

# 赋值表达式:
tsk2 = ""
(a := 5)
""

print(timeit(tsk1, number=10_000_000))
print(timeit(tsk2, number=10_000_000))

重复 10,000,000 次的 tsk1 耗时:0.7440573999992921 秒

重复 10,000,000 次的 tsk2 耗时:1.0597749000007752 秒

我进行了多次测试,结果总是大致相同。


最后,这是我应该在 Python 中实现 restrict_TypeVar_for_object(typeVar, object) 函数的方式:

def restrict_TypeVar_for_object(ty, obj):
    if not isTypeVar(ty):  # 为了确保 ty 是 TypeVar
        raise ValueError('期望 TypeVar 作为参数')

    if not ty.__constraints__:
        return type(obj)

    lCons = [cons for cons in ty.__constraints__ if check_type(obj, cons)]
    return (TypeVar(ty.__name__, *lCons)
            if len(lCons) > 1 else
            [*lCons, NullType][0])

这个函数,重复 10,000,000 次,耗时:383.57044309999765 秒

然而,正如您所看到的,仍然需要太多时间,所以我尝试了吃符号操作符:

def restrict_by_object(ty, obj):
    if not isTypeVar(ty):  # 为了确保 ty 是 TypeVar
        raise ValueError('期望 TypeVar 作为参数')

    if not ty.__constraints__:
        return type(obj)

    return (TypeVar(ty.__name__, *lCons)
            if len(lCons:=[cons for cons in ty.__constraints__ if check_type(obj, cons)]) > 1 else
            [*lCons, NullType][0])

这需要的时间是:373.867099199997 秒

我仍然不知道为什么吃符号操作符在这种情况下更好,但一旦我理解了,将会更新这个答案。

英文:

First of all, as tdelaney pointed out, the difference in term of their execution time comes from the creation of many list in the 3rd version :

  • [c for c in ty.__constraints__ if check_type(obj,c)] which is common to all version.
  • [NullType], the list resulting from the concatenation lCons + [NullType] and lCons[:-1] which are exclusive to the 3rd version.

Now comes the 2nd part of my question, about the time consumption of the walrus operator.

The walrus operator takes more time than a simple assignation as shown by the test below :

from timeit import timeit

#Assignement statement :
tsk1="""
a = 5
"""

#Assignement expression :
tsk2="""
(a := 5)
"""

print(timeit(tsk1 , number=10_000_000))
print(timeit(tsk2 , number=10_000_000))

tsk1 repeated 10 000 000 takes : 0.7440573999992921 s

tsk2 repeated 10 000 000 takes : 1.0597749000007752 s

I did the test many time and the result are always approximately the same.


Finally, here is how I should implement my restrict_TypeVar_for_object(typeVar , object) function in Python :

def restrict_TypeVar_for_object(ty , obj):
    if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
        raise ValueError ('Expected a TypeVar as an argument')

    if not ty.__constraints__ :
        return type(obj)
    
    lCons = [cons for cons in ty.__constraints__ if check_type(obj , cons)]
    return (TypeVar(ty.__name__ , *lCons)
            if len(lCons)>1 else
            [*lCons , NullType][0])

Which, repeated 10 000 000 times takes : 383.57044309999765

However as you can see it still takes too much time so I have tried with the walrus operator :

def restrict_by_object(ty , obj):
    if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
        raise ValueError ('Expected a TypeVar as an argument')

    if not ty.__constraints__ :
        return type(obj)

    return (TypeVar(ty.__name__ , *lCons)
            if len(lCons:=[cons for cons in ty.__constraints__ if check_type(obj , cons)])>1 else
            [*lCons , NullType][0])

This takes : 373.867099199997 s

I still don't know why the walrus operator is better in this case but will update this answer once I understand it.

huangapple
  • 本文由 发表于 2023年5月7日 10:07:18
  • 转载请务必保留本文链接:https://go.coder-hub.com/76191928.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定