英文:
Why are those algorithm so different in term of their execution time ? Is the walrus operator less time consuming in Python?
问题
以下是您提供的内容的翻译部分:
在制作一个用来操作typing模块中的TypeVar
的函数时,我遇到了一个问题。我的函数虽然能够工作,但我想让它更快,所以我使用了timeit模块来测试一些不同的版本。然而,结果并不如我所期望的那样。
澄清一些可能的疑问:
check_type(object, type)
是一个用来测试对象是否与某种类型相符的函数(目前非常耗时)。
isTypeVar(type)
用于测试类型是否为TypeVar。
NullType
是一种不能描述对象的类型,因此当它与check_type
进行测试时,它总是返回False。
restrict_TypeVar_for_object(typeVar, object)
是我创建的函数,用于移除不符合对象的类型变量约束的所有约束。
我如何测量我的函数:
from timeit import timeit
func_version = """..."""
task="""..."""
timeit(task, setups=func_version, number=10_000_000)
这是我计时的任务(正如您上面所看到的,我重复了它10000000次):
Number = int | float
T = TypeVar("T", int, str, Number)
restrict_TypeVar_for_object(T, 5)
# --> TypeVar("T", int, Number)
我的测试:
- 第一版本:
def restrict_TypeVar_for_object(ty, obj):
if not isTypeVar(ty): #只是为了确保ty是TypeVar
raise ValueError ('Expected a TypeVar as an argument')
if not ty.__constraints__:
return type(obj)
lCons=[cons for cons in ty.__constraints__ if check_type(obj, cons)]
return (TypeVar(ty.__name__, *lCons)
if len(lCons)>1 else
lCons[0]
if lCons else
NullType)
耗时:378.5483768秒
- 第二版本:
def restrict_TypeVar_for_object(ty, obj):
if not isTypeVar(ty): #只是为了确保ty是TypeVar
raise ValueError ('Expected a TypeVar as an argument')
if not ty.__constraints__:
return type(obj)
return (TypeVar(ty.__name__, *lCons)
if len(lCons:=[cons for cons in ty.__constraints__ if check_type(obj, cons)])>1 else
lCons[0]
if lCons else
NullType)
耗时:376.9706139秒
- 第三版本:
def restrict_TypeVar_for_object(ty, obj):
if not isTypeVar(ty): #只是为了确保ty是TypeVar
raise ValueError ('Expected a TypeVar as an argument')
if not ty.__constraints__:
return type(obj)
return (TypeVar(ty.__name__, *lCons[:-1])
if len(lCons:=[c for c in ty.__constraints__ if check_type(obj,c)]+[NullType]) > 2
else
lCons[0])
耗时:391.5145658000001秒
正如您所看到的,第二个版本是最快的。然而,我没有预料到第三个版本会与其他版本相比如此之慢。坦率地说,我甚至期望它会是最快的,因为在这个版本中我使用了更少的if语句。
我有两种假设可以解释这种情况:
-
首先,是海象运算符,但如果是这样,为什么第二个版本会最快。
-
其次,是我在
lCons
的末尾添加了[NullType]
,但我不知道为什么这会耗费这么多时间。
英文:
While making a function to manipulate the TypeVar
from the typing module, I came across a problem.
My function do work but I wanted to make it faster so I used the timeit module to test some different versions.
However the result aren't what I expected.
Clearing some potential interrogation :
check_type(object , type)
is a function meant to test if an object correspond to a type (quite time consuming currently).
isTypeVar(type)
test if type is a TypeVar.
NullType
is a type that cannot describe an object so when it's tested with check_type
it always return False.
restrict_TypeVar_for_object(typeVar , object)
is the function that I have made in order to remove all of the constraints of my typeVar that doesn't correspond to object.
How I time my functions :
from timeit import timeit
func_version = """..."""
task="""..."""
timeit(task , setups=func_version , number=10_000_000)
Here is the task that I have timed (as you can see above I have repeated it 10 000 000 times) :
Number = int | float
T = TypeVar("T" , int , str , Number)
restrict_TypeVar_for_object(T , 5)
# --> TypeVar("T" , int , Number)
My test :
- 1st version :
def restrict_TypeVar_for_object(ty , obj):
if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
raise ValueError ('Expected a TypeVar as an argument')
if not ty.__constraints__ :
return type(obj)
lCons=[cons for cons in ty.__constraints__ if check_type(obj , cons)]
return (TypeVar(ty.__name__ , *lCons)
if len(lCons)>1 else
lCons[0]
if lCons else
NullType)
Takes : 378.5483768 s
- 2nd version :
def restrict_TypeVar_for_object(ty , obj):
if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
raise ValueError ('Expected a TypeVar as an argument')
if not ty.__constraints__ :
return type(obj)
return (TypeVar(ty.__name__ , *lCons)
if len(lCons:=[cons for cons in ty.__constraints__ if check_type(obj , cons)])>1 else
lCons[0]
if lCons else
NullType)
Takes : 376.9706139 s
- 3rd version :
def restrict_TypeVar_for_object(ty , obj):
if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
raise ValueError ('Expected a TypeVar as an argument')
if not ty.__constraints__ :
return type(obj)
return (TypeVar(ty.__name__ , *lCons[:-1])
if len(lCons:=[c for c in ty.__constraints__ if check_type(obj,c)]+[NullType]) > 2
else
lCons[0])
Takes : 391.5145658000001 s
As you can see the 2nd one is the quickest.
However I wasn't expecting the 3rd one to be that slow compared to the other.
To be frank I was even expecting it to be the quickest one because I did less if statement in this one.
I have 2 hypothesis for why this is the case :
-First, the walrus operator but if it is, why is the 2nd version the quickest.
-Second, the fact that I add [NullType]
at the end of lCons
but I don't know why it would be this time consuming.
答案1
得分: 1
以下是您要翻译的内容:
首先,正如 tdelaney 指出的,它们执行时间上的差异来自于第三个版本中创建许多列表:
[c for c in ty.__constraints__ if check_type(obj,c)]
这对所有版本都是通用的。[NullType]
,由lCons + [NullType]
和lCons[:-1]
连接而成的列表,这些是第三个版本独有的。
现在是我问题的第二部分,关于 吃符号操作符的时间消耗。
吃符号操作符比简单赋值操作花费更多的时间,如下面的测试所示:
from timeit import timeit
# 赋值语句:
tsk1 = ""
a = 5
""
# 赋值表达式:
tsk2 = ""
(a := 5)
""
print(timeit(tsk1, number=10_000_000))
print(timeit(tsk2, number=10_000_000))
重复 10,000,000 次的 tsk1 耗时:0.7440573999992921 秒
重复 10,000,000 次的 tsk2 耗时:1.0597749000007752 秒
我进行了多次测试,结果总是大致相同。
最后,这是我应该在 Python 中实现 restrict_TypeVar_for_object(typeVar, object)
函数的方式:
def restrict_TypeVar_for_object(ty, obj):
if not isTypeVar(ty): # 为了确保 ty 是 TypeVar
raise ValueError('期望 TypeVar 作为参数')
if not ty.__constraints__:
return type(obj)
lCons = [cons for cons in ty.__constraints__ if check_type(obj, cons)]
return (TypeVar(ty.__name__, *lCons)
if len(lCons) > 1 else
[*lCons, NullType][0])
这个函数,重复 10,000,000 次,耗时:383.57044309999765 秒
然而,正如您所看到的,仍然需要太多时间,所以我尝试了吃符号操作符:
def restrict_by_object(ty, obj):
if not isTypeVar(ty): # 为了确保 ty 是 TypeVar
raise ValueError('期望 TypeVar 作为参数')
if not ty.__constraints__:
return type(obj)
return (TypeVar(ty.__name__, *lCons)
if len(lCons:=[cons for cons in ty.__constraints__ if check_type(obj, cons)]) > 1 else
[*lCons, NullType][0])
这需要的时间是:373.867099199997 秒
我仍然不知道为什么吃符号操作符在这种情况下更好,但一旦我理解了,将会更新这个答案。
英文:
First of all, as tdelaney pointed out, the difference in term of their execution time comes from the creation of many list in the 3rd version :
[c for c in ty.__constraints__ if check_type(obj,c)]
which is common to all version.[NullType]
, the list resulting from the concatenationlCons + [NullType]
andlCons[:-1]
which are exclusive to the 3rd version.
Now comes the 2nd part of my question, about the time consumption of the walrus operator.
The walrus operator takes more time than a simple assignation as shown by the test below :
from timeit import timeit
#Assignement statement :
tsk1="""
a = 5
"""
#Assignement expression :
tsk2="""
(a := 5)
"""
print(timeit(tsk1 , number=10_000_000))
print(timeit(tsk2 , number=10_000_000))
tsk1 repeated 10 000 000 takes : 0.7440573999992921 s
tsk2 repeated 10 000 000 takes : 1.0597749000007752 s
I did the test many time and the result are always approximately the same.
Finally, here is how I should implement my restrict_TypeVar_for_object(typeVar , object)
function in Python :
def restrict_TypeVar_for_object(ty , obj):
if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
raise ValueError ('Expected a TypeVar as an argument')
if not ty.__constraints__ :
return type(obj)
lCons = [cons for cons in ty.__constraints__ if check_type(obj , cons)]
return (TypeVar(ty.__name__ , *lCons)
if len(lCons)>1 else
[*lCons , NullType][0])
Which, repeated 10 000 000 times takes : 383.57044309999765
However as you can see it still takes too much time so I have tried with the walrus operator :
def restrict_by_object(ty , obj):
if not isTypeVar(ty) : #Just to be sure that ty is a TypeVar
raise ValueError ('Expected a TypeVar as an argument')
if not ty.__constraints__ :
return type(obj)
return (TypeVar(ty.__name__ , *lCons)
if len(lCons:=[cons for cons in ty.__constraints__ if check_type(obj , cons)])>1 else
[*lCons , NullType][0])
This takes : 373.867099199997 s
I still don't know why the walrus operator is better in this case but will update this answer once I understand it.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论