如何在测试之间暂停执行常见任务,然后继续进行测试断言。

huangapple go评论79阅读模式
英文:

How to pause between tests to do a common task, then resume for assert in tests

问题

目前我有一个场景,每个测试中都发生了一个资源密集型的任务。

我想要做的是,让这个任务在所有测试都生成后执行,并在执行之间暂停。最后再断言一个条件。

例如,当前我有一个任务(假设是foo)正在运行。目前执行如下:

foo(var):
  SendAPIrequest_to_store(val) #--->> 这需要很长时间

test_A():
  var = 生成的数据
  foo(var)
  断言服务器中存在生成的数据

test_B():
  var = 生成的值
  foo(var)
  断言服务器中存在生成的数据

....

test_X():
  var = 生成的值
  foo(var)
  断言服务器中存在生成的数据

这个服务器可以一次添加多个数据 - 因此,我正在寻找一种实现以下行为的方式:

add_data(x):
  val = val + "," + x

foo(var):
  # 此函数将在所有测试数据已添加且所有测试当前都处于暂停状态时运行
  SendAPIrequest_to_store(val) #----> 这需要很长时间
  # 现在我们将继续执行从暂停的地方开始的测试
  # 因此,为test_A断言数据,然后继续为test_B等进行断言



test_A():
  var = 生成的数据
  add_data(var)
  暂停并转到下一个测试
  断言服务器中存在生成的数据

test_B():
  var = 生成的值
  add_data(var)
  暂停并转到下一个测试
  断言服务器中存在生成的数据

....

test_X():
  var = 生成的值
  add_data(var)
  暂停并转到下一个测试
  断言服务器中存在生成的数据

任何帮助或指针将不胜感激。提前感谢您。

英文:

Currently I have a scenario where there is a resource intensive task occurring in each test.
What I want to do is make this task execute after all tests have generated and pause between executions. Then finally assert a condition.
For example currently I have the task (let's say foo) running. This is currently executed as:

foo(var):
  SendAPIrequest_to_store(val) #--->> this takes a long time

test_A():
  var = generated data
  foo(var)
  assert generated data present in server

test_B():
  var = generated value
  foo(var)
  assert generated data present in server

....

test_X():
  var = generated value
  foo(var)
  assert generated data present in server

<br><br>
This server can, let's say add multiple data at once - hence instead of running the whole task individually, I am looking for a way to achieve the following behaviour:
<br>

add_data(x):
  val = val + &quot;,&quot; + x

foo(var):
  # this function will be run after all the test data has been added with all tests currently being suspended till generation
  SendAPIrequest_to_store(val) #----&gt; this takes a long time
  # now we will continue the execution for the tests from where it was paused in sequence
  # so assert data for test_A then continue assertion for test_B and so-on



test_A():
  var = generated data
  add_data(var)
  PAUSE and move to the next test
  assert generated data present in server

test_B():
  var = generated value
  add_data(var)
  PAUSE and move to the next test
  assert generated data present in server

....

test_X():
  var = generated value
  add_data(var)
  PAUSE and move to the next test
  assert generated data present in server

Any help or pointers will be greatly appreciated.
Thank you in advance

答案1

得分: 2

以下是您要的翻译内容:

这实际上是可以在测试中执行一个耗时函数,只需要在特定操作完成后执行(比如生成数据)。但是这个解决方案必须被视为一种权宜之计,而不是正式的实现(请参见末尾的注意事项)。

第一步是在模块级别定义两个fixture,这些fixture将在同一Python文件的测试之间共享方法和数据。

@pytest.fixture(scope="module")
def post_asserts() -> List[Callable[[], None]]:
    return []

@pytest.fixture(scope="module")
def data(post_asserts: List[Callable[[], None]]) -> Generator[List[str], None, None]:
    d = []  # 为在测试中聚合数据创建一个容器
    yield d
    foo(d)  # 耗时步骤,只运行一次。

    # 从模块中的测试运行所有后置断言语句
    for pa in post_asserts:
        pa()

具体来说:

  • fixture post_asserts 是一个指向将在测试结束时定义为内部函数的函数列表。每个测试将负责添加其后置断言函数(主要由断言语句组成的函数)。
  • fixture data 是一个列表,用作在测试中聚合数据的容器。该容器被yield,允许在fixture完成时运行代码(在模块的末尾,因为fixture是模块范围的,请参阅此帖子)。在这里,它用于运行所有后置断言方法。

每个使用foo函数的测试都必须将其数据和后置断言方法添加到两个fixture容器中。例如:

def test_A(post_asserts: List[Callable[[], None]], data: List[str]):
    data.append(any_generated_data_A)

    def test_A_post_asserts():
        assert True

    post_asserts.append(test_A_post_asserts)

def test_B(post_asserts: List[Callable[[], None]], data: List[str]):
    data.append(any_generated_data_B)

    def test_B_post_asserts():
        assert True

    post_asserts.append(test_B_post_asserts)

def test_X(post_asserts: List[Callable[[], None]], data: List[str]):
    data.append(any_generated_data_X)

    def test_X_post_asserts():
        assert False

    post_asserts.append(test_X_post_asserts)

运行测试时,每个测试都将其数据添加到共享的data容器和共享的post_asserts容器中。在完成data fixture时,将使用data容器中的聚合内容调用foo函数,并运行每个后置断言方法。这个包装过程只执行一次。

一个重要的注意事项是在测试运行后进行断言似乎不是标准做法。因此,pytest将不会检测到测试失败,而是检测到“流水线”的最后阶段发生了错误。由于后置断言方法在测试体中定义,您将能够检测到哪个测试失败了。但是,第一个失败的后置断言将阻止后续的后置断言运行。一个快速的解决方法是使用try/except,如下所示(使用traceback来打印堆栈,请参见此帖子):

@pytest.fixture(scope="module")
def data(post_asserts: List[Callable[[], None]]) -> Generator[List[str], None, None]:
    d = []
    yield d
    foo(d)

    exc_container: List[str] = []
    for pa in post_asserts:
        try:
            pa()
        except:
            exc_container.append(traceback.format_exc())

    if len(exc_container) > 0:
        raise AssertionError([str(exc) for exc in exc_container])
英文:

It is actually possible to run a time-consuming function once and after specific actions were completed in test (such as generating data). But this solution must be considered as a workaround and not a proper implementation (see the notice at the end).

The first step is to define two fixtures on the module level, that will share methods and data across tests from the same python file.

@pytest.fixture(scope=&quot;module&quot;)
def post_asserts() -&gt; List[Callable[[], None]]:
    return []


@pytest.fixture(scope=&quot;module&quot;)
def data(post_asserts: List[Callable[[], None]]) -&gt; Generator[List[str], None, None]:
    d = [] # create a container for aggregating data in tests
    
    yield d
    
    foo(d) # time consuming step, that is run once.

    # running all post assert statements from test in the module
    for pa in post_asserts:
        pa()

Namely:

  • the fixture post_asserts is a list pointing toward function that will be defined as inner function at test ends. Each test will be responsible for adding its post assert function (a function mainly composed by assert statement).
  • the fixture data is a list, acting as a container for aggregating data across tests. The container is yielded, allowing to run code when the fixture wraps up (at the end of the module, as the fixture is module scoped, see this post). Here, it is used to run all post assert methods.

Each tests that uses the foo function must add their data and post assert method to both fixture containers. For instance:

def test_A(post_asserts: List[Callable[[], None]], data: List[str]):
    data.append(any_generated_data_A)

    def test_A_post_asserts():
        assert True

    post_asserts.append(test_A_post_asserts)


def test_B(post_asserts: List[Callable[[], None]], data: List[str]):
    data.append(any_generated_data_B)

    def test_B_post_asserts():
        assert True

    post_asserts.append(test_B_post_asserts)


def test_X(post_asserts: List[Callable[[], None]], data: List[str]):
    data.append(any_generated_data_X)

    def test_X_post_asserts():
        assert False

    post_asserts.append(test_X_post_asserts)

When running tests, each test will add its data to the shared data container and the shared post_asserts container. When wrapping up the data fixture, the foo function will be called with aggregated content in the data container and each post assert method will be run. This wrapping-up process is done once.

An important notice is that asserting after tests run seems not standard. Thus, pytest will not detect that the test has failed, but that an error occurred in the final stage of the "pipeline". As the post assert method is defined in the test body, you will be able to detect which test has failed. However, the first failing post assert will prevent subsequent post assert from running. A quick workaround is to use a try/except as such (using traceback to print stack, see this post):

@pytest.fixture(scope=&quot;module&quot;)
def data(post_asserts: List[Callable[[], None]]) -&gt; Generator[List[str], None, None]:
    d = []
    yield d
    foo(d)

    exc_container: List[str] = []
    for pa in post_asserts:
        try:
            pa()
        except:
            exc_container.append(traceback.format_exc())

    if len(exc_container) &gt; 0:
        raise AssertionError([str(exc) for exc in exc_container])

答案2

得分: 1

你的测试至少是函数,因此你可以手动调用它们,例如:

import pytest

def test_a():
    a = 1
    test_b()
    assert 1 == 1

def test_b():
    a = 2
    assert 2 == 1

然而,我不建议采用这种方式。测试相互依赖并不理想。如果我正确理解你的问题,与其收集测试并使用这种方法来设置一个变量,然后继续运行所有测试,不如在所有测试之前执行一个步骤。这一步骤将设置需要设置的内容,然后执行测试,顺序将不再重要。

为了实现这一点,你可以使用带有 autousesession 的 fixture。示例如下:

import pytest

@pytest.fixture(scope="session")
def value():
    return []

# 这将在测试会话中首先执行
@pytest.fixture(autouse=True, scope="session")
def first(value):
    value.append(1)
    value.append(2)
    return value

def test_a(value):
    assert value[0] == 1

def test_b(value):
    assert value[1] == 2
英文:

Your tests are at least functions, so you can call them manually, for example:

import pytest

def test_a():
    a = 1
    test_b()
    assert 1 == 1

def test_b():
    a = 2
    assert 2 == 1

However, I would not have your approach. Having tests depending on each others is not ideal. If I understand correctly your problem, instead of collecting tests, using this "mecanism" to set a variable, and then resume all tests, you could just have a step that executes before all tests. This step will set the content of whatever needs to be set, and then the tests will be executed and the order will not matter.

To achieve that, you could use fixtures with autouse and session

For example:

import pytest

@pytest.fixture(scope=&quot;session&quot;)
def value():
    return []

# this will be executed at first for the test session
@pytest.fixture(autouse=True, scope=&quot;session&quot;)
def first(value):
    value.append(1)
    value.append(2)
    return value

def test_a(value):
    assert value[0] == 1

def test_b(value):
    assert value[1] == 2

huangapple
  • 本文由 发表于 2023年4月10日 21:28:57
  • 转载请务必保留本文链接:https://go.coder-hub.com/75977554.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定