coverage per test with gradle using jacoco offline instrumentation – java.lang.IllegalStateException: JaCoCo agent not started

huangapple
go评论75阅读模式
英文:

coverage per test with gradle using jacoco offline instrumentation - java.lang.IllegalStateException: JaCoCo agent not started

问题

我正在尝试使用Gradle + Jacoco实现每个测试的覆盖率统计。

思路是创建一个org.gradle.api.tasks.testing.TestListener的实现，在常规事件（测试开始和测试结束）时重置会话ID并生成覆盖率报告。

（完整的项目示例可在https://github.com/jayanmn/gradle-coverage-per-test-debug/tree/main找到）

使用Ant任务在Gradle内部执行Jacoco的离线仪器（offline instrumentation）。由于测试环境包括某些测试（有点集成类型），因此采用离线方式。

构建脚本（build.gradle）部分如下：

buildscript {
    repositories {
        jcenter()
        mavenCentral()
    }
    dependencies {
        classpath group: 'org.jacoco', name: 'org.jacoco.agent', version: '0.8.5', classifier: 'runtime'
        classpath group: 'org.jacoco', name: 'org.jacoco.core', version: '0.8.5'
    }
}

dependencies {
    // 离线仪器通过Ant任务
    jacocoAnt group: 'org.jacoco', name: 'org.jacoco.ant', version: '0.8.5', classifier: 'nodeps'
    // Jacoco运行时（用于RT.getAgent()）
    jacocoRuntime group: 'org.jacoco', name: 'org.jacoco.agent', version: '0.8.5', classifier: 'runtime'
    testImplementation 'junit:junit:4.12'
}

测试任务（sampletest）部分如下：

task sampletest(type: Test) {
    ignoreFailures = true
    println classpath.asPath

    forkEvery 1
    maxParallelForks 1

    doFirst {
        println "org.jacoco.agent.rt.RT loaded from " + RT.class.getProtectionDomain().getCodeSource()

        systemProperty 'jacoco-agent.destfile', buildDir.path + '/jacoco/firsttests.exec'
        classpath = files(instrument.outputDir) + classpath + configurations.jacocoRuntime
    }

    beforeTest { TestDescriptor descriptor ->
        println "org.jacoco.agent.rt.RT loaded from " + RT.class.getProtectionDomain().getCodeSource()
    }

    afterTest { TestDescriptor descriptor ->
        systemProperty 'jacoco-agent.destfile', buildDir.path + '/jacoco/aftertests.exec'
        
        def agent = RT.getAgent()
        agent.sessionId = descriptor.name
        agent.dump(true)
    }
}

您提到的问题可能与“buildscript”和“test”块的类加载器有关。添加“jacoco-agent-runtime”的正确方法是通过在“dependencies”部分中声明该依赖，就像上面的代码示例中所做的那样。

英文:

I am trying to do a coverage per test using gradle+jacoco.

Idea is to have an implementation of org.gradle.api.tasks.testing.TestListener, and reset session-id at regular events (test start and test end) and dump coverage.

(Entire project to demo the problem is at https://github.com/jayanmn/gradle-coverage-per-test-debug/tree/main )

Instrumentation is done via jacoco-offline instrumentation using ant tasks inside gradle. (offline as the testing environment includes some tests - somewhat integration type ).

buildscript {
	repositories {
		jcenter()
		mavenCentral()
	}
	dependencies {
		classpath group: 'org.jacoco', name: 'org.jacoco.agent', version: '0.8.5', classifier: 'runtime'
		classpath group: 'org.jacoco', name: 'org.jacoco.core', version: '0.8.5'
	}
}

The dependency section is like below

	dependencies {
	//offline instrument via ant task
	jacocoAnt group: 'org.jacoco', name: 'org.jacoco.ant', version: '0.8.5', classifier: 'nodeps'

	//the jacoco runtime private for RT.getAgent()
	jacocoRuntime group: 'org.jacoco', name: 'org.jacoco.agent', version: '0.8.5', classifier: 'runtime'

	testImplementation 'junit:junit:4.12'
}

Running of test (sampletest)

task sampletest(type: Test) {
	ignoreFailures = true
	//jars used every step is same.
	println classpath.asPath

	forkEvery 1
	maxParallelForks 1

	doFirst {
		println "org.jacoco.agent.rt.RT loaded from " + RT.class.getProtectionDomain().getCodeSource()

		systemProperty 'jacoco-agent.destfile', buildDir.path + '/jacoco/firsttests.exec'
		classpath = files(instrument.outputDir) + classpath + configurations.jacocoRuntime

	}
	beforeTest { TestDescriptor descriptor ->
		println "org.jacoco.agent.rt.RT loaded from " + RT.class.getProtectionDomain().getCodeSource()
	}

	afterTest { TestDescriptor descriptor ->
		//get agent will fail with Agent not started.
		systemProperty 'jacoco-agent.destfile', buildDir.path + '/jacoco/aftertests.exec'

		//agent not started error
		def agent = RT.getAgent()
		agent.sessionId = descriptor.name
		agent.dump(true)

	}
}

For some reason

> java.lang.IllegalStateException: JaCoCo agent not started.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':sampletest'.
		at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:207)
		at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:263)
		at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:205)
		at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:186)
		at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:114)
		at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
		at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:62)
		at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
		at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
		at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
		at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
		at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
		at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
		at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:409)
		at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:399)
		at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:157)
		at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:242)
		at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:150)
		at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:94)
		at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
		at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
		at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:41)
		at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:356)
		at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
		at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
		at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
		at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
		at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
		at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
		at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
		at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
		at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
		at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:56)
Caused by: java.lang.IllegalStateException: JaCoCo agent not started.
		at org.jacoco.agent.rt.internal_43f5073.Agent.getInstance(Agent.java:77)
		at org.jacoco.agent.rt.RT.getAgent(RT.java:33)
		at org.jacoco.agent.rt.RT$getAgent.call(Unknown Source)
		at build_q0s35v9r0an7s8a7ayxelgpr$_run_closure6$_closure13.doCall(C:\repos\number-utils\build.gradle:91)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at org.gradle.listener.ClosureBackedMethodInvocationDispatch.dispatch(ClosureBackedMethodInvocationDispatch.java:41)
		at org.gradle.listener.ClosureBackedMethodInvocationDispatch.dispatch(ClosureBackedMethodInvocationDispatch.java:25)
		at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
		at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:245)
		at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:157)
		at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:58)
		at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:346)
		at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:249)
		at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:141)
		at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
		at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
		at com.sun.proxy.$Proxy69.afterTest(Unknown Source)
		at org.gradle.api.internal.tasks.testing.results.TestListenerAdapter.completed(TestListenerAdapter.java:50)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
		at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
		at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
		at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:245)
		at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:157)
		at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:58)
		at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:346)
		at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:249)
		at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:141)
		at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
		at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
		at com.sun.proxy.$Proxy71.completed(Unknown Source)
		at org.gradle.api.internal.tasks.testing.results.StateTrackingTestResultProcessor.completed(StateTrackingTestResultProcessor.java:96)
		at org.gradle.api.internal.tasks.testing.results.AttachParentTestResultProcessor.completed(AttachParentTestResultProcessor.java:56)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)

Problem is likely to be with classloaders of "buildscript" and test block? What is the correct way to add jacoco-agent-runtime ?

答案1

得分: 1

afterTest org.gradle.api.tasks.testing.TestListener 是由 Gradle 在 Gradle VM 中执行的，但不会在使用 JaCoCo 进行测试的 VM 中执行。

作为证据：

对于 src/test/java/ExampleTest.java

public class ExampleTest {
  @org.junit.Test
  public void test() {
    System.out.println(java.lang.management.ManagementFactory.getRuntimeMXBean().getName());
  }
}

以及 build.gradle

plugins {
  id "java"
}

repositories {
    mavenCentral()
}

dependencies {
    testImplementation "junit:junit:4.12"
}

test {
    testLogging.showStandardStreams = true
    afterTest {
        System.println(java.lang.management.ManagementFactory.getRuntimeMXBean().getName())
    }
}

执行命令 gradle build 将会产生类似以下输出：

> Task :test

ExampleTest > test STANDARD_OUT
    33542@Godin.local
33018@Godin.local

其中 33542 和 33018 是 Java 进程的标识，可以看到它们是不同的。

org.jacoco.agent.rt.RT 只能与在同一 VM 中运行的 JaCoCo 进行通信。

要与另一个 VM 中的 JaCoCo 进行通信，可以使用代理的 output 模式 tcpclient 和 tcpserver，或者通过使用代理选项 jmx=true 来通过 JMX 进行通信 - 参见 https://www.jacoco.org/jacoco/trunk/doc/agent.html 对于离线 instrument 的情况，相同的选项也适用 - 请参阅 https://www.eclemma.org/jacoco/trunk/doc/offline.html

思路是实现一个 org.gradle.api.tasks.testing.TestListener，并在定期事件（测试开始和测试结束）时重置会话 ID 并进行覆盖率转储。

JUnit 4 有 org.junit.runner.notification.RunListener，它由 JUnit 在测试的 VM 中执行，在 https://github.com/SonarSource/sonar-java/blob/5.14.0.18788/sonar-jacoco-listeners/src/main/java/org/sonar/java/jacoco/JUnitListener.java 可以找到类似于您想法的实现示例。

不幸的是，根据 https://github.com/gradle/gradle/issues/1330，截至今天，在 Gradle 中没有简单的方法来配置 JUnit 4 使用 org.junit.runner.notification.RunListener，但似乎可以配置 JUnit 5 来使用其 org.junit.platform.launcher.TestExecutionListener。

英文:

afterTest org.gradle.api.tasks.testing.TestListener is executed by Gradle in the VM for Gradle, but not in the VM for tests with JaCoCo.

For the proof:

for src/test/java/ExampleTest.java

public class ExampleTest {
  @org.junit.Test
  public void test() {
    System.out.println(java.lang.management.ManagementFactory.getRuntimeMXBean().getName());
  }
}

and build.gradle

plugins {
  id "java"
}

repositories {
    mavenCentral()
}

dependencies {
    testImplementation "junit:junit:4.12"
}

test {
    testLogging.showStandardStreams = true
    afterTest {
        System.println(java.lang.management.ManagementFactory.getRuntimeMXBean().getName())
    }
}

execution of command gradle build will produce something like

> Task :test

ExampleTest > test STANDARD_OUT
    33542@Godin.local
33018@Godin.local

two numbers 33542 and 33018 are IDs of Java processes, and as you can see they are different.

org.jacoco.agent.rt.RT can be used to communicate only with JaCoCo running in the same VM.

To communicate with JaCoCo in another VM you can use agent output modes tcpclient and tcpserver, or via JMX using agent option jmx=true - see https://www.jacoco.org/jacoco/trunk/doc/agent.html And the same options available for the case of offline instrumentation - see https://www.eclemma.org/jacoco/trunk/doc/offline.html

> Idea is to have an implementation of org.gradle.api.tasks.testing.TestListener, and reset session-id at regular events (test start and test end) and dump coverage.

JUnit 4 has org.junit.runner.notification.RunListener which is executed inside VM for tests by JUnit, an example of its implementation similar to your idea can be found at https://github.com/SonarSource/sonar-java/blob/5.14.0.18788/sonar-jacoco-listeners/src/main/java/org/sonar/java/jacoco/JUnitListener.java

Unfortunately according to https://github.com/gradle/gradle/issues/1330
seems that as of today there is no easy way in Gradle to configure JUnit 4 to use org.junit.runner.notification.RunListener, however seems possible to configure JUnit 5 to use its org.junit.platform.launcher.TestExecutionListener.

答案2

得分: -1

你还需要初始化 Jacoco 插件。以下是 Maven 的示例，但我相信 Gradle 也有类似的东西：

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>${jacoco.version}</version>
    <executions>
        <execution>
            <id>jacoco-initialize</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
            <configuration>
                <destFile>${basedir}\target\jacoco.exec</destFile>
                <propertyName>argLine</propertyName>
            </configuration>
        </execution>
        <execution>
            <id>jacoco-site</id>
            <phase>package</phase>
            <goals>
                <goal>report</goal>
            </goals>
            <configuration>
                <dataFile>${basedir}\target\jacoco.exec</dataFile>
                <outputDirectory>${basedir}\target</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>

如果有其他需要，请告诉我。

英文:

You also need thejacoco agent initialize plugin. Here's the maven example, but I am sure there must be something similar for gradle:

   &lt;plugin&gt;
                    &lt;groupId&gt;org.jacoco&lt;/groupId&gt;
                    &lt;artifactId&gt;jacoco-maven-plugin&lt;/artifactId&gt;
                    &lt;version&gt;${jacoco.version}&lt;/version&gt;
                    &lt;executions&gt;
                        &lt;execution&gt;
                            &lt;id&gt;jacoco-initialize&lt;/id&gt;
                            &lt;goals&gt;
                                &lt;goal&gt;prepare-agent&lt;/goal&gt;
                            &lt;/goals&gt;
                            &lt;configuration&gt;
                                &lt;destFile&gt;${basedir}\target\jacoco.exec&lt;/destFile&gt;
                                &lt;propertyName&gt;argLine&lt;/propertyName&gt;
                            &lt;/configuration&gt;
                        &lt;/execution&gt;
                        &lt;execution&gt;
                            &lt;id&gt;jacoco-site&lt;/id&gt;
                            &lt;phase&gt;package&lt;/phase&gt;
                            &lt;goals&gt;
                                &lt;goal&gt;report&lt;/goal&gt;
                            &lt;/goals&gt;
                            &lt;configuration&gt;
                                &lt;dataFile&gt;${basedir}\target\jacoco.exec&lt;/dataFile&gt;
                                &lt;outputDirectory&gt;${basedir}\target&lt;/outputDirectory&gt;
                            &lt;/configuration&gt;
                        &lt;/execution&gt;
                    &lt;/executions&gt;
                &lt;/plugin&gt;

huangapple
  • 本文由 发表于 2020年10月17日 20:17:01
  • 转载请务必保留本文链接：https://go.coder-hub.com/64402342.html
  • classloader
  • gradle
  • instrumentation
  • jacoco
  • java
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定