Gradle任务间歇性失败

huangapple go评论79阅读模式
英文:

Gradle task fails intermittently

问题

task runIntProxy(dependsOn: [':int-proxy:getVersions', ':infrastructure:dev:startNginx']) {
  group 'start';
  description 'Starts int-proxy';
  doLast {
    file(logsDir).mkdirs();
    def imageTag = readVersion('int_proxy_image_tag');
    command("docker-compose ${composeFiles} up -d", ['env': ['INT_PROXY_IMAGE_TAG': imageTag]]);
    command("docker-compose ${composeFiles} logs -f", ['printOutput': false, 'waitForExit': false, 'outputFile': "$logsDir/int-proxy.log"]);
    quiet "Running healthchecks:";
    println(intProxyStatus);
    try {
      def response = new URL(intProxyStatus).getContent().text;
      println("output from URL:" + response);
      if (response.contains("alive")) {
        quiet("Int-proxy is up and running");
      } else {
        throw new GradleException("Int-proxy is not up!");
      }
    } catch (GradleException e) { }
  }
}

if (!waitForNoConnection(intProxyStatus, 5, 5, 15)) {
  throw new GradleException("Couldn't stop int-proxy. You might need to forcefully kill it yourself.");
}

Error code:

* What went wrong:
Execution failed for task ':int-proxy:runIntProxy'.
> java.net.SocketException: Connection reset

* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':int-proxy:runIntProxy'.
  ...
Caused by: org.gradle.api.UncheckedIOException: java.net.SocketException: Connection reset
  ...
Caused by: java.net.SocketException: Connection reset
  ...

Also, we poll to see if the service is up or not:

if (!waitForNoConnection(intProxyStatus, 5, 5, 15)) {
  throw new GradleException("Couldn't stop int-proxy. You might need to forcefully kill it yourself.");
}
英文:

While running a gradle task it fails intermittently (1-2 out of 10 times on avg which does not makes sense)

task runIntProxy(dependsOn: [':int-proxy:getVersions',':infrastructure:dev:startNginx']) {
  group 'start'
  description 'Starts int-proxy'
    doLast {
        file(logsDir).mkdirs()
        def imageTag = readVersion('int_proxy_image_tag')

        command("docker-compose ${composeFiles} up -d", ['env': ['INT_PROXY_IMAGE_TAG': imageTag]])
        command("docker-compose ${composeFiles} logs -f", ['printOutput': false, 'waitForExit': false, 'outputFile': "$logsDir/int-proxy.log"])

        quiet "Running healthchecks:"
        println(intProxyStatus)
        try {
        def response = new URL(intProxyStatus).getContent().text
        println("output from URL:" + response)
      
        if(response.contains("alive")){
          quiet("Int-proxy is up and running")
        } else {
          throw new GradleException("Int-proxy is not up!")
        }
        } catch (GradleException e) { }
    }
}

since it was failing falsely (while performing same task manually it works fine but with jenkins it fails) I added catch (GradleException e) { } but still it fails intermittently can anyone point out what am I missing?

error code is below

* What went wrong:
Execution failed for task ':int-proxy:runIntProxy'.
> java.net.SocketException: Connection reset
* Try:
Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':int-proxy:runIntProxy'.
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$2.accept(ExecuteActionsTaskExecuter.java:121)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$2.accept(ExecuteActionsTaskExecuter.java:117)
at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:184)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:110)
at org.gradle.api.internal.tasks.execution.ResolveIncrementalChangesTaskExecuter.execute(ResolveIncrementalChangesTaskExecuter.java:84)
at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:92)
at org.gradle.api.internal.tasks.execution.FinishSnapshotTaskInputsBuildOperationTaskExecuter.execute(FinishSnapshotTaskInputsBuildOperationTaskExecuter.java:51)
at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:103)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionStateTaskExecuter.execute(ResolveBeforeExecutionStateTaskExecuter.java:75)
at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:110)
at org.gradle.api.internal.tasks.execution.ResolveBeforeExecutionOutputsTaskExecuter.execute(ResolveBeforeExecutionOutputsTaskExecuter.java:67)
at org.gradle.api.internal.tasks.execution.StartSnapshotTaskInputsBuildOperationTaskExecuter.execute(StartSnapshotTaskInputsBuildOperationTaskExecuter.java:52)
at org.gradle.api.internal.tasks.execution.ResolveAfterPreviousExecutionStateTaskExecuter.execute(ResolveAfterPreviousExecutionStateTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:94)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:95)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:63)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:49)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:46)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:416)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:406)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:102)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:43)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:355)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:343)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:336)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:322)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:134)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker$1.execute(DefaultPlanExecutor.java:129)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:202)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:193)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:129)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: org.gradle.api.UncheckedIOException: java.net.SocketException: Connection reset
at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:61)
at org.gradle.internal.UncheckedException.throwAsUncheckedException(UncheckedException.java:41)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$4.run(ExecuteActionsTaskExecuter.java:367)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:402)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:394)
at org.gradle.internal.operations.DefaultBuildOperationExecutor$1.execute(DefaultBuildOperationExecutor.java:165)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:250)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:158)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:92)
at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:326)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:311)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.access$200(ExecuteActionsTaskExecuter.java:75)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$TaskExecution.execute(ExecuteActionsTaskExecuter.java:158)
at org.gradle.internal.execution.impl.steps.ExecuteStep.execute(ExecuteStep.java:46)
at org.gradle.internal.execution.impl.steps.CancelExecutionStep.execute(CancelExecutionStep.java:34)
at org.gradle.internal.execution.impl.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:69)
at org.gradle.internal.execution.impl.steps.TimeoutStep.execute(TimeoutStep.java:49)
at org.gradle.internal.execution.impl.steps.CatchExceptionStep.execute(CatchExceptionStep.java:34)
at org.gradle.internal.execution.impl.steps.CreateOutputsStep.execute(CreateOutputsStep.java:49)
at org.gradle.internal.execution.impl.steps.SnapshotOutputStep.execute(SnapshotOutputStep.java:42)
at org.gradle.internal.execution.impl.steps.SnapshotOutputStep.execute(SnapshotOutputStep.java:28)
at org.gradle.internal.execution.impl.steps.CacheStep.executeWithoutCache(CacheStep.java:136)
at org.gradle.internal.execution.impl.steps.CacheStep.lambda$execute$5(CacheStep.java:86)
at org.gradle.internal.execution.impl.steps.CacheStep.execute(CacheStep.java:85)
at org.gradle.internal.execution.impl.steps.CacheStep.execute(CacheStep.java:37)
at org.gradle.internal.execution.impl.steps.PrepareCachingStep.execute(PrepareCachingStep.java:33)
at org.gradle.internal.execution.impl.steps.StoreSnapshotsStep.execute(StoreSnapshotsStep.java:38)
at org.gradle.internal.execution.impl.steps.StoreSnapshotsStep.execute(StoreSnapshotsStep.java:23)
at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:95)
at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.lambda$execute$0(SkipUpToDateStep.java:88)
at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:52)
at org.gradle.internal.execution.impl.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:36)
at org.gradle.internal.execution.impl.DefaultWorkExecutor.execute(DefaultWorkExecutor.java:34)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:109)
... 40 more
Caused by: java.net.SocketException: Connection reset
at java_net_URL$getContent$0.call(Unknown Source)
at build_4hqoiddv5luerpje0uiotdsrg$_run_closure4$_closure14.doCall(/home/jenkins/workspace/development/int-proxy/build.gradle:63)
at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:644)
at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:617)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$4.run(ExecuteActionsTaskExecuter.java:337)
... 72 more

also we poll to see if service is up or not

if(!waitForNoConnection(intProxyStatus, 5, 5, 15)) {
throw new GradleException("Couldn't stop int-proxy. You might need to forcefully kill it yourself.")
}

答案1

得分: 1

根本原因是java.net.SocketException: Connection reset

这意味着网络出现了问题:

  • 可能服务器未运行或无法访问
  • 可能尝试连接错误的 IP/主机名
  • 可能服务器因某些内部错误关闭了连接
  • 可能传输速度过慢,导致连接超时
  • 可能防火墙设置有问题等等。

您是否已经手动尝试了足够多次,以确信手动操作时是否也出现了这个问题?您是否尝试在相同的机器上,使用相同版本的 Java、Gradle 和其他所有内容进行了尝试?您是否确认问题与时间或速度无关?

我最初的想法是您正在启动一个 Docker 容器,然后在此之后对该服务发起 URL 请求,但您忽略了容器启动是异步的这一事实。换句话说,在 docker-compose 完成时,并不能保证服务实际上已经在运行。但这只是一个猜测。

尝试在发起 URL 请求之前添加延迟,看看是否会有任何变化。正确的做法是在循环中轮询该 URL,直到服务启动或超时为止。

英文:

The root cause is java.net.SocketException: Connection reset.

This means that there was something wrong with the network:

  • Maybe the server is not running or reachable
  • Maybe you are trying to connect to the wrong IP/hostname
  • Maybe the server closed the connection because of some internal error
  • Maybe the transfer was too slow and connection timed out
  • Maybe you have a shitty firewall, etc.

Have you tried it enough times manually to be 99.9% sure that it is not also happening manually? Have you tried it on the same machine, with the same version of Java, Gradle and everything else? Have you made sure that the issue is not timing- or speed-related?

My initial thought was that you are starting a docker container, after which you make a URL request to the service, but you are forgetting that container start is asynchronous. In other words, you have no guarantee that the service will actually be running when docker-compose finishes. But this is just a guess.

Try adding a delay before making a URL request to see if it makes any change. The proper way to do it would be to poll the URL in a loop until the service either comes up or times out.

答案2

得分: 0

问题是容器已经启动,但是某种原因服务没有启动,并且需要一些时间,添加了15秒的延迟,然后一切都开始正常运行。

英文:

The issue was container was coming up but somehow service was not coming up and was taking some time, added a sleep of 15sec and it all started to go fine.

huangapple
  • 本文由 发表于 2020年9月16日 01:35:19
  • 转载请务必保留本文链接:https://go.coder-hub.com/63907145.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定