问题

以下是您提供的内容的翻译：

尝试通过java -jar target/assembly.jar向Dataflow提交作业，但出现以下错误，但使用直接运行器（direct runner）正常工作。

Exception in thread "main" java.lang.IllegalArgumentException: 期望 DoFn 为具有 URN beam:dofn:javasdk:0.1 的 FunctionSpec，但实际 URN 为
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:477)
at org.apache.beam.runners.core.construction.ParDoTranslation.doFnWithExecutionInformationFromProto(ParDoTranslation.java:697)
at org.apache.beam.runners.core.construction.ParDoTranslation.getSchemaInformation(ParDoTranslation.java:389)
at org.apache.beam.runners.core.construction.ParDoTranslation.getSchemaInformation(ParDoTranslation.java:374)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$11.translateTyped(DataflowPipelineTranslator.java:1158)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$11.translate(DataflowPipelineTranslator.java:1148)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$11.translate(DataflowPipelineTranslator.java:1144)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.visitPrimitiveTransform(DataflowPipelineTranslator.java:484)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:664)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:463)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.translate(DataflowPipelineTranslator.java:423)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translate(DataflowPipelineTranslator.java:182)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:910)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:195)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:303)
at com.Pipeline.main(Pipeline.java:74)



```<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>pipeline</artifactId>
    <version>0.1-SNAPSHOT</version>
    <properties>
        <beam.version>2.24.0</beam.version>
    </properties>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <addClasspath>true</addClasspath>
                            <classpathPrefix>lib/</classpathPrefix>
                            <mainClass>com.Pipeline</mainClass>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
            <!-- 其他插件... -->
        </plugins>
    </build>
    <dependencies>
        <!-- 依赖列表... -->
    </dependencies>
</project>


<details>
<summary>英文:</summary>

trying to submit job to dataflow via java -jar target/assembly.jar but failing with error below, but with direct runner it&#39;s working fine.

Exception in thread "main" java.lang.IllegalArgumentException: Expected DoFn to be FunctionSpec with URN beam:dofn:javasdk:0.1, but URN was
at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument(Preconditions.java:477)
at org.apache.beam.runners.core.construction.ParDoTranslation.doFnWithExecutionInformationFromProto(ParDoTranslation.java:697)
at org.apache.beam.runners.core.construction.ParDoTranslation.getSchemaInformation(ParDoTranslation.java:389)
at org.apache.beam.runners.core.construction.ParDoTranslation.getSchemaInformation(ParDoTranslation.java:374)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$11.translateTyped(DataflowPipelineTranslator.java:1158)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$11.translate(DataflowPipelineTranslator.java:1148)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$11.translate(DataflowPipelineTranslator.java:1144)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.visitPrimitiveTransform(DataflowPipelineTranslator.java:484)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:664)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.visit(TransformHierarchy.java:656)
at org.apache.beam.sdk.runners.TransformHierarchy$Node.access$600(TransformHierarchy.java:317)
at org.apache.beam.sdk.runners.TransformHierarchy.visit(TransformHierarchy.java:251)
at org.apache.beam.sdk.Pipeline.traverseTopologically(Pipeline.java:463)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator.translate(DataflowPipelineTranslator.java:423)
at org.apache.beam.runners.dataflow.DataflowPipelineTranslator.translate(DataflowPipelineTranslator.java:182)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:910)
at org.apache.beam.runners.dataflow.DataflowRunner.run(DataflowRunner.java:195)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:303)
at com.Pipeline.main(Pipeline.java:74)


```&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;
&lt;project xmlns=&quot;http://maven.apache.org/POM/4.0.0&quot;
xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot;
xsi:schemaLocation=&quot;http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd&quot;&gt;
&lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
&lt;groupId&gt;org.example&lt;/groupId&gt;
&lt;artifactId&gt;pipeline&lt;/artifactId&gt;
&lt;version&gt;0.1-SNAPSHOT&lt;/version&gt;
&lt;properties&gt;
&lt;beam.version&gt;2.24.0&lt;/beam.version&gt;
&lt;/properties&gt;
&lt;build&gt;
&lt;plugins&gt;
&lt;plugin&gt;
&lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
&lt;artifactId&gt;maven-jar-plugin&lt;/artifactId&gt;
&lt;configuration&gt;
&lt;archive&gt;
&lt;manifest&gt;
&lt;addClasspath&gt;true&lt;/addClasspath&gt;
&lt;classpathPrefix&gt;lib/&lt;/classpathPrefix&gt;
&lt;mainClass&gt;com.Pipeline&lt;/mainClass&gt;
&lt;/manifest&gt;
&lt;/archive&gt;
&lt;/configuration&gt;
&lt;/plugin&gt;
&lt;plugin&gt;
&lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
&lt;artifactId&gt;maven-compiler-plugin&lt;/artifactId&gt;
&lt;configuration&gt;
&lt;source&gt;8&lt;/source&gt;
&lt;target&gt;8&lt;/target&gt;
&lt;/configuration&gt;
&lt;/plugin&gt;
&lt;plugin&gt;
&lt;groupId&gt;net.revelc.code.formatter&lt;/groupId&gt;
&lt;artifactId&gt;formatter-maven-plugin&lt;/artifactId&gt;
&lt;version&gt;2.12.0&lt;/version&gt;
&lt;configuration&gt;
&lt;encoding&gt;UTF-8&lt;/encoding&gt;
&lt;/configuration&gt;
&lt;executions&gt;
&lt;execution&gt;
&lt;goals&gt;
&lt;goal&gt;format&lt;/goal&gt;
&lt;/goals&gt;
&lt;/execution&gt;
&lt;/executions&gt;
&lt;/plugin&gt;
&lt;plugin&gt;
&lt;groupId&gt;org.xolstice.maven.plugins&lt;/groupId&gt;
&lt;artifactId&gt;protobuf-maven-plugin&lt;/artifactId&gt;
&lt;version&gt;0.6.1&lt;/version&gt;
&lt;configuration&gt;
&lt;protocExecutable&gt;/usr/local/bin/protoc&lt;/protocExecutable&gt;
&lt;protoSourceRoot&gt;${basedir}/src/main/external/&lt;/protoSourceRoot&gt;
&lt;/configuration&gt;
&lt;executions&gt;
&lt;execution&gt;
&lt;goals&gt;
&lt;goal&gt;compile&lt;/goal&gt;
&lt;goal&gt;test-compile&lt;/goal&gt;
&lt;/goals&gt;
&lt;/execution&gt;
&lt;/executions&gt;
&lt;/plugin&gt;
&lt;plugin&gt;
&lt;groupId&gt;org.codehaus.mojo&lt;/groupId&gt;
&lt;artifactId&gt;build-helper-maven-plugin&lt;/artifactId&gt;
&lt;executions&gt;
&lt;execution&gt;
&lt;phase&gt;generate-sources&lt;/phase&gt;
&lt;goals&gt;
&lt;goal&gt;add-source&lt;/goal&gt;
&lt;/goals&gt;
&lt;configuration&gt;
&lt;sources&gt;
&lt;source&gt;${project.build.directory}/generated-sources/protobuf/&lt;/source&gt;
&lt;/sources&gt;
&lt;/configuration&gt;
&lt;/execution&gt;
&lt;/executions&gt;
&lt;/plugin&gt;
&lt;plugin&gt;
&lt;groupId&gt;org.flywaydb&lt;/groupId&gt;
&lt;artifactId&gt;flyway-maven-plugin&lt;/artifactId&gt;
&lt;version&gt;7.0.0&lt;/version&gt;
&lt;configuration&gt;
&lt;url&gt;url&lt;/url&gt;
&lt;user&gt;user&lt;/user&gt;
&lt;password&gt;a&lt;/password&gt;
&lt;placeholders&gt;
&lt;socketFactory&gt;com.google.cloud.sql.postgres.SocketFactory&lt;/socketFactory&gt;
&lt;/placeholders&gt;
&lt;/configuration&gt;
&lt;/plugin&gt;
&lt;plugin&gt;
&lt;artifactId&gt;maven-assembly-plugin&lt;/artifactId&gt;
&lt;version&gt;3.3.0&lt;/version&gt;
&lt;configuration&gt;
&lt;descriptorRefs&gt;
&lt;descriptorRef&gt;jar-with-dependencies&lt;/descriptorRef&gt;
&lt;/descriptorRefs&gt;
&lt;appendAssemblyId&gt;false&lt;/appendAssemblyId&gt;
&lt;finalName&gt;${artifactId}-${version}-assembly&lt;/finalName&gt;
&lt;archive&gt;
&lt;manifest&gt;
&lt;addClasspath&gt;true&lt;/addClasspath&gt;
&lt;classpathPrefix&gt;lib/&lt;/classpathPrefix&gt;
&lt;mainClass&gt;com.Pipeline&lt;/mainClass&gt;
&lt;/manifest&gt;
&lt;/archive&gt;
&lt;/configuration&gt;
&lt;executions&gt;
&lt;execution&gt;
&lt;id&gt;make-assembly&lt;/id&gt;
&lt;phase&gt;package&lt;/phase&gt;
&lt;goals&gt;
&lt;goal&gt;single&lt;/goal&gt;
&lt;/goals&gt;
&lt;/execution&gt;
&lt;/executions&gt;
&lt;/plugin&gt;
&lt;/plugins&gt;
&lt;/build&gt;
&lt;dependencies&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
&lt;artifactId&gt;beam-runners-google-cloud-dataflow-java&lt;/artifactId&gt;
&lt;version&gt;${beam.version}&lt;/version&gt;
&lt;scope&gt;runtime&lt;/scope&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
&lt;artifactId&gt;beam-sdks-java-core&lt;/artifactId&gt;
&lt;version&gt;${beam.version}&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
&lt;artifactId&gt;beam-runners-direct-java&lt;/artifactId&gt;
&lt;version&gt;${beam.version}&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
&lt;artifactId&gt;beam-sdks-java-extensions-sql&lt;/artifactId&gt;
&lt;version&gt;${beam.version}&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
&lt;artifactId&gt;beam-sdks-java-io-google-cloud-platform&lt;/artifactId&gt;
&lt;version&gt;${beam.version}&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.beam&lt;/groupId&gt;
&lt;artifactId&gt;beam-sdks-java-io-jdbc&lt;/artifactId&gt;
&lt;version&gt;${beam.version}&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;com.google.code.gson&lt;/groupId&gt;
&lt;artifactId&gt;gson&lt;/artifactId&gt;
&lt;version&gt;2.8.6&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;joda-time&lt;/groupId&gt;
&lt;artifactId&gt;joda-time&lt;/artifactId&gt;
&lt;version&gt;2.10.6&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;com.google.protobuf&lt;/groupId&gt;
&lt;artifactId&gt;protobuf-java&lt;/artifactId&gt;
&lt;version&gt;3.11.4&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.parquet&lt;/groupId&gt;
&lt;artifactId&gt;parquet-avro&lt;/artifactId&gt;
&lt;version&gt;1.10.1&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.apache.avro&lt;/groupId&gt;
&lt;artifactId&gt;avro-protobuf&lt;/artifactId&gt;
&lt;version&gt;1.9.1&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;com.google.api-client&lt;/groupId&gt;
&lt;artifactId&gt;google-api-client-gson&lt;/artifactId&gt;
&lt;version&gt;1.30.10&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;com.google.apis&lt;/groupId&gt;
&lt;artifactId&gt;google-api-services-secretmanager&lt;/artifactId&gt;
&lt;version&gt;v1-rev8-1.25.0&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;org.postgresql&lt;/groupId&gt;
&lt;artifactId&gt;postgresql&lt;/artifactId&gt;
&lt;version&gt;42.2.16&lt;/version&gt;
&lt;/dependency&gt;
&lt;dependency&gt;
&lt;groupId&gt;com.google.cloud.sql&lt;/groupId&gt;
&lt;artifactId&gt;postgres-socket-factory&lt;/artifactId&gt;
&lt;version&gt;1.1.0&lt;/version&gt;
&lt;/dependency&gt;
&lt;/dependencies&gt;
&lt;/project&gt;```
</details>
# 答案1
**得分**: 1
我认为这个问题可能会在生成的 JAR 文件中没有包含一些在 META-INF/services 中的服务实现时发生。请尝试使用以下的 `maven-shade-plugin` 来替代 `maven-assembly-plugin` 进行打包。
我在从 [Beam Quickstart](https://beam.apache.org/get-started/quickstart-java/#get-the-wordcount-code) 生成的 pom.xml 中找到了插件配置。
```xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<finalName>${project.artifactId}-bundled-${project.version}</finalName>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/LICENSE</exclude>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>

英文:

I believe this issue can happen when the generated jar doesn't include some service implementations in META-INFO/services. Please try the following maven-shade-plugin instead of maven-assembly-plugin for packaging.

I found the plugin configuration in pom.xml generated from Beam Quickstart .

      &lt;plugin&gt;
&lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
&lt;artifactId&gt;maven-shade-plugin&lt;/artifactId&gt;
&lt;version&gt;3.1.0&lt;/version&gt;
&lt;executions&gt;
&lt;execution&gt;
&lt;phase&gt;package&lt;/phase&gt;
&lt;goals&gt;
&lt;goal&gt;shade&lt;/goal&gt;
&lt;/goals&gt;
&lt;configuration&gt;
&lt;finalName&gt;${project.artifactId}-bundled-${project.version}&lt;/finalName&gt;
&lt;filters&gt;
&lt;filter&gt;
&lt;artifact&gt;*:*&lt;/artifact&gt;
&lt;excludes&gt;
&lt;exclude&gt;META-INF/LICENSE&lt;/exclude&gt;
&lt;exclude&gt;META-INF/*.SF&lt;/exclude&gt;
&lt;exclude&gt;META-INF/*.DSA&lt;/exclude&gt;
&lt;exclude&gt;META-INF/*.RSA&lt;/exclude&gt;
&lt;/excludes&gt;
&lt;/filter&gt;
&lt;/filters&gt;
&lt;transformers&gt;
&lt;transformer implementation=&quot;org.apache.maven.plugins.shade.resource.ServicesResourceTransformer&quot;/&gt;
&lt;/transformers&gt;
&lt;/configuration&gt;
&lt;/execution&gt;
&lt;/executions&gt;
&lt;/plugin&gt;

通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。

预期的 DoFn 应该是具有 URN beam:dofn:javasdk:0.1 的 FunctionSpec，但是 URN 是

问题

验证方法 public 的查询失败，Spring JPA？

在Java中声明ConcurrentHashMap时，我是否必须添加final关键字？

Java创建客户函数生成随机值

访问 JavaFX SceneBuilder 应用中的字段

What's the correct way to type hint an empty list as a literal in python?

如何在Highcharts Gantt中更改本地化的星期名称

如何在同一个流中使用多个过滤器和映射函数？

如何使用Map/Set来将代码优化到O(n)？

.NET MAUI Android在GitHub Actions上构建失败，错误代码为1。

如何在Playwright视觉比较中屏蔽多个定位器？

在C++中，可以使用可变模板参数来检索类型的内部类型。

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: stale element not found

Creating and opening a URL to log in to Website via Basic Auth with Robot Framework/Selenium (Python)

AG Grid 在上下文菜单中以大文本形式打开

发表评论