在创建 GCP Dataproc 集群时配置 trino-jvm 属性。

huangapple go评论53阅读模式
英文:

Configure trino-jvm properties in GCP Dataproc on cluster create

问题

I'm trying to configure trino-jvm properties while creating a Dataproc cluster. I'm following Google's documentation and am able to successfully create a cluster without any special JVM configuration, but am receiving an error when attempting to configure JVM properties.

Here's the CLI command that running:

gcloud dataproc clusters create test-dataproc-cluster \
    --project=MY_PROJECT \
    --optional-components=TRINO \
    --region=region \
    --enable-component-gateway \
    --region=us-central1 \
    --image-version=2.1 \
    --properties="trino-jvm:XX:+HeapDumpOnOutOfMemoryError"

Here's the error that I receive:

ERROR: (gcloud.dataproc.clusters.create) argument --properties: Bad syntax for dict arg: [trino-jvm:XX:+HeapDumpOnOutOfMemoryError]. Please see `gcloud topic flags-file` or `gcloud topic escaping` for information on providing list or dictionary flag values with special characters.

It looks like Dataproc expects the value to the --properties argument to be in dictionary form, i.e., --properties=TYPE:KEY=VALUE. I'm able to successfully configure other properties that have a Key/Value syntax. However, I'm unable to configure JVM properties that do not follow that Key/Value form.

How can I configure trino-jvm properties in Dataproc?

英文:

I'm trying to configure trino-jvm properties while creating a Dataproc cluster. I'm following Google's documentation and am able to successfully create a cluster without any special JVM configuration, but am receiving an error when attempting to configure JVM properties.

Here's the CLI command that running:

gcloud dataproc clusters create test-dataproc-cluster \
    --project=MY_PROJECT \
    --optional-components=TRINO \
    --region=region \
    --enable-component-gateway \
    --region=us-central1 \
    --image-version=2.1 \
    --properties="trino-jvm:XX:+HeapDumpOnOutOfMemoryError"

Here's the error that I receive:

ERROR: (gcloud.dataproc.clusters.create) argument --properties: Bad syntax for dict arg: [trino-jvm:XX:+HeapDumpOnOutOfMemoryError]. Please see `gcloud topic flags-file` or `gcloud topic escaping` for information on providing list or dictionary flag values with special characters.

It looks like Dataproc expects the value to the --properties argument to be in dictionary form, i.e. --properties=TYPE:KEY=VALUE. I'm able to successfully configure other properties that have a Key/Value syntax. However, I'm unable to configure JVM properties that do not follow that Key/Value form.

How can I configure trino-jvm properties in Dataproc?

答案1

得分: 1

你可以使用 `--properties` 标志命令来指定 Trino JVM 属性 trino.jvm-extras=-XX:+HeapDumpOnOutOfMemoryError。该属性以 key=value 的格式提供。

--properties trino-env-config=trino.jvm-extras=-XX:+HeapDumpOnOutOfMemoryError


除了 **--properties 标志**,修复这个错误的另一个方法是使用 `--metadata 标志`,因为它可以接受多个键值对。它可以在创建 Dataproc 集群时为 Trino 提供必要的 JVM 属性。

--metadata trino-jvm=XX:+HeapDumpOnOutOfMemoryError

英文:

You can use this --properties flag command flag to specify the Trino JVM property trino.jvm-extras=-XX:+HeapDumpOnOutOfMemoryError. The property is provided in the format key=value.

--properties trino-env-config=trino.jvm-extras=-XX:+HeapDumpOnOutOfMemoryError

Aside from --properties flag, one workaround to fix the error it by using the --metadata flag because it can accept multiple key-value pairs. It can provide the necessary JVM properties for Trino in Dataproc clusters during cluster creation.

--metadata trino-jvm=XX:+HeapDumpOnOutOfMemoryError

huangapple
  • 本文由 发表于 2023年7月10日 23:50:53
  • 转载请务必保留本文链接:https://go.coder-hub.com/76655391.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定