如何使用PySpark在Databricks中将变量插入方法

huangapple go评论62阅读模式
英文:

How to insert variable into methods with Databricks using PySpark

问题

The title of this question is terrible, as I couldn't find the correct words to describe exactly what I would like in a sentence.

Anyway, I have a variable which returns two values separated by a comma, see below:

如何使用PySpark在Databricks中将变量插入方法

At the moment I'm reading in a database in Databricks as follows, df = spark.table("samples.cdcTest.cdcmergetest")

I would like to use the variable 'regName' instead of typing cdcTest.cdcmergetest.

I have tried the following:

df = spark.table("samples".regName)

But I get the error AttributeError: 'str' object has no attribute 'regName'

I also tried the following

如何使用PySpark在Databricks中将变量插入方法

This looks like an indexing and slicing issue, but not sure

Ok, I've tried a few things that appear to be getting me closer to my goal see below 如何使用PySpark在Databricks中将变量插入方法

The problem I have is combining all the parts together:

df = spark.table("samples", db_name, table_name)

Any thoughts much appreciated.

英文:

The title of this question is terrible, as I couldn't find the correct words to describe exactly what I would like in a sentence.

Anyway, I have a variable which returns two values separated by a comma, see below:

如何使用PySpark在Databricks中将变量插入方法

At the moment I'm reading in a database in Databricks as follows, df = spark.table("samples.cdcTest.cdcmergetest")

I would like to use the variable 'regName' instead of typing cdcTest.cdcmergetest.

I have tried the followiong:

df = spark.table("samples".regName)

But I get the error AttributeError: 'str' object has no attribute 'regName'

I also trid the following

如何使用PySpark在Databricks中将变量插入方法

This looks like an indexing and slicing issue, but not sure

Ok, I've tried a few things that appear to be getting me closer to my goal see below
如何使用PySpark在Databricks中将变量插入方法

The problem I have is combining all the parts together:

df = spark.table("samples",db_name,table_name)

Any thoughts much appreciated.

答案1

得分: 1

以下是翻译好的部分:

  • Just concatenate names: "samples" + "." + regName -> 只需连接名称:"samples" + "." + regName
  • Use f-strings f"samples.{regName}" -> 使用 f-strings f"samples.{regName}"
  • join the list: ".".join(["samples",db_name,table_name]) -> 连接列表:".".join(["samples",db_name,table_name])
英文:

You have multiple possibilities to generate a full name:

  • Just concatenate names: "samples" + "." + regName
  • Use f-strings `f"samples.{regName}"
  • join the list: ".".join(["samples",db_name,table_name])

huangapple
  • 本文由 发表于 2023年5月23日 01:13:48
  • 转载请务必保留本文链接:https://go.coder-hub.com/76308519.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定