英文:
Set "table" in spark session
问题
You can set salaryData as a table in a SparkSession using the createOrReplaceTempView method like this:
salaryData.createOrReplaceTempView("salaries")
After this, you can fetch it from the SparkSession as you mentioned:
val salaryDataframeFromSession = spark.table("salaries")
This will allow you to access the salaryData DataFrame as a table named "salaries" within your SparkSession.
英文:
In some spark/scala code (I cannot link the source) I see a dataframe being created like this:
val someDataFrame = spark.table("dataframeName")
where spark is a SparkSession. How can I set a table in the spark session? i.e. if I have a frame like this:
val salaryData = Seq(("James", "Sales", 3000),
("Michael", "Sales", 4600),
("Robert", "Sales", 4100),
("Maria", "Finance", 3000),
("James", "Sales", 3000),
)
val salaryData = simpleData.toDF("employee_name", "department", "salary")
how can I set salaryData as a table in a SparkSession and fetch it like this:
val salaryDataframeFromSession = spark.table("salaries")
答案1
得分: 0
我成功解决了这个问题，通过以下代码：
salaryData.createOrReplaceTempView("salaries")
英文:
I was able to solve this via
salaryData.createOrReplaceTempView("salaries")
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论