英文:
Trying to convert a string to a date column in databricks SQL
问题
I'm providing translations for the non-code parts of your text:
"有一张包含大部分字符串列的表格,其中一列以103格式(dd-mm-yyyy)作为字符串列出日期,想要在Databricks SQL中将其转换为日期列,但我找不到常规方法来实现。如果答案是将其转换为yyyy-mm-dd,我也不介意。只是想知道如何做到这一点,我已经尝试过alter列和甚至是pyspark的with列方法,但似乎一点用也没有。任何帮助都将不胜感激。
编辑:我想永久性地修改表中的列,而不是作为选择查询。
我尝试使用alter列得到了这个结果:
ALTER TABLE table_name
ALTER COLUMN colname TYPE to_date(colname,'dd/mm/yyyy')
%python
from pyspark.sql.functions import col
(spark.read.table("tablename")
.withColumn("colname", col("colname").cast('date'))
.write
.mode("overwrite")
.option("overwriteSchema", "true")
.saveAsTable("tablename")
)
这些代码返回了空值。"
英文:
have a table with information that's mostly consisted of string columns, one column has the date listed in the 103 format (dd-mm-yyyy) as a string, would like to convert it to a date column in databricks sql, but I can't find a conventional method to do so. I don't mind if the answer lies in it being converted to yyyy-mm-dd. Just wondering how to do it, I've tried alter column and even the pyspark with column method, but it doesn't seem to work at all. Any help is appreciated thanks.
Edit: I'd like to alter the column in the table permanently, not as a select query.
I've tried to use alter column with this result:
ALTER TABLE table_name
ALTER COLUMN colname TYPE to_date(colname,'dd/mm/yyyy')
Error in SQL statement: ParseException:
mismatched input 'colname' expecting INTEGER_VALUE(line 2, pos 37)
== SQL ==
ALTER TABLE tablename
ALTER COLUMN colname TYPE to_date(colname,'dd/mm/yyyy')
As well as
%python
from pyspark.sql.functions import col
(spark.read.table("tablename")
.withColumn("colname", col("colname").cast('date'))
.write
.mode("overwrite")
.option("overwriteSchema", "true")
.saveAsTable("tablename")
)
Which returns me null values
答案1
得分: 1
你可以在pyspark中像下面这样使用。
df2.withColumn("Date_w_type", to_date("date","dd/MM/yyyy")).show()
%sql
ALTER TABLE tbl_tmp ADD COLUMN new_date_column DATE;
UPDATE tbl_tmp
SET new_date_column = CAST(TO_DATE(curr_date, 'dd/MM/yyyy') AS DATE);
SELECT * from tbl_tmp;
英文:
You use like below in pyspark.
df2.withColumn("Date_w_type", to_date("date","dd/MM/yyyy")).show()
In sql,
%sql
ALTER TABLE tbl_tmp ADD COLUMN new_date_column DATE;
UPDATE tbl_tmp
SET new_date_column = CAST(TO_DATE(curr_date, 'dd/MM/yyyy') AS DATE);
SELECT * from tbl_tmp;
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论