尝试在Databricks SQL中将字符串转换为日期列。

huangapple go评论68阅读模式
英文:

Trying to convert a string to a date column in databricks SQL

问题

I'm providing translations for the non-code parts of your text:

"有一张包含大部分字符串列的表格,其中一列以103格式(dd-mm-yyyy)作为字符串列出日期,想要在Databricks SQL中将其转换为日期列,但我找不到常规方法来实现。如果答案是将其转换为yyyy-mm-dd,我也不介意。只是想知道如何做到这一点,我已经尝试过alter列和甚至是pyspark的with列方法,但似乎一点用也没有。任何帮助都将不胜感激。

编辑:我想永久性地修改表中的列,而不是作为选择查询。

我尝试使用alter列得到了这个结果:

ALTER TABLE table_name
ALTER COLUMN colname TYPE to_date(colname,'dd/mm/yyyy')
%python
from pyspark.sql.functions import col

(spark.read.table("tablename")
  .withColumn("colname", col("colname").cast('date'))
  .write
  .mode("overwrite")
  .option("overwriteSchema", "true")
  .saveAsTable("tablename")
)

这些代码返回了空值。"

英文:

have a table with information that's mostly consisted of string columns, one column has the date listed in the 103 format (dd-mm-yyyy) as a string, would like to convert it to a date column in databricks sql, but I can't find a conventional method to do so. I don't mind if the answer lies in it being converted to yyyy-mm-dd. Just wondering how to do it, I've tried alter column and even the pyspark with column method, but it doesn't seem to work at all. Any help is appreciated thanks.

Edit: I'd like to alter the column in the table permanently, not as a select query.

I've tried to use alter column with this result:

ALTER TABLE table_name
ALTER COLUMN colname TYPE to_date(colname,'dd/mm/yyyy')


Error in SQL statement: ParseException: 
mismatched input 'colname' expecting INTEGER_VALUE(line 2, pos 37)

== SQL ==
ALTER TABLE tablename
ALTER COLUMN colname TYPE to_date(colname,'dd/mm/yyyy')
                                  

As well as

%python
from pyspark.sql.functions import col


(spark.read.table("tablename")
  .withColumn("colname", col("colname").cast('date'))
  .write
  .mode("overwrite")
  .option("overwriteSchema", "true")
  .saveAsTable("tablename")
)

Which returns me null values

答案1

得分: 1

你可以在pyspark中像下面这样使用

df2.withColumn("Date_w_type", to_date("date","dd/MM/yyyy")).show()
%sql

ALTER  TABLE tbl_tmp ADD  COLUMN new_date_column DATE;

UPDATE tbl_tmp
SET new_date_column =  CAST(TO_DATE(curr_date, 'dd/MM/yyyy') AS  DATE);

SELECT  *  from tbl_tmp;
英文:

You use like below in pyspark.

df2.withColumn("Date_w_type", to_date("date","dd/MM/yyyy")).show()

尝试在Databricks SQL中将字符串转换为日期列。

In sql,

%sql

ALTER  TABLE tbl_tmp ADD  COLUMN new_date_column DATE;

UPDATE tbl_tmp
SET new_date_column =  CAST(TO_DATE(curr_date, 'dd/MM/yyyy') AS  DATE);

SELECT  *  from tbl_tmp;

尝试在Databricks SQL中将字符串转换为日期列。

huangapple
  • 本文由 发表于 2023年7月10日 19:18:08
  • 转载请务必保留本文链接:https://go.coder-hub.com/76653199.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定