数据处理在Spark中使用Java

huangapple go评论99阅读模式
英文:

Data processing on spark using java

问题

  1. 我有一个 DataSet
  2. +-------------------+
  3. | 日期 |
  4. +-------------------+
  5. |1970-01-01 06:00:00|
  6. +-------------------+
  7. 我想要只提取小时和分钟。列 "日期" 是时间戳类型。
  8. 我尝试了以下代码:
  9. String format = "HH:mm";
  10. df.withColumn("时:分",unix_timestamp(col("日期"),format)).show(100);
  11. 但对我来说没有起作用。
  12. 谢谢。
英文:

I have a DataSet.

  1. +-------------------+
  2. | Date |
  3. +-------------------+
  4. |1970-01-01 06:00:00|
  5. +-------------------+

I want to take only the hour and the minute.The column Date is on timestamp.
I tried the following code :

  1. String format = "HH:mm";
  2. df.withColumn("hour:minute",unix_timestamp(col("Date"),format)).show(100);

But doesn't work for me .

Thank you .

答案1

得分: 1

unix_timestamp 函数将时间戳转换为 epoch/unix 时间

  • 使用 date_formatfrom_unixtime(unix_timestamp()) 函数之一来提取 hour:minute

示例:

  1. df.withColumn("hour:minute", date_format(col("Date"), "HH:mm")).show()
  2. df.withColumn("hour:minute", from_unixtime(unix_timestamp(col("Date"), "yyyy-MM-dd HH:mm:ss"), "HH:mm")).show()
  3. //+-------------------+-----------+
  4. //| Date|hour:minute|
  5. //+-------------------+-----------+
  6. //|1970-01-01 06:00:00| 06:00|
  7. //+-------------------+-----------+
英文:

unix_timestamp converts timestamp to epoch/unix time.

  • Use either date_format,from_unixtime(unix_timestamp()) functions to extract hour:minute.

Example:

  1. df.withColumn("hour:minute",date_format(col("Date"),"HH:mm")).show()
  2. df.withColumn("hour:minute",from_unixtime(unix_timestamp(col("Date"),"yyyy-MM-dd HH:mm:ss"),"HH:mm")).show()
  3. //+-------------------+-----------+
  4. //| Date|hour:minute|
  5. //+-------------------+-----------+
  6. //|1970-01-01 06:00:00| 06:00|
  7. //+-------------------+-----------+

huangapple
  • 本文由 发表于 2020年5月5日 01:44:27
  • 转载请务必保留本文链接:https://go.coder-hub.com/61598411.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定