Pyspark Table Name with Timestamp

I'm working in databricks coding in Pyspark. I'm encountering issues with renaming an existing table and adding a time stamp. The table is located mydatabase.tableOne, and I want to save it as another table called mydatabase.tableOne_20230625. This will allow the new process to run, creating a new version of mydatabase.tableOne

df_my_loc = 'mydatabase.tableOne'
timestamp_suffix = date_format(current_timestamp(), 'yyyyMMdd')
df_my_loc_new = df_my_loc + '_' + timestamp_suffix
df_arch = spark.table(df_my_loc)
df_arch.write.format("delta").mode("ignore").saveAsTable(df_my_loc_new)

I get an error that says the column is not iterable. It seems like adding the timestamp as a suffix is giving me the error. Before running a new process, the goal is to archive the previous table with a timestamp and run a notebook for the new output where the table names do not change.

date_format and current_timestamp are Spark SQL functions. If used in Python, like in your example, their return type is Column. This:

This:

df_my_loc_new = df_my_loc + '_' + timestamp_suffix

is a Python expression - concatenation of simple strings. You cannot concatenate String and Column of type String. You can either:

• concatenate Python string using + operator
• or concatenate SQL strings using concat function

Since building a table name is pure Python you just need to do:

from datetime import datetime
timestamp_suffix = datetime.now().strftime('%Y%m')

This will yield timestamp_suffix as a simple string in YYYYMM format and your code will work.