PySpark / Snowpark random column name during left anti join problem

I am trying to compare two dataframes, to get new records to be inserted into an incremental table.

I am following previously asked questions, example https://stackoverflow.com/questions/72181011/how-to-compare-two-dataframes-and-extract-unmatched-rows-in-pyspark

But I'm having this problem, where even though I used the alias function, it seems that the output dataframe is having random column names. At least that is what I see in the Snowflake query history.

Sample code:

main = session.table("source")
incremental = session.table("target")

new_customer = (incremental
        .join(main, incremental.customer_id==main.customer_id, "leftanti" )
        .select(main.customer_id.alias("customer_id"))
                        ).show()

Error:

snowflake.snowpark.exceptions.SnowparkSQLException: 1304): 01acd0d6-3201-cee0-0000-6b1502059ff6: 000904 (42000): SQL compilation error: error line 1 at position 7
invalid identifier '"r_tu5s_customer_id".

Query history:

SELECT "r_tu5s_customer_id" AS "customer_id" FROM ( SELECT  *  FROM ( SELECT "customer_id" .........

Turning the comment into an answer to be useful for others.

The leftanti is similar to the join functionality, but it returns only columns from the left DataFrame for non-matched records.

So the solution is just swtiching the two dataframes so you can get the new records in main df that don't exist in incremental df

main.join(incremental, incremental.customer_id==main.customer_id, "leftanti" )