英文:
Disable double quotes
问题
The code you provided is adding double quotes around the data because of the "quoteAll" option set to true. To achieve the desired format without double quotes, you can set the "quoteAll" option to false as follows:
df_test.repartition(1).write.option('header', 'false').option("delimiter", '|').option("quoteAll", 'false').mode("overwrite").csv(path_of_file)
This should result in the data in the file appearing as:
A|1
B|2
Please make sure to update your code accordingly.
英文:
When I use the below code, it is adding double quotes at the starting and end of the data. Ideally, this is valid as I am telling it is a pipe delimeted file and pipe is coming in the data. But I have a requirement not to get double quote added in the starting and end of the data. Is there a way to acheive this?
data = [{"Cnt": 'A|1'},{"Cnt": 'B|2'}]
rdd = sc.parallelize(data)
df_test = rdd.toDF()
df_test.repartition(1).write.option('header','false').option("delimiter",'|').option("quoteAll", 'false').option("quote", None).mode("overwrite").csv(path_of_file)
Data in the file looks like below after exporting
"A|1"
"B|2"
But I need the data in the file like below.
A|1
B|2
Apache Spark version - 3.3.1
答案1
得分: 2
I reproduced the same code what you have done, even i got the data in quotes as shown below
Later, I tried saving in text format then i got output without quotes.
data = [{"Cnt": 'A|1'}, {"Cnt": 'B|2'}]
rdd = sc.parallelize(data)
df_test = rdd.toDF()
df_test.repartition(1).write.option('header','false').option("delimiter",'|').option("quoteAll", 'false').option("quote", None).mode("overwrite").text("/demot/")
英文:
I reproduced the same code what you have done,
even i got the data in quotes as shown
below
Later, I tried saving in text format then i got output without quotes.
data = [{"Cnt": 'A|1'},{"Cnt": 'B|2'}]
rdd = sc.parallelize(data)
df_test = rdd.toDF()
df_test.repartition(1).write.option('header','false').option("delimiter",'|').option("quoteAll", 'false').option("quote", None).mode("overwrite").text("/demot/")
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论