0 votes
Writing spark df into csv along with headers repartitions the df into 1 by default. So it takes a lot of time while writing considering the dataset is large because only 1 partition is active. How do I write spark dataframe to csv in hdfs with column headers and multiple partitions, so that it runs faster?
by

Please log in or register to answer this question.

1,296 questions
1,324 answers
1,506 comments
11,862 users

┬ęDataiku 2012-2018 - Privacy Policy