Coming soon: We’re working on a brand new, revamped Community experience. Want to receive updates? Sign up now!

0 votes
Writing spark df into csv along with headers repartitions the df into 1 by default. So it takes a lot of time while writing considering the dataset is large because only 1 partition is active. How do I write spark dataframe to csv in hdfs with column headers and multiple partitions, so that it runs faster?

Please log in or register to answer this question.

1,337 questions
1,362 answers
11,912 users

©Dataiku 2012-2018 - Privacy Policy