Coming soon: We’re working on a brand new, revamped Community experience. Want to receive updates? Sign up now!

0 votes
Writing spark df into csv along with headers repartitions the df into 1 by default. So it takes a lot of time while writing considering the dataset is large because only 1 partition is active. How do I write spark dataframe to csv in hdfs with column headers and multiple partitions, so that it runs faster?
by

Please log in or register to answer this question.

1,339 questions
1,365 answers
1,557 comments
11,916 users

©Dataiku 2012-2018 - Privacy Policy