0 votes
Hi All,

What I am trying to do is replace empty cells with '0'. After running the recipe, only a few rows are getting replaced with 0 and others remain blank.

Please note that this is only happening when I select Spark engine. Other engines do not cause this problem. Is this a known problem with spark engine in prepare recipe?
asked by Sid
Hi,
This is not a known issue and we are not able to reproduce it. Could you give more details about your input and output data ? (dataset type for both, format type for both, column storage types for both, DSS and Spark versions, ...). What did you put exactly in your prepare recipe ?
Hi,

Input file - Excel

Output file - HDFS Parquet format

DSS version - 4.0.1

Not sure about the spark engine version

Formula used - Fill empty cells function with pattern _(.*) and replace value as 0 -- There are multiple columns like _2002, _2003 etc.

Let me know if you need any other details.

1 Answer

0 votes
Hi,

We confirm this as a known issue when using "All" or "Pattern" columns selection mode in Spark engine: filling of value will only start after the first non-empty value. We are working on this.
answered by
657 questions
655 answers
490 comments
414 users