0 votes

 what defines the list of available engines for data processing recipes such as prepare? I have a HDFS dataset created by Impala, then a prepare or sync to another HDFS dataset, but only Spark/MR (or LocalStream) is available. Why the DSS is not allowing to use SQL based engines? The source dataset has a hive synced table definition


asked by

1 Answer

0 votes
Best answer

For prepare recipe you only have the choice between Streaming, Hadoop/Mapreduce or Spark.

You don't have SQL engine because we generate java code we can not push in a SQL db.


For sync recipe, as the purpose is to move data from one system to another, in some case we have to stream the data.

answered by
selected by
971 questions
998 answers
2,361 users

┬ęDataiku 2012-2018 - Privacy Policy