Interrow calculations in Analysis

UserBird
Dataiker
Interrow calculations in Analysis
Does the Analyse feature support interrow calculations similar to what we have with window functions in SQL. As a simple example get the column value of the previous row in an ordered dataset.
0 Kudos
2 Replies
jereze
Community Manager
Community Manager

Hi Uli,



No, in an Analysis, each row or line is calculated separately (as it can be multi-threaded). So you cannot use window functions in an Analysis.



You should use a SQL, Python or R recipe to do that.

(Similarly to the sorting.)



We have on the roadmap a Window recipe (similar to the Grouping recipe) that will work on datasets managed by SQL databases supporting such functions (PostgreSQL, etc.).



I hope that helps,



Jeremy

 

Jeremy, Product Manager at Dataiku
0 Kudos
ubethke
Level 2
Hi Jeremy.

Thanks for that.
Let's assume I am using Spark as my execution engine and let's assume that I apply a Grouping recipe and then an Analysis where I replace a value in a column for example. When this is pushed down into Spark does it require two passes over the data or is there some optimization in the background that generates code with one pass only?
0 Kudos

Labels

?
Labels (2)
A banner prompting to get Dataiku