DSS auto save Json String To Array

tmgiang
Level 2
DSS auto save Json String To Array
I use Python to read data from Mongodb. But when I save to DSS, the json data automatically store as "array" like that:

[{u'keyed_data': [{u'indexed_EDLetterCode_orig': ...

(actually, the json data is array)

Before use DSS, I code the recipe on Jupyter Notebook and it works well (I use json_normalize to read json data to panda frame). But after DSS automatically store the json in "array" type, I can't use json normalize

I found that the preparation recipe can fold / unfold the json data to rows, but unsuccesfully.

How can I read the json data to rows? Can I force DSS does not store json data like "array" ?
0 Kudos
3 Replies
Alex_Combessie
Dataiker Alumni
Hi,

We have a native MongoDB connector: https://www.dataiku.com/learn/guide/connecting/dss-and-mongodb.html. Could you try to switch to it? Let us know if it solves the issue you are having with the custom Python connection.

Cheers,

Alex
0 Kudos
tmgiang
Level 2
Author
thanks your answer but I can not create data set with mongo connection (i created mongo connection already). Maybe I need to upgrade my account to premium
0 Kudos
Alex_Combessie
Dataiker Alumni
Hello,
Indeed, native MongoDB connection is not available in the Community edition. You can read more on this page: https://www.dataiku.com/dss/editions/.
To pursue with the custom Python connection, have you tried writing the dataframe resulting from pandas.io.json_normalize into a dataiku.Dataset using`write_from_dataframe`? Feel free to paste some code if that is OK with you.
0 Kudos

Labels

?
Labels (3)
A banner prompting to get Dataiku