0 votes

I'm trying to create a custom transformation but haven't been successful. The sample code provided works fine but when I define my own function with the same transformation it fails. See below for the exact snippets.

Works:

from sklearn import preprocessing
import numpy as np

# Applies log transformation to the feature
processor = preprocessing.FunctionTransformer(np.log1p)



Does not work:

from sklearn import preprocessing
import numpy as np

def CustomT(X):
    return np.log1p(X)

# Applies log transformation to the feature
processor = preprocessing.FunctionTransformer(CustomT)

 

Error (more detailed logs available if required):

Failed to train : <type 'exceptions.TypeError'> : expected string or Unicode object, NoneType found

 

I used a pared down dataset to generate this log: https://we.tl/t-YWJABcMO5s

by
edited by
Could you include the full logs please? If they are too large, you can use a file transfer service such as WeTransfer.

1 Answer

0 votes
Hello, (Repost from our official customer support channel) In a custom preprocessing, you cannot declare a function nor class as those won't be pickled. You need to define it in a library, at either the project level or instance level, see https://doc.dataiku.com/dss/latest/python/reusing-code.html, and then import and use it in the custom preprocessing code. Hope it helps, Alex
ago by
1,078 questions
1,123 answers
1,246 comments
10,710 users

┬ęDataiku 2012-2018 - Privacy Policy