Invalid argument: MySQL Schema too wide

Question

I'm trying to transfer data from the file based architecture to a MariaDB ColumnStore database.

 

I however come across this error:

"An invalid argument has been encountered : MySQL Schema too wide (maximum is 64K - strings alone currently require 92696)"

Am I importing too many columns? Or too many characters in total? Why wouldn't I be able to import as many columns as I'd like? It seems to be limiting my import.

I suspect it to be a Dataiku problem since I am sure I've had more columns in there before.

 

See attached screenshot for a complete view:

 

Thanks in advance

Comments

How many string columns do you have in your dataset?

Answer 1

Hi Casper,

By design, MySQL restricts datasets to a maximum schema size of 64K which is actually 16K characters. Right now, your output dataset schema is more than 64k, so that’s why this error is thrown. 

Please go to the schema settings of the output dataset and shrink the values of the columns. I’ve attached a screenshot of the schema settings and where you are supposed to change the value. Please note that this will most likely take a bit of trial and error, so you would have to adjust the values in a few columns and then re-run recipe to see if the error is thrown again.