I have a little python streamlit webapp I've made in code studio that outputs data generated to a snowflake table that's in the same project. The way I'm attempting to do it (code below) is taking up to 2 minutes to upload the data but it's only ever a couple of rows of data (<10), 20 columns and no large fields. I would have thought for data this small that it would take under 30s. Is there a faster way of doing this?
with st.spinner('Uploading to Snowflake...'):
output_ds = dataiku.Dataset("output_dataset")
output_ds.write_with_schema(df_to_write)
st.success('Successful Upload into Snowflake!', icon="โ
")
