Snowflake SQL Driver Option

[nmbryant]

Is there any interest in having Snowflake as a driver option in the SQL processor/output? The driver can be found here: https://github.com/snowflakedb/gosnowflake

[Jeffail]

Hey @nmbryant yes absolutely, any idea if there's a way to test it without using a snowflake account?

[nmbryant]

Unfortunately it doesn't seem like testing can be without an account from what I have found. I think adding the driver option would be a small change, should I just make the change without tests?

[ironchef]

Snowflake is pretty good with open source folks from what I've heard. May be worth a shout out to someone like mpilman. If not, I have access to snowflake and have used benthos in the past ... if I can swing some cycles I could potentially help.

[mihaitodor]

I'm exploring the library right now and trying to build an output. @ironchef do you have experience with the Snowflake platform? I'd love to have a chat with someone who can provide some feedback on how people expect this to work in practice, especially from a data streaming perspective.

[ironchef]

Yup yup .. mostly from a "hey .. execute this query and give me results" perspective on the golang side. But I've integrated snowflake with AWS + S3, airflow, dbt, etc. (a bunch of other tooling). I'll dm / email ya.

[mihaitodor]

@ironchef I created a PR for adding a snowflake_put output in #1160. If you have some time to try it out, I'd really appreciate some feedback.

Potentially, we can also have a snowflake_insert output, but there are some caveats to that as described in this issue I opened on the gosnowflake client repo: snowflakedb/gosnowflake#540

[mihaitodor]

Update: The snowflake_put output was merged in #1160. I think it's worth adding a corresponding input and, potentially, some alternatives based on SELECT / INSERT. Leaving this open for feedback.

[mihaitodor]

Closing this since, besides the existing snowflake_put output, Snowflake support was added to the various sql_* components in v4.10.0.