Version 1.1.0

This release adds logging capability to Stream Loader and adds Sentry integration to Stream Loader, Mutator, and Repeater.

New functionality

• Implement new metrics reporter for stdout (#238)
• Stream Loader: integrate Sentry (#240)
• Repeater: integrate Sentry (#241)
• Mutator: integrate Sentry (#242)

Dependency management

• Bump log4cats to 1.4.0 (#246)

Under the hood

• Repeater: move statistics logs under metrics (#239)
• Mutator: use logger instead of println (#244)
• Stream Loader: use logger instead of println (#245)
• Extend copyright to 2022 (#243)

Version 1.0.2

Prepare for 1.0.2 release

Version 1.0.1

A patch release, with a few small improvements over the previous release. In particular, the library bumps should make the streamloader more resilient to any pubsub exceptions.

Changelog

• Use sbt-dynver plugin (#227)
• Repeater: change filename logging level to info (#222)
• Repeater: fix typos in cli options' help text (#221)
• Add monitoring defaults to application.conf (#220)
• Bump fs2-google-cloud-pubsub to 0.18.0 (#226)
• Bump google-cloud-pubsub to 1.114.3 (#225)
• Bump google-cloud-storage to 2.1.1 (#224)
• Bump google-cloud-bigquery to 2.1.6 (#223)

Version 1.0.0

The main highlight of this release is a production-ready StreamLoader. For more information on breaking changes and new features, see the Discourse post.

Breaking changes

Switch from JSON to HOCON in config file (#186)

New functionality

Make the --config flag optional (#214)
Add load_tstamp field (#140)
Add metrics in StreamLoader and Repeater (#165)
Mutator: add table partitioning to create command (#113)

Improvements and bugfixes

StreamLoader: drop intermediary queue for writing out observed types (#183)
StreamLoader: batch events in BigQuery write request (#192)
Repeater: fix logging of number of inserted records (#143)
Repeater: remove PayloadParser (#215)
Repeater: make EventContainer codecs isomorphic (#141)
Loader: drop waitUntilDone from runner (#145)

Dependency management

Bump schema-ddl to 0.14.2 (#197)
Bump netty-codec version to 4.1.48.Final (#198)
Bump metrics-core to 4.1.25 (#199)
Bump slf4j-simple to 1.7.32 (#200)
Bump cats-core to 2.6.1 (#201)
Bump cats-effect to 2.5.4 (#202)
Bump circe to 0.14.1 (#203)
Bump decline to 1.4.0 (#204)
Bump fs2-core to 2.5.9 (#205)
Bump netty-codec-http version to 4.1.61.Final (#206)
Bump http4s-async-http-client to 0.22.5 (#207)
Bump snowplow-badrows to 2.1.1 (#208)
Bump iglu-scala-client to 1.1.1 (#209)
Bump iglu-core-circe to 1.0.1 (#210)
Bump specs2-core to 4.12.12 (#212)
Bump sbt to 1.5.5 (#213)
Remove scalacheck dependency (#211)

Under the hood

Change base image to adoptopenjdk:8-jre-hotspot-focal (#188)
Publish arm64 and amd64 docker images (#193)
Github action to scan docker image with lacework (#195)

Deprecations

Forwarder: remove from working tree (#121)

Thanks

Thanks to Alex Fainshtein for contributing to this release.

Version 0.6.4

A bugfix release, fixing invalid casting of null: [array, null] into STRING

Bugfixes

• Bump schema-ddl to 0.13.0 (#176)

Under the hood

• Bump sbt to 1.5.0 (#175)

Version 0.6.3

Performance improvements thanks to updated iglu-scala-client version and using optimizedWrites in Loader.

Changelog

• Loader: use optimizedWrites (#172)
• Add TransformJsonBenchmark, FromEventBenchmark and GroupContextsBenchmark (#162)
• Bump iglu-scala-client to 1.0.2 (#163)
• Use snowplow/base-debian image from Docker Hub (#164)

Discourse

• https://discourse.snowplowanalytics.com/t/bigquery-loader-0-6-3-released/4960

Version 0.6.2

• Discourse post: https://discourse.snowplowanalytics.com/t/bigquery-loader-0-6-2-released/4711

Docker images can now be found on Docker Hub and are no longer hosted on Bintray.

Fatjars are attached to these release notes, or you can build the jars yourself following instructions in the README.

Build

• Migrate from Travis to Github Actions (#149)

Dependency management

• Bump snowplow-scala-analytics-sdk to 2.1.0 (#150)
• Bump scio-core to 0.9.6 (#151)
• Bump metrics-core to 4.1.17 (#152)
• Bump cats-effect to 2.3.1 (#153)
• Bump fs2-core to 2.5.0 (#154)
• Bump http4s-async-http-client to 0.21.15 (#155)

Misc

• Add benchmark module (#148)
• Extend copyright to 2021 (#158)

Thanks

• talyz

Version 0.6.1

Fixes a bug introduced in 0.6.0.

• Discourse post: https://discourse.snowplowanalytics.com/t/bigquery-loader-0-6-1-released/4302

Bugs

Common: do not drop contexts when creating TableRow from enriched event (#136)

Version 0.6.0

We have discovered a bug in version 0.6.0 that can lead to contexts being dropped from the event. Please do not upgrade to this version. We’re working on a fix and will update soon.

Adds BigQuery StreamLoader.

• Discourse post: https://discourse.snowplowanalytics.com/t/bigquery-loader-0-6-0-released/4272

New features

Add Snowplow BigQuery StreamLoader (#116)

Bugs

Fix ignored project id from the config (#104)

Security

Integrate Snyk (#117)
Common: fix jackson-databind version to 2.9.10.6 (#128)
Common: fix netty-codec version to 4.1.46.Final (#129)
Common: fix netty-codec-http version to 4.1.44.Final (#130)
Common: fix fastjson version to 1.2.69 (#131)
Common: fix google-oauth-client version to 1.31.0 (#132)

Misc

Bump Scala to 2.13 (#115)
Bump scio-core to 0.9.4 (#114)

Thanks

@simplylizz

Version 0.5.1

Bug fixes

Remove LoaderCache.scala (#107)