Skip to content

CHANGELOG.md

Latest commit

 

History

History
529 lines (374 loc) · 33.5 KB

CHANGELOG.md

File metadata and controls

529 lines (374 loc) · 33.5 KB

Changelog

  • All notable changes to this project will be documented in this file.
  • The format is based on Keep a Changelog.
  • We do not follow semantic versioning.
  • All changes are solely tracked by date and have a git tag available (from 2021-10-19 onwards):
  • The latest master is considered stable and should be periodically merged into our customer projects.

Unreleased

Changed

2021-10-22

Changed

  • Fixes minor Makefile typos.
  • New go-starter releases are now git tagged (starting from the previous release go-starter-2021-10-19 onwards). See FAQ: What's the process of a new go-starter release?
  • You may now specify a specific tag/branch/commit from the upstream go-starter project while running make git-fetch-go-starter, make git-compare-go-starter and make git-merge-go-starter. This will especially come in handy if you want to do a multi-phased merge (for projects that haven't been updated in a long time):
    • Merge with the latest: make git-merge-go-starter
    • Merge with a specific tag, e.g. the tag go-starter-2021-10-19: GIT_GO_STARTER_TARGET=go-starter-2021-10-19 make git-merge-go-starter
    • Merge with a specific branch, e.g. the branch mr/housekeeping: GIT_GO_STARTER_TARGET=go-starter/mr/housekeeping make git-merge-go-starter (heads up! it's go-starter/<branchname>)
    • Merge with a specific commit, e.g. the commit e85bedb94c3562602bc23d2bfd09fca3b13d1e02: GIT_GO_STARTER_TARGET=e85bedb94c3562602bc23d2bfd09fca3b13d1e02 make git-merge-go-starter
  • The primary GitHub Action pipeline .github/workflows/build-test.yml has been synced to include most validation tasks from our internal .drone.yml pipeline. Furthermore:
    • Avoid Build & Test GitHub Action running twice (on push and on pull_request).
    • Add trivy scan to our base Build & Test pipeline (as we know also build and test the app target docker image).
    • Our GitHub Action pipeline will no longer attempt to cache the previously built Docker images by other pipelines, as extracting/restoring from cache (docker buildx) typically takes longer than fully rebuilding the whole image. We will reinvestigate caching mechanisms in the future if GitHub Actions provides a speedier and official integration for Docker images.

2021-10-19

Changed

  • BREAKING Upgrades to Go 1.17.1 golang:1.17.1-buster
    • Switch to //go:build <tag> from // +build <tag>.
    • Migrates go.mod via go mod tidy -go=1.17 (pruned module graphs).
    • Do the following to upgrade:
      1. make git-merge-go-starter
      2. ./docker-helper --rebuild
      3. Manually remove the new second require block (with all the // indirect modules) within your go.mod
      4. Execute go mod tidy -go=1.17 once so the second require block appears again.
      5. Find // +build <tag> and replace it with //go:build <tag>.
      6. make all.
      7. Recheck your go.mod that the newly added // indirect transitive dependencies are the proper version as you were previously using (e.g. via the output from make get-licenses and make get-embedded-modules). Feel free to move any // indirect tagged dependencies in your first require block to the second block. This is where they should live.
  • BREAKING You now need to take special care when it comes to parsing semicolons (;) in query strings via net/url and net/http from Go >1.17!
    • Anything before the semicolon will now be stripped. e.g. example?a=1;b=2&c=3 would have returned map[a:[1] b:[2] c:[3]], while now it returns map[c:[3]]
    • See Go 1.17 URL query parsing.
    • You may need to manually migrate your handlers/tests regarding this new default handling.

2021-09-27

Changed

2021-08-17

Changed

  • Hotfix: We will pin the Dockerfile development and builder stage to golang:1.16.7-buster (+ -buster) for now, as currently the new debian bullseye release within the go official docker images breaks some tooling. The upgrade to debian bullseye and Go 1.17 will happen simultaneously separately within go-starter in the following weeks.

2021-08-16

Changed

2021-08-06

Changed

  • Bump golang from 1.16.6 to 1.16.7 (requires ./docker-helper.sh --rebuild).
  • Adds util.GetEnvAsStringArrTrimmed and minor util test coverage upgrades.

2021-08-04

Changed

  • README.md badges for go-starter.
  • Fix some misspellings of English words within internal/test/*.go comments.
  • Upgrades
    • Bump github.com/labstack/echo/v4 from 4.4.0 to 4.5.0:
      • Switch from github.com/dgrijalva/jwt-go to github.com/golang-jwt/jwt to mitigate CVE-2020-26160.
      • Note that it might take some time until the former dep fully leaves our dependency graph, as it is also a transitive dependency of various versions of github.com/spf13/viper.
      • However, even though this functionality was never used by go-starter, this change fixes an important part: The original github.com/dgrijalva/jwt-go is no longer included in the final app binary, it is fully replaced by github.com/golang-jwt/jwt.
      • Our .trivyignore still excludes CVE-2020-26160 as trivy cannot skip checking transitive dependencies.
      • Breaking: If you have actually directly depended upon github.com/dgrijalva/jwt-go, please switch to github.com/golang-jwt/jwt via the following command: find -type f -name "*.go" -exec sed -i "s/dgrijalva\/jwt-go/golang-jwt\/jwt/g" {} \;

2021-07-30

Changed

  • Upgrades:
    • Bump golang from 1.16.5 to 1.16.6
    • Bump github.com/labstack/echo/v4 from 4.3.0 to 4.4.0 (adds binder.BindHeaders support, not affecting our goswagger runtime.Validatable bind helpers)
    • Bump github.com/gabriel-vasile/mimetype from 1.3.0 to 1.3.1
    • Bump github.com/spf13/cobra from 1.1.3 to 1.2.1 (and see all the big completion upgrades in 1.2.0)
    • Bump google.golang.org/api from 0.49.0 to 0.52.0
    • Bump gotestsum to 1.7.0 (adds handy keybindings while you are in make watch-tests mode, see While in watch mode, pressing some keys will perform an action)
    • Bump watchexec to 1.17.0
    • Bump golang.org/x/crypto to v0.0.0-20210711020723-a769d52b0f97

2021-07-29

Changed

  • Fixed Makefile has disregarded pipefails in executed targets (e.g. make sql-spec-migrate previously returned exit code 0 even if there were migration errors as its output was piped internally). We now set -cEeuo pipefail for make's shell args, preventing these issues.

2021-06-30

Changed

  • BREAKING Switched from golint to revive
    • golint is deprecated.
    • revive is considered to be a drop-in replacement for golint, however this change still might lead to breaking changes in your codebase.
  • BREAKING make lint no longer uses --fast when calling golangci-lint
    • Up until now, make lint also ran golangci-lint using the --fast flag to remain consistent with the linting performed by VSCode automatically.
    • As running only fast linters in both steps meant skipping quite a few validations (only 4/13 enabled linters are actually active), a decision has been made to break consistency between the two lint-steps and perform "full" linting during the build pipeline.
    • This change could potentially bring up additional warnings and thus fail your build until fixed.
  • BREAKING gosec is now also applied to test packages
    • All linters are now applied to every source code file in this project, removing the previous exclusion of gosec from test files/packages
    • As gosec might (incorrectly) detect some hardcoded credentials in your tests (variable names such as passwordResetLink get flagged), this change might require some fixes after merging.
  • Extended auth middleware to allow for multiple auth token sources
    • Default token validator uses access token table, maintaining previous behavior without any changes required.
    • Token validator can be changed to e.g. use separate API keys for specific endpoints, allowing for more flexibility if so desired.
  • Changed util.LogFromContext to always return a valid logger
    • Helper no longer returns a disabled logger if context provided did not have an associated logger set (e.g. by middleware). If you still need to disable the logger for a certain context/function, use util.DisableLogger(ctx, true) to force-disable it.
    • Added request ID to context in logger middleware.
  • Extended DB query helpers
    • Fixed TSQuery escaping, should now properly handle all type of user input.
    • Implemented helper for JSONB queries (see ExampleWhereJSON for implementation details).
    • Added LeftOuterJoin helper, similar to already existing LeftJoin variants.
    • Managed transactions (via WithTransaction) can now have their options configured via WithConfiguredTransaction.
    • Added util to combine query mods with OR expression.
  • Implemented middleware for parsing Cache-Control header
    • Allows for cache handling in relevant services, parsed directive is stored in request context.
    • New middleware is enabled by default, can be disabled via env var (SERVER_ECHO_ENABLE_CACHE_CONTROL_MIDDLEWARE).
  • Added extra misc. helpers
    • Extra helpers for slice handling and generating random strings from a given character set have been included (util.ContainsAllString, util.UniqueString, util.GenerateRandomString).
    • Added util to check whether current execution runs inside a test environment (util.RunningInTest).
  • Test and snapshot util improvements
    • Added snapshoter.SaveU as a shorthand for updating a single test
    • Implemented GenericArrayPayload with respective request helpers for array payloads in tests
    • Added VScode launch task for updating all snapshots in a single test file

2021-06-29

Changed

  • We now directly bake the gsdev cli "bridge" (it actually just runs go run -tags scripts /app/scripts/main.go "$@") into the development stage of our Dockerfile and create it at /usr/bin/gsdev (requires ./docker-helper.sh --rebuild).
    • gsdev was previously symlinked to /app/bin from /app/scripts/gsdev (within the projects' workspace) and chmod +x via the Makefile during init.
    • However this lead to problems with WSL2 VSCode related development setups (always dirty git workspaces as WSL2 tries to prevent +x flags).
    • BREAKING encountered at 2021-06-30: Upgrading your project via make git-merge-go-starter if you already have installed our previous gsdev approach from 2021-06-22 may require additional steps:
      • It might be necessary to unlink the current gsdev symlink residing at /app/bin/gsdev before merging up (as this symlinked file will no longer exist)!
      • Do this by issuing rm -f /app/bin/gsdev which will remove the symlink which pointed to the previous (now gone bash script) at /app/scripts/gsdev.
      • It might also be handy to install the newer variant directly into your container (without requiring a image rebuild). Do this by:
        • sudo su to become root in the container,
        • issuing the following command: printf '#!/bin/bash\nset -Eeo pipefail\ncd /app && go run -tags scripts ./scripts/main.go "$@"' > /usr/bin/gsdev && chmod 755 /usr/bin/gsdev (in sync with what we do in our Dockerfile) and
        • [CTRL + c] to return to being the development user within your container.

2021-06-24

Changed

  • Introduces GitHub Actions docker layer caching via docker buildx. For details see .github/workflows/build-test.yml.
  • Upgrades:
    • Bump golang from 1.16.4 to 1.16.5
    • golangci-lint@v1.41.1
    • Bump github.com/rs/zerolog from 1.22.0 to 1.23.0
    • Bump github.com/go-openapi/runtime from 0.19.28 to 0.19.29
    • Bump github.com/volatiletech/sqlboiler/v4 from 4.5.0 to 4.6.0
    • Bump github.com/rubenv/sql-migrate v0.0.0-20210408115534-a32ed26c37ea to v0.0.0-20210614095031-55d5740dbbcc
    • Bump github.com/spf13/viper v1.7.1 to v1.8.0
    • Bump golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a to v0.0.0-20210616213533-5ff15b29337e
    • Bump golang.org/x/sys v0.0.0-20210525143221-35b2ab0089ea to v0.0.0-20210616094352-59db8d763f22
    • Bump google.golang.org/api v0.47.0 to v0.49.0
  • Fixes linting within /scripts/**/*.go, now activated by default.

2021-06-22

Changed

  • Development scripts are no longer called via go run [script] but via gsdev:
    • The gsdev cli is our new entrypoint for development workflow specific scripts, these scripts are not available in the final app binary.
    • All previous go run scripts have been moved to their respective /scripts/cmd cli entrypoint + internal implementation within /scripts/internal/**.
    • Please use gsdev --help to get an overview of available development specific commands.
    • gsdev relys on a tiny helper bash script scripts/gsdev which gets symlinked to /app/bin on make init.
    • Use make test-scripts to run tests regarding these internal scripts within /scripts/**/*_test.go.
    • We now enforce that all /scripts/**/*.go files set the // +build scripts build tag. We do this to ensure these files are not directly depended upon from the actual app source-code within /internal.
  • VSCode's .devcontainer/devcontainer.json now defines that the go tooling must use the scripts build tag for its IntelliSense. This is neccessary to still get proper code-completion when modifying resources at /scripts/**/*.go. You may need to reattach VSCode and/or run ./docker-helper.sh --rebuild.

Added

  • Scaffolding tool to quickly generate generic CRUD endpoint stubs. Usage: gsdev scaffold [resource name] [flags], also see gsdev scaffold --help.

2021-05-26

Changed

  • Scans for CVE-2020-26160 also match for our final app binary, however, we do not use github.com/dgrijalva/jwt-go as part of our auth logic. This dependency is mostly here because of child dependencies, that yet need to upgrade to >=v4.0.0. Therefore, we currently disable this CVE for scans in this project (via .trivyignore).
  • Upgrades Dockerfile: watchexec@v1.16.1, lichen@v0.1.4 (requires ./docker-helper.sh --rebuild).

2021-05-18

Changed

  • Upgraded Dockerfile to golang:1.16.4, gotestsum@v1.6.4, golangci-lint@v1.40.1, watchexec@v1.16.0 (requires ./docker-helper.sh --rebuild).
  • Upgraded go.mod:
  • GitHub Actions:
    • Pin to actions/checkout@v2.3.4.
    • Remove unnecessary git checkout HEAD^2 in CodeQL step (Code Scanning recommends analyzing the merge commit for best results).
    • Limit trivy and codeQL actions to push against master and pull_request against master to overcome read-only access workflow errors.

2021-04-27

Added

  • Adds test.WithTestDatabaseFromDump*, test.WithTestServerFromDump methods for writing tests based on a database dump file that needs to be imported first:
    • We dynamically setup IntegreSQL pools for all combinations passed through a test.DatabaseDumpConfig{} object:
      • DumpFile string is required, absolute path to dump file
      • ApplyMigrations bool optional, default false, automigrate after installing the dump
      • ApplyTestFixtures bool optional, default false, import fixtures after (migrating) installing the dump
    • test.ApplyDump(ctx context.Context, t *testing.T, db *sql.DB, dumpFile string) error may be used to apply a dump to an existing database connection.
    • As we have dedicated IntegreSQL pools for each combination, testing performance should be on par with the default IntegreSQL database pool.
  • Adds test.WithTestDatabaseEmpty* methods for writing tests based on an empty database (also a dedicated IntegreSQL pool).
  • Adds context aware test.WithTest*Context methods reusing the provided context.Context (first arg).
  • Adds make sql-dump command to easily create a dump of the local development database to /app/dumps/development_YYYY-MM-DD-hh-mm-ss.sql (.gitignored).

Changed

  • test.ApplyMigrations(t *testing.T, db *sql.DB) (countMigrations int, err error) is now public (e.g. for usage with test.WithTestDatabaseEmpty* or test.WithTestDatabaseFromDump*)
  • test.ApplyTestFixtures(ctx context.Context, t *testing.T, db *sql.DB) (countFixtures int, err error) is now public (e.g. for usage with test.WithTestDatabaseEmpty* or test.WithTestDatabaseFromDump*)
  • internal/test/test_database_test.go and /app/internal/test/test_server_test.go were massively refactored to allow for better extensibility later on (non breaking, all method signatures are backward-compatible).

2021-04-12

Added

  • Adds echo NoCache middleware: Use middleware.NoCache() and middleware.NoCacheWithConfig(Skipper) to explicitly force browsers to never cache calls to these handlers/groups.

Changed

  • /swagger.yml and /-/* now explicity set no-cache headers by default, forcing browsers to re-execute calls each and every time.
  • Upgrade watchexec@v1.15.0 (requires ./docker-helper.sh --rebuild).

2021-04-08

Added

  • Live-Reload for our swagger-ui is now available out of the box:
    • allaboutapps/browser-sync acts as proxy at localhost:8081.
    • Requires ./docker-helper.sh --up.
    • Best used in combination with make watch-swagger (still refreshes make all or make swagger of course).

Changed

2021-04-07

Changed

  • Moved /api/main.yml to /api/config/main.yml to overcome path resolve issues (../definitions) with the VSCode 42crunch.vscode-openapi extension (auto-included in our devContainer) and our go-swagger concat behaviour.
  • Updated api/README.md information about /api/swagger.yml generation logic and changed make swagger-concat accordingly

2021-04-02

Changed

2021-04-01

Changed

2021-03-30

Changed

2021-03-26

Changed

2021-03-25

Changed

2021-03-24

Changed

  • We no longer do explicit calls to t.Parallel() in our go-starter tests (except autogenerated code). For the reasons why see FAQ: Should I use t.Parallel() in my tests?.
  • Switched to github.com/uw-labs/lichen for getting license information of embedded dependencies in our final ./bin/app binary.
  • The following make targets are no longer flagged as (opt) and thus move into the main make help target (use make help-all to see all targets):
    • make lint: Runs golangci-lint and make check-*.
    • make go-test-print-slowest: Print slowest running tests (must be done after running tests).
    • make get-licenses: Prints licenses of embedded modules in the compiled bin/app.
    • make get-embedded-modules: Prints embedded modules in the compiled bin/app.
    • make clean: Cleans ./tmp and ./api/tmp folder.
    • make get-module-name: Prints current go module-name (pipeable).
  • make check-gen-dirs now ignores .DS_Store within /internal/models/**/* and /internal/types/**/* and echo an errors detailing what happened.
  • Upgrade to github.com/go-openapi/runtime@v0.19.27

2021-03-16

Changed

  • make all no longer executes make info as part of its targets chain.
    • It's very common to use make all multiple times per day during development and thats fine! However, the output of make info is typically ignored by our engineers (if they explicitly want this information, they use make info). So make all was just too spammy in it's previous form.
    • make info does network calls and typically takes around 5sec to execute. This slowdown is not acceptable when running make all, especially if the information it provides isn't used anyways.
    • Thus: Just trigger make info manually if you need the information of the [spec DB] structure, current [handlers] and [go.mod] information. Furthermore you may also visit tmp/.info-db, tmp/.info-handlers and tmp/.info-go after triggering make info as we store this information there after a run.

2021-03-15

Changed

  • Upgrades go.mod:
  • make help no longer reports (opt) flagged targets, use make help-all instead.
  • make tools now executes go install {} in parallel
  • make info now fetches information in parallel
  • Seeding: Switch to db|dbUtil.WithTransaction instead of manually managing the db transaction. Note: We will enforce using WithTransaction instead of manually managing the life-cycle of db transactions through a custom linter in an upcoming change. It's way safer and manually managing db transactions only makes sense in very very special cases (where you will be able to opt-out via linter excludes). Also see What's WithTransaction, shouldn't I use db.BeginTx, db.Commit, and db.Rollback?.

Fixed

  • The correct implementation of (util|scripts).GetProjectRootDir() string now gets automatically selected based on the scripts build tag.
    • We currently have 2 different GetProjectRootDir() implementations and each one is useful on its own:
      • util.GetProjectRootDir() gets used while app or go test runs and resolves in the following way: use PROJECT_ROOT_DIR (if set), else default to the resolved path to the executable unless we can't resolve that, then panic!
      • scripts.GetProjectRootDir() gets used while generation time (make go-generate) and resolves in the following way: use PROJECT_ROOT_DIR (if set), otherwise default to /app (baked, as we can assume we are in the development container).
    • /internal/util/(get_project_root_dir.go|get_project_root_dir_scripts.go) is now introduced to automatically switch to the proper implementation based on the // +build !scripts or // +build scripts build tag, thus it's now consistent to import util.GetProjectRootDir(), especially while handler generation time (make go-generate).

2021-03-12

Changed

  • Upgrades to golang@v1.16.2 (use ./docker-helper.sh --rebuild).
  • Silence resolve of GO_MODULE_NAME if go was not found in path (typically host env related).

2021-03-11

Added

  • make build (make go-build) now sets internal/config.ModuleName, internal/config.Commit and internal/config.BuildDate via -ldflags.
    • /-/version (mgmt key auth) endpoint is now available, prints the same as app -v.
    • app -v is now available and prints out buildDate and commit. Sample:
app -v
github.com/mwieser/go-micro @ 19c4cdd0da151df432cd5ab33c35c8987b594cac (2021-03-11T15:42:27+00:00)

Changed

  • Upgrades to golang@v1.16.1 (use ./docker-helper.sh --rebuild).
  • Updates google.golang.org/api@v0.41.0, github.com/gabriel-vasile/mimetype@v1.2.0 (new supported formats), golang.org/x/sys
  • Removed **/.git from .dockerignore (builder stage) as we want the local git repo available while running make go-build.
  • app --help now prominently includes the module name of the project.
  • Prominently recommend make force-module-name after running make git-merge-go-starter to fix all import paths.

2021-03-09

Added

  • Introduces CHANGELOG.md

Changed

2021-03-08

Added

Changed

2021-02-23

Deprecated

  • util.BindAndValidate is now marked as deprecated as labstack/echo@v4.2.0 exposes a more granular binding through its DefaultBinder.

Added

  • The more specialized variants util.BindAndValidatePathAndQueryParams and util.BindAndValidateBody are now available. See /internal/util/http.go.

Changed

2021-02-16

Changed

  • Upgrades to pgFormatter@v5.0.0 + forces VSCode to use that version within the devcontainer through it's extension.

2021-02-09

Changed

  • golang@v1.15.8, go-swagger@v0.26.1

2021-02-01

Changed

- Dockerfile updates:
  - golang@1.15.7
  - apt add icu-devtools (VSCode live sharing)
  - gotestsum@1.6.1
  - golangci-lint@v1.36.0
  - goswagger@v0.26.0
- go.mod:
  - sqlboiler@4.4.0
  - swag@0.19.3
  - strfmt@0.20.0
  - testify@1.7.0
  - go-openapi/runtime@v0.19.26
  - go-openapi/swag@v0.19.13
  - go-openapi/validate@v0.20.1
  - jordan-wright/email
  - rogpeppe/go-internal@v1.7.0
  - golang.org/x/crypto
  - golang.org/x/sys
  - google.golang.org/api@v0.38.0

Fixed

  • disabled goswagger generate server flag --keep-spec-order as relative resolution of its temporal created yml file is broken - see go-swagger/go-swagger#2216

2020-11-04

Added

  • make watch-swagger and make watch-sql

Changed

  • sqlboiler@4.3.0

2020-11-02

Added

  • make watch-tests: Watches .go files and runs package tests on modifications.

2020-09-30

Added

2020-09-24

Added

2020-09-22

Added

  • app probe readiness and app probe liveness sub-commands.
  • /-/ready and /-/healthy handlers.

2020-09-16

Changed

  • Force VSCode to use our installed version of golang-cilint
  • All *.go files in /scripts now use the build tag scripts so we can ensure they are not compiled into the final app binary.

Added

  • go.not file to ensure certain generation- / test-only dependencies don't end up in the final app binary. Automatically checked though make (sub-target make check-embedded-modules-go-not).

2020-09-11