Support loading configuration from both YAML files and env vars

[lippserd]

I will keep the original PR description as a reference, but the implementation now uses config.Load() from our Go libs.

With this PR Icinga DB supports configuration loading in three scenarios:

1. Load configuration solely from YAML files when no relevant environment variables are set.
2. Combine YAML file and environment variable configurations, allowing environment variables to supplement or override possible incomplete YAML configurations.
3. Load entirely from environment variables if the default YAML config file is absent and no specific config path is provided by the user.

config.FromYAMLFile() is still called first but continuation with config.FromEnv() is allowed by handling:

• ErrInvalidConfiguration: The configuration may be incomplete and will be revalidated in config.FromEnv().
• Non-existent file errors: If no explicit config path is set, fallback to environment variables is allowed.

config.FromEnv() is called regardless of the outcome from config.FromYAMLFile(). If no environment variables are set, configuration relies entirely on YAML. Otherwise, environment variables can supplement, override YAML settings, or serve as the sole source. config.FromEnv() also includes validation, ensuring completeness after considering both sources.

Possible alternative implementations:

• If the config path is the default, os.Stat() could be used before calling config.FromYAMLFile(), rather than handling non-existent file errors. This approach would split the logic into two sections and add an additional if block.
• Don't call Validate() in the config package automatically. Instead, require it to be called manually. I appreciate that library-wise, both config.FromYAMLFile() and config.FromEnv() include validation allowing them to be used without needing an additional function call on their own. When combining them, I think it's straightforward to use errors.Is() to check for ErrInvalidConfiguration, i.e. errors from Validate().

Requires:

• build(deps): bump github.com/icinga/icinga-go-library from 0.4.0 to 0.5.0 #879
• Support loading configuration from both YAML files and env vars icinga-go-library#87
• config#Load(): Let error indicate absence of the default config file icinga-go-library#112

closes #756

[oxzi]

Having documentation (like human readable docs) for this PR would be nice. Even just stating that this is possible in the 03-Configuration section.

[oxzi]

I have taken another look at why the CI fails and after some digging, it became quite obvious. The Icinga DB container bailed out as it cannot parse its config.

can't parse YAML file /icingadb.yml: does not implemented Unmarshaler

However, the generated config looked fine.

database:
  type: pgsql
  host: 172.20.0.4
  port: 5432
  database: d1
  user: u1
  password: 7f96cc1ac9d3ad03
redis:
  host: 172.20.0.2
  port: 6379
logging:
  level: debug
  interval: 1s
retention:
  history-days: 7
  sla-days: 30
  options:
    acknowledgement: 0
    comment: 1
    downtime: 2

So I went down the YAML parsing and ended up at a faulty YAML parsing for the RetentionOptions.

Turns out, the custom encoding.TextUnmarshaler is not enough on its own, but a yaml.InterfaceUnmarshaleris also required for custom types.

I pushed a hopefully fix to the just created config-from-environment-variables-fix-alvar branch in fe94784. At least on my machine all tests went through.
Edit: Pushed commit in this PR's branch after talking about with @lippserd and deleted my other branch.

It would be good to have unit test for this, similar to RetentionOptions to catch this earlier on.

[lippserd]

@oxzi Thanks for the correction. We already had a similar issue in our Go library and I'll link it here for reference:

Icinga/icinga-go-library#84

From my point of view, the changes from the PR can be merged now.

Edit:

I'll push another set of tests:

It would be good to have unit test for this, similar to RetentionOptions to catch this earlier on.

[oxzi]

LGTM, Thanks!

I have just canceled the docker CI job since it seemed stalled since over two hours and reran it. Unless it fails, feel free to merge!