Add layered configuration files

AGENTS.md

@@ -19,6 +19,8 @@ The human author of a change is fully responsible for its correctness, scope, an
 
 Agents and humans alike must follow the canonical project documentation, which can be found as markdown (.md) files in the repo, particularly under /docs. This file does not define separate rules for AI systems; it clarifies that no such separate rules exist.
 
+In particular, you must have read README.md and CODESTYLE.md, and you must check docs/ when necessary.
+
 # Scope and autonomy limits
 
 AI tools must not:

docs/api_milc.md

@@ -21,7 +21,8 @@ def __init__(name: Optional[str] = None,
              author: Optional[str] = None,
              version: Optional[str] = None,
              logger: Optional[logging.Logger] = None,
-             env_prefix: Optional[str] = None) -> None
+             env_prefix: Optional[str] = None,
+             config_file: Optional[Union[str, Path]] = None) -> None
 ```
 
 Initialize the MILC object.
@@ -281,10 +282,10 @@ Save a single config option to the config file.
 #### save\_config
 
 ```python
-def save_config() -> None
+def save_config(config_file: Optional[Union[str, Path]] = None) -> None
 ```
 
-Save the current configuration to the config file.
+Save the current configuration to the config file or an explicit path.
 
 <a id="milc.MILC.__call__"></a>
 

docs/api_milc_interface.md

@@ -24,7 +24,8 @@ def milc_options(*,
                  author: Optional[str] = None,
                  version: Optional[str] = None,
                  logger: Optional[Logger] = None,
-                 env_prefix: Optional[str] = None) -> None
+                 env_prefix: Optional[str] = None,
+                 config_file: Optional[Union[str, Path]] = None) -> None
 ```
 
 Configure MILC before the entrypoint runs.
@@ -38,6 +39,7 @@ Call this before `cli()` or any imports that reference `cli`. It may be called m
 - `version` - The version string reported by `--version`.
 - `logger` - A custom logger instance to use instead of MILC's default logger.
 - `env_prefix` - A string prefix that enables environment variable defaults. When set, each `--flag` can be configured via a `<PREFIX>_<FLAG>` environment variable.
+- `config_file` - A system configuration file to read before the platformdirs user configuration file.
 
 <a id="milc_interface.MILCInterface.subcommand_name"></a>
 
@@ -184,10 +186,10 @@ Decorator to add an argument to a MILC command or subcommand.
 #### save\_config
 
 ```python
-def save_config() -> None
+def save_config(config_file: Optional[Union[str, Path]] = None) -> None
 ```
 
-Save the current configuration to the config file.
+Save the current configuration to the config file or an explicit path.
 
 <a id="milc_interface.MILCInterface.__call__"></a>
 

docs/api_subcommand_config.md

@@ -53,6 +53,10 @@ Set a config key in the running config.
                    '--all',
                    action='store_true',
                    help='Show all configuration options.')
+@milc.cli.argument('-o',
+                   '--output',
+                   arg_only=True,
+                   help='Write the current configuration to this file.')
 @milc.cli.argument('-ro',
                    '--read-only',
                    arg_only=True,

docs/configuration.md

@@ -35,7 +35,7 @@ You can create new values by simply assigning to them. This only works with dict
 
 # Writing Configuration Files
 
-Use `cli.save_config()` to save the user's configuration file. It will be written to the location specified by `cli.config_file`.
+Use `cli.save_config()` to save the user's configuration file. It writes to the user's config location specified by `cli.config_file`, unless `--config-file` was supplied. Pass a path to `cli.save_config(path)` to write the current configuration elsewhere.
 
 # Configuration File Location
 
@@ -53,6 +53,16 @@ This will (usually) result in the following config file locations:
 * macOS: `~/Library/Application Support/Florzelbop`
 * Windows: `C:\Users\<User>\AppData\Local\Florzelbop\Florzelbop`
 
+# System Configuration File
+
+Applications can read a system configuration file before the platformdirs user configuration file:
+
+```python
+cli.milc_options(config_file='/etc/my_app.conf')
+```
+
+When the system file exists, MILC reads it first. It then always reads the platformdirs user config, whose settings override matching system settings. `cli.config_file` remains the platformdirs location, and `cli.save_config()` writes there. An explicit `--config-file` command-line argument instead reads and writes only that path, bypassing both layered locations.
+
 # Where Did A Value Come From?
 
 Sometimes you need to know how a configuration value was set. You can use `cli.config_source` to find out.

docs/metadata.md

@@ -19,6 +19,7 @@ You should only do this once, and you should do it as early in your program's ex
 * `author` — The author string, used in the config file path on some platforms.
 * `logger` — A custom logger instance to use instead of MILC's default logger.
 * `env_prefix` — A string prefix that enables [environment variable defaults](environment_variables.md). When set, each `--flag` can be configured via a `<PREFIX>_<FLAG>` environment variable. See [Environment Variables](environment_variables.md) for full details.
+* `config_file` — A system configuration file to read before the platformdirs user configuration file. The user configuration overrides matching settings. `--config-file` bypasses both and uses only its supplied path.
 
 !!! warning
     If you have spread your program among several files, or you are using `milc.subcommand.config`, you need to use `cli.milc_options()` before you import those modules.

docs/subcommand_config.md

@@ -106,6 +106,12 @@ You can read configuration values for all set options, the entire configuration,
 
     my_cli config user general.verbose general.log_format
 
+## Exporting Configuration
+
+Use `--output` (or `-o`) to write the current configuration to another file. This does not change the normal platformdirs save location:
+
+    my_cli config --output /path/to/config.ini
+
 ## Deleting Configuration Values
 
 You can delete a configuration value by setting it to the special string `None`.

milc/configuration.py

@@ -1,3 +1,6 @@
+from configparser import RawConfigParser
+from decimal import Decimal
+from pathlib import Path
 from typing import TYPE_CHECKING, Any, Generator, Hashable, List, Tuple
 
 if TYPE_CHECKING:
@@ -78,6 +81,37 @@ def _config_navigate(config_root: 'Configuration', dotted_path: str) -> 'Configu
     return section
 
 
+def _read_config_file(config_file: Path, config: Configuration, config_source: Configuration) -> None:
+    """Merge a configuration file into the running configuration."""
+    raw_config = RawConfigParser()
+
+    raw_config.read(str(config_file))
+
+    # Iterate over the config file options and write them into config.
+    # Section names may be dotted (e.g. [remote.add]) for nested subcommands.
+    for section in raw_config.sections():
+        config_section = _config_navigate(config, section)
+        config_source_section = _config_navigate(config_source, section)
+
+        for option in raw_config.options(section):
+            value = raw_config.get(section, option)
+
+            if value.lower() in ['yes', 'true', 'on']:
+                value = True
+            elif value.lower() in ['no', 'false', 'off']:
+                value = False
+            elif value.lower() in ['none']:
+                continue
+            elif value.replace('.', '').isdigit():
+                if '.' in value:
+                    value = Decimal(value)
+                else:
+                    value = int(value)
+
+            config_section[option] = value
+            config_source_section[option] = 'config_file'
+
+
 def _collect_config_sections(section: 'Configuration', prefix: str = '') -> Generator[Tuple[str, str, Any], None, None]:
     """Recursively yield (dotted_section_name, option_name, value) for all leaf values.
 

milc/milc.py

@@ -6,7 +6,6 @@
 import subprocess
 import sys
 from configparser import RawConfigParser
-from decimal import Decimal
 from pathlib import Path
 from platform import platform
 from tempfile import NamedTemporaryFile
@@ -27,7 +26,7 @@
 from ._in_argv import _in_argv, _index_argv
 from .ansi import MILCFormatter, ansi_colors, ansi_config, ansi_escape, format_ansi
 from .attrdict import AttrDict
-from .configuration import Configuration, SubparserWrapper, _collect_config_sections, _config_navigate, get_argument_name, get_argument_strings, handle_store_boolean
+from .configuration import Configuration, SubparserWrapper, _collect_config_sections, _config_navigate, _read_config_file, get_argument_name, get_argument_strings, handle_store_boolean
 
 P = ParamSpec("P")
 R = TypeVar("R")
@@ -36,7 +35,7 @@
 class MILC(object):
     """MILC - An Opinionated Batteries Included Framework
     """
-    def __init__(self, name: Optional[str] = None, author: Optional[str] = None, version: Optional[str] = None, logger: Optional[logging.Logger] = None, env_prefix: Optional[str] = None) -> None:
+    def __init__(self, name: Optional[str] = None, author: Optional[str] = None, version: Optional[str] = None, logger: Optional[logging.Logger] = None, env_prefix: Optional[str] = None, config_file: Optional[Union[str, Path]] = None) -> None:
         """Initialize the MILC object.
         """
         # Set some defaults
@@ -66,6 +65,8 @@ def __init__(self, name: Optional[str] = None, author: Optional[str] = None, ver
         self._initialized = False
         self.ansi = ansi_colors
         self.arg_only: Dict[str, List[str]] = {}
+        self._config_file_explicit = _in_argv('--config-file')
+        self.system_config_file = Path(config_file).expanduser().resolve() if config_file is not None else None
         self.config_file = self.find_config_file()
         self.default_arguments: Dict[str, Dict[str, Optional[str]]] = {}
         self.env_prefix = env_prefix
@@ -537,35 +538,20 @@ def read_config_file(self) -> Tuple[Configuration, Configuration]:
         """
         config = Configuration()
         config_source = Configuration()
+        config_files = []
 
-        if self.config_file.exists():
-            raw_config = RawConfigParser()
-            raw_config.read(str(self.config_file))
-
-            # Iterate over the config file options and write them into config.
-            # Section names may be dotted (e.g. [remote.add]) for nested subcommands.
-            for section in raw_config.sections():
-                config_section = _config_navigate(config, section)
-                config_source_section = _config_navigate(config_source, section)
-
-                for option in raw_config.options(section):
-                    value = raw_config.get(section, option)
-
-                    # Coerce values into useful datatypes
-                    if value.lower() in ['yes', 'true', 'on']:
-                        value = True
-                    elif value.lower() in ['no', 'false', 'off']:
-                        value = False
-                    elif value.lower() in ['none']:
-                        continue
-                    elif value.replace('.', '').isdigit():
-                        if '.' in value:
-                            value = Decimal(value)
-                        else:
-                            value = int(value)
-
-                    config_section[option] = value
-                    config_source_section[option] = 'config_file'
+        if self._config_file_explicit:
+            config_files.append(self.config_file)
+        else:
+            for config_file in (self.system_config_file, self.config_file):
+                if config_file:
+                    config_files.append(config_file)
+
+        for config_file in config_files:
+            if not config_file.exists():
+                continue
+
+            _read_config_file(config_file, config, config_source)
 
         return config, config_source
 
@@ -627,35 +613,39 @@ def merge_args_into_config(self) -> None:
 
         self.release_lock()
 
-    def _save_config_file(self, config: Configuration) -> None:
+    def _save_config_file(self, config: Configuration, config_file: Optional[Path] = None) -> None:
         """Write config to disk.
         """
+        config_file = config_file or self.config_file
+        config_dir = config_file.parent
+        sane_config = RawConfigParser()
+        tmpfile_name = None
+
         # Generate a sanitized version of our running configuration.
         # _collect_config_sections recurses into nested ConfigurationSection objects,
         # emitting (dotted_section_name, option_name, value) for every leaf.
-        sane_config = RawConfigParser()
         for section_name, option_name, value in _collect_config_sections(config):
             if not sane_config.has_section(section_name):
                 sane_config.add_section(section_name)
             config_source_section = _config_navigate(self.config_source, section_name)
             if config_source_section[option_name] == 'config_file' and value is not None:
                 sane_config.set(section_name, option_name, str(value))
 
-        if not self.config_dir.exists():
-            self.config_dir.mkdir(parents=True, exist_ok=True)
+        if not config_dir.exists():
+            config_dir.mkdir(parents=True, exist_ok=True)
 
         # Write the config file atomically.
         self.acquire_lock()
-        tmpfile_name = None
+
         try:
-            with NamedTemporaryFile(mode='w', dir=str(self.config_dir), delete=False) as tmpfile:
+            with NamedTemporaryFile(mode='w', dir=str(config_dir), delete=False) as tmpfile:
                 tmpfile_name = tmpfile.name
                 sane_config.write(tmpfile)
 
             if os.path.getsize(tmpfile_name) > 0:
-                os.replace(tmpfile_name, str(self.config_file))
+                os.replace(tmpfile_name, str(config_file))
             else:
-                self.log.warning('Config file saving failed, not replacing %s with %s.', str(self.config_file), tmpfile_name)
+                self.log.warning('Config file saving failed, not replacing %s with %s.', str(config_file), tmpfile_name)
         finally:
             try:
                 if tmpfile_name and os.path.exists(tmpfile_name):
@@ -682,18 +672,22 @@ def write_config_option(self, section: str, option: Any) -> None:
         # Housekeeping
         self.log.info('Wrote configuration to %s', shlex.quote(str(self.config_file)))
 
-    def save_config(self) -> None:
-        """Save the current configuration to the config file.
+    def save_config(self, config_file: Optional[Union[str, Path]] = None) -> None:
+        """Save the current configuration to the config file or an explicit path.
         """
-        self.log.debug("Saving config file to '%s'", str(self.config_file))
+        save_path = Path(config_file).expanduser().resolve() if config_file is not None else self.config_file
 
-        if not self.config_file:
+        self.log.debug("Saving config file to '%s'", str(save_path))
+
+        if not save_path:
             self.log.warning('%s.config_file file not set, not saving config!', self.__class__.__name__)
+
             return
 
         # Write config to disk
-        self._save_config_file(self.config)
-        self.log.info('Wrote configuration to %s', shlex.quote(str(self.config_file)))
+        self._save_config_file(self.config, save_path)
+
+        self.log.info('Wrote configuration to %s', shlex.quote(str(save_path)))
 
     def check_deprecated(self) -> None:
         entry_name = getattr(self._entrypoint, '__name__')

milc/milc_interface.py

@@ -27,8 +27,9 @@ def __init__(self) -> None:
         self._version: Optional[str] = None
         self._logger: Optional[Logger] = None
         self._env_prefix: Optional[str] = None
+        self._config_file: Optional[Union[str, Path]] = None
 
-    def milc_options(self, *, name: Optional[str] = None, author: Optional[str] = None, version: Optional[str] = None, logger: Optional[Logger] = None, env_prefix: Optional[str] = None) -> None:
+    def milc_options(self, *, name: Optional[str] = None, author: Optional[str] = None, version: Optional[str] = None, logger: Optional[Logger] = None, env_prefix: Optional[str] = None, config_file: Optional[Union[str, Path]] = None) -> None:
         """Configure MILC before the entrypoint runs.
 
         Call this before `cli()` or any imports that reference `cli`. It may be called multiple times; each call updates only the supplied arguments.
@@ -39,6 +40,7 @@ def milc_options(self, *, name: Optional[str] = None, author: Optional[str] = No
             version: The version string reported by `--version`.
             logger: A custom logger instance to use instead of MILC's default logger.
             env_prefix: A string prefix that enables environment variable defaults. When set, each `--flag` can be configured via a `<PREFIX>_<FLAG>` environment variable.
+            config_file: A system configuration file to read before the platformdirs user configuration file.
         """
         if self._milc and self._milc._initialized:
             raise RuntimeError('You must run cli.milc_options() before cli() or anything else!')
@@ -53,7 +55,9 @@ def milc_options(self, *, name: Optional[str] = None, author: Optional[str] = No
             self._logger = logger
         if env_prefix is not None:
             self._env_prefix = env_prefix
-        self._milc = MILC(self._name, self._author, self._version, self._logger, self._env_prefix)
+        if config_file is not None:
+            self._config_file = config_file
+        self._milc = MILC(self._name, self._author, self._version, self._logger, self._env_prefix, self._config_file)
 
     @property
     def milc(self) -> MILC:
@@ -184,10 +188,10 @@ def argument(self, *args: Any, **kwargs: Any) -> Callable[[Callable[P, R]], Call
         """
         return self.milc.argument(*args, **kwargs)
 
-    def save_config(self) -> None:
-        """Save the current configuration to the config file.
+    def save_config(self, config_file: Optional[Union[str, Path]] = None) -> None:
+        """Save the current configuration to the config file or an explicit path.
         """
-        return self.milc.save_config()
+        return self.milc.save_config(config_file)
 
     def __call__(self) -> Any:
         """Execute the entrypoint function.