Allow custom wordlists, fix #84.

Follow XDG base file specification for configuration files and to find
additional wordlist files.

CHANGES.rst

@@ -14,8 +14,16 @@ We now follow the XDG base directory specification, that tells where config
 unset, `${HOME}/.config/diceware/diceware.ini`. The traditional location
 `${HOME}/.diceware.ini` is still supported.
 
+Furthermore we read `${XDG_DATA_HOME}/diceware/` (or
+`${HOME}/.local/share/diceware/` if `${XDG_DATA_HOME}` is empty or unset) to
+lookup further wordlists.
+
+New option `--show-wordlist-dirs` lists all directory locations we search for
+contained wordlists.
+
 - Officially support Python 3.10 to 3.12.
-- Fixed #86: Follow `XDG`_ base directory specification.
+- Fixed #86: Follow `XDG <https://specifications.freedesktop.org/basedir-spec/latest/>`_ base directory specification.
+- Fixed #84: Allow to store wordlists also in custom directories.
 - Use `ruff` as linter, drop `flake8`.
 - Renew `tox` configuration.
 - Switch to `pyproject`-based project layout, away from using `setup.py`.

diceware.1

@@ -1,6 +1,6 @@
 .\" Man page generated from reStructuredText.
 .
-.TH DICEWARE 1 "July 2024" "diceware 1.0.dev0" "User Commands"
+.TH DICEWARE 1 "September 2024" "diceware 1.0.dev0" "User Commands"
 .SH NAME
 diceware \- create passphrases
 .
@@ -104,15 +104,118 @@ Number of sides of dice. Default: 6
 .UNINDENT
 .UNINDENT
 .UNINDENT
+.SH ENVIRONMENT VARIABLES
+.INDENT 0.0
+.TP
+.B \fBXDG_CONFIG_HOME\fP
+If set and not empty, this variable determines the directory to use for
+user\-local configuration files. We then lookup
+\fI${XDG_CONFIG_HOME}/diceware/diceware.ini\fP and values set here override
+system\-wide config files.
+.TP
+.B \fBXDG_CONFIG_DIRS\fP
+If set and not empty, this variable is interpreted as colon\-separated list
+of directories, that might contain system\-wide configuration files. We
+lookup \fI<DIR>/diceware/diceware.ini\fP for each directory set in
+\fI$XDG_CONFIG_DIRS\fP\&.
+.TP
+.B \fBXDG_DATA_HOME\fP
+.INDENT 7.0
+.INDENT 3.5
+If set and not empty, this variable determines a directory to search for
+additional wordlists. We then lookup \fI${XDG_DATA_HOME}/diceware\fP for any
+existing wordlist files.
+.UNINDENT
+.UNINDENT
+.INDENT 7.0
+.TP
+.B \fBXDG_DATA_DIRS\fP
+If set and not empty, this variable is interpreted as colon\-separated list
+of directories, that might contain additional wordlist files. See below. We
+lookup \fI<DIR>/diceware/\fP then for each directory set in the list.
+.UNINDENT
+.UNINDENT
 .SH FILES
+.sp
+Depending on environment variables set (or not set) we lookup certain
+directories for configuration files called \fBdiceware.ini\fP and for wordlist
+files.
+.SS CONFIGURATION FILES
+.sp
+Configuration settings for \fBdiceware\fP can be spread over several
+configuration files. We parse configuration values from the files given below,
+but values set in former files take precedence over values set in latter ones.
 .INDENT 0.0
 .TP
 .B \fI~/.diceware.ini\fP
-Your personal diceware configuration file.
+Your personal diceware configuration file. Values set here override values
+from any other configuration file.
+.TP
+.B \fI$XDG_CONFIG_HOME/diceware/diceware.ini\fP
+Additional location for your personal diceware configuration. Values set
+here will override any system\-wide valid values but can be overridden by
+\fI~/.diceware.ini\fP\&.
+.TP
+.B \fI$HOME/.config/diceware/diceware.ini\fP
+Alternative location for diceware configuration, only used if
+\fI${XDG_CONFIG_HOME}\fP is empty or unset.
+.TP
+.B \fI/etc/xdg/diceware/diceware.ini\fP
+If \fB$XDG_CONFIG_DIRS\fP is not set or empty, we look here for a system\-wide
+configuration file. Values set here take least precedence.
 .UNINDENT
+.SS WORDLIST FILES AND WORDLIST DIRECTORIES:
+.sp
+\fBdiceware\fP comes with a set of wordlists but enables you to add new wordlists
+by putting them into certain directories. The paths where the lists are stored
+(including the built\-in ones) is shown using \fB\-\-show\-wordlist\-dirs\fP\&.
+.sp
+Wordlist files are expected to contain lines with one term on each
+line and they must have a certain filenames to be found.
+.sp
+Wordlist filenames have to follow the pattern: \fBwordlist_<NAME>.txt\fP
+where \fB<NAME>\fP can be any name consisting of letters, numbers, underscores and
+hyphens. For instance \fBwordlist_en_eff.txt\fP is the filename of the EFF
+(electronic frontier foundation) word list. \fBen_eff\fP is the name of this list.
 .sp
-\fBdiceware\fP also comes with a set of wordlists. The path where these lists are
-stored is showed with \fB\-\-help\fP\&.
+We support \fB\&.txt\fP and \fB\&.asc\fP as filename extensions for wordlists, where
+\fB\&.txt\fP files are expected to be plain wordlists and \fB\&.asc\fP files should
+provide a PGP\-signature.
+.sp
+If wordlists with the same name are found in different directories then the one
+in the directory with the highest precedence is taken only. The following
+locations are ordered by precedence (highest first). Therefore built\-in
+wordlists cannot be overridden by custom wordlists. You can, however, use
+custom wordlists with a different name.
+.sp
+Directories we look up that do not exist (in part or completely) are silently
+skipped when searching for wordlist files.
+.INDENT 0.0
+.TP
+.B \fI<INSTALL\-DIR>/wordlists/\fP
+The directory containing the built\-in wordlists as part of the
+installation. These are the wordlists that are always available, regardless
+of configuration values and their exact location depends on the
+installation location of the \fBdiceware\fP package.
+.TP
+.B \fI$XDG_DATA_HOME/diceware/\fP
+If $XDG_DATA_HOME is set and not empty, we look in this directory for
+wordlists.
+.TP
+.B \fI$HOME/.local/share/diceware/\fP
+If $XDG_DATA_HOME is unset or empty, we look  into this directory for
+wordlists.
+.TP
+.B \fI<DIR>/diceware\fP from \fI$XDG_DATA_DIRS\fP
+If $XDG_DATA_DIR is set and not empty, it is interpreted as a
+colon\-separated list of directories with \fI/diceware\fP appended. So,
+\fI/foo/bar:/baz\fP will make us look into \fI/foo/bar/diceware/\fP and
+\fI/baz/diceware/\fP in that order.
+.TP
+.B \fI/usr/local/share/diceware/\fP, \fI/usr/share/diceware\fP
+If $XDG_DATA_DIRS is unset or empty, we look into these two directories for
+wordlists.
+.UNINDENT
 .SH EXAMPLES
 .INDENT 0.0
 .TP

diceware/__init__.py

@@ -24,7 +24,7 @@
 from diceware.config import get_config_dict
 from diceware.logger import configure
 from diceware.wordlist import (
-    WordList, get_wordlist_path, get_wordlists_dir, get_wordlist_names,
+    WordList, get_wordlist_path, get_wordlist_dirs, get_wordlist_names,
     )
 
 #: Special chars inserted on demand
@@ -60,6 +60,13 @@ def print_version():
     print(GPL_TEXT)
 
 
+def print_wordlist_dirs():
+    """Output all dirs we look up for wordlists.
+    """
+    for entry in get_wordlist_dirs():
+        print(entry)
+
+
 def get_random_sources():
     """Get a dictionary of all entry points called diceware_random_source.
 
@@ -92,7 +99,7 @@ def handle_options(args):
     defaults = get_config_dict()
     parser = argparse.ArgumentParser(
         description="Create a passphrase",
-        epilog="Wordlists are stored in %s" % get_wordlists_dir()
+        epilog="Use --show-wordlist-dirs to list directories where you can store custom wordlists."
         )
     parser.add_argument(
         '-n', '--num', default=6, type=int,
@@ -121,7 +128,7 @@ def handle_options(args):
         metavar="NAME", nargs='*',
         help=(
             "Use words from this wordlist. Possible values: `%s'. "
-            "Wordlists are stored in the folder displayed below. "
+            "Wordlists are stored in the folders displayed below. "
             "Default: en_eff" % "', `".join(wordlist_names)))
     realdice_group = parser.add_argument_group(
         "Arguments related to `realdice' randomsource",
@@ -139,7 +146,11 @@ def handle_options(args):
         help='Be verbose. Use several times for increased verbosity.')
     parser.add_argument(
         '--version', action='store_true',
-        help='output version information and exit.',
+        help='Output version information and exit.',
+        )
+    parser.add_argument(
+        '--show-wordlist-dirs', action='store_true',
+        help='Output directories we look up to find wordlists and exit.',
         )
     for plugin in plugins.values():
         if hasattr(plugin, "update_argparser"):
@@ -219,6 +230,9 @@ def main(args=None):
     if options.version:
         print_version()
         raise SystemExit(0)
+    elif options.show_wordlist_dirs:
+        print_wordlist_dirs()
+        raise SystemExit(0)
     try:
         print(get_passphrase(options))
     except (OSError, IOError) as infile_error:

docs/config.rst

@@ -47,16 +47,92 @@ Please note, that all options must be set within a section
 ``[diceware]``.
 
 
-Config File Name and Path
--------------------------
+Config Files Name and Path
+--------------------------
 
-Currently, we look for configuration files only in the calling users'
-home directory. The file must be called::
+Prior to version 1.0 we looked for a single configuration file in the calling
+users' home directory only. The file had to be called::
 
  .diceware.ini
 
-(please note the leading dot). If such a file is missing, build-in
-defaults apply.
+(please note the leading dot). If such a file were missing, buildt-in
+defaults applied.
+
+Since version 1.0 we look into several additional locations, but values set in
+``.diceware.ini`` still always override settings from other configuration files
+found.
+
+In order of precedence (with highest priority first) we look into the following
+paths::
+
+    ${HOME}/.diceware.ini
+
+Values set here override settings in any of the following files. If
+``${XDG_CONFIG_HOME}`` is defined and not empty, we then look into
+
+::
+
+    ${XDG_CONFIG_HOME}/diceware/diceware.ini
+
+
+or otherwise into
+
+::
+
+    ${HOME}/.config/diceware/diceware.ini
+
+Finally, if a colon-separated and not empty list of directories is set
+in
+
+::
+
+    ${XDG_CONFIG_DIRS}
+
+we look up any directory in this list, appended by
+
+::
+
+    /diceware/diceware.ini
+
+
+If none of the above files exist, default settings apply. Using this scheme we
+follow the `XDG Base Directory Specification
+<https://specifications.freedesktop.org/basedir-spec/latest/>`_.
+
+
+Examples
+........
+
+If you set the environment variable ``${XDG_CONFIG_DIRS}`` to ``/foo:/etc``,
+and then create a file ``diceware.ini`` in directory
+``/foo/diceware/`` with contents like this::
+
+    [diceware]
+    num = 2
+
+then `diceware` will create passphrases with two terms, except other
+configuration files or commandline options overrule this setting.
+
+Precedence of paths in ``${XDG_CONFIG_DIRS}`` is from least to highest
+priority. Therefore, if you create a file ``/etc/diceware/diceware.ini`` with
+content
+
+::
+
+    [diceware]
+    num = 4
+
+then this setting will override ``num = 2`` from the file above. Still any
+setting in ``${XDG_CONFIG_HOME}/diceware/diceware.ini`` or
+``${HOME}/.config/diceware/diceware.ini`` as explained above will take
+precedence while options set in ``${HOME}/.diceware.ini`` or on the commandline
+like
+
+::
+
+    diceware -n 6
+
+will still have highest priority.
 
 
 Option Values