‼️ Improve needs default field application (via needs_global_options) (#1478)

Previously defaults would be applied to any fields of a need with a
"falsy" value, e.g. `None`, `False`, `0` , `""`, `[]`. ...

This is an issue, if the user wants to specifically set fields to these
values, without them being overridden by defaults. Therefore, now
defaults are only applied to fields with a missing or `None` value.

In the need directive context, this means that defaults will only be set
for fields not specifically set in the directive options.

In the external/import need context, defaults will only be applied to
missing keys in the `needs.json`, or fields with the value set to
`null`.

For predicate defaults, the context passed to the expression has been
restricted to a subset of fields, and these values are no longer mutated
by previous defaults, i.e. each predicate gets the same immutable
context data.
This makes these predicates more isolated / understandable, and allows
for them potentially later being co-located with the type information.

----

‼️ Breaking

This PR may be breaking:

1. For users expecting defaults to be applied to external/import
`needs.json`, although I would suggest this was never the intention for
these defaults and that `needs.json` should, at least by default, be
seen as static ingest data which has already had defaults etc applied.

2. For users with "exotic" predicate expressions, that potentially rely
on the application of previous defaults.

Author: chrisjsewell

docs/configuration.rst

@@ -349,7 +349,22 @@ Defaults will be used if the field is not set specifically by the user and thus
    }
 
 To set a default based on a one or more predicates, use the ``predicates`` key.
-These predicates are a list of (:ref:`filter string <filter_string>`, value), evaluated in order, with the first match set as the default value.
+These predicates are a list of ``(match expression, value)``, evaluated in order, with the first match set as the default value.
+
+A match expression is a string, using Python syntax, that will be evaluated against data from each need (before the resolution of defaults or dynamic functions etc):
+
+- `id` (`str`)
+- `type` (`str`)
+- `title` (`str`)
+- `tags` (`tuple[str, ...]`)
+- `status` (`str | None`)
+- `docname` (`str | None`)
+- `is_external` (`bool`)
+- `is_import` (`bool`)
+- :ref:`needs_extra_options` (`str`)
+- :ref:`needs_extra_links` (`tuple[str, ...]`)
+- :ref:`needs_filter_data`
+
 If no predicates match, the ``default`` value is used (if present).
 
 .. code-block:: python
@@ -380,17 +395,6 @@ If no predicates match, the ``default`` value is used (if present).
               "option1": {"default": '[[copy("id")]]'}
       }
 
-.. warning::
-
-   The filter string gets executed against the current need only and has no access to other needs.
-   So avoid any references to other needs in the filter string.
-
-   If you need access to other needs for complex filtering, you can maybe provide your own :ref:`dynamic_functions`
-   and perform the filtering there.
-
-   Default replacements are done, for each field, in the order they are defined in the configuration,
-   so a filter string should not depend on the value of a field below it in the configuration.
-
 .. _`needs_report_dead_links`:
 
 needs_report_dead_links