Multiple dsearch improvements

doc/doc-docbook/spec.xfpt

@@ -6830,20 +6830,28 @@ each element starting with a tag name and an equals.
 
 Two options are supported, for the return value and for filtering match
 candidates.
-The "ret" option requests an alternate result value of
-the entire path for the entry. Example:
+The "ret=" options request alternative result values: "ret=key" is the default,
+and just returns the key (filename), "ret=full" requests the entire path for
+the entry, and "ret=dir" requests the directory alone.
+Example:
 .code
+${lookup {passwd} dsearch,ret=key {/etc}}
 ${lookup {passwd} dsearch,ret=full {/etc}}
+${lookup {passwd} dsearch,ret=dir {/etc}}
 .endd
-The default result is just the requested entry.
-The "filter" option requests that only directory entries of a given type
-are matched. The match value is one of "file", "dir" or "subdir" (the latter
-not matching "." or ".."). Example:
-.code
-${lookup {passwd} dsearch,filter=file {/etc}}
-.endd
-The default matching is for any entry type, including directories
-and symlinks.
+gives "passwd", "/etc/passwd" and "/etc".
+
+By default all directory entries are matched, regardless of type. The "filter"
+option requests that only directory entries of the given type(s) are matched:
+"file", "dir", "symlink", "pipe", "socket", "tty", "bdev". The filter option
+may be repeated to allow multiple types.
+
+In addition, "filter=nodots" excludes "." and "..", and "filter=subdir" is
+shorthand for "filter=dir,filter=nodots".
+
+The "follow" option follows any symlinks before applies other checks, and
+dangling symlinks will report as nonexistent; otherwise symlinks are rejected
+if any other file-type filter is applied.
 
 An example of how this
 lookup can be used to support virtual domains is given in section

doc/doc-txt/ChangeLog

@@ -125,6 +125,18 @@ JH/26 For a ${readsocket } in TLS mode, send a TLS Close Alert before the TCP
 JH/27 Fix ${srs_encode ..}.  Previously it would give a bad result for one day
       every 1024 days.
 
+MK/08 Numerous changes to dsearch lookups: (a) now performs taint check before
+      other validations to prevent an attacker from making inferences from the
+      varying error messages; (b) no longer require "read" permission on the
+      directory (so you can now chmod a-r the dir to improve system security);
+      (c) uses fstatat on systems that support it (reducing the syscall count);
+      (d) additional filter options to support matching all inode types; (e)
+      new ret=dir option. See documentation for full list.
+MK/09 Bug 2916: dsearch lookups no longer requires "read" permission on the
+      directory.
+
+MK/10 New dsearch lookup features: (a) ret=dir option; (b) filter options for
+      all inode types; (c) can use fstatat(2) where supported.
 
 Exim version 4.96
 -----------------

doc/doc-txt/NewStuff

@@ -9,7 +9,7 @@ the documentation is updated, this file is reduced to a short list.
 Version 4.97
 ------------
 
- 1. The expansion-test faciility (exim -be) can set variables.
+ 1. The expansion-test facility (exim -be) can set variables.
 
  2. An event on a failing SMTP AUTH, for both client and server operations.
 
@@ -32,6 +32,25 @@ Version 4.97
  11. An option for the ${readsocket } expansion to set an SNI for TLS.
 
  12. The ACL remove_header modifier can take a pattern.
+ 3. Dsearch lookup will now check for taint violations before doing anything
+    else (this prevents an attacker from inferring the locations of files from
+    differences in error messages).
+
+ 4. Dsearch lookup no longer requires read access to directories; consider
+    revoking read access using chmod go-r /path/to/containing/dir/.
+
+ 5. Dsearch lookup will use fstatat() on systems that support it.
+
+ 6. Dsearch lookup has new option "ret=dir" to return only the name of the
+    containing directory without the target (this simplifies some kinds of
+    nested loopup expansions).
+
+ 7. Dsearch lookup now has filter= parameters to match all common inode types
+    (file, dir, symlink, pipe, socket, tty, & bdev).
+
+ 8. Dsearch lookup has new option "filter=nodots" to exclude "." and ".."
+    even when directory matching is not required; equivalent to "filter=subdir"
+    but without requiring that the target actually be a directory.
 
 Version 4.96
 ------------

src/README.UPDATING

@@ -26,6 +26,15 @@ The rest of this document contains information about changes in 4.xx releases
 that might affect a running system.
 
 
+Exim version 4.97
+-----------------
+
+The dsearch lookup type no longer requires read permission on its target
+directories; scan permission alone now suffices. If you were relying on
+unreadable directories to block dsearch lookups, you should now make other
+arrangements. Conversely, you may now tighten up the permissions on any
+directories that used to require world or group read access.
+
 Exim version 4.95
 -----------------
 

src/src/exim.h

@@ -668,5 +668,12 @@ not get the TIME_WAIT */
 # define SERVERSIDE_CLOSE_NOWAIT
 #endif
 
+/* Stand-in for an unused function parameter */
+#ifdef __GNUC__
+#define UNUSED(NAME) NAME __attribute__((unused))
+#else
+#define UNUSED(NAME) NAME
+#endif
+
 #endif
 /* End of exim.h */

src/src/log.c

@@ -106,6 +106,7 @@ static const uschar * exim_errstrings[] = {
   [- ERRNO_UTF8_FWD] =		US"target not supporting SMTPUTF8",
   [- ERRNO_HOST_IS_LOCAL] =	US"host is local",
   [- ERRNO_TAINT] =		US"tainted filename",
+  [- ERRNO_MISMATCH] =		US"not requested type",
 
   [- ERRNO_RRETRY] =		US"Not time for routing",
 

src/src/lookups/Makefile

@@ -30,47 +30,47 @@ lf_check_file.o: $(HDRS) lf_check_file.c  lf_functions.h
 lf_quote.o:      $(HDRS) lf_quote.c       lf_functions.h
 lf_sqlperform.o: $(HDRS) lf_sqlperform.c  lf_functions.h
 
-cdb.o:           $(HDRS) cdb.c
-dbmdb.o:         $(HDRS) dbmdb.c
-dnsdb.o:         $(HDRS) dnsdb.c
-dsearch.o:       $(HDRS) dsearch.c
-ibase.o:         $(HDRS) ibase.c
-ldap.o:          $(HDRS) ldap.c
+cdb.o:           $(HDRS) cdb.c            lf_functions.h
+dbmdb.o:         $(HDRS) dbmdb.c          lf_functions.h
+dnsdb.o:         $(HDRS) dnsdb.c          lf_functions.h
+dsearch.o:       $(HDRS) dsearch.c        lf_functions.h
+ibase.o:         $(HDRS) ibase.c          lf_functions.h
+ldap.o:          $(HDRS) ldap.c           lf_functions.h
 lmdb.o:          $(HDRS) lmdb.c
-json.o:          $(HDRS) json.c
-lsearch.o:       $(HDRS) lsearch.c
-mysql.o:         $(HDRS) mysql.c
-nis.o:           $(HDRS) nis.c
-nisplus.o:       $(HDRS) nisplus.c
+json.o:          $(HDRS) json.c           lf_functions.h
+lsearch.o:       $(HDRS) lsearch.c        lf_functions.h
+mysql.o:         $(HDRS) mysql.c          lf_functions.h
+nis.o:           $(HDRS) nis.c            lf_functions.h
+nisplus.o:       $(HDRS) nisplus.c        lf_functions.h
 oracle.o:        $(HDRS) oracle.c
 passwd.o:        $(HDRS) passwd.c
-pgsql.o:         $(HDRS) pgsql.c
-readsock.o:      $(HDRS) readsock.c
-redis.o:         $(HDRS) redis.c
-spf.o:           $(HDRS) spf.c
-sqlite.o:        $(HDRS) sqlite.c
-testdb.o:        $(HDRS) testdb.c
+pgsql.o:         $(HDRS) pgsql.c          lf_functions.h
+readsock.o:      $(HDRS) readsock.c       lf_functions.h
+redis.o:         $(HDRS) redis.c          lf_functions.h
+spf.o:           $(HDRS) spf.c            lf_functions.h
+sqlite.o:        $(HDRS) sqlite.c         lf_functions.h
+testdb.o:        $(HDRS) testdb.c         lf_functions.h
 whoson.o:        $(HDRS) whoson.c
 
-cdb.so:           $(HDRS) cdb.c
-dbmdb.so:         $(HDRS) dbmdb.c
-dnsdb.so:         $(HDRS) dnsdb.c
-dsearch.so:       $(HDRS) dsearch.c
-ibase.so:         $(HDRS) ibase.c
-json.so:          $(HDRS) json.c
+cdb.so:           $(HDRS) cdb.c           lf_functions.h
+dbmdb.so:         $(HDRS) dbmdb.c         lf_functions.h
+dnsdb.so:         $(HDRS) dnsdb.c         lf_functions.h
+dsearch.so:       $(HDRS) dsearch.c       lf_functions.h
+ibase.so:         $(HDRS) ibase.c         lf_functions.h
+json.so:          $(HDRS) json.c          lf_functions.h
 ldap.so:          $(HDRS) ldap.c
-lmdb.so:          $(HDRS) lmdb.c
-lsearch.so:       $(HDRS) lsearch.c
-mysql.so:         $(HDRS) mysql.c
-nis.so:           $(HDRS) nis.c
-nisplus.so:       $(HDRS) nisplus.c
+lmdb.so:          $(HDRS) lmdb.c          lf_functions.h
+lsearch.so:       $(HDRS) lsearch.c       lf_functions.h
+mysql.so:         $(HDRS) mysql.c         lf_functions.h
+nis.so:           $(HDRS) nis.c           lf_functions.h
+nisplus.so:       $(HDRS) nisplus.c       lf_functions.h
 oracle.so:        $(HDRS) oracle.c
 passwd.so:        $(HDRS) passwd.c
-pgsql.so:         $(HDRS) pgsql.c
-redis.so:         $(HDRS) redis.c
-spf.so:           $(HDRS) spf.c
-sqlite.so:        $(HDRS) sqlite.c
-testdb.so:        $(HDRS) testdb.c
+pgsql.so:         $(HDRS) pgsql.c         lf_functions.h
+redis.so:         $(HDRS) redis.c         lf_functions.h
+spf.so:           $(HDRS) spf.c           lf_functions.h
+sqlite.so:        $(HDRS) sqlite.c        lf_functions.h
+testdb.so:        $(HDRS) testdb.c        lf_functions.h
 whoson.so:        $(HDRS) whoson.c
 
 # End