Merge pull request #12037 from poettering/oom-state

add cgroupv2 oom killer event handling to service management

Author: poettering

TODO

@@ -62,17 +62,13 @@ Features:
 
 * bootctl,sd-boot: actually honour the "architecture" key
 
-* set memory.oom.group in cgroup v2 for all leaf cgroups (kernel v4.19+)
-
 * add a new syscall group "@esoteric" for more esoteric stuff such as bpf() and
   usefaultd() and make systemd-analyze check for it.
 
 * paranoia: whenever we process passwords, call mlock() on the memory
   first. i.e. look for all places we use string_erase()/string_free_erase() and
   augment them with mlock(). Also use MADV_DONTDUMP
 
-* whenever oom_kill memory.event event is triggered print a nice log message
-
 * Move RestrictAddressFamily= to the new cgroup create socket
 
 * support the bind/connect/sendmsg cgroup stuff for sandboxing, and possibly

catalog/systemd.catalog.in

@@ -391,3 +391,16 @@ The following "tags" are possible:
 - "overflowgid-not-65534" — the kernel group ID used for "unknown" users (with
   NFS or user namespaces) is not 65534
 Current system is tagged as @TAINT@.
+
+-- fe6faa94e7774663a0da52717891d8ef
+Subject: A process of @UNIT@ unit has been killed by the OOM killer.
+Defined-By: systemd
+Support: %SUPPORT_URL%
+
+A process of unit @UNIT has been killed by the Linux kernel out-of-memory (OOM)
+killer logic. This usually indicates that the system is low on memory and that
+memory needed to be freed. A process associated with @UNIT@ has been determined
+as the best process to terminate and has been forcibly terminated by the
+kernel.
+
+Note that the memory pressure might or might not have been caused by @UNIT@.

man/systemd-system.conf.xml

@@ -364,6 +364,17 @@
         limits are only defaults for units, they are not applied to PID 1
         itself.</para></listitem>
       </varlistentry>
+
+      <varlistentry>
+        <term><varname>DefaultOOMPolicy=</varname></term>
+
+        <listitem><para>Configure the default policy for reacting to processes being killed by the Linux
+        Out-Of-Memory (OOM) killer. This may be used to pick a global default for the per-unit
+        <varname>OOMPolicy=</varname> setting. See
+        <citerefentry><refentrytitle>systemd.service</refentrytitle><manvolnum>5</manvolnum></citerefentry>
+        for details. Note that this default is not used for services that have <varname>Delegate=</varname>
+        turned on.</para></listitem>
+      </varlistentry>
     </variablelist>
   </refsect1>
 

man/systemd.exec.xml

@@ -651,11 +651,17 @@ CapabilityBoundingSet=~CAP_B CAP_C</programlisting>
       <varlistentry>
         <term><varname>OOMScoreAdjust=</varname></term>
 
-        <listitem><para>Sets the adjustment level for the Out-Of-Memory killer for executed processes. Takes an integer
-        between -1000 (to disable OOM killing for this process) and 1000 (to make killing of this process under memory
-        pressure very likely). See <ulink
-        url="https://www.kernel.org/doc/Documentation/filesystems/proc.txt">proc.txt</ulink> for
-        details.</para></listitem>
+        <listitem><para>Sets the adjustment value for the Linux kernel's Out-Of-Memory (OOM) killer score for
+        executed processes. Takes an integer between -1000 (to disable OOM killing of processes of this unit)
+        and 1000 (to make killing of processes of this unit under memory pressure very likely). See <ulink
+        url="https://www.kernel.org/doc/Documentation/filesystems/proc.txt">proc.txt</ulink> for details. If
+        not specified defaults to the OOM score adjustment level of the service manager itself, which is
+        normally at 0.</para>
+
+        <para>Use the <varname>OOMPolicy=</varname> setting of service units to configure how the service
+        manager shall react to the kernel OOM killer terminating a process of the service.  See
+        <citerefentry><refentrytitle>systemd.service</refentrytitle><manvolnum>5</manvolnum></citerefentry>
+        for details.</para></listitem>
       </varlistentry>
 
       <varlistentry>

man/systemd.service.xml

@@ -963,6 +963,29 @@
         above.</para></listitem>
       </varlistentry>
 
+      <varlistentry>
+        <term><varname>OOMPolicy=</varname></term>
+
+        <listitem><para>Configure the Out-Of-Memory (OOM) killer policy. On Linux, when memory becomes scarce
+        the kernel might decide to kill a running process in order to free up memory and reduce memory
+        pressure. This setting takes one of <constant>continue</constant>, <constant>stop</constant> or
+        <constant>kill</constant>. If set to <constant>continue</constant> and a process of the service is
+        killed by the kernel's OOM killer this is logged but the service continues running. If set to
+        <constant>stop</constant> the event is logged but the service is terminated cleanly by the service
+        manager. If set to <constant>kill</constant> and one of the service's processes is killed by the OOM
+        killer the kernel is instructed to kill all remaining processes of the service, too. Defaults to the
+        setting <varname>DefaultOOMPolicy=</varname> in
+        <citerefentry><refentrytitle>system.conf</refentrytitle><manvolnum>5</manvolnum></citerefentry> is
+        set to, except for services where <varname>Delegate=</varname> is turned on, where it defaults to
+        <constant>continue</constant>.</para>
+
+        <para>Use the <varname>OOMScoreAdjust=</varname> setting to configure whether processes of the unit
+        shall be considered preferred or less preferred candidates for process termination by the Linux OOM
+        killer logic. See
+        <citerefentry><refentrytitle>systemd.exec</refentrytitle><manvolnum>5</manvolnum></citerefentry> for
+        details.</para></listitem>
+      </varlistentry>
+
     </variablelist>
 
     <para>Check