Skip to content

Commit

Permalink
add a timestamp to the events of the event interface
Browse files Browse the repository at this point in the history
  • Loading branch information
@julien6387
julien6387 committed Jan 5, 2025
1 parent 2dcf5a0 commit 8c9ab90
Show file tree
Hide file tree
Showing 18 changed files with 223 additions and 164 deletions.
8 changes: 7 additions & 1 deletion CHANGES.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
when navigating to another node.

* Add a session cookie to **Supvisors** Web UI client.
All statistics images served by the **Supvisors** Web UI are xxx enough to enable multiple auto-refreshed views
All statistics images served by the **Supvisors** Web UI are renamed to allow multiple auto-refreshed views
in the browser tabs.

* Add the XML-RPCs `get_instance_state_modes` and `get_all_instances_state_modes` in support of getting detailed state
Expand All @@ -33,6 +33,12 @@

* Add the process PID to the process statistics event.

* Add a timestamp to the events of the **Supvisors** event interface.
This timestamp is also added to the XML-RPCs `get_supvisors_state`, `get_instance_state_modes`,
`get_all_instances_state_modes`, `get_application_info`, `get_all_applications_info`, `get_process_info`,
`get_all_process_info`.
The `last_event_time` in the Process status event is renamed as `last_event_mtime`.

* Cancel individual subscriptions from the Supervisor events rather than perform a global clear.

* Consider variable CPU frequency value returned by `psutil`.
Expand Down
20 changes: 16 additions & 4 deletions docs/event_interface.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ Key Type Value
'identifier' ``str`` The identifier of the |Supvisors| instance.
'nick_identifier' ``str`` The |Supvisors| instance nick name, as set in the ``supvisors_list`` option,
or a copy of the |Supvisors| identifier if not set.
'now_monotonic' ``float`` The monotonic time of the event, in the reference time of the local host.
'fsm_statecode' ``int`` The state of |Supvisors|, in [0;7].
'fsm_statename' ``str`` The string state of |Supvisors|, among { ``'SYNCHRONIZATION'``, ``'ELECTION'``,
``'DISTRIBUTION'``, ``'OPERATION'``, ``'CONCILIATION'``, ``'RESTARTING'``,
Expand Down Expand Up @@ -102,6 +103,8 @@ Application status
Key Type Value
================== ================= ==================
'application_name' ``str`` The Application name.
'managed' ``bool`` True if the Application is managed in |Supvisors|.
'now_monotonic' ``float`` The monotonic time of the event, in the reference time of the local host.
'statecode' ``int`` The Application state, in [0;3].
'statename' ``str`` The Application state as string, among { ``'STOPPED'``, ``'STARTING'``,
``'RUNNING'``, ``'STOPPING'`` }.
Expand All @@ -119,6 +122,9 @@ Key Type Value
================== ================= ==================
'application_name' ``str`` The Application name.
'process_name' ``str`` The Process name.
'now_monotonic' ``float`` The monotonic time of the event, in the reference time of the local host.
'last_event_mtime' ``float`` The local monotonic time of the last process event received for this process,
regardless of the |Supvisors| instance that sent the event.
'statecode' ``int`` The Process state, in {0, 10, 20, 30, 40, 100, 200, 1000}.
A special value -1 means that the process has been deleted as a consequence
of an XML-RPC ``update_numprocs``.
Expand All @@ -127,8 +133,6 @@ Key Type Value
A special value ``DELETED`` means that the process has been deleted as a consequence
of an XML-RPC ``update_numprocs``.
'expected_exit' ``bool`` True if the exit status is expected (only when state is ``'EXITED'``).
'last_event_time' ``float`` The date of the last process event received for this process, regardless
of the originating |Supvisors| instance.
'identifiers' ``list(str)`` The identifiers of the |Supvisors| instances where the process is running.
'extra_args' ``str`` The additional arguments passed to the command line of the process.
================== ================= ==================
Expand All @@ -140,21 +144,27 @@ Key Type Value
* `#1150 - Why do event listeners not report the process exit status when stopped/crashed?
<https://github.com/Supervisor/supervisor/issues/1150>`_


Process event
~~~~~~~~~~~~~

================== ================= ==================
Key Type Value
================== ================= ==================
'identifier' ``str`` The identifier of the |Supvisors| instance that sent the initial event.
'nick_identifier' ``str`` The |Supvisors| instance nick name, as set in the ``supvisors_list`` option,
or a copy of the |Supvisors| identifier if not set.
'group' ``str`` The Application name.
'name' ``str`` The Process name.
'state' ``int`` The Process state, in {0, 10, 20, 30, 40, 100, 200, 1000}.
A special value -1 means that the process has been deleted as a consequence
of an XML-RPC ``update_numprocs``.
'expected' ``bool`` True if the exit status is expected (only when state is 100 - ``EXITED``).
'now' ``float`` The monotonic time of the event in the reference time of the host.
'spawnerr' ``str`` The description of error that occurred during spawn, or empty string if none.
'now' ``float`` The POSIX time of the event, in the reference time of the remote host.
'now_monotonic' ``float`` The monotonic time of the event, in the reference time of the remote host.
'event_mtime' ``float`` The monotonic time of the event, in the reference time of the local host.
'pid' ``int`` The UNIX process ID (only when state is 20 - ``RUNNING`` or 40 - ``STOPPING``).
'identifier' ``str`` The identifier of the |Supvisors| instance that sent the initial event.
'extra_args' ``str`` The additional arguments passed to the command line of the process.
'disabled' ``bool`` True if the process is disabled on the |Supvisors| instance.
================== ================= ==================
Expand All @@ -167,6 +177,7 @@ Host statistics
Key Type Value
================== ========================= ==================
'identifier' ``str`` The identifier of the |Supvisors| instance.
'now_monotonic' ``float`` The monotonic time of the event, in the reference time of the local host.
'target_period' ``float`` The configured integration period, in seconds.
'period' ``list(float)`` The start and end uptimes of the integration period, as a list of 2 values
in seconds.
Expand All @@ -189,6 +200,7 @@ Key Type Value
'namespec' ``str`` The Process namespec.
'identifier' ``str`` The identifier of the |Supvisors| instance where the process is running.
'pid' ``int`` The Process PID.
'now_monotonic' ``float`` The monotonic time of the event, in the reference time of the local host.
'target_period' ``float`` The configured integration period, in seconds.
'period' ``list(float)`` The start and end uptimes of the integration period, as a list of 2 values in seconds.
'cpu' ``float`` The CPU (IRIX mode) of the process on the node, in percent.
Expand Down
8 changes: 6 additions & 2 deletions docs/xml_rpc.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ Status
'identifier' ``str`` The |Supvisors| local instance identifier (``host_id:http_port``).
'nick_identifier' ``str`` The |Supvisors| local instance nick name, or a copy of the |Supvisors|
identifier if not set.
'now_monotonic' ``float`` The monotonic time, in the reference time of the local host.
'fsm_statecode' ``int`` The |Supvisors| state, in [0;8].
'fsm_statename' ``str`` The |Supvisors| state as string, in [``'OFF'``, ``'SYNCHRONIZATION'``,
``ELECTION``, ``'DISTRIBUTION'``, ``'OPERATION'``, ``'CONCILIATION'``,
Expand All @@ -64,6 +65,7 @@ Status
'identifier' ``str`` The |Supvisors| instance identifier (``host_id:http_port``).
'nick_identifier' ``str`` The |Supvisors| instance nick name, or a copy of the |Supvisors|
identifier if not set.
'now_monotonic' ``float`` The monotonic time, in the reference time of the local host.
'fsm_statecode' ``int`` The |Supvisors| state, in [0;8].
'fsm_statename' ``str`` The |Supvisors| state as string, in [``'OFF'``, ``'SYNCHRONIZATION'``,
``ELECTION``, ``'DISTRIBUTION'``, ``'OPERATION'``, ``'CONCILIATION'``,
Expand Down Expand Up @@ -156,6 +158,7 @@ Status
Key Type Description
================== ========= ===========
'application_name' ``str`` The Application name.
'now_monotonic' ``float`` The monotonic time, in the reference time of the local host.
'statecode' ``int`` The Application state, in [0;4].
'statename' ``str`` The Application state as string, in [``'UNKNOWN'``, ``'STOPPED'``,
``'STARTING'``, ``'STOPPING'``, ``'RUNNING'``].
Expand All @@ -172,13 +175,14 @@ Status
================== =============== ===========
'application_name' ``str`` The Application name the process belongs to.
'process_name' ``str`` The Process name.
'now_monotonic' ``float`` The monotonic time, in the reference time of the local host.
'last_event_mtime' ``float`` The monotonic time of the last event received for this process,
in the reference time of the local host.
'statecode' ``int`` The Process state, in {0, 10, 20, 30, 40, 100, 200, 1000}.
'statename' ``str`` The Process state as string, in [``'STOPPED'``, ``'STARTING'``,
``'RUNNING'``, ``'BACKOFF'``, ``'STOPPING'``, ``'EXITED'``, ``'FATAL'``,
``'UNKNOWN'``].
'expected_exit' ``bool`` A status telling if the process has exited expectedly.
'last_event_time' ``float`` The local monotonic time of the last event received for this process,
in seconds.
'identifiers' ``list(str)`` The identifiers of all |Supvisors| instances where the process is
running.
'extra_args' ``str`` The extra arguments used in the command line of the process.
Expand Down
6 changes: 4 additions & 2 deletions supvisors/application.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@

import ast
import re
import time
from typing import Any, Dict, List, Optional, Sequence, Union

from supervisor.loggers import Logger
Expand Down Expand Up @@ -506,9 +507,10 @@ def get_operational_status(self) -> str:
def serial(self) -> Payload:
""" Get a serializable form of the application status.
:return: the application status in a dictionary
:return: the application status in a dictionary.
"""
return {'application_name': self.application_name, 'managed': self.rules.managed,
return {'application_name': self.application_name,
'managed': self.rules.managed, 'now_monotonic': time.monotonic(),
'statecode': self.state.value, 'statename': self.state.name,
'major_failure': self.major_failure, 'minor_failure': self.minor_failure}

Expand Down
4 changes: 2 additions & 2 deletions supvisors/commander.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -737,7 +737,7 @@ def distribute_to_single_node(self) -> None:
:return: None
"""
# FIXME: what if any process of this application is already running (started manually) ?
# TBC: what if any process of this application is already running (started manually) ?
# get all ProcessStartCommand of the application
commands = [process for sequence in self.planned_jobs.values() for process in sequence]
# find the applicable Supvisors instances iaw strategy
Expand Down Expand Up @@ -773,7 +773,7 @@ def distribute_to_single_instance(self) -> None:
:return: None
"""
# FIXME: what if any process of this application is already running (started manually) ?
# TBC: what if any process of this application is already running (started manually) ?
# get all ProcessStartCommand of the application
commands = [process for sequence in self.planned_jobs.values() for process in sequence]
# find the applicable Supvisors instances iaw strategy
Expand Down
7 changes: 6 additions & 1 deletion supvisors/context.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -590,7 +590,10 @@ def on_process_removed_event(self, status: SupvisorsInstanceStatus, event: Paylo
application_impacted = False
# get process targets
application, event_process = app_proc
processes = [event_process] if event_process else application.get_instance_processes(status.identifier)
if event_process:
processes = [event_process]
else:
processes = application.get_instance_processes(status.identifier)
for process in processes:
# WARN: process_failures are not triggered here as the processes have been properly stopped
# as a consequence of the user action
Expand Down Expand Up @@ -694,6 +697,8 @@ def on_process_state_event(self, status: SupvisorsInstanceStatus, event: Payload
application.update()
# publish process event, status and application status
if self.external_publisher:
# timestamp the event in the local reference time
event['event_mtime'] = time.monotonic()
self.external_publisher.send_process_event(event)
self.external_publisher.send_process_status(process.serial())
self.external_publisher.send_application_status(application.serial())
Expand Down
4 changes: 1 addition & 3 deletions supvisors/initializer.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,5 @@ def __init__(self, supervisor: Supervisor, **config) -> None:
self.listener = SupervisorListener(self)
# create state machine
self.fsm = FiniteStateMachine(self)
# TODO: HTTP sessions
# TODO: no expiry date (1 session cookie per browser, independent from number of pages), but close all on stopping ?
# TODO: name the process png ?
# HTTP sessions manager
self.sessions = SessionViews(self)
10 changes: 5 additions & 5 deletions supvisors/listener.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -282,8 +282,8 @@ def on_process_state(self, event: events.ProcessStateEvent) -> None:
def _get_local_process_info(self, namespec: str) -> Payload:
""" Use the Supvisors RPCInterface to get local information on this process.
:param namespec: the process namespec
:return: the process local information
:param namespec: the process namespec.
:return: the process local information.
"""
try:
rpc_intf = self.supvisors.supervisor_data.supvisors_rpc_interface
Expand All @@ -295,8 +295,8 @@ def _get_local_process_info(self, namespec: str) -> Payload:
def on_process_added(self, event: ProcessAddedEvent) -> None:
""" Called when a process has been added due to a numprocs change.
:param event: the ProcessAddedEvent object
:return: None
:param event: the ProcessAddedEvent object.
:return: None.
"""
try:
namespec = make_namespec(event.process.group.config.name, event.process.config.name)
Expand Down Expand Up @@ -540,7 +540,7 @@ def force_process_state(self, process: ProcessStatus, identifier: str,
nick_identifier = ''
if identifier:
nick_identifier = self.supvisors.mapper.instances[identifier].nick_identifier
# create payload from event
# create payload from event (the 'forced' entry is the only one added vs a normal payload)
payload = {'identifier': identifier, 'nick_identifier': nick_identifier,
'group': process.application_name, 'name': process.process_name,
'state': forced_state, 'forced': True,
Expand Down
Loading

0 comments on commit 8c9ab90

Please sign in to comment.