Funded by the EU.
The Notification-manager component keeps track of the current state of each running workflow and sends notification messages to the WFE dashboard so that users are made aware of their workflows’ progress. Notification-manager receives messages from three distinct sources: Provision-service, workflow components, and Prometheus Alertmanager. Provision-service sends the schema of a newly created workflow to Notification-manager. The schema includes a list of WFE node IDs (each node/component in the WFE is assigned a unique identifier). Notification-manager then uses the schema to create an internal state machine for the workflow with the following states: RUNNING, FAILED, COMPLETED. This state machine is updated each time a message from a workflow component is received. These messages include a certain status: STARTED, INFO, WARNING, ERROR, CRITICAL, and SUCCESS. When Notification-manager encounters a CRITICAL status, the whole workflow is flagged as a failure and its state machine changes to the FAILED state. Every workflow component has to send messages of STARTED and SUCCESS status, in order for a workflow to change state from RUNNING to COMPLETED. When a workflow is first created, its state machine starts in the RUNNING state. Notification-manager makes use of the components’ WFE node IDs to distinguish between their messages. With each update to the state machine, Notification-manager sends a message to WFE which serves as the single source of truth for the state of any workflow running on the platform. Notification-manager treats alert-based messages just like any other status update and updates the corresponding workflow’s state accordingly—typically marking it as FAILED in the case of critical alerts related to infrastructure or processing stalls.