Skip to content

Notifications

Introduction#

The Piveau Metrics Notifications is part of the Metadata Quality Assurance (MQA) of Piveau. It notifies a publisher per email when their catalogues score is decreasing.

How Change the mail sending interval#

The time and interval for sending notification emails can be configured dynamically using the Data Provider setting feature. When the Data Provider provides frequency information, the system will calculate the Next Check Date accordingly.

The frequency determines how often checks are run and notifications are sent to recipients and consists of two key fields:

unit: Specifies the interval type. Accepted values are: day, week, month

value: Specifies the number of units (days, weeks, or months) from the current date when the notification should be sent.

How It Works

Once the Data Provider specifies their intended frequency settings, the system automatically calculates the Next Check Date based on the provided frequency relative to the current date. This ensures that notifications are sent at the exact intervals the Data Provider desires.

For example, if the frequency is set to unit: "day" and value: 3, and the current date is February 1, 2025, the system will calculate the Next Check Date as February 4, 2025.

Alternatively a check can be triggered manually via the API at /notification/check/now.

What triggers a mail?#

Here’s the updated version with the equation added:

The Data Provider can now set the threshold dynamically using the Data Provider setting feature.

On the Next Check Date, the system will compare the current score with the previously saved score. If the current score is significantly lower than the previous score, as determined by the formula:

current score < previous score - threshold

The system will:

  • Update and save the new score.
  • Send a notification to the recipients.

This ensures notifications are sent only when there is a notable drop in the score.

How to subscribe a new user#

The Notification will be sent to the publisher of the catalogue. The publisher is specified in the Linked Data Database. However, not every catalogue is checked initially. If the catalogue is already enabled, another notification recipient can be added, by adding them to the publisher list in the metadata.

If the user is already a publisher for a catalogue, but the catalogue is not yet active, the catalogue has to be added to the list of active catalogues in an environment variable called NOTIFICATIONS_ACTIVE_CATALOGUES.

How to unsubscribe a user#

If a user is a publisher of an enabled catalogue, but does not want to receive notifications, they can be set on a blacklist. To do that, an API PUT request can be created on this endpoint: /user/{mail}/blacklist/{catalogue} where {mail} has to be replaced with the users mail address and {catalogue} has to be replaced with the name of the catalogue. This API endpoint is protected with an API key.

To remove the user from this blacklist, the same endpoint has to be called with an DELETE request.

How to use Data Provider Setting#

  • If a Data Provider does not have a setting beforehand, calling the GET API will create a new instance. If they want to visualize their current settings, they can use the same GET API to do so.

To make this request, an API GET call can be made to /catalogue/{catalogueName}/setting, where {catalogueName} should be replaced with the actual catalogue name.

  • If a Data Provider wants to add new data to their settings, such as the recipient list, threshold, or frequency, they must first activate the catalogue. Without activation, all fields in the settings will remain disabled.

To activate the catalogue and enable the settings page, an API POST request can be made to /catalogue/{catalogueName}/setting, where {catalogueName} should be replaced with the actual catalogue name, along with the request body: {"activeStatus": true"}

To deactivate the catalogue and disable the report settings page, a similar POST request can be made to /catalogue/{catalogueName}/setting, replacing {catalogueName} with the actual catalogue name, and using the request body:{"activeStatus": false"}

  • After activating the catalogue and enabling Setting page, data provider can add other fields, such as receiver emails, threshold, and frequency.

To do this, a POST request can be made to /catalogue/{catalogueName}/setting, replacing {catalogueName} with the actual catalogue name, and using the request body:

{
  "receiverEmailList": [],
  "frequency": {
    "unit": "",
    "value": 0L
  },

  "threshold": 0L,
  "activeStatus": true
}
Replace the receiverEmailList, frequency, threshold and any other fields with the appropriate values when making the request.

  • If the setting page for the catalogue is no longer required, data provider can delete it by making a DELETE request.

To do so, a DELETE request can be made to /catalogue/{catalogueName}/setting, replacing {catalogueName} with the actual catalogue name.

How Data Provider Setting Feature Works#

Data Provider Setting feature allows Data Providers to configure notification settings, such as frequency, thresholds, and recipient lists. These settings determine how often notifications are sent and under what conditions. Below is a detailed explanation of how the system works, including the role of the TimerManagementVerticle and its interaction with other components.

On Startup#

When the system starts, the following steps are executed:

  • Retrieve Catalogue Settings: The system fetches the settings for all active catalogues from the database.

These settings include:

  • Frequency: How often notifications should be sent (e.g., after 3 days).
  • Threshold: The minimum score drop required to trigger a notification.

  • Recipient List: The list of email addresses to receive notifications.

  • Initialize Timers: For each active catalogue, the system calculates the Next Check Date based on the configured frequency. A timer is created for each catalogue using the TimerManagementVerticle. This timer ensures that checks are performed at the specified intervals.

  • Default Values: If no settings are found for a catalogue, default values (e.g., frequency of 60 minutes) are used.

When the Timer Fires for a Catalogue#

When the timer for a specific catalogue triggers, the following workflow is executed:

  • Retrieve Metadata: The system retrieves the old metadata (previous score and metrics) from metrics cache.It also fetches the current metadata from the database.

  • Compare Scores: The system compares the current score with the previous score using the formula:

current score < previous score - threshold

If the current score is significantly lower than the previous score (based on the configured threshold), a notification is triggered.

  • Send Notification: If the score drop meets the threshold condition:

  • A notification email is sent to the recipients listed in the settings.

  • The new metrics data is saved to the database to update the historical record.

  • Calculate Next Check Date: The system calculates the Next Check Date based on the configured frequency.This date is stored in the database to ensure the next check occurs at the correct interval.

Responsibilities of TimerManagementVerticle#

The TimerManagementVerticle is responsible for managing timers for all active catalogues.

It performs the following tasks:

  • Start Timers:

    When the system starts, the TimerManagementVerticle retrieves the list of active catalogues and creates timers for each one. The timers are configured based on the frequency specified in the catalogue settings.

  • Handle Timer Events:

    When a timer fires, the TimerManagementVerticle triggers a check for the corresponding catalogue. It sends a message to the NotificationVerticle to perform the score comparison and send notifications if necessary.

  • Reset or Stop Timers:

    If the settings for a catalogue are updated (e.g., frequency or threshold changes), the TimerManagementVerticle resets the timer.

    If a catalogue is deactivated, the timer is stopped.

  • Active Timers Management:

    The TimerManagementVerticle maintains a map of active timers (activeTimers), which tracks the timer IDs for each catalogue. This allows the system to manage timers dynamically (e.g., stop, reset, or update them).

Key Technologies#