Skip to content

Configuring and customizing the metrics#

Unreleased preview — metrics v2

This section describes metrics v2, which is not yet released. Current piveau deployments run metrics v1 (documented in the pages above). The content here is published in advance so operators can prepare; details may still change until the release.

One shared metric-definition file#

In v1, each metrics service that needed to know about scoring weights or property checks carried its own configuration. In v2, all of that lives in a single, shared metric-definition file — a Turtle (RDF) document listing every (metric, entity type) binding: which RDF property it checks, how important it is, how heavily it is weighted, and what kind of check to run.

This one file is read by:

  • the Annotator, to know which properties to check and how to check them;
  • the Accessibility service, to know which URLs to probe;
  • the Score service, to know each check's weight when computing entity and roll-up scores;
  • the Cache service, to label coverage results with the same metric names and dimensions.

The following diagram shows how the shared definition file fans out to the four consuming services, and how the override variable replaces it:

flowchart TD
    DEF["piveau-metrics-definitions.ttl - shared metric definitions"]
    OVR["override: PIVEAU_METRICS_DEFINITIONS = /path/to/custom.ttl"]
    OVR -.-> DEF
    DEF -->|"which checks to run"| ANN["Annotator"]
    DEF -->|"which URLs to probe"| ACC["Accessibility"]
    DEF -->|"weights per metric"| SCO["Score"]
    DEF -->|"labels/weights for coverage"| CAC["Cache"]

Because every service reads the same file, the property catalogue is defined once and stays consistent across the whole pipeline. The Scoring methodology page documents the default set of 56 bindings this file ships with.

The binding schema#

Each binding is a small record with the following fields:

Field Meaning
metric A stable identifier for the check (e.g. "keyword availability"). This is the value written into the dqv:isMeasurementOf property of every measurement the pipeline produces.
appliesTo Which entity type the binding is for: dcat:Dataset, dcat:Distribution, or dcat:DataService.
property The RDF property being checked (e.g. dcat:keyword).
Dimension (via dqv:inDimension) Which of the four FAIR dimensions the check is labeled with (Findability, Accessibility, Interoperability, Reusability) — informational only, reusing the standard DQV dimension property rather than a metrics-specific one.
importance Mandatory, Recommended, or Optional — determines the weight.
weight A decimal number: 1.0 for Mandatory, 0.5 for Recommended, 0.25 for Optional.
checkKind How the check is evaluated: set, valid-url, vocabulary, or accessibility (see Check kinds).
checkVocabulary For vocabulary checks, which controlled vocabulary to check against (e.g. file-type, iana, licence, access-right).
checkFlag For some vocabulary checks, an additional flag narrowing the check (e.g. nonProprietary, machineReadable).

A complete example binding, in the same Turtle shape the shared file uses:

@prefix pmd:  <https://piveau.eu/ns/metrics-def#> .
@prefix pv:   <https://piveau.eu/ns/voc#> .
@prefix dct:  <http://purl.org/dc/terms/> .
@prefix dcat: <http://www.w3.org/ns/dcat#> .
@prefix dqv:  <http://www.w3.org/ns/dqv#> .
@prefix xsd:  <http://www.w3.org/2001/XMLSchema#> .

[] a pmd:MetricBinding ;
   pmd:metric pv:formatMediaTypeNonProprietary ;
   pmd:appliesTo dcat:Distribution ;
   pmd:property dct:format ;
   dqv:inDimension pv:interoperability ;
   pmd:importance "Recommended" ;
   pmd:weight "0.5"^^xsd:decimal ;
   pmd:checkKind "vocabulary" ;
   pmd:checkVocabulary "file-type" ;
   pmd:checkFlag "nonProprietary" .

This one binding says: for every dcat:Distribution, check whether its dct:format value is flagged as non-proprietary in the file-type vocabulary; the check is Recommended importance (weight 0.5), and the result is labeled with the Interoperability dimension.

Overriding the definition file#

To use a custom metric-definition file instead of the bundled default, set the environment variable PIVEAU_METRICS_DEFINITIONS to the path of your Turtle file — on each of the four consuming services: the Annotator, the Accessibility service, the Score service, and the Cache service. Point all four at the same file so their checks, weights, and labels stay consistent with each other; if they diverge, the services will disagree about how a given check is weighted or even what it means.

What you can safely change#

  • Adjust weights. Change a property's importance (and correspondingly its weight) to make it count for more or less in the overall score.
  • Add or remove property checks. Add a new binding for a property you care about, or remove one you don't want evaluated, per entity type.

Consequences to keep in mind:

  • Changing the set of bindings or their weights changes each entity type's maximum possible score — it will no longer necessarily be 7.5. The default 56-binding set is tuned so that Dataset, Distribution, and Data Service each sum to exactly 7.5, but this is a convention of the default configuration, not a rule enforced by the services. If you customize the file, the roll-up and catalogue formulas in Scoring methodology still apply — they just operate on whatever maximum your configuration produces.
  • Keep the file identical across all four services. A mismatch means, for example, that the Score service could weight a check differently than the Cache service labels it, producing confusing or inconsistent results.

Excluding individual metrics per service#

Independently of the shared definition file, the Annotator and Accessibility services each accept an EXCLUDE_ANNOTATIONS environment variable: a comma-separated list of metric URIs to skip evaluating, without editing the definition file itself. This is useful for quickly disabling a single check on one service (for example, if you want the Annotator to keep checking a property but don't want the Accessibility service probing its URL, or vice versa) without maintaining a second copy of the definition file.