Data model and interfaces

Unreleased preview — metrics v2

This section describes metrics v2, which is not yet released. Current piveau deployments run metrics v1 (documented in the pages above). The content here is published in advance so operators and publishers can prepare; details may still change until the release.

• Data model and interfaces
  • The metrics graph
    • Version marker
    • Boolean property measurements (Annotator / Accessibility output)
    • Accessibility diagnostics
    • Score measurements
  • What the cache serves
  • Aggregation semantics

The metrics graph

Every DCAT-AP resource metrics v2 evaluates — a Dataset, Distribution, or Data Service — gets its measurements written into a DQV (Data Quality Vocabulary) metrics graph alongside it. All v2 measurements share the same basic shape, and every v2 metrics graph carries a version marker so downstream services (and operators inspecting the data directly) can tell a v2 graph from a v1 one.

The diagram below shows the shape of a v2 metrics named graph:

graph TD
    subgraph NG["Metrics named graph"]
        G["Graph resource"] -->|"dct:conformsTo"| V["piveau.eu/ns/metrics/2.0.0 - version marker"]
        ENT["Entity: Dataset / Distribution / DataService"] -->|"dqv:hasQualityMeasurement"| M["Measurement"]
        M -->|"dqv:isMeasurementOf"| MT["Metric"]
        M -->|"dqv:value"| VAL["Value: boolean or decimal"]
        M -->|"dqv:computedOn"| ENT
    end

Version marker

@prefix dct: <http://purl.org/dc/terms/> .

<urn:example-metrics-graph> dct:conformsTo <https://piveau.eu/ns/metrics/2.0.0> .

Every v2-producing service checks for this triple before doing anything, and every v2-producing service writes it once it has run. A graph without this marker is treated as not-yet-v2 (or v1) and is passed through untouched rather than scored.

Boolean property measurements (Annotator / Accessibility output)

Every property check described in Scoring methodology is written as one DQV measurement per (entity, metric) pair:

@prefix dqv:  <http://www.w3.org/ns/dqv#> .
@prefix xsd:  <http://www.w3.org/2001/XMLSchema#> .
@prefix pv:   <https://piveau.eu/ns/voc#> .

<https://example.org/dataset/1> dqv:hasQualityMeasurement _:m .

_:m a dqv:QualityMeasurement ;
    dqv:isMeasurementOf pv:keywordAvailability ;
    dqv:computedOn <https://example.org/dataset/1> ;
    dqv:value "true"^^xsd:boolean .

dqv:value is always xsd:boolean for these checks — true if the check passed, false (or absent) if it did not. There is no partial credit.

Accessibility diagnostics

The Accessibility service's binary reachability result follows the same measurement shape, but additionally carries the raw HTTP diagnostics it observed, so operators can see why a URL was marked unreachable:

@prefix dqv:  <http://www.w3.org/ns/dqv#> .
@prefix xsd:  <http://www.w3.org/2001/XMLSchema#> .
@prefix pv:   <https://piveau.eu/ns/voc#> .
@prefix dct:  <http://purl.org/dc/terms/> .
@prefix http: <http://www.w3.org/2011/http#> .

<https://example.org/distribution/1> dqv:hasQualityMeasurement _:m .

_:m a dqv:QualityMeasurement ;
    dqv:isMeasurementOf pv:accessUrlStatusCode ;
    dqv:computedOn <https://example.org/distribution/1> ;
    dqv:value "false"^^xsd:boolean ;
    http:statusCodeValue "404" ;
    http:methodName "HEAD" ;
    dct:description "Not Found" .

A few details worth knowing if you consume this data directly:

• http:statusCodeValue is a plain string literal, not a typed integer — parse it as text. It can repeat if more than one URL was probed for the same check.
• A sentinel value of "1100" means no HTTP response was received at all (not a real status code).
• http:methodName and dct:description are best-effort diagnostics and are only present when the underlying probe actually returned that information.
• The overall dqv:value is true only if every probed URL for that check returned a 2xx status.

Score measurements

The Score service condenses the boolean checks above into numeric scores, written as xsd:decimal measurements on the same entities:

@prefix dqv: <http://www.w3.org/ns/dqv#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix pv:  <https://piveau.eu/ns/voc#> .

<https://example.org/dataset/1> dqv:hasQualityMeasurement _:score .

_:score a dqv:QualityMeasurement ;
    dqv:isMeasurementOf pv:scoring ;
    dqv:computedOn <https://example.org/dataset/1> ;
    dqv:value "4.25"^^xsd:decimal .

Every scored entity also gets one measurement per FAIR dimension it has at least one check in (pv:findabilityScoring, pv:accessibilityScoring, pv:interoperabilityScoring, pv:reusabilityScoring), in the same shape.

In addition, every top-level unit — a dataset, or a standalone data service — gets a pv:finalScore measurement:

<https://example.org/dataset/1> dqv:hasQualityMeasurement _:final .

_:final a dqv:QualityMeasurement ;
    dqv:isMeasurementOf pv:finalScore ;
    dqv:computedOn <https://example.org/dataset/1> ;
    dqv:value "3.75"^^xsd:decimal .

For a dataset, pv:finalScore is the dataset roll-up value. For a standalone data service, it is simply that service's own score (there is nothing to roll up). Distributions and embedded data services never get a pv:finalScore — only their own pv:scoring and per-dimension values.

What the cache serves

The Cache service aggregates these measurements per catalogue (and per country, and platform-wide) and serves them over a REST API. The exact response schema is still being finalized ahead of release, but the shape is expected to look like the following.

Subject to change

The JSON shapes below are illustrative, based on the current design, and may still change before metrics v2 is released. Treat field names and structure as indicative, not final.

A catalogue-level response:

{
  "metricsVersion": "2.0.0",
  "catalogue": { "id": "example-catalogue", "uri": "https://example.org/catalogue/example" },
  "score": 4.55,
  "counts": {
    "datasets": 1200,
    "distributions": 3400,
    "dataServices": 95,
    "datasetsExcludedAllDeprecated": 12
  },
  "coverage": {
    "dataset": [
      { "metric": "titleAvailability", "property": "http://purl.org/dc/terms/title",
        "coverage": 99.8, "evaluated": 1200, "importance": "Mandatory", "weight": 1.0 }
    ],
    "distribution": [
      { "metric": "licenceAvailability", "property": "http://purl.org/dc/terms/license",
        "coverage": 64.5, "evaluated": 3120, "importance": "Recommended", "weight": 0.5 }
    ],
    "dataService": [
      { "metric": "endpointUrlAvailability", "property": "http://www.w3.org/ns/dcat#endpointURL",
        "coverage": 88.4, "evaluated": 95, "importance": "Mandatory", "weight": 1.0 }
    ]
  }
}

A single-dataset response, showing per-metric results and the roll-up:

{
  "metricsVersion": "2.0.0",
  "dataset": {
    "id": "example-dataset",
    "uri": "https://example.org/dataset/example-dataset",
    "score": 4.25,
    "maxScore": 7.5,
    "metrics": [
      { "metric": "titleAvailability", "property": "http://purl.org/dc/terms/title",
        "importance": "Mandatory", "weight": 1.0, "result": 1, "score": 1.0 }
    ]
  },
  "distributions": [
    { "id": "1", "uri": "https://example.org/dataset/example-dataset/distribution/1",
      "score": 3.25, "maxScore": 7.5, "metrics": [ "..." ] }
  ],
  "dataServices": [],
  "datasetFinal": 3.75
}

Aggregation semantics

A catalogue's overall score is the plain average of pv:finalScore across its member datasets and standalone data services; per-metric coverage is computed as the percentage of evaluated resources of that type for which the check passed. Historical values are stored as a time series: score values are numerically averaged when bucketed into a coarser time resolution (e.g. daily readings rolled up into a monthly figure), while counts and coverage — which are structured objects, not single numbers — keep the most recent entry in each bucket rather than being averaged.