piveau Profile#

Experimental Feature

The described features, configuration and APIs are work in progress.

A piveau Profile is central configuration that allows you to customize the metadata model of piveau instance. Hence, it enables you to modify, extend and configure the DCAT-AP-based core model. Furthermore it allows you add additional metadata types beyond catalogs, datasets and distributions. Currently, piveau Profile is supported by hub-repo and hub-search and if used it needs to be enabled in both services.

Concept#

A piveau Profile is a directory that contains a manifest file package.json and one or more SHACL files. The SHACL files defines the metadata model and are using a domain-specific properties.

.
└─ my-profile
    ├─ piveau.json
    ├─ shacl-file-1.ttl
    ├─ shacl-file-2.ttl
    └─ samples

The profile will take effect on the following aspects:

The mapping in Elasticsearch
The indexing process from RDF to the mapping
The OpenAPI description of the hub-search API
The availability of additional entities in hub-repo and hub-search

Enabling the Feature#

The following config applied for both hub-repo and hub-search will enable and configure a piveau Profile.

{
    "PIVEAU_FEATURE_FLAGS": {
        "piveau_profile": true
    },
    "PIVEAU_PROFILE": {
        "type": "directory",
        "path": "/path/to/my/profile/my-profile"
    }
}

piveau.json#

The piveau.json has the following minimal structure:

{
  "version": "1",
  "id": "my-profile",
  "core": [
    {
      "id": "dataset",
      "description": "My model for a dataset",
      "path": "my-profile.ttl",
      "name": "dataset",
      "shapeUri": "http://data.europa.eu/r5r/Dataset_Shape"
    }
  ],
  "resources": [
    {
      "id": "my-resource",
      "description": "My Resource",
      "path": "my-resource.ttl",
      "name": "my-resource",
      "typeUri": "https://piveau.eu/ns/voc#CustomResource",
      "shapeUri": "https://piveau.eu/ns/voc#CustomResourceShape"
    }
  ]
}

Models with SHACL#

The foundation of a piveau Profile are SHACL files that use domain-specific properties.

@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix dct: <http://purl.org/dc/terms/> .
@prefix dcat: <http://www.w3.org/ns/dcat#> .
@prefix dcatap: <http://data.europa.eu/r5r/> .
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix pv: <https://piveau.eu/ns/voc#> .

pv:My_Profile
    a pv:PiveauProfile ;
    pv:profileVersion "1" ;
    dct:title "My Profile"@en ;
    dct:description "Based and inspired by DCAT-AP 2.1.1"@en ;
    owl:versionInfo "0.0.1" .


dcatap:Catalog_Shape
    a sh:NodeShape ;    
    sh:name "Catalog"@en ;
    sh:property [
        pv:mappingClass "SimpleMultiLangTitle" ;
        pv:mappingName "title" ;
        sh:minCount 1 ;
        sh:nodeKind sh:Literal ;
        sh:path dct:title ;
        sh:severity sh:Violation ;
    ] ;
    sh:targetClass dcat:Catalog .

Danger

Installing or updating a piveau Profile requires a restart of the services. You cannot change the mapping of an existing field (e.g., change text to keyword, or change an analyzer). This would invalidate already indexed data. If you need to do this, you typically have to reindex your data into a new index with the correct mapping. You can add new, entirely new top-level fields to an existing index's mapping. For example, if your index initially only had product_id and name, you could add description or price without reindexing.

You can also add new fields (sub-fields) to an existing text field if those sub-fields are based on analysis that can be derived from the already tokenized text field, or if they are keyword type (which only needs the raw string, which is typically also stored). For example, if you have a name field of type text, you can later add a name.keyword sub-field, because the _source contains the original string, from which the keyword version can be derived upon reindexing.