Skip to content

Features#

piveau comes with all batteries included for building an entire metadata catalog pipeline. While metadata is the core business, piveau has some features for actual data in store!

Getting the data - consus#

  • Support for many data protocols, sources and formats: OAI-PMH, RDF, CKAN, uData, OwnCloud, JSON, SPARQL, Socrata, Drupal.
  • Super scalable up to hundreds of thousands of datasets per source.
  • Dynamic and programmable data transformation with JavaScript or XSLT.
  • Harvesting runs can be individually scheduled to be executed regularly on time.
  • Exports into DCAT(-AP) and derived standards are included.
  • A flexible configuration-based orchestration allows for easy extension with custom processing steps and even third-party services.

Storing and browsing the data - hub#

  • Direct storage of the DCAT metadata as RDF in a triplestore.
  • Support for Virtuoso and Fuseki as storage backends.
  • URI harmonization to create a consistent knowledge graph.
  • Integration and linking of external vocabularies and ontologies.
  • Powerful search for metadata based on Elasticsearch.
  • Support for managing actual data with S3 or filesystem.
  • Rich, customizable and multilingual frontend to search and filter for the metadata.
  • Extensive backend tool to create and manage the metadata.
  • Ready for integrating DOI providers.

Checking the quality - metrics#

  • Periodic creation of quality assessments based on SHACL validations and the FAIR principles.
  • Storage of the assessments alongside the metadata with the Data Quality Vocabulary (DQV).
  • Extensive frontend to browse the assessments and download reports in PDF, ODS or XLSX.

Access control#

  • piveau integrates with the industry-proven identity and access management Keycloak.
  • It can be easily connected to existing login and sign-on solutions.

Operations#

  • piveau is completely containerized with Docker.
  • It runs great on Kubernetes and suitable Helm charts are ready to use.

Technologies#

  • The backends are mostly written in Java and Kotlin with the great Vert.x framework.
  • The frontends are developed with Vue.js.