kanselarij/ttl-import-service repository overview

⁠ttl import service

A service that watches a directory for newly appearing RDF-dumps (.ttl or .nt) and imports those into the database. Uses sudo-headers⁠ and imports in batches, so it can be used through mu-authorization⁠.

⁠Configuration

⁠Environment variables

The service can be configured through the following environment variables:

• GRAPH: graph to write to, by default http://mu.semte.ch/application
• BATCH_SIZE: Amount of triples imported in each insert statement. Particularly important when querying through mu-authorization. Defaults to 100.
• IMPORT_DIR: path to monitor, by default /data/imports
• (MU_SPARQL_ENDPOINT): SPARQL endpoint to use (default provided by mu-javascript-template⁠)

⁠docker-compose snippet

  ttl-import:
    build: https://github.com/kanselarij-vlaanderen/ttl-import-service.git
    volumes:
      - "./data/my-import-folder:/data/imports"
    environment:
      GRAPH: "http://mu.semte.ch/graphs/staatsblad"

⁠FAQ

No. This service is scoped to monitoring for newly appearing files. A one-time import can easily be accomplished through the mu-migrations-service⁠.

⁠Related services

Other services that have inspired this one, or have a slightly different take on importing data are:

• mandaten-import-service⁠
• ttl-to-delta⁠ in conjuction with themis-publication-consumer⁠.