Public | Automated Build

Last pushed: 5 months ago
Short Description
Preprocessing docker to create database from Knowledgestore overview.
Full Description


Preprocessing module for the QueryBuilder stack. Provides commands to create a database which is ready for use by the server.


The query builder preprocessor is written in python and makes heavy use of the Java code in


Automated installation

Please note that the installation and execution process is usually fully automated by Docker and Docker-Compose. For information on installing the full stack, see the repository for more information.

Manual installation and useage

With Docker
A Dockerfile has been provided for your convenience. Please refer to for installation of docker.

    docker build -t nlescstoryteller/query-builder-preprocessing


Given an empty query-builder-client interface, the user may want to check the following in this docker container:

  1. Connect to the docker container for troubleshooting


    sudo docker exec -v data:/data -it nlescstoryteller/query-builder-preprocessing /bin/bash


    winpty docker exec -v data:/data -ti nlescstoryteller/query-builder-preprocessing //bin/bash
  1. Check the /data-tmp folder for json files

     ls -al /data-tmp

    The user is expected to find a number of JSON data files here, with sizes > 0kb.

  2. Check the /data folder for a storyteller.db file

     ls -al /data

    The user is expected to find a storyteller.db file here, with size > 0kb.

  3. If either 2. or 3. (or both) are not the case, the getoverview part of the script in has most likely failed:

     cd /src/StoryTeller/scripts
     chmod +x
     mkdir /data-tmp
     cp token.index.gz $DATA_DIR/token.index.gz
     mv *.json /data-tmp

    To fix this issue, try to remove the contents of the /data and /data-tmp directory and any json files in the /src/StoryTeller/scripts directory, and run the script again.

The Full QueryBuilder / Knowledgestore visualization system

The full system consists of 7 parts:

  1. KnowledgeStore
    • RDF triple store. The NewsReader KnowledgeStore is a scalable, fault-tolerant, and Semantic Web grounded storage system to jointly store, manage, retrieve, and semantically query, both structured and unstructured data (see
  2. cltl/StoryTeller (
    • Support library for querying the knowledge store and creating JSON data from the results.
  3. QueryBuilder Preprocessing (
    • written in python
    • uses cltl/Storyteller to get an overview of possible queries from the knowledgestore
    • processes the overview into a sqlite3 database that can be used by the server.
  4. QueryBuilder Server (
    • written in javascript, with express.js + sqlite3 database
    • incudes a custom database trigger written in C on INSERT statements to the queries table.
    • upon receiving a new query string from QueryBuilder, the Server
      • stores the received query in a database
      • runs the received query against the KnowledgeStore using the QueryBuilder Daemon, which updates the list of previous queries and their results, indexed by an identifier.
  5. QueryBuilder Daemon (
    • written in Java with Xenon (
    • receives requests for knowledgestore queries from the database trigger in the server.
    • queries the Knowledgestore using cltl/Storyteller
    • updates the list of previous queries and their results, indexed by an identifier.
  6. QueryBuilder Client (this repository)
    • written in Typescript and React.js
    • shows a list of all possible concepts, actors or events from the server
    • helps the user to compose a sparql query to run against the KnowledgeStore by selecting items of interest, be it concepts, actors, events, etc.
  7. UncertaintyVisualization (
    • written in javascript and Angular 1 (legacy)
    • allows the user to select from a list of previously run queries (communicates with Server) to select one for visualization.
    • visualizes the results selected
Docker Pull Command