teamdigitale/pdnd-dashboard-scripts repository overview

⁠Dashboard data ingestion scripts for Piattaforma Digitale Nazionale Dati (PDND), previously DAF

The scripts in this repository are used to collect data from different external services and store them in a MongoDB database.

⁠What is the PDND (previously DAF)?

PDND stays for "Piattaforma Digitale Nazionale Dati" (the Italian Digital Data Platform), previously known as Data & Analytics Framework (DAF).

You can find more informations about the PDND on the official Digital Transformation Team website⁠.

⁠What the dashboard data ingestion scripts do?

The scripts in this repository are used to collect data from different external services and store them in a MongoDB database.

Data are then exposed through Metabase⁠. They can be quired and visualized.

⁠External project dependencies

This project depends by the following components:

• Metabase version v0.25.2, available here⁠.

• PostgreSQL version 9.2, available as Docker image here⁠.

• MongoDB version 3.4, available as Docker image here⁠.

⁠How to build and test the PDND dashboard scripts

The dashboard scripts container and the related visualization tools are distributed as a set of Docker containers interacting with one each other.

The Dockerfile and the docker-compose.yaml files are in the root of this repository.

To build the local test environment (build the CKAN container and download dependencies) run:

docker-compose up -d

While the ingestion scripts are called on a regular basis in production, in this test environment they need to be called manually, so that some test data can be manually sourced and injected into MongoDB (see how in the paragraphs below).

Also, once the build process is complete, the Metabase dashboard can be accessed at http://localhost:3000.

To bring down the test environment and remove the containers use

docker-compose down

⁠How to contribute

Contributions are welcome. Feel free to open issues⁠ and submit a pull request⁠ at any time, but please read our handbook⁠ first.

⁠License

Copyright (c) 2019 Presidenza del Consiglio dei Ministri

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/⁠.