Scripts to collect data from external services and save them in a Mongo database.
50K+
The scripts in this repository are used to collect data from different, external sources and store them in a MongoDB database.
More informations about the Italian Department of Innovation can be found on the Department website
The scripts in this repository are used to collect data from different external sources and store them in a MongoDB database.
The dashboard scripts container and the related visualization tools are distributed as a set of Docker containers interacting with one each other.
The Dockerfile and the docker-compose.yaml files are in the root of this repository.
To build the local test environment run:
docker-compose up -d
NOTE: the
docker-compose.yamlfile sets different environment variables that could be used to adapt and customized many platform functionalities.
While the ingestion scripts are called on a regular basis in production, in this test environment they need to be called manually, so that some test data can be manually sourced and injected into MongoDB (see how in the paragraphs below).
Also, once the build process is complete, the Metabase dashboard can be accessed at http://localhost:3000.
NOTE: Credentials should be changed after the first login.
NOTE: the
/datafolder contains dummy data, useful to be able to carry out tests.
To bring down the test environment and remove the containers use
docker-compose down
Contributions are welcome. Feel free to open issues and submit a pull request at any time.
Copyright (c) 2020 Presidenza del Consiglio dei Ministri
This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/.
Content type
Image
Digest
sha256:03c1ce23c…
Size
283 MB
Last updated
about 2 years ago
docker pull teamdigitale/dpt-services-dashboard-scripts