indiepartners/crowdwatch repository overview

⁠CrowdWatch

⁠Set-up

1. Install NPM packages:

• PRODUCTION: docker-compose run app npm install and docker-compose run webpack npm install
• DEVELOPMENT: docker-compose run app npm install and cd sources/webpack && npm install

The reason there is a discrepancy between these two is that docker does not seem to play nice with hot reload of the webpack dev server. Therefore I need to run it locally, and node-sass uses a c++ binding that binds to a specific system architecture. As docker's sys architecture doesn't always match the host architecture, this causes an issue if the install command is run from within docker.

1. Create a .env file and fill it in as such:

• DEVELOPMENT: NODE_ENV=dev
• STAGING: NODE_ENV=staging
• PRODUCTION: NODE_ENV=prod

1. Add the aws.json config file that is in Dashlane to sources/moderation/config

2. Update the config for your environment in sources/moderation/*yourenv*.json, see dev.json as an example.

⁠Configuration parameters:

• @param {string} name - Application name
• @param {string} backend.port - The port used to connect to the back-end by the front-end client
• @param {string} backend.auth.jwt.secretKey - The secret key used to sign the JWT tokens
• @param {string} backend.tokenLifetime - Lifetime of the JWT token
• @param {number} statisticsCacheTimeout - Amount of time in milliseconds before the application's statistics shown on the front-end are refreshed on the back-end's cache
• @param {string} clients[clientName].publishingUrl - Where should the result of work items be reported?
• @param {number} clients[clientName].requestLimitPerWindow - How many results of work items do we publish per window?
• @param {number} clients[clientName].requestTimeWindow - How often do we automatically check for new results to be published? Time in milliseconds
• @param {number} clients[clientName].timeBetweenRequests - The delay in milliseconds between the requests as described above
• @param {string} clients[clientName].api_output_username - The username used to communicate with the publishingUrl mentioned above
• @param {string} clients[clientName].api_output_password - The password used to communicate with the publishingUrl mentioned above
• @param {string} clients[clientName].api_input_username - The username used to with CrowdWatch (e.g.: recording work items, and any other protected endpoints)
• @param {string} clients[clientName].api_input_password - The password used to communicate with CrowdWatch
• @param {bool} clients[clientName].shouldAutoPublishResults - If set to false, will not automatically find work item results to publish
• @param {number} clients[clientName].policy.minVotes - The amount of votes needed for a work item to be considered as "processed".
• @param {number} clients[clientName].policy.confirmationLockTimeout - When an item is pulled in the conflict queue, it will not be shown to other people for the duration of this timeout in milliseconds. This is to prevent duplicate votes.
• @param {number} clients[clientName].policy.votingTimeout - The amount of time in milliseconds after a vote before the front-end pulls a new item automatically for the moderator
• @param {string} logDir - The directory to store application logs
• @param {string} db.host - The URL (and port) of the database
• @param {string} db.username - The username used to authenticate with the database
• @param {string} db.password - The password used to authenticate with the database

⁠Running the app:

• PRODUCTION/STAGING:

1. docker-compose run webpack run npm build
2. docker-compose up app

• DEVELOPMENT:

1. docker-compose up app
2. cd sources/webpack && npm run webpack-dev-server

⁠Optional - Docker build steps to deploy to staging

1. Build docker images - make sure that the node_modules are not installed already! Otherwise it will upload these also. Git clone into a new directory if needed

• docker build --rm -f Dockerfile.crowdwatch -t crowdwatch-src .

• docker build --rm -f Dockerfile.moderation -t crowdwatch-front .

• docker tag crowdwatch-src:latest indiepartners/tools:crowdwatch-src-1.0

• docker tag crowdwatch-front:latest indiepartners/tools:crowdwatch-front-1.0

• docker push indiepartners/tools:crowdwatch-src-1.0

• docker push indiepartners/tools:crowdwatch-front-1.0

• Or, in one command:

• • docker build --rm -f Dockerfile.crowdwatch -t crowdwatch-src . && docker build --rm -f Dockerfile.moderation -t crowdwatch-front . && docker tag crowdwatch-src:latest indiepartners/tools:crowdwatch-src-1.0 && docker tag crowdwatch-front:latest indiepartners/tools:crowdwatch-front-1.0 && docker push indiepartners/tools:crowdwatch-src-1.0 && docker push indiepartners/tools:crowdwatch-front-1.0

1. To run it locally if you want:

• docker-compose -f compose.yml up -d

⁠Commenting legend:

• @TODO: Priority notes on improvements and known limitations
• @IMPROVEMENT: Less-urgent notes to signal improvements and known limitations
• @DISABLED: Disabled parts of the codebase