chrisengelhardt/ir-split repository overview

⁠Image Convolution

This workflow performs industrial optical QA on a large set of input images.

⁠Overview

This repository contains a parallel implementation, orchestrated with the Abstract Function Choreography Language and runnable on the Apollo Engine⁠

Fig 1: workflow.yaml control and data flow

⁠Get the code

git clone https://github.com/Apollo-Workflows/Image-Recognition
cd Image-Recognition

⁠Get an input dataset

Then, update input.json with the desired parallelism. The default is 2. This yields a 100 images per 1 convolution function ratio for the 200⁠ dataset.

{
  "desired_num_splits": 1     // <---
  "s3bucket": "YOUR_BUCKET",
  "s3prefix": "YOUR/PREFIX",
  "topic_arn": "YOUR_TOPIC",
  "region": "us-east-2",
}

⁠Autodeploy

1. Save the credentials for your cloud provider in the according subfolder:
   • AWS: Put credential file under aws/credentials
   • IBM:
     • Add ibmcloud_api_key to ibm/terraform.tfvars
     • Add S3 credentials from AWS to ibm/s3Credentials.json
2. • A: Deploy to all providers: Run from root dir docker run --rm -it --entrypoint=/app/deployAll.sh -v ${PWD}:/app/ chrisengelhardt/apollo-autodeploy
   • B: Deploy single provider with custom settings: Run docker run --rm -v ${PWD}:/app/ chrisengelhardt/apollo-autodeploy --help from within the directory of your chosen cloud provider

Note: For IBM you have to create a namespace first and place it into ibm.tf at line namespace = "YOURNAMESPACE".

Usage: /app/deploy.sh [--help] [--region region] [--url] [--mapping]

Commands:
        --help                  Show this help output.
        --region region         Sets a specific region for the deployment. Use a region from:
                                https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html
        --bucket bucket_name    Sets a specific bucket name for the deployment.
        --url                   Prints out all deployment urls
        --mappings              Creates typeMapping.json with the deployment urls

⁠Run the containers

To run the containers (preprocess-imgs, ir-split, ir-convolution.-educe) you must mount your input and credentials file into the correct dir as shown below:

docker run --rm -it -v ${PWD}/input.json:/usr/src/app/input.json -v ${PWD}/credentials:/root/.aws/credentials ir-preprocess