developmentseed/building_parts repository overview

⁠Overview

This building parts model is an object detection model for identifying building parts in street view images (see detectable classes at the end). It was developed to support the World Bank in the Housing Passports project. The model itself was developed with the TF Object Detection API using the ResNet and SSD model components for feature extraction and object proposals, respectively. The model is a "generalist" in that it was trained simultaneously on 4 neighborhoods in South America. The goal was to generate a model that performed well across a variety of geospatial regions.

⁠Inputs and Outputs

The docker image takes in a json-like dict (via a POST request) with base64 encoded images. Images should have been read from a standard format (like png or jpg) where the valid pixel range is 0-255. The images should be 512x512 pixels with 3 channels (RGB).

This model will return a json response with each line corresponding to a single detection. The format of these responses is a dictionary with specifying at least the detected classes, detected scores (i.e., confidences), and detection boxes (giving [x_min, y_min, x_max, y_max] in image normalize coordinates). It is suggested to threshold predictions based on model confidence (0.5 is a good default).

⁠Example

Use this code snippet as a starting point for getting predictions from a local, running container. You could also run the container on a separate remote machine as long as the port is exposed.

# Set the url of the running Docker container
url_endpoint='http://localhost:8501/v1/models/building_parts:predict'

# Iterate through groups of images
for img_fname in img_fnames:

    ###################################
    # Load an image  to predict

    instances = []
    with open(op.join(image_directory, img_fname), 'rb') as image_file:
        b64_image = base64.b64encode(image_file.read())
        instances.append({'b64': b64_image.decode('utf-8')})

    ################
    # Run prediction
    payload = json.dumps({"instances": instances})
    resp = requests.post(url_endpoint, data=payload)
    preds = json.loads(resp.content)['predictions']

⁠Classes

For descriptions and examples of the classes, see the associated labeling guide⁠. Classes the model is capable of detecting and their class number:

 "window":1,
  "door":2,
  "garage":3,
  "disaster_mitigation":4