Public | Automated Build

Last pushed: 11 days ago
Short Description
Indexes V2 content in Elasticsearch for use by SAPI v1
Full Description

content-rw-elasticsearch

Introduction

Indexes V2 content in Elasticsearch for use by SAPI V1

Installation

Download the source code, dependencies and test dependencies:

    go get -u github.com/kardianos/govendor
    go get -u github.com/Financial-Times/content-rw-elasticsearch
    cd $GOPATH/src/github.com/Financial-Times/content-rw-elasticsearch
    govendor sync
    go build .

Running locally

  1. Run the tests and install the binary:

     govendor sync
     govendor test -v -race
     go install
    
  2. Run the binary (using the help flag to see the available optional arguments):

     $GOPATH/bin/content-rw-elasticsearch [--help]
    

Options:

    --app-system-code="content-rw-elasticsearch"            System Code of the application ($APP_SYSTEM_CODE)
    --app-name="Content RW Elasticsearch"                   Application name ($APP_NAME)
    --port="8080"                                           Port to listen on ($APP_PORT)
    --aws-access-key=""                                     AWS ACCES KEY ($AWS_ACCESS_KEY_ID)
    --aws-secret-access-key=""                              AWS SECRET ACCES KEY ($AWS_SECRET_ACCESS_KEY)
    --elasticsearch-sapi-endpoint="http://localhost:9200"   AES endpoint ($ELASTICSEARCH_SAPI_ENDPOINT)
    --index-name="ft"                                       The name of the elaticsearch index ($ELASTICSEARCH_SAPI_INDEX)
    --kafka-proxy-address="http://localhost:8080"           Addresses used by the queue consumer to connect to the queue ($KAFKA_PROXY_ADDR)
    --kafka-consumer-group="default-consumer-group"         Group used to read the messages from the queue ($KAFKA_CONSUMER_GROUP)
    --kafka-topic="CombinedPostPublicationEvents"           The topic to read the meassages from ($KAFKA_TOPIC)
    --kafka-header="kafka"                                  The header identifying the queue to read the messages from ($KAFKA_HEADER)
    --kafka-concurrent-processing=false                     Whether the consumer uses concurrent processing for the messages ($KAFKA_CONCURRENT_PROCESSING)
  1. Test:

There are no service endpoints to test.

Build and deployment

Healthchecks

Admin endpoints are:

`/__gtg`

Returns 503 if any if the checks executed at the /__health endpoint returns false

`/__health`

There are several checks performed:

  • Elasticsearch cluster connectivity
  • Elasticsearch cluster health
  • Elastic schema validation
  • Kafka queue topic check
`/__health-details`

Shows ES cluster health details

`/__build-info` 

Other information

An example of event structure is here testdata/exampleEnrichedContentModel.json

The reference mappings for Elasticsearch are found here runtime/referenceSchema.json

Docker Pull Command
Owner
coco

Comments (0)