Public | Automated Build

Last pushed: 8 days ago
Short Description
content-collection-unfolder
Full Description

content-collection-unfolder

Introduction

UPP Service that forwards mapped content collections to the content-collection-rw-neo4j.
If a 200 answer is received from the writer, it retrieves the elements in the collection from
the document-store-api and places them in Kafka on the Post Publication topic so that notifications
will be created for them.

Dependencies are:

  1. content-collection-rw-neo4j
  2. document-store-api
  3. kafka

Installation

Download the source code, dependencies and test dependencies:

    go get -u github.com/kardianos/govendor
    go get -u github.com/Financial-Times/content-collection-unfolder
    cd $GOPATH/src/github.com/Financial-Times/content-collection-unfolder
    govendor sync
    go build .

Running locally

  1. Run the tests and install the binary:

     govendor sync
     govendor test -v -race
     go install
    
  2. Run the binary (using the help flag to see the available optional arguments):

     $GOPATH/bin/content-collection-unfolder [--help]
    

Options:

    --app-system-code="content-collection-unfolder"                                          System Code of the application ($APP_SYSTEM_CODE)
    --app-name="Content Collection Unfolder"                                                 Application name ($APP_NAME)
    --app-port="8080"                                                                        Port to listen on ($APP_PORT)
    --unfolding-whitelist=["content-package"]                                                Collection types for which the unfolding process should be performed ($UNFOLDING_WHITELIST)
    --writer-uri="http://localhost:8080/__content-collection-rw-neo4j/content-collection/"   URI of the Writer ($WRITER_URI)
    --writer-health-uri="http://localhost:8080/__content-collection-rw-neo4j/__health"       URI of the Writer health endpoint ($WRITER_HEALTH_URI)
    --content-resolver-uri="http://localhost:8080/__document-store-api/content/"             URI of the Content Resolver ($CONTENT_RESOLVER_URI)
    --content-resolver-health-uri="http://localhost:8080/__document-store-api/__health"      URI of the Content Resolver health endpoint ($CONTENT_RESOLVER_HEALTH_URI)
    --kafka-write-topic="PostPublicationEvents"                                              The topic to write the messages to ($Q_WRITE_TOPIC)
    --kafka-proxy-address="http://localhost:8080"                                            Addresses of the kafka proxy ($Q_ADDR)
    --kafka-proxy-hostname="kafka"                                                           The hostname of the kafka proxy (for hostname based routing) ($Q_HOSTNAME)
    --kafka-authorization=""                                                                 Authorization for kafka ($Q_AUTHORIZATION)
  1. Test:

Create a file with the following content collection contents:

  {
    "uuid": "45163790-eec9-11e6-abbc-ee7d9c5b3b90",
    "items": [
      {
        "uuid": "d4986a58-de3b-11e6-86ac-f253db7791c6"
      },
      {
        "uuid": "d9b4c4c6-dcc6-11e6-86ac-f253db7791c6"
      }
    ],
    "publishReference": "tdi23377744",
    "lastModified": "2017-01-31T15:33:21.687Z"
  }

Assuming that the file you just is cc.json, you can run the following curl command to test the unfolder

  curl -X PUT --data "@cc.json"  localhost:8080/content-collection/content-package/45163790-eec9-11e6-abbc-ee7d9c5b3b90

If you've setup everything correctly, you should receive a 200 response. You can also watch the app logs for errors as the
unfolder will return a 200 even if kafka related processing fails.

Build and deployment

How can I build and deploy it (lots of this will be links out as the steps will be common)

Service endpoints

PUT

Using curl:

curl -X PUT --data "@cc.json"  localhost:8080/content-collection/content-package/45163790-eec9-11e6-abbc-ee7d9c5b3b90

The expected response is a simple 200 with no response body. In case a error takes place, a json response body will be provided,
similar to the following example:

{"msg\":"Something bad happened"}

As a rule of thumb, the unfolder will return the exact response status code and body received from the content-collection-neo4j-rw app in
case a non 200 response is received.

The flow of the unfolder is the following:

  1. the PUT request is received and is forwarded to the content-collection-neo4j-rw
  2. the response from the RW app is evaluated, if it is not a 200 response, the writer response is sent to the unfolder client
  3. in case the collection type is not one that needs unfolding (at the moment only content-package is unfolded), a 200 response is returned
  4. all the UUIDs from content collection as resolved using the document-store-api
  5. for each piece of content retrieved from the DSAPI, a new message is created and placed on the configured kafka topic

Healthchecks

Admin endpoints are:

/__gtg

/__build-info

/__ping

/__health

There are following checks are performed when the /__health is called:

  1. content-collection-neo4j-rw connectivity check
  2. document-store-api connectivity check
  3. kafka connectivity check and topic existence check

The /__gtg endpoint will return a 200 in case all above health checks are successful.

Docker Pull Command
Owner
coco

Comments (0)