Public | Automated Build

Last pushed: 2 years ago
Short Description
kafka-pipeline-lambdas-node-sink
Full Description

data-pipeline-lambda sink docker image

This image is the base for nodejs code in data-pipelines.
This is the repository for sink-lambdas in nodejs

Installation

The basic setup consists of a index.js and a Dockerfile

 docker pull julianbei/data-pipeline-lambdas-sink

Example Usage

create two files:

  • index.js
  • Dockerfile

index.js

var Promise = require('bluebird');

var config = {
 RECEIVE_TOPIC     : process.env.RECEIVE_TOPIC,
 QUEUE_URL         : process.env.SQS_URL,
 AWS_ACCESS_KEY    : process.env.AWS_ACCESS_KEY,
 AWS_ACCESS_SECRET : process.env.AWS_ACCESS_SECRET,
 AWS_REGION        : process.env.AWS_REGION
};
module.exports.config = config;

var processor = function(message){
  return new Promise(function(resolve, reject) {
    // Processing logic goes here for example add fields to message objects:
    message.active = true;
    // after processing just resolve the final object
    resolve(message);
  });
};
module.exports.processor = processor;

var callback = function(result){
  // do something with the processing result
  console.log('processing result: '+result);
};
module.exports.callback = callback;

Dockerfile

FROM julianbei/data-pipeline-lambdas-sink

RUN mkdir -p /app/lambda

# Copy the main application.
COPY . ./app/lambda

WORKDIR /app

Options

If you need own dependencies, just create a package.json and install it with your Dockerfile

FROM julianbei/data-pipeline-lambdas-sink

RUN mkdir -p /app/lambda

COPY ./package.json /app/package.json
WORKDIR /app
RUN npm install

# Copy the main application.
COPY . ./app/lambda/

If the js module.exports.config in your index.js is not specified, the config gets automatically loaded from your environment.

Docker-Compose Template:

sink-lambda:
  build: sink-lambda/.
  command: forever -m 5 /app/index.js
  environment:
    - RECEIVE_TOPIC=
    - SQS_URL=
    - AWS_ACCESS_KEY=
    - AWS_ACCESS_SECRET=
    - AWS_REGION=

You can optionally reload the app an each change in your app folder.
This might be an option for development:

volumes:
  - ./sink-lambda:/app/lambda/
Docker Pull Command
Owner
hitfox
Source Repository