Sign inSign up

stackateam/foxeye_worker

By stackateam

•Updated over 6 years ago

FOXEYE worker: generates a DLF from any file

Image
0

534

stackateam/foxeye_worker repository overview

alt text

TODO: re.sub(' +', ' ', 'The quick brown fox') To remove double space for ocr

⁠FOXEYE WORKER

Runs on EC2-docker AWS instance

⁠Foxeye process description:

  1. the rabbitMQ consummed is "foxeye_master"
  2. The worker consumes this element from the queue :
}
  "docid": O,

  "key": "media/10038_02_24.03.2005.pdf",

  "bucket": "stackafront-static",

  "conv_key": "string",

  "conv_bucket": "string",

  "version": "caseip",

  "translate": true
}
  1. Download of the file from key+bucket
  2. Foxeye processing of this file :
report, dlf,  conv_pdf_path = foxeye(req["docid"], "./file_to_process", translate=req["translate"], version=req["version"],debug=True)
  1. Push of the produced data towards POSTGRES-DB
  2. Acknowledgment sent to rabbit so that it can delete the element from the queue

⁠How to use me:

⁠Build :
docker build -t stackateam/foxeye_worker:0.5 . --rm
⁠Run :
docker run --env-file env.list stackateam/foxeye_worker:<tag_version>
⁠env.list Format:

The file "env.list" should contain the following variables:

AWS_SECRET_ACCESS_KEY=
AWS_ACCESS_KEY_ID=
RABBIT_QUEUE=
RABBIT_USER=
RABBIT_PASSWORD=
AMPQ_URL=
HEROKU_API_TOKEN=
HEROKU_API=

⁠Process description

alt text

Tag summary

Content type

Image

Digest

Size

831.5 MB

Last updated

over 6 years ago

docker pull stackateam/foxeye_worker