convmind/soco-core-indexer repository overview

⁠soco-core-indexer

A task-queue based server used to index/modify data in the NES index. It is designed to be distributed for horizontal scalability.

⁠Install

pip install -r requirements.txt

⁠Run

docker-compose.yml -f docker/docker-compose.yml up --scale worker=3

⁠Configuration

• ENV REGION US
• ENV USE_GPU on
• ENV ES_URL ec2.convmind.ai:9200
• ENV MRC_URL http://lambda.tepper.cmu.edu:7777⁠
• ENV QG_URL http://lambda.tepper.cmu.edu:6312⁠
• ENV REDIS_URL redis://redis:6379/0
• ENV REDIS_QUEUES default

⁠Run Test:

pytest --cov=./
codecov --token=a5313626-6ef1-418e-9718-ee4ff3297aea

⁠Build an annotator image

docker build -t convmind/soco-core-indexer:0.14-qa -f annotators/xx/Dockerfile .