Sign inSign up

thescobber/ims-processor

By thescobber

•Updated about 1 year ago

Image
0

8.5K

thescobber/ims-processor repository overview

⁠IMS

⁠Environment Variables
NameFunctionModule
COSMOS_MYSQL_USERDatabase UsernameAll
COSMOS_MYSQL_PASSDatabase PasswordAll
COSMOS_MYSQL_HOSTDatabase HostnameAll
COSMOS_MYSQL_DBDatabase Default CatalogAll
COSMOS_MQTT_BROKERM2M Event ChannelAll
COSMOS_EMAIL_HOSTIMAP Email ServerEmail
COSMOS_EMAIL_USERIMAP Email UsernameEmail
COSMOS_EMAIL_PASSIMAP Email PasswordEmail

Completed / Betatesting Modules:

⁠importer.py
⁠Used to import generic scanned documents into the array

Creates file pointers and a 'book' pointer in the necessary tables. A book is one or more pages scanned into the system. Actions are executed against books or images in the system independent of eachother. for instance, file actions executed on a image will not affect any processing at a book level. Images can also be multi-homed into books.

⁠Usage
Command Line
docker run -it --entrypoint "python3 /app/fulltextindex.py" thescobber/ims-processor:latest

Stack Component  
version: '3.2'
services:
  importer2:
    image: thescobber/ims-processor
    environment:
      COSMOS_MYSQL_USER: ${COSMOS_MYSQL_USER}
      COSMOS_MYSQL_PASS: ${COSMOS_MYSQL_PASSWORD}
      COSMOS_MYSQL_DB: ${COSMOS_MYSQL_DB}
      COSMOS_MYSQL_HOST: ${COSMOS_MYSQL_HOST}
      COSMOS_MQTT_BROKER: ${COSMOS_MQTT_BROKER}
      PYTHONUNBUFFERED: 1
    entrypoint: ["python3", "/app/importer.py"]
    networks:
      - cosmos-net
    logging:
      driver: local
    deploy:
      mode: global
networks:
  cosmos-net:
    driver: overlay
    attachable: true
    ipam:
      config:
        - subnet: 25.30.0.0/24

⁠fulltextindex.py

Creates a index of every image file hosted in the production/imagefile database This is responsible for crawling the index, and providing the return data back to the database. The file is marked in the ftidone field with a 1 when successful.

⁠Usage
Command Line
docker run -it --entrypoint "python3 /app/fulltextindex.py" thescobber/ims-processor:latest

Stack Component  
version: '3.2'
services:
  ftinode2:
    image: thescobber/ims-processor
    environment:
      COSMOS_MYSQL_USER: ${COSMOS_MYSQL_USER}
      COSMOS_MYSQL_PASS: ${COSMOS_MYSQL_PASSWORD}
      COSMOS_MYSQL_DB: ${COSMOS_MYSQL_DB}
      COSMOS_MYSQL_HOST: ${COSMOS_MYSQL_HOST}
      COSMOS_MQTT_BROKER: ${COSMOS_MQTT_BROKER}
      PYTHONUNBUFFERED: 1
    entrypoint: ["python3", "/app/fulltextindex.py"]
    networks:
      - cosmos-net
    logging:
      driver: local
    deploy:
      mode: global
networks:
  cosmos-net:
    driver: overlay
    attachable: true
    ipam:
      config:
        - subnet: 25.30.0.0/24

⁠emailimport.py

Downloads email from a IMAP email server database table Imagetask

Attributevalue
tasktask name can be anything
subtaskemailimport
galaxyEML
pathimap path to scrape
arcpathimap path for archival
outputpathdisk path to place attachments
arcaction0-1
enabled0-1
debug0-1
unpack0-1
actiondlattachement
priority0-255
⁠Usage
Command Line
docker run -it --entrypoint "python3 /app/emailimport.py" thescobber/ims-processor:latest

Stack Component  
version: '3.2'
services:
  emailnode2:
    image: thescobber/ims-processor
    environment:
      COSMOS_MYSQL_USER: ${COSMOS_MYSQL_USER}
      COSMOS_MYSQL_PASS: ${COSMOS_MYSQL_PASSWORD}
      COSMOS_MYSQL_DB: ${COSMOS_MYSQL_DB}
      COSMOS_MYSQL_HOST: ${COSMOS_MYSQL_HOST}
      COSMOS_EMAIL_HOST: ${COSMOS_EMAIL_HOST}
      COSMOS_EMAIL_USER: ${COSMOS_EMAIL_USER}
      COSMOS_EMAIL_PASS: ${COSMOS_EMAIL_PASS}
      COSMOS_MQTT_BROKER: ${COSMOS_MQTT_BROKER}
      PYTHONUNBUFFERED: 1
    entrypoint: ["python3", "/app/emailimport.py"]
    networks:
      - cosmos-net
    logging:
      driver: local
    deploy:
      mode: global
networks:
  cosmos-net:
    driver: overlay
    attachable: true
    ipam:
      config:
        - subnet: 25.30.0.0/24

⁠Database Defs

⁠imagetask
RowVarUsageexamples
attributecompileCompiles input pdf as a book
attributeflatIgnores building a book, and imports each image flat

Tag summary

Content type

Image

Digest

Size

317.5 MB

Last updated

over 5 years ago

docker pull thescobber/ims-processor