traxair/logstash repository overview

⁠TraxIT

⁠Creating new microservices

Go to the skeletons directory⁠ and select the microservice type you want to create

(Django REST microservice, Celery microservice, ...). Instructions are located inside each directory.

⁠Code Checks

⁠Build Statuses

Microservice	Status
Base
base_gpu
ci-slave
ci-slave-acceptance
identifications
youtube_insight
ingestion
api_gateway
claim
claims
compute_url_from_semantic
compute_waveform_from_shared_mem
decode
decode_from_blob_to_shared_mem
decode_from_uri_to_shared_mem
extended_content_id
fingerprint
historun
identify_fingerprinting_streaming
match_candidates
match_scores
microtracks
picture_uploader
post_process
store_tracklist
tracklist_manager
traxitpro_video_creator
url_resolver
waveform_manager
fingerprint_from_blob_to_es
fingerprint_from_shared_mem_to_blob
fingerprint_from_shared_mem_to_shared_mem
traxit_services
semantic_labelling
ci-slave
traxitpro
video_maker

config.json:

{
  "packages": [
    {
      "pkg_type": "whl",
      "pkg_name": "traxit_core",
      "test_type": "pytest",
      "extra_args": "--runslow",
      "exclude_dependencies_on": ["BLD"],
      "type": "celery-microservice",
      "components": {
        "elasticsearch":false,
        "mongo":false,
        "redis":false
      }
    },
    ...
  ]
}

When you want to update the config.json you have to update the packages and sometimes the environment or integration test parts.

##Environment Some tests need environment variables for running, you can indicate here this variables

##Packages

A package in the config.json is represent by a json object, who contains:

• pkg_type: Determine what is the package type to build. If not present, the package will not be built and we suppose there is no app directory.
• pkg_name: the package name in the github root and the output package name
• test_type: What is the test to execute for this package. If no present, no testing phase will be executed. This is useful for docker base images
• extra_args (optional): extra arguments for pytest like runslow
• exclude_dependencies_only_on_keywords (optional): if this set to ['BLD', 'CLN'] and all commits' message in a pull request start with BLD: or CLN: or even BLD/CLN:, then exclude this package's dependencies if they were not modified. This is case-sensitive.
• type: Type is either a skeleton name ('celery-microservices') or the term 'skeleton' for skeletons.
• components (optional): Determine what components is nececessary for the test (if it not indicate, all components are set to true)

pkg_type can be set to:

• microservice: container docker
• deb: debian package
• whl: wheel package
• browser_extension: firefox extension (xpi) and chrome extension (zip)

test_type can be set:

• make: for code managed with a Makefile. The Makefile must have the ci method.
• pytest: Deprecated soon.
• grunt: Deprecated soon.
• gulp: Deprecated soon.

components have 3 keys:

• elasticsearch: can be set to true or false
• mongo: can be set to true or false
• redis: can be set to true or false If a component must be start for running test set it to true