mavroudo/siesta-query repository overview

⁠Overview

The SIESTA Query Processor is a component of the SIESTA (Scalable Infrastructure for Sequential Pattern Analysis) project. It is designed to efficiently handle pattern queries using indices built by the Preprocess Component. This application supports functionalities such as Pattern Detection, Pattern Continuation, and Basic Statistics, with additional methods like Signature and Set-Containment.

---

⁠Environmental Variables

When running the SIESTA Query Processor via Docker, several environment variables need to be set. These variables configure the application's connection to databases and other critical settings:

• master.uri: Specifies the URI for the Spark master. Use local[4] for four threads or local[*] to use all available threads.

• database: Determines the database type. Options include cassandra-rdd for Cassandra or s3 for S3.

  For S3 (MinIO) Configuration:

  • s3.endpoint: The endpoint URL for the S3 storage (e.g., http://minio:9000).
  • s3.user: The username for accessing the S3 storage.
  • s3.key: The password or key for accessing the S3 storage.
  • s3.timeout: The timeout setting for S3 operations in milliseconds.

  For Cassandra Configuration:

  • cassandra.max_requests_per_local_connection: Maximum requests per local connection.
  • cassandra.max_requests_per_remote_connection: Maximum requests per remote connection.
  • cassandra.connections_per_host: Number of connections per host.
  • cassandra.max_queue_size: Maximum queue size for requests.
  • cassandra.connection_timeout: Connection timeout in milliseconds.
  • cassandra.read_timeout: Read timeout in milliseconds.
  • spring.data.cassandra.contact-points: Contact points for connecting to Cassandra.
  • spring.data.cassandra.port: Port for connecting to Cassandra.
  • spring.data.cassandra.user: Username for Cassandra authentication.
  • spring.data.cassandra.password: Password for Cassandra authentication.

• server.port: Port on which the application will run (default is 8090).

---

⁠Volumes

To cache dependencies and improve build times, the following volume can be mounted to avoid re-downloading the dependencies every time the image is build:

• ./build:/root/.m2: Maps the local ./build directory to the Maven repository inside the container.

---

⁠Ports

Expose the application's port to the host:

• 8080:8080: Maps port 8080 on the container to port 8080 on the host.

---

⁠SIESTA Query Types and Endpoints

• /health: Checks if the application is up and running.
• /lognames: Returns the names of the different log databases.
• /eventTypes: Returns the names of different event types for a specific log database.
• /refreshData: Reloads metadata after a new log file is appended.
• /metadata: Returns metadata for a specific log database.
• /stats: Returns basic statistics for each consecutive event-pair in the query pattern.
• /detection: Returns traces that contain an occurrence of the query pattern.
• /explore: Returns the most probable continuation of the query pattern with different modes of operation (fast, hybrid, accurate).
• /set-containment/detect: Accesses the detection mechanisms using the Set-Containment method (Cassandra only).
• /signatures/detect: Accesses the detection mechanisms using the Signature method (Cassandra only).

---

⁠Documentation and Testing

For comprehensive documentation and testing of the endpoints, Swagger is integrated and can be accessed at the /swagger-ui/ endpoint. This interface provides a complete list of available endpoints and allows users to test them easily.