apache/otava repository overview

⁠Apache Otava – Change Detection for Continuous Performance Engineering

Apache Otava (incubating) performs statistical analysis of performance test results stored in CSV files, PostgreSQL, BigQuery, or Graphite database. It finds change-points and notifies about possible performance regressions.

A typical use-case of otava is as follows:

• A set of performance tests is scheduled repeatedly, such as after each commit is pushed.
• The resulting metrics of the test runs are stored in a time series database (Graphite) or appended to CSV files.
• Otava is launched by a Jenkins/Cron job (or an operator) to analyze the recorded metrics regularly.
• Otava notifies about significant changes in recorded metrics by outputting text reports or sending Slack notifications.

Otava is capable of finding even small, but persistent shifts in metric values, despite noise in data. It adapts automatically to the level of noise in data and tries to notify only about persistent, statistically significant changes, be it in the system under test or in the environment.

Unlike in threshold-based performance monitoring systems, there is no need to setup fixed warning threshold levels manually for each recorded metric. The level of accepted probability of false-positives, as well as the minimal accepted magnitude of changes are tunable. Otava is also capable of comparing the level of performance recorded in two different git histories. This can be used for example to validate a feature branch against the main branch, perhaps integrated with a pull request.

See the documentation in https://otava.apache.org/docs/overview/⁠.

⁠Quick Start

• Pull the image:

docker pull apache/otava:latest

• Create a configuration file (see Configuration⁠)

• Run analysis:

docker run \
    -v /path/to/config:/config \
    -v /path/to/data:/data \
    -e OTAVA_CONFIG=/config/otava.yaml \
  apache/otava:latest analyze <test-name>

⁠Available Tags

• apache/otava:latest - Latest stable release
• apache/otava:<version> - Specific version (e.g., apache/otava:0.7.0)

To see all available tags, visit the Apache Otava Docker Hub page⁠.

⁠Configuration

Before running Apache Otava, you need to create a configuration file that defines your data sources and tests. The configuration file should be mounted into the container.

⁠Minimal Configuration Example

Create an otava.yaml file:

tests:
  local.sample:
    type: csv
    file: /data/local_sample.csv
    time_column: time
    attributes: [commit]
    metrics: [metric1, metric2]
    csv_options:
      delimiter: ","
      quotechar: "'"

For detailed configuration options, see the Getting Started Documentation⁠ and configuration examples⁠.

⁠Usage Examples

⁠Analyzing CSV Data

# List available tests
docker run \
    -v $(pwd)/config:/config \
    -v $(pwd)/data:/data \
    -e OTAVA_CONFIG=/config/otava.yaml \
  apache/otava:latest list-tests

# Run analysis on a specific test
docker run \
    -v $(pwd)/config:/config \
    -v $(pwd)/data:/data \
    -e OTAVA_CONFIG=/config/otava.yaml \
  apache/otava:latest analyze local.sample

⁠Analyzing Database Data

For PostgreSQL or other database sources:

docker run --network host \
  -v $(pwd)/config:/config \
  -e OTAVA_CONFIG=/config/otava.yaml \
  apache/otava:latest analyze database-test

⁠Docker Compose

For more complex setups, use Docker Compose:

services:
  otava:
    image: apache/otava:latest
    environment:
      OTAVA_CONFIG: /config/otava.yaml
    volumes:
      - ./config:/config:ro
      - ./data:/data:ro
    command: analyze my-test

Run with: docker-compose run otava

⁠Environment Variables

• OTAVA_CONFIG - Path to the configuration file inside the container (required)

⁠Volume Mounts

Common volume mount patterns:

• Configuration: -v /host/path/to/config:/config:ro - Mount configuration directory as read-only
• Data: -v /host/path/to/data:/data:ro - Mount data directory as read-only

⁠Getting Help

• View available commands: docker run apache/otava:latest --help
• Command-specific help: docker run apache/otava:latest analyze --help
• Documentation: https://otava.apache.org/docs/overview/⁠
• Examples: https://github.com/apache/otava/tree/master/examples⁠
• Issues: https://github.com/apache/otava/issues⁠
• Mailing list: [email protected]⁠
• Slack: #otava channel in official ASF Slack. Sign up for an ASF Slack account⁠.

⁠License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

⁠Disclaimer

Apache Otava (incubating) is an effort undergoing incubation at the Apache Software Foundation (ASF), sponsored by the Apache Incubator PMC.

Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects.

While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.