retzkek/condorbeat repository overview

⁠Condorbeat

Condorbeat is a tool to collect raw ClassAds from HTCondor on a defined period and publish them to a variety of outputs, including Kafka, Logstash, and Elasticsearch. Supported ClassAds include:

• Curent jobs in the queue (condor_q)
• Completed jobs (condor_history)
• Machine, daemons, etc (condor_status)

⁠Running

The fastest way to get condorbeat running is with Docker, using the retzkek/condorbeat⁠ image off Docker Hub and bind-mount in your configuration file:

docker pull retzkek/condorbeat
docker run -v /path/to/condorbeat.yml:/condorbeat/condorbeat.yml retzkek/condorbeat

See instructions below for building the binary from source otherwise.

⁠Configuration

Condorbeat-specific options are under the condorbeat section. For all other options see the Beats documentation⁠

• period time between ClassAd collection in Go time.Duration format⁠ (default 68s).
• pool HTCondor collector/central manager to query. Leave blank for local machine (default).
• checkpoint_file name of file to store checkpoints (last data collection) under data directory (default checkpoints).

⁠Queue

Current jobs in the queue (condor_q).

• classads collect job classads (default true).

⁠History

Collect completed jobs (condor_history). Note that condor_history is not very efficient at collecting "jobs completed since time t" so it's preferable to use filebeat directly on the schedd to collect the history spool file in realtime.

• classads collect job classads (default true).
• limit maximum number of job histories to collect during each period.

⁠Status

Machine, daemon, and any other classads (condor_status). Provide a list of ad types to collect. Default:

- type: Collector
- type: Scheduler
- type: Negotiator

• type daemon/classAd type (MyType).
• constraint optional constraint to apply

⁠Building

To build a stand-alone executable for your current platform run make.

TODO: upgrade to libbeat 6.5+ to take advantage of the better cross-platform release build.