Sign inSign up

ryanmaclean/esrally

By ryanmaclean

•Updated over 8 years ago

Elastic's ES Rally Elasticsearch benchmarking tool to be run against a remote ES node/cluster

Image
2

10K+

ryanmaclean/esrally repository overview

⁠esrally-docker

CircleCI

An Elastic ES Rally Docker container image that allows you to run ESRally in a container and point it at a remote Elasticsearch host/cluster in order to run and re-run benchmarks.

Based on the work found here: https://github.com/honestbee/docker-esrally⁠ but much less grandiose in scope, and more automated.

It does a good job of punishing smaller clusters, here's a sample from the Elastic Cloud trial cluster:

Elastic Cloud Dashboard Elastic Cloud Dashboard

⁠ES Rally in Docker

This docker container is expected to be used as a remote target benchmark.

Warning: it will download a 250MB+ file and expand it to 3.30GB. It will ingest this data into the Elasticsearch cluster that you point it at.

As the disclaimer will warn you:

************************************************************************
************** WARNING: A dark dungeon lies ahead of you  **************
************************************************************************

⁠More Info About ES Rally

This project came about after watching the fantastic ES Rally intro video from Daniel Mitterdorfer: https://www.youtube.com/watch?v=HriBY2zoChw⁠

The ES Rally docs can be found here: http://esrally.readthedocs.io/en/latest/⁠

⁠Usage

There's are three ENV variables, one PIPELINE, which defaults to --pipeline=benchmark-only, an ENDPOINT var that defaults to 192.168.99.100:9200 for Kitematic use, but you can over-ride it as follows:

docker run -it -e ENDPOINT="54.32.263.2:9200" ryanmaclean/esrally:latest

Another is CREDENTIALS, and is left blank, but may be needed when testing X-Pack-enabled clusters. That can be done as follows:

docker run -it -e CREDENTIALS=--client-options="basic_auth_user:'user',basic_auth_password:'password'" ryanmaclean/esrally:latest

This assumes that 54.32.263.2 is the primary endpoint of your cluster, and that you're using 9200 as the Elasticsearch port.

⁠Usage with Elastic Cloud

Elastic Cloud has a remote server connection and security as well as certs enabled, so our string gets a bit long. On top of this, if you're using the trial, the budget allocated for two weeks means that the cluster will be yellow, requiring an final third CLUSTERHEALTH Docker ENV var that translates to an esrally parameter which will allow the test to continue.

For example:

docker run -it \
   - e CREDENTIALS=\
      --client-options=\
         use_ssl:true,\
         verify_certs:true,\
         basic_auth_user:'elastic',\
         basic_auth_password:'lkjsadioioasdlkjas822'
   -e ENDPOINT=\
      0983209823883892039265cd8046e4.us-west-1.aws.found.io:9243
   -e CLUSTERHEALTH=\
      --cluster-health=yellow\
   ryanmaclean/esrally:latest

⁠Sample Output

This is from a terminal session running against the official Elastic Docker container for version 5.4 on a Mid 2015 maxed-out Macbook Pro (not pretty).

It took quite a while to run, and you can see that the boot2docker host should have been tweaked beforehand, but this should be a good baseline of a badly tuned cluster.

    ____        ____
   / __ \____ _/ / /_  __
  / /_/ / __ `/ / / / / /
 / _, _/ /_/ / / / /_/ /
/_/ |_|\__,_/_/_/\__, /
                /____/

INFO:rally.config:Upgrading configuration from version [6] to [8].
INFO:rally.config:Creating a backup of the current config file at [/root/.rally/rally.ini].
INFO:rally.config:Successfully self-upgraded configuration to version [8]
[INFO] Writing logs to /root/.rally/logs/rally_out_20170527T222757Z.log
Cloning into '/root/.rally/benchmarks/tracks/default'...
remote: Counting objects: 1057, done.
remote: Total 1057 (delta 0), reused 0 (delta 0), pack-reused 1057
Receiving objects: 100% (1057/1057), 755.53 KiB | 1.46 MiB/s, done.
Resolving deltas: 100% (799/799), done.
Checking connectivity... done.

************************************************************************
************** WARNING: A dark dungeon lies ahead of you  **************
************************************************************************

Rally does not have control over the configuration of the benchmarked
Elasticsearch cluster.

Be aware that results may be misleading due to problems with the setup.
Rally is also not able to gather lots of metrics at all (like CPU usage
of the benchmarked cluster) or may even produce misleading metrics (like
the index size).

************************************************************************
****** Use this pipeline only if you are aware of the tradeoffs.  ******
*************************** Watch your step! ***************************
************************************************************************

[INFO] Racing on track [geonames], challenge [append-no-conflicts] and car [external]

[INFO] Downloading data from [http://benchmarks.elasticsearch.org.s3.amazonaws.com/corpora/geonames/documents-2.json.bz2] (252 MB) to [/root/.rally/benchmarks/data/geonames/documents-2.json.bz2] ... [OK]
[INFO] Decompressing track data from [/root/.rally/benchmarks/data/geonames/documents-2.json.bz2] to [/root/.rally/benchmarks/data/geonames/documents-2.json] (resulting size: 3.30 GB) ... [OK]
[INFO] Preparing file offset table for [/root/.rally/benchmarks/data/geonames/documents-2.json] ... [OK]
Running index-append                                                           [100% done]
Running force-merge                                                            [100% done]
Running index-stats                                                            [100% done]
Running node-stats                                                             [100% done]
Running default                                                                [100% done]
Running term                                                                   [100% done]
Running phrase                                                                 [100% done]
Running country_agg_uncached                                                   [100% done]
Running country_agg_cached                                                     [100% done]
Running scroll                                                                 [100% done]
Running expression                                                             [100% done]
Running painless_static                                                        [100% done]
Running painless_dynamic                                                       [100% done]
Running large_terms                                                            [100% done]
Running large_filtered_terms                                                   [100% done]
Running large_prohibited_terms                                                 [100% done]

------------------------------------------------------
    _______             __   _____
   / ____(_)___  ____ _/ /  / ___/_________  ________
  / /_  / / __ \/ __ `/ /   \__ \/ ___/ __ \/ ___/ _ \
 / __/ / / / / / /_/ / /   ___/ / /__/ /_/ / /  /  __/
/_/   /_/_/ /_/\__,_/_/   /____/\___/\____/_/   \___/
------------------------------------------------------
LapMetricOperationValueUnit
AllIndexing time41.7569min
AllMerge time39.541min
AllRefresh time15.867min
AllFlush time0.0291167min
AllMerge throttle time0.995683min
AllTotal Young Gen GC525.372s
AllTotal Old Gen GC12.543s
AllHeap used for segments17.9723MB
AllHeap used for doc values0.11013MB
AllHeap used for terms16.7452MB
AllHeap used for norms0.0749512MB
AllHeap used for points0.213588MB
AllHeap used for stored fields0.828438MB
AllSegment count102
AllMin Throughputindex-append4045.99docs/s
AllMedian Throughputindex-append4200.45docs/s
AllMax Throughputindex-append4817.98docs/s
All50.0th percentile latencyindex-append8832.61ms
All90.0th percentile latencyindex-append13494.6ms
All99.0th percentile latencyindex-append19720.5ms
All99.9th percentile latencyindex-append22807ms
All100th percentile latencyindex-append23539.8ms
All50.0th percentile service timeindex-append8832.61ms
All90.0th percentile service timeindex-append13494.6ms
All99.0th percentile service timeindex-append19720.5ms
All99.9th percentile service timeindex-append22807ms
All100th percentile service timeindex-append23539.8ms
Allerror rateindex-append0%
AllMin Throughputforce-merge3.30106ops/s
AllMedian Throughputforce-merge3.30106ops/s
AllMax Throughputforce-merge3.30106ops/s
All100th percentile latencyforce-merge302.75ms
All100th percentile service timeforce-merge302.75ms
Allerror rateforce-merge0%
AllMin Throughputindex-stats99.9312ops/s
AllMedian Throughputindex-stats100.049ops/s
AllMax Throughputindex-stats100.093ops/s
All50.0th percentile latencyindex-stats4.56831ms
All90.0th percentile latencyindex-stats5.37362ms
All99.0th percentile latencyindex-stats10.4568ms
All99.9th percentile latencyindex-stats16.8071ms
All100th percentile latencyindex-stats20.6186ms
All50.0th percentile service timeindex-stats4.2707ms
All90.0th percentile service timeindex-stats4.89174ms
All99.0th percentile service timeindex-stats9.63112ms
All99.9th percentile service timeindex-stats16.1339ms
All100th percentile service timeindex-stats20.4829ms
Allerror rateindex-stats0%
AllMin Throughputnode-stats99.7653ops/s
AllMedian Throughputnode-stats100.085ops/s
AllMax Throughputnode-stats100.535ops/s
All50.0th percentile latencynode-stats4.55892ms
All90.0th percentile latencynode-stats6.2264ms
All99.0th percentile latencynode-stats85.0515ms
All99.9th percentile latencynode-stats153.335ms
All100th percentile latencynode-stats159.538ms
All50.0th percentile service timenode-stats4.22034ms
All90.0th percentile service timenode-stats5.08135ms
All99.0th percentile service timenode-stats10.5145ms
All99.9th percentile service timenode-stats31.6439ms
All100th percentile service timenode-stats159.403ms
Allerror ratenode-stats0%
AllMin Throughputdefault35.7368ops/s
AllMedian Throughputdefault38.1349ops/s
AllMax Throughputdefault38.7257ops/s
All50.0th percentile latencydefault6194.75ms
All90.0th percentile latencydefault8757.89ms
All99.0th percentile latencydefault9139.22ms
All99.9th percentile latencydefault9168.4ms
All100th percentile latencydefault9176.96ms
All50.0th percentile service timedefault23.1244ms
All90.0th percentile service timedefault32.4216ms
All99.0th percentile service timedefault42.7273ms
All99.9th percentile service timedefault47.2948ms
All100th percentile service timedefault47.3488ms
Allerror ratedefault0%
AllMin Throughputterm199.627ops/s
AllMedian Throughputterm199.953ops/s
AllMax Throughputterm200.106ops/s
All50.0th percentile latencyterm3.45829ms
All90.0th percentile latencyterm9.61196ms
All99.0th percentile latencyterm25.0415ms
All99.9th percentile latencyterm30.6498ms
All100th percentile latencyterm31.8718ms
All50.0th percentile service timeterm2.67088ms
All90.0th percentile service timeterm3.56639ms
All99.0th percentile service timeterm17.2176ms
All99.9th percentile service timeterm24.8637ms
All100th percentile service timeterm26.781ms
Allerror rateterm0%
AllMin Throughputphrase181.584ops/s
AllMedian Throughputphrase193.872ops/s
AllMax Throughputphrase199.885ops/s
All50.0th percentile latencyphrase165.916ms
All90.0th percentile latencyphrase272.091ms
All99.0th percentile latencyphrase293.071ms
All99.9th percentile latencyphrase297.487ms
All100th percentile latencyphrase299.297ms
All50.0th percentile service timephrase3.25947ms
All90.0th percentile service timephrase7.3253ms
All99.0th percentile service timephrase25.5142ms
All99.9th percentile service timephrase32.5812ms
All100th percentile service timephrase36.5469ms
Allerror ratephrase0%
AllMin Throughputcountry_agg_uncached1.10785ops/s
AllMedian Throughputcountry_agg_uncached1.24299ops/s
AllMax Throughputcountry_agg_uncached1.29339ops/s
All50.0th percentile latencycountry_agg_uncached605116ms
All90.0th percentile latencycountry_agg_uncached805617ms
All99.0th percentile latencycountry_agg_uncached854591ms
All99.9th percentile latencycountry_agg_uncached859400ms
All100th percentile latencycountry_agg_uncached859942ms
All50.0th percentile service timecountry_agg_uncached703.377ms
All90.0th percentile service timecountry_agg_uncached741.63ms
All99.0th percentile service timecountry_agg_uncached800.875ms
All99.9th percentile service timecountry_agg_uncached870.371ms
All100th percentile service timecountry_agg_uncached894.24ms
Allerror ratecountry_agg_uncached0%
AllMin Throughputcountry_agg_cached100.026ops/s
AllMedian Throughputcountry_agg_cached100.063ops/s
AllMax Throughputcountry_agg_cached100.141ops/s
All50.0th percentile latencycountry_agg_cached3.57384ms
All90.0th percentile latencycountry_agg_cached4.93876ms
All99.0th percentile latencycountry_agg_cached16.0598ms
All99.9th percentile latencycountry_agg_cached22.5004ms
All100th percentile latencycountry_agg_cached26.4915ms
All50.0th percentile service timecountry_agg_cached3.15232ms
All90.0th percentile service timecountry_agg_cached4.06166ms
All99.0th percentile service timecountry_agg_cached15.8608ms
All99.9th percentile service timecountry_agg_cached22.1849ms
All100th percentile service timecountry_agg_cached26.2606ms
Allerror ratecountry_agg_cached0%
AllMin Throughputscroll39.7068ops/s
AllMedian Throughputscroll40.5957ops/s
AllMax Throughputscroll41.0411ops/s
All50.0th percentile latencyscroll259404ms
All90.0th percentile latencyscroll371192ms
All99.0th percentile latencyscroll395720ms
All100th percentile latencyscroll398442ms
All50.0th percentile service timescroll598.62ms
All90.0th percentile service timescroll625.847ms
All99.0th percentile service timescroll651.197ms
All100th percentile service timescroll658.699ms
Allerror ratescroll0%
AllMin Throughputexpression0.67594ops/s
AllMedian Throughputexpression0.678439ops/s
AllMax Throughputexpression0.680025ops/s
All50.0th percentile latencyexpression293177ms
All90.0th percentile latencyexpression369571ms
All99.0th percentile latencyexpression386814ms
All100th percentile latencyexpression388714ms
All50.0th percentile service timeexpression1459.99ms
All90.0th percentile service timeexpression1492.65ms
All99.0th percentile service timeexpression1522.07ms
All100th percentile service timeexpression1527.01ms
Allerror rateexpression0%
AllMin Throughputpainless_static0.495775ops/s
AllMedian Throughputpainless_static0.514832ops/s
AllMax Throughputpainless_static0.519771ops/s
All50.0th percentile latencypainless_static433935ms
All90.0th percentile latencypainless_static570563ms
All99.0th percentile latencypainless_static603637ms
All100th percentile latencypainless_static607317ms
All50.0th percentile service timepainless_static2003.61ms
All90.0th percentile service timepainless_static2512.4ms
All99.0th percentile service timepainless_static2795.82ms
All100th percentile service timepainless_static2993.33ms
Allerror ratepainless_static0%
AllMin Throughputpainless_dynamic0.463116ops/s
AllMedian Throughputpainless_dynamic0.471679ops/s
AllMax Throughputpainless_dynamic0.488911ops/s
All50.0th percentile latencypainless_dynamic487720ms
All90.0th percentile latencypainless_dynamic629961ms
All99.0th percentile latencypainless_dynamic657571ms
All100th percentile latencypainless_dynamic660803ms
All50.0th percentile service timepainless_dynamic2132.71ms
All90.0th percentile service timepainless_dynamic2692.83ms
All99.0th percentile service timepainless_dynamic3210.24ms
All100th percentile service timepainless_dynamic3589.56ms
Allerror ratepainless_dynamic0%
AllMin Throughputlarge_termsops/s
AllMedian Throughputlarge_termsops/s
AllMax Throughputlarge_termsops/s
All50.0th percentile latencylarge_terms203.65ms
All90.0th percentile latencylarge_terms368.147ms
All99.0th percentile latencylarge_terms567.241ms
All100th percentile latencylarge_terms906.133ms
All50.0th percentile service timelarge_terms201.502ms
All90.0th percentile service timelarge_terms362.732ms
All99.0th percentile service timelarge_terms517.201ms
All100th percentile service timelarge_terms905.241ms
Allerror ratelarge_terms100%
AllMin Throughputlarge_filtered_terms0.461976op

Tag summary

Content type

Image

Digest

Size

145.6 MB

Last updated

over 8 years ago

docker pull ryanmaclean/esrally