[{"_1":2,"_174":-5,"_175":-5},"loaderData",{"_3":4,"_17":18,"_25":26,"_150":151},"root",{"_5":-5,"_6":-7,"_7":8,"_9":-7,"_10":11,"_12":-7,"_13":14,"_15":16},"currentUser","primaryEmail","siteVerification","6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI","toast","enableDev",false,"incidentStatus","theme","system","contrast","normal","routes/_layout",{"_19":-5,"_20":21,"_22":21,"_23":11,"_24":21},"username","isAskGordonEnabled",true,"isUsingNewTemplate","showMinimalLanding","showHoneypotTrap","routes/_layout.r.$namespace.$reponame",{"_27":28,"_29":-5,"_30":11,"_31":11,"_32":33,"_34":35,"_36":-5,"_37":11,"_38":39,"_98":-5,"_99":100,"_139":-5,"_140":11,"_141":21,"_142":11,"_143":-5,"_144":11,"_145":-5,"_146":147,"_148":101,"_149":-7},"canonicalUrl","https://hub.docker.com/r/johanostberg/neoloadcompare/tags","mcpServer","canSeeDockerfile","canSeeAutobuilds","namespace","johanostberg","reponame","neoloadcompare","cloudBuildSource","isAutobuildsAvailable","repo",{"_40":41,"_53":54,"_56":57,"_58":59,"_60":11,"_61":62,"_63":64,"_65":66,"_43":35,"_32":33,"_68":69,"_70":55,"_71":72,"_73":74,"_75":76,"_77":-5,"_78":72,"_79":80,"_81":11,"_82":33,"_83":84,"_89":11,"_90":91,"_95":33,"_96":97},"categories",[42,47,50],{"_43":44,"_45":46},"name","Integration & delivery","slug","integration-and-delivery",{"_43":48,"_45":49},"Developer tools","developer-tools",{"_43":51,"_45":52},"Monitoring & observability","monitoring-and-observability","content_types",[55],"image","date_registered","2024-02-07T05:37:22.949082Z","description","NeoLoadCompare compares performance test results against a baseline using the NeoLoad Web API","is_private","last_updated","2024-11-15T13:28:01.626219Z","last_modified","2024-11-15T14:52:30.294107Z","media_types",[67],"application/vnd.docker.container.image.v1+json","pull_count",317,"repository_type","star_count",0,"status",1,"status_description","active","source","collaborator_count","full_description","

NeoloadCompare

\n\n

NeoloadCompare is a powerful tool designed to automatically compare performance test results against a predefined baseline using the NeoLoad Web API. This tool ensures your system's performance remains consistent with expectations by analyzing key metrics such as average response times and percentiles.

\n\n

Features

Performance Comparison: Compares average response times and percentiles (e.g., 95th percentile) between the latest test and a baseline.
Transaction Filtering: Ignores \"init\" transactions and focuses on transactions in the \"actions\" part of the user flow.
Pipeline Integration: Run it after a performance test in a CI/CD pipeline.
Failing Criteria: Fails the test if any transaction's response time deviates more than a set percentage from the baseline.

\n\n

Usage

After completing your performance test, run NeoloadCompare to automatically compare the results against a predefined baseline.

\n\n

API Parameters

workspace: The name of the workspace.
BaselineTest: The test ID of the baseline to compare against.
scenario: The name of the scenario being run.
element: The performance metric to compare (e.g., avgDuration, percentile90, percentile95, percentile99). Use false if no comparison is needed.
percentage: The maximum allowable deviation percentage from the baseline before the test fails.

\n\n

Example Request

http://your_endpoint/NeoLoadCompare?workspace=demo&scenario=demo_scenario&baseline=1&percentage=10&element=avgduration

\n\n

Example Responses

\n\n

200 OK (Successful Comparison)

{\n    \"workspace\": {\n        \"name\": \"demo\",\n        \"id\": \"62568bb580cec421c189ff01\"\n    },\n    \"comparison\": {\n        \"percentageDifference\": \"1\",\n        \"element\": \"avgDuration\"\n    },\n    \"latestTest\": {\n        \"name\": \"#11\",\n        \"testId\": \"4cc0ccc3-f3f2-4325-b201-9f5f68607a61\"\n    },\n    \"baseline\": {\n        \"name\": \"6\",\n        \"testId\": \"080a9251-c653-4279-abd3-7fc5e17a54ad\"\n    },\n    \"status\": \"Check against baseline OK!\"\n}

\n\n

Explanation of Fields

workspace: The workspace details.
comparison: Shows the percentage difference and the metric compared.
latestTest: The details of the latest test run.
baseline: The details of the baseline test.
status: The result of the comparison.

\n\n

500 Internal Server Error (Comparison Failed)

{\n    \"percentage\": \"50%\",\n    \"baselinetest\": \"6\",\n    \"transactions\": [\n        {\n            \"baselineValue\": \"542.0\",\n            \"latestValue\": \"869.0\",\n            \"increase\": \"60.0%\",\n            \"transaction\": \"get-stub-wiremock-faker\"\n        }\n    ],\n    \"info\": \"Test failed because the increase from baseline is bigger than the allowed percentage value\",\n    \"element\": \"avgDuration\",\n    \"currenttest\": \"#11\"\n}

\n\n

Explanation of Fields

percentage: The allowed percentage difference that was set for the test.
baselinetest: The ID of the baseline test that the current test is compared to.
transactions: A list of transactions that failed the comparison due to exceeding the allowed percentage difference.
baselineValue: The baseline value for the transaction.
latestValue: The value from the latest test for the transaction.
increase: The percentage increase between the baseline value and the latest test value.
transaction: The name or ID of the transaction that caused the failure.
info: A message explaining why the test failed.
element: The performance metric that was compared (e.g., avgDuration).
currenttest: The ID or name of the current test being compared to the baseline.

\n\n

Running with Docker

To run the application, use the following command:

\ndocker run -d \\\n  -p 8080:8080 \\\n  -e Server=neoloadapi_server \\\n  -e Token=admin_token \\\n  johanostberg/neoloadcompare:latest\n

\n\n

Additional Information

For more details and source code, visit the GitHub repository:

NeoloadCompare GitHub Repository

\n","has_starred","hub_user","immutable_tags_settings",{"_85":11,"_86":87},"enabled","rules",[88],".*","is_automated","permissions",{"_92":21,"_93":11,"_94":11},"read","write","admin","user","storage_size",572797913,"orgData","recentTags",[101],{"_102":55,"_103":104,"_105":106,"_107":108,"_109":110,"_111":112,"_61":128,"_129":104,"_130":33,"_131":67,"_43":132,"_133":134,"_135":119,"_136":128,"_137":76,"_138":21},"content_type","creator",12250181,"digest","sha256:b9dbbfbbf7ca88e71d1a4f3717c3cf05db64e0739f78274f5b901191ca4c4b82","full_size",358604349,"id",599128790,"images",[113],{"_114":115,"_105":106,"_116":117,"_118":119,"_120":121,"_122":123,"_124":117,"_125":-5,"_126":108,"_73":76,"_127":-5},"architecture","amd64","features","","last_pulled","2026-09-22T11:18:24.042775737Z","last_pushed","2024-11-15T13:28:02Z","os","linux","os_features","os_version","size","variant","2024-11-15T13:28:01.34142Z","last_updater","last_updater_username","media_type","latest","repository",23136448,"tag_last_pulled","tag_last_pushed","tag_status","v2","sbxKitMetadata","isPotentialSpam","isHubSbxKitsEnabled","showSbxLogoOverride","sbxTrademarkText","displaySecurityMetadata","tagAttestations","tagsHaveAttestations",[],"selectedTag","attestationsUrl","routes/_layout.r.$namespace.$reponame.tags",{"_38":152,"_162":163,"_146":172,"_173":11},{"_40":153,"_53":157,"_56":57,"_58":59,"_60":11,"_61":62,"_63":64,"_65":158,"_43":35,"_32":33,"_68":69,"_70":55,"_71":72,"_73":74,"_75":76,"_77":-5,"_78":72,"_79":80,"_81":11,"_82":33,"_83":159,"_89":11,"_90":161,"_95":33,"_96":97},[154,155,156],{"_43":44,"_45":46},{"_43":48,"_45":49},{"_43":51,"_45":52},[55],[67],{"_85":11,"_86":160},[88],{"_92":21,"_93":11,"_94":11},"tags",{"_164":74,"_165":-5,"_166":-5,"_167":168},"count","next","previous","results",[169],{"_102":55,"_103":104,"_105":106,"_107":108,"_109":110,"_111":170,"_61":128,"_129":104,"_130":33,"_131":67,"_43":132,"_133":134,"_135":119,"_136":128,"_137":76,"_138":21},[171],{"_114":115,"_105":106,"_116":117,"_118":119,"_120":121,"_122":123,"_124":117,"_125":-5,"_126":108,"_73":76,"_127":-5},[],"canDeleteTags","actionData","errors"]

johanostberg/neoloadcompare

Tags for johanostberg/neoloadcompare