kam1kaze/netdata repository overview

⁠netdata

Netdata is distributed, real-time, performance and health monitoring for systems and applications. It is a highly optimized monitoring agent you install on all your systems and containers.

Netdata provides unparalleled insights, in real-time, of everything happening on the systems it runs (including web servers, databases, applications), using highly interactive web dashboards. It can run autonomously, without any third party components, or it can be integrated to existing monitoring tool chains (Prometheus, Graphite, OpenTSDB, Kafka, Grafana, etc).

Netdata is fast and efficient, designed to permanently run on all systems (physical & virtual servers, containers, IoT devices), without disrupting their core function.

Netdata is free, open-source software and it currently runs on Linux, FreeBSD, and MacOS.

Netdata is in the Cloud Native Computing Foundation (CNCF) landscape⁠ and it is the 3rd most starred open-source project. Check the CNCF TOC Netdata presentation⁠.

People get addicted to netdata.
Once you use it on your systems, there is no going back! You have been warned...

⁠Contents

1. How it looks⁠ - have a quick look at it
2. User base⁠ - who uses netdata?
3. Quick Start⁠ - try it now on your systems
4. Why Netdata⁠ - why people love netdata, how it compares with other solutions
5. News⁠ - latest news about netdata
6. How it works⁠ - high level diagram of how netdata works
7. infographic⁠ - everything about netdata, in a page
8. Features⁠ - what features does it have
9. Visualization⁠ - unique visualization features
10. What does it monitor⁠ - which metrics it collects
11. Documentation⁠ - read the docs
12. Community⁠ - discuss with others and get support
13. License⁠ - check the license of netdata
14. Is it any good?⁠ - Yes
15. Is it awesome?⁠ - Yes

⁠How it looks

The following animated image, shows the top part of a typical netdata dashboard.

A typical netdata dashboard, in 1:1 timing. Charts can be panned by dragging them, zoomed in/out with SHIFT + mouse wheel, an area can be selected for zoom-in with SHIFT + mouse selection. Netdata is highly interactive and real-time, optimized to get the work done!

⁠User base

Netdata is used by hundreds of thousands of users all over the world. Check our GitHub watchers list⁠. You will find people working for Amazon, Atos, Baidu, Cisco Systems, Citrix, Deutsche Telekom, DigitalOcean, Elastic, EPAM Systems, Ericsson, Google, Groupon, Hortonworks, HP, Huawei, IBM, Microsoft, NewRelic, Nvidia, Red Hat, SAP, Selectel, TicketMaster, Vimeo, and many more!

⁠Docker pulls

We provide docker images for the most common architectures. These are statistics reported by docker hub:

⁠Registry

When you install multiple netdata, they are integrated into one distributed application, via a netdata registry⁠. This is a web browser feature and it allows us to count the number of unique users and unique netdata servers installed. The following information comes from the global public netdata registry we run:

in the last 24 hours:

⁠Quick Start

You can quickly install netdata on a Linux box (physical, virtual, container, IoT) with the following command:

# make sure you run `bash` for your shell
bash

# install netdata, directly from github sources
bash <(curl -Ss https://my-netdata.io/kickstart.sh)

The above command will:

1. install any required packages on your system (it will ask you to confirm before doing so),
2. compile it, install it and start it

More installation methods and additional options can be found at the installation page⁠.

To try netdata in a docker container, run this:

docker run -d --name=netdata \
  -p 19999:19999 \
  -v /proc:/host/proc:ro \
  -v /sys:/host/sys:ro \
  -v /var/run/docker.sock:/var/run/docker.sock:ro \
  --cap-add SYS_PTRACE \
  --security-opt apparmor=unconfined \
  netdata/netdata

For more information about running netdata in docker, check the docker installation page⁠.

From Netdata v1.12 and above, anonymous usage information is collected by default and sent to Google Analytics. To read more about the information collected and how to opt-out, check the anonymous statistics page⁠.

⁠Why Netdata

Netdata has a quite different approach to monitoring.

Netdata is a monitoring agent you install on all your systems. It is:

• a metrics collector - for system and application metrics (including web servers, databases, containers, etc)
• a time-series database - all stored in memory (does not touch the disks while it runs)
• a metrics visualizer - super fast, interactive, modern, optimized for anomaly detection
• an alarms notification engine - an advanced watchdog for detecting performance and availability issues

All the above, are packaged together in a very flexible, extremely modular, distributed application.

This is how netdata compares to other monitoring solutions:

Netdata is open-source, free, super fast, very easy, completely open, extremely efficient, flexible and integrate-able.

It has been designed by SysAdmins, DevOps and Developers for troubleshooting performance problems, not just visualize metrics.

⁠News

May 21st, 2019 - netdata v1.15.0 released!⁠

Release v1.15.0 contains 11 bug fixes and 30 improvements.

We are very happy and proud to be able to include two major improvements in this release: The aggregated node view and the new database engine⁠.

Aggregated node view

The No. 1 request from our community has been a better way to view and manage their Netdata installations, via an aggregated view. The node menu with the simple list of hosts on the agent UI just didn't do it for people with hundreds, or thousands of instances. This release introduces the node view, which uses the power of Netdata Cloud⁠ to deliver powerful views of a Netdata-based monitoring infrastructure. You can read more about Netdata Cloud and the future of netdata here⁠.

New database engine

Historically, Netdata has required a lot of memory for long-term metrics storage. To mitigate this we've been building a new DB engine for several months and will continue improving until it can become the default memory mode for new Netdata installations. The version included in release v1.15.0 already permits longer-term storage of compressed data and we'll continue reducing the required memory in following releases.

Other major additions

We have added support for the AWS Kinesis backend⁠ and new collectors for OpenVPN⁠, the Tengine web server⁠, ScaleIO (VxFlex OS)⁠, ioping-like latency metrics⁠ and Energi Core node instances⁠.

We now have a new, "text-only" chart type⁠, cpu limits for v2 cgroups⁠, docker swarm metrics⁠ and improved documentation⁠.

We continued improving the Kubernetes helmchart⁠ with liveness probes for slaves, persistence options, a fix for a Cannot allocate memory issue and easy configuration for the kubelet, kube-proxy and coredns collectors.

Finally, we built a process to quickly replace any problematic nightly builds and added more automated CI tests to prevent such builds from being published in the first place.

Apr 26th, 2019 - netdata v1.14.0 released!⁠

Release 1.14 contains 14 bug fixes and 24 improvements.

The release introduces major additions to Kubernetes monitoring, with tens of new charts for Kubelet⁠, kube-proxy⁠ and coredns⁠ metrics, as well as significant improvements to the netdata helm chart⁠.

Two new collectors were added, to monitor Docker hub⁠ and Docker engine⁠ metrics.

Finally, v1.14 adds support for version 2 cgroups⁠, OpenLDAP over TLS⁠, NVIDIA SMI free and per process memory⁠ and configurable syslog facilities⁠.

Mar 14th, 2019 - netdata v1.13.0 released!⁠

Release 1.13.0 contains 14 bug fixes and 8 improvements.

netdata has taken the first step into the world of Kubernetes, with a beta version of a Helm chart⁠ for deployment to a k8s cluster and proper naming⁠ of the cgroup containers. We have big plans⁠ for Kubernetes, so stay tuned!

A major refactoring of the python.d plugin⁠ has resulted in a dramatic decrease of the required memory, making netdata even more resource efficient.

We also added charts for IPC shared memory segments and total memory used.

Feb 28th, 2019 - netdata v1.12.2 released!⁠

Patch release 1.12.2 contains 7 bug fixes and 4 improvements.

The main motivation behind a new patch release is the introduction of a stable release channel. A "stable" installation and update channel was always on our roadmap, but it became a necessity when we realized that our users in China could not use the nightly releases published on Google Cloud. The "stable" channel is based on our official GitHub releases and uses assets hosted on GitHub.

We are also introducing a new Oracle DB collector module, implemented in Python.

Feb 21st, 2019 - netdata v1.12.1 released!⁠

Patch release 1.12.1 contains 22 bug fixes and 8 improvements.

Feb 14th, 2019 - netdata v1.12.0 released!⁠

Release 1.12 is made out of 211 pull requests and 22 bug fixes. The key improvements are:

• Introducing netdata.cloud, the free netdata service for all netdata users
• High performance plugins with go.d.plugin (data collection orchestrator written in Go)
• 7 new data collectors and 11 rewrites of existing data collectors for improved performance
• A new management API for all netdata servers
• Bind different functions of the netdata APIs to different ports
• Improved installation and updates

Nov 22nd, 2018 - netdata v1.11.1 released!⁠

• Improved internal database to support values above 64bit.
• New data collection plugins: openldap⁠, tor⁠, nvidia_smi⁠.
• Improved data collection plugins: netdata now supports monitoring network interface aliases, smartd_log⁠, cpufreq⁠, sensors⁠.
• Health monitoring improvements: network interface congestion alarm restored, alerta.io⁠, conntrack_max.
• my-netdatamenu has been refactored.
• Packaging: openrc service definition got a few improvements.

Sep 18, 2018 - netdata has its own organization

Netdata used to be a firehol.org⁠ project, accessible as firehol/netdata.

Netdata now has its own github organization netdata, so all github URLs are now netdata/netdata. The old github URLs, repo clones, forks, etc redirect automatically to the new repo.

⁠How it works

Netdata is a highly efficient, highly modular, metrics management engine. Its lockless design makes it ideal for concurrent operations on the metrics.

This is how it works:

The result is a highly efficient, low latency system, supporting multiple readers and one writer on each metric.

⁠Infographic

This is a high level overview of netdata feature set and architecture. Click it to to interact with it (it has direct links to documentation).

⁠Features

This is what you should expect from Netdata:

⁠General

• 1s granularity - the highest possible resolution for all metrics.
• Unlimited metrics - collects all the available metrics, the more the better.
• 1% CPU utilization of a single core - it is super fast, unbelievably optimized.
• A few MB of RAM - by default it uses 25MB RAM. You size it⁠.
• Zero disk I/O - while it runs, it does not load or save anything (except error and access logs).
• Zero configuration - auto-detects everything, it can collect up to 10000 metrics per server out of the box.
• Zero maintenance - You just run it, it does the rest.
• Zero dependencies - it is even its own web server, for its static web files and its web API (though its plugins may require additional libraries, depending on the applications monitored).
• Scales to infinity - you can install it on all your servers, containers, VMs and IoTs. Metrics are not centralized by default, so there is no limit.
• Several operating modes - Autonomous host monitoring (the default), headless data collector, forwarding proxy, store and forward proxy, central multi-host monitoring, in all possible configurations. Each node may have different metrics retention policy and run with or without health monitoring.

⁠Health Monitoring & Alarms

• Sophisticated alerting - comes with hundreds of alarms, out of the box! Supports dynamic thresholds, hysteresis, alarm templates, multiple role-based notification methods.
• Notifications: alerta.io⁠, amazon sns⁠, discordapp.com⁠, email⁠, flock.com⁠, irc⁠, kavenegar.com⁠, messagebird.com⁠, pagerduty.com⁠, prowl⁠, pushbullet.com⁠, pushover.net⁠, rocket.chat⁠, slack.com⁠, smstools3⁠, syslog⁠, telegram.org⁠, twilio.com⁠, web⁠ and custom notifications⁠.

⁠Integrations

• time-series dbs - can archive its metrics to graphite, opentsdb, prometheus, json document DBs, in the same or lower resolution (lower: to prevent it from congesting these servers due to the amount of data collected).

⁠Visualization

• Stunning interactive dashboards - mouse, touchpad and touch-screen friendly in 2 themes: slate (dark) and white.
• Amazingly fast visualization - responds to all queries in less than 1 ms per metric, even on low-end hardware.
• Visual anomaly detection - the dashboards are optimized for detecting anomalies visually.
• Embeddable - its charts can be embedded on your web pages, wikis and blogs. You can even use Atlassian's Confluence as a monitoring dashboard⁠.
• Customizable - custom dashboards can be built using simple HTML (no javascript necessary).

⁠Positive and negative values

To improve clarity on charts, netdata dashboards present positive values for metrics representing read, input, inbound, received and negative values for metrics representing write, output, outbound, sent.

![positive-and-negative-values](https://user-images.githubusercontent.com/2662304/48309090-7c5c6180-e57a-11e8-8⁠