mjaksn/nettail repository overview

⁠nettail

A NetFlow and IPFIX collector that prints flow records to the console in a readable table, annotating addresses with hostnames where it can find them. tail -f for your network.

Built for pointing a UniFi Dream Machine Pro at a workstation and actually seeing what the network is doing. Works with any exporter that speaks NetFlow v5, NetFlow v9, or IPFIX (v10).

Two dependencies, both of them halves of this program that grew up and left: netflume⁠ reads the wire, and lanname⁠ turns an address into a hostname. Neither has dependencies of its own, so pip install nettail brings in three pure Python packages and nothing else.

TIME         EXPORTER        PROTO  SOURCE                                     DESTINATION                                 PKTS    BYTES     DUR  FLAGS
13:40:03.000 10.0.0.1        TCP    192.168.1.42:51234 (macbook-pro)         ↑ 140.82.114.4:443/https (github)               23     4.1K   4.90s  ...AP.SF
13:40:03.000 10.0.0.1        UDP    192.168.1.77:5353/mdns (hue)             ⇄ 224.0.0.251:5353/mdns                          2     180B   4.90s
13:39:56.103 10.0.0.1        TCP    10.0.1.5:44321 (nas)                     ↑ 104.244.42.1:443/https (twitter-edge)        412    57.5K  12.50s  ...AP...
13:40:08.453 10.0.0.1        TCP    1.1.1.1:853/domain-s                     ↓ 10.0.1.5:39012 (nas)                           4     320B   0.15s  ......S.

⁠Contents

• Requirements⁠
• Quick start⁠
• Configuring the UDM Pro⁠
• Command line options⁠
• Settings file⁠
• The status bar⁠
• Keyboard controls⁠
• The web interface⁠
• The traffic summary⁠
• Output format⁠
• Hostname resolution⁠
• Country marking⁠
• JSON output⁠
• Installing it⁠
• Running as a service⁠
• Running in Docker⁠
• How it works⁠
• Troubleshooting⁠
• Tests⁠
• Limitations⁠
• Extending it⁠

⁠Requirements

• Python 3.11 or newer
• netflume⁠, the wire decoder
• lanname⁠, the hostname lookups

Neither has dependencies of its own, so this brings in three pure Python packages and nothing else.

pip install nettail

Or, from a checkout, either of:

pip install -e .        # the nettail command, pointed at the working tree
pip install lanname netflume && python -m nettail

Binding to the default port 2055 does not require root on Linux, since it is above 1024. You only need elevated privileges if you choose a port below 1024.

⁠Quick start

# Listen on the default port
nettail

# Pick a port and only show flows that touch the internet
nettail --port 2055 --external-only

# Passive name resolution plus your own static mappings
nettail --resolve dns --hosts ./lan-hosts

# Dump every decoded field under each flow
nettail --verbose

# Spell out each template an exporter sends, without the field lines
nettail --templates

# Machine readable, one object per line
nettail --json > flows.jsonl

# Or to a file of its own, with the table still on the screen
nettail --json flows.jsonl

# Keep thirty days of flow history rather than fourteen, for the browser to
# scroll back into and to replay a tab that was in the background
nettail --flow-retention-days 30

# Or keep no flow history at all
nettail --flow-store off

Press Ctrl-C to stop. A summary prints on exit with datagram counts, template statistics, name resolution hit rates, and the top ten external addresses by volume.

⁠Configuring the UDM Pro

In the UniFi Network application:

Settings > CyberSecure > Traffic Logging

On some firmware versions this lives under Settings > System > Integrations instead. Look for a section named NetFlow or NetFlow (IPFIX).

⁠Sampling must be off

This matters more than anything else on the page. Sampled export throws away most flows before they leave the router. A malware beacon is a handful of small flows per hour, and at 1:1024 sampling you will never see it. Sampling is fine for bandwidth graphs and useless for security monitoring.

If your firmware does not let you disable sampling, the flow data is not trustworthy for detection work and you should look at a mirror port with Zeek instead.

You do not have to take the router's word for it. The collector reads the sampling rate an exporter advertises and says so on stderr the first time it sees one:

10.0.0.1 reports 1-in-1000 sampling. The byte and packet counts shown are a sample,
so real traffic is roughly 1000x higher. Turn sampling off at the exporter for true
counts.

The rate is repeated in a Sampling section of the exit summary. It is read from whichever form the exporter uses: samplingInterval or samplerRandomInterval on v9, samplingPacketInterval with samplingPacketSpace or samplingSize out of samplingPopulation on IPFIX, and the sampling field of the header on v5. Nothing is printed when an exporter reports no sampling, or a rate of one in one.

If an exporter later reports that sampling is off, meaning an interval of one or a selection that skips nothing, the remembered rate is dropped and a line says the counts are complete again. An option record that says nothing about sampling, such as one carrying interface names, never changes what is remembered.

Detection depends on the exporter actually sending options data. Silence is not proof that sampling is off, so check the UniFi UI as well. On v5 there are no options records at all: sampling is read from the header, and an exporter that stops sampling simply clears the field, which is reported as nothing rather than as a change.

⁠Local traffic may never arrive

Selecting every network in the table above reads like a request for visibility into all of them. On a UDM Pro it is not what you get. Traffic to and from the internet arrives. Traffic between two local hosts may not, and the two reasons for that are worth telling apart, because only one of them is surprising.

Two hosts on the same VLAN never produce a flow record at all. They talk through a switch and the gateway routes nothing between them, so there is nothing for it to account. This is not a fault and no setting changes it. A flow exporter running on the router cannot see traffic that never reaches the router, and every tool reading NetFlow from a gateway has this property, this one included.

Two hosts on different VLANs are routed by the gateway and may still be missing. This is the surprising one. Traffic between separate subnets has to pass through the UDM Pro, and a tracert or traceroute between the hosts shows it as a hop, so the router demonstrably handles the packets. They can still be absent from the export, with every network selected.

Two mechanisms would account for that and they cannot be told apart from outside the box. The routing may be offloaded to hardware along a path the flow accounting does not sit on, or the accounting may be scoped to the internet-facing path by design. The consequence is the same either way, which is why the distinction matters less than it looks: neither this collector nor anything in the export configuration changes it.

This is behaviour observed on a UDM Pro rather than anything Ubiquiti documents, so establish it on your own hardware before concluding. Three checks, in order of effort:

1. tracert or traceroute between the two hosts. The UDM's address as a hop means it routes them, and the rest of this section applies. A switch address instead means a layer 3 switch is doing the routing and the gateway has nothing to report, which is a different problem with a different answer.
2. Insights > Flows in the UniFi Network application. Sessions listed there and absent from your export mean the data exists on the box and only the export scope is holding it back. That is the version worth raising with Ubiquiti, being a demonstrable gap rather than a suspicion.
3. This collector's own exit summary. Compare flows decoded under Summary with flows under External traffic. Two equal figures mean every flow that arrived had a public endpoint, so nothing internal is being exported at all.

If it is east-west visibility you need, a gateway is the wrong vantage point for it. Mirror the VLANs you care about to a capture host at the switch and read packets rather than flow records, which is the same advice as for sampled export and for a different reason.

⁠Other notes

• Make sure a host firewall on the collector is not dropping inbound UDP on your chosen port. sudo ufw allow 2055/udp or the nftables equivalent.
• If flows appear with the UDM's own WAN address as the source, you are seeing post-NAT records and have lost internal host attribution. Check which networks are selected in the export configuration.

⁠Command line options

usage: nettail [-h] [--version] [--config FILE | --save-config [FILE]]
               [--bind BIND] [--port PORT] [--external-only] [--names]
               [--macs] [--verbose] [--templates] [--json [FILE]]
               [--flow-store [FILE]] [--flow-retention-days DAYS]
               [--flow-prune-every SECONDS] [--flow-commit-every SECONDS]
               [--colour WHEN] [--no-color] [--header-every HEADER_EVERY]
               [--sticky-header] [--hide-status] [--no-supplemental-services]
               [--web] [--web-port PORT] [--web-bind ADDR] [--web-host NAME]
               [--web-token TOKEN] [--web-colour WHEN] [--web-readonly]
               [--web-detail-refresh SECONDS] [--size-scale-max BYTES |
               --size-scale-dynamic] [--size-scale-window FLOWS] [--country]
               [--country-db FILE] [--update-country-db]
               [--country-style {auto,flag,code}] [--resolve {off,dns,all}]
               [--hosts FILE] [--resolve-public] [--fqdn]
               [--resolve-workers RESOLVE_WORKERS]
               [--resolve-timeout RESOLVE_TIMEOUT]

⁠General

⁠Web interface

All off unless --web is given. See The web interface⁠.

⁠Sticky header

--sticky-header keeps the column header on the top row while flows scroll underneath it, so you never lose track of which column is which. It works by setting a VT100 scroll region (DECSTBM) covering every row but the first, then letting output scroll inside that region as usual.

The k key moves the same setting while the collector runs, in both directions, which is worth having for the first item below: pin the header while you read the columns, let it go when you want the scrollback back. Letting it go leaves the flows where they are, and pinning it again scrolls one row rather than clearing the screen, so an hour of history is not the price of pressing a key. In a window too short to hold a header the key says so and changes nothing, as the b key does for the bar.

Two things to know before turning it on:

• You lose scrollback. Most terminals discard lines that scroll out of a margin region instead of pushing them into the scrollback buffer, so you can only see what is currently on screen. If you want to scroll back through past flows, leave the flag off and stay with --header-every, or keep the display and write the flows to a file with --json flows.jsonl.
• It needs a real terminal. On Windows the script enables virtual terminal processing automatically, which covers Windows Terminal and modern conhost. If stdout is redirected to a file or a pipe, or the window is too short, the flag prints a notice and falls back to --header-every. The header alone needs six rows; with the status bar up, which is the default, it needs eight.

The header is redrawn when the window is resized (the size is re-measured every 16 flow lines), and the scroll region is released on Ctrl-C so the summary and your shell prompt are not trapped inside it. --header-every is ignored while the header is pinned, since the repeats would be redundant.

⁠Flow size colour

See Size colour scale⁠ for what the colours mean.

⁠Country marking options

See Country marking⁠ for where the database comes from and what is marked.

⁠Hostname resolution options

⁠Settings file

Everything the collector takes on the command line it takes in a file instead, under the same name without its leading dashes. There is no second list of what can be set: the file is read against the same parser the command line is, so an option that exists is settable, and one that does not is reported as an unknown key.

# nettail.conf
port = 2055
external-only = true
web = true
web-bind = 127.0.0.1
flow-retention-days = 30
resolve = dns
hosts =
    /etc/hosts.lan
    /etc/hosts.iot

A [nettail] header is allowed and is not needed: there is one section and nowhere else for a setting to be, so a file that opens with port = 2055 is read as though it had one. A key may be written with dashes as the flag is or with underscores as Python spells it, and an option with two spellings answers to both: colour and color are one setting, as they are one flag. Writing one option twice under two of its names is reported rather than silently resolved, and the first is the one used. A switch takes true or false, and yes, no, on, off, 1 and 0 besides. An option that may be repeated takes one value a line, indented under the key. A comment is a line beginning with # or ;, and a % is a % rather than anything clever.

The command line wins. What the file says becomes the default and anything typed overrides it, so a config file is where the settings you always want live and the command line is for today. What neither says falls back to what the program shipped with, which is how a default this program changes later still reaches you.

The one exception is an option that may be repeated, --hosts and --web-host. Typing one of those adds to what the file listed rather than replacing it, because that is what repeatable means everywhere else here. A run that wants none of them wants --config pointed at a file that lists none.

The durable flow store is a different shape of option. It is on unless something turns it off, so a file needs to say nothing to have one; flow-store = /var/lib/nettail/flows.sqlite3 writes there instead of in the default per-user path, and flow-store = off keeps no history at all. false and no mean off too, since a file reads them as switches, and true means the default path, as --flow-store on its own does. The retention and cadence settings are ordinary values, so flow-retention-days = 30, flow-prune-every = 3600, and flow-commit-every = 2.5 all work exactly as the flags do.

Options that are alternatives win the same way, and it is worth saying because they are the one place where winning means the file's setting is dropped rather than replaced. With size-scale-max in your file, --size-scale-dynamic on the command line gives you a dynamic scale and the file's fixed top is set aside for that run, exactly as though it had not been there. The same goes for --size-scale-window, which rules out a fixed top too. Neither is an error: you asked for one of a pair of alternatives and got it.

There is a second thing worth knowing before you rely on it: a switch turned on in a file cannot be turned off from the command line, because switches here have no --no- form. Colour is not quite an exception and is worth spelling out, since it is the setting most likely to be in a file: colour is a choice rather than a switch, so colour = never in a file still yields to --colour always like any other value. no-color = true is a switch, so it yields to nothing, and it beats --colour always besides, because --no-color means --colour never however it arrived. Set colour, not no-color, in a file you want to argue with later. Anything else, and the answer is a different file.

⁠Where it is looked for

The first of these that exists is read, and only that one. They are not merged: a setting that comes from two files at once is a sett