choppsv1/bgptest repository overview

• BGPTEST

  This is a set of tools and a simulation to stress test a BGP implmentation. 2 tools are used to test genrt.py and bgp-inject.py. Also included is a simulation setup that runs the test using docker to simulate 4 BGP routers.

** Running Requirements *** Requirements for using tools: - Ubuntu 18 with some packages installed. - sudo apt-get update -qy - sudo apt-get install bash python3 python3-dev python3-venv *** Requirements for simulation - Additionally add latest docker package. - curl -fsSL get.docker.com | sh - sudo usermod -aG docker $(whoami) - logout and back in so you are part of the unix group.

** Description of Tools *** genrt.py Generates routes in various formats. It can create raw BGP Update messages to inject directly into a BGP connection (using bgp-inject.py). It can also create a list of static route definitions that can be included in BIRD (or other) routing configs. Finally it can create MRT table dumps of routes that can be used with gobgp to inject routes. MRT table dumps are also used by RIPE and others to record BGP sessions and routing tables.

*** bgp-inject.py bgp-inject.py Opens an EBGP connection and then sends the contents of file over the connection. The file content (BGP update messages) can be created by the genrt.py tool. Finally it enters a Keep Alive loop to keep the BGP connection established.

** Running the simulation. To generate all the required configs and bringup the simulation one can simply

: make run

All the configs and update messages files and 4 routers are created:

- ~bgput~ - The BGP under test everything connects to this router (AS30).
- ~st-1~ - BGP Static 1 - 2.5M routes (AS10)
- ~st-2~ - BGP Static 2 - 2.5M routes (AS11)
- ~st-3~ - BGP Static 3 - 2.5M routes (AS12)
- ~ibgp~ - IBGP speaker to check for presence of routes from ~bgput~

One can use the birdc6 CLI tool to examine state of the routers. For example to look at the protocol stats (including route counts) you could use the following command

: docker-compose exec ibgp birdc6 show proto all

Or to just look at the BGP speaker on IBGP

: docker-compose exec ibgp birdc6 show proto all bgp_30

The fifth router (FT) is you, running the bgp-inject.py tool directly from the shell. This would be similar to testing a real router setup.

To bring down the simualtion use make stop.

** Running the test. These instructions assume the simulation has been launched with make run from above. In a physical router setup the connectivity would need to be in place and the addresses adjusted accordingly (if needed).

After the update messages raw file has been generated and the routers (or simulation) brought up you are ready to inject the routes over EBGP this can be done with the following command

: ./bgp-inject.py -i data/mpath-0-1000.raw fc20::1,30

This creates an EBGP speaker AS 20 router ID 10.0.0.20, connecting to fc20::1 remote AS 30. Once connection is ESTABLISHED the tool will send the content of the file (data/mpath-0-1000.raw in this case) to the remote speaker.

We generate a few variants of 2.5M route updates. Here's the format:

: --

The type is either

• onepath - uses the same set of AS path for all routes
• mpath - the first AS in the path is incremented for each update message.

The mod is a modulus applied to the incremented AS number. This effectively limits the number of unique AS paths to this value. 0 means no modulus is applied.

• 0 mod no modulus is done all paths will be unique (if incrementing)
• 100 mod this means that every after 100 increments the AS number will wrap so there will be 100 unique AS paths.
• 1000 mod this means that every after 1000 increments the AS number will wrap so there will be 1000 unique AS paths.

The mpack value is the number of prefixes to pack into a single update message. By definition all these prefix will share the same path.

We generated the following files automatically:

• data/mpath-0-1.raw
• data/mpath-0-10.raw
• data/mpath-0-1000.raw
• data/mpath-100-1.raw
• data/mpath-100-10.raw
• data/mpath-100-1000.raw
• data/mpath-1000-1.raw
• data/mpath-1000-10.raw
• data/mpath-1000-1000.raw
• data/onepath-0-1.raw
• data/onepath-0-10.raw
• data/onepath-0-1000.raw

** Run Notes:

*** Shell cmd to query IBGP: Use this command to query the IBGP instance in a loop looking at route counts. This is used below to track route count during the runs.

#+begin_src bash while sleep 1; do
printf "%s: " "$(date)";
docker-compose exec ibgp birdc6 show proto all bgp_30 | grep Routes:;
done #+end_src

*** 2.5M routes, 1000 paths 100 prefix per update.

Letting the simulation come up and settle, then running the inject with 1000 unique paths and 100 prefix per update (thus using same path) takes

• ~14s to send the raw data:
• ~18s to see them in BIRD iBGP instance.

The sending time is definitely affected by the redistribution in BIRD. The next section after this one shows a test with just the BGPUT and the FT injecting routes.

**** Output when using ./bgp-inject.py -i data/mpath-1000-100.raw fc20::1,30

#+begin_src bash

[ simulation startup ] Sat Jul 28 11:32:08 EDT 2018: Routes: 0 imported, 0 exported, 0 preferred Sat Jul 28 11:32:09 EDT 2018: Routes: 6 imported, 1 exported, 4 preferred Sat Jul 28 11:32:11 EDT 2018: Routes: 788361 imported, 1 exported, 788359 preferred Sat Jul 28 11:32:13 EDT 2018: Routes: 1992842 imported, 1 exported, 1992840 preferred Sat Jul 28 11:32:15 EDT 2018: Routes: 3092105 imported, 1 exported, 3092103 preferred Sat Jul 28 11:32:17 EDT 2018: Routes: 4286332 imported, 1 exported, 4286330 preferred Sat Jul 28 11:32:18 EDT 2018: Routes: 5515840 imported, 1 exported, 5515838 preferred Sat Jul 28 11:32:20 EDT 2018: Routes: 6746259 imported, 1 exported, 6746257 preferred Sat Jul 28 11:32:21 EDT 2018: Routes: 7864326 imported, 1 exported, 7864324 preferred Sat Jul 28 11:32:23 EDT 2018: Routes: 7864326 imported, 1 exported, 7864324 preferred Sat Jul 28 11:32:24 EDT 2018: Routes: 7864326 imported, 1 exported, 7864324 preferred Sat Jul 28 11:32:26 EDT 2018: Routes: 7864326 imported, 1 exported, 7864324 preferred Sat Jul 28 11:32:27 EDT 2018: Routes: 7935026 imported, 1 exported, 7935024 preferred [ Started ./bgp-inject.py ] Sat Jul 28 11:32:29 EDT 2018: Routes: 8358526 imported, 1 exported, 8358524 preferred Sat Jul 28 11:32:30 EDT 2018: Routes: 8650214 imported, 1 exported, 8650212 preferred Sat Jul 28 11:32:32 EDT 2018: Routes: 8910014 imported, 1 exported, 8910012 preferred Sat Jul 28 11:32:33 EDT 2018: Routes: 9133502 imported, 1 exported, 9133500 preferred Sat Jul 28 11:32:35 EDT 2018: Routes: 9335502 imported, 1 exported, 9335500 preferred Sat Jul 28 11:32:36 EDT 2018: Routes: 9534002 imported, 1 exported, 9534000 preferred Sat Jul 28 11:32:38 EDT 2018: Routes: 9726502 imported, 1 exported, 9726500 preferred Sat Jul 28 11:32:39 EDT 2018: Routes: 9921002 imported, 1 exported, 9921000 preferred Sat Jul 28 11:32:41 EDT 2018: Routes: 10129078 imported, 1 exported, 10129076 preferred Sat Jul 28 11:32:42 EDT 2018: Routes: 10363078 imported, 1 exported, 10363076 preferred Sat Jul 28 11:32:44 EDT 2018: Routes: 10485766 imported, 1 exported, 10485764 preferred Sat Jul 28 11:32:45 EDT 2018: Routes: 10485766 imported, 1 exported, 10485764 preferred Sat Jul 28 11:32:47 EDT 2018: Routes: 10485766 imported, 1 exported, 10485764 preferred

#+end_src

**** Output from ./bgp-inject.py -i data/mpath-1000-100.raw fc20::1,30 #+begin_src bash

[...] 2018-07-28 11:32:27,860: INFO: ESTABLISHED (IPv6Address('fc20::1'), 179) (assuming 4-octet AS numbers) 2018-07-28 11:32:27,860: INFO: Loading raw data from data/mpath-1000-100.raw 2018-07-28 11:32:27,877: INFO: Sending 19529767 bytes of raw data to (IPv6Address('fc20::1'), 179) 2018-07-28 11:32:42,215: INFO: Done sending after 14.355535507202148 to (IPv6Address('fc20::1'), 179) #+end_src

*** FT+BGPUT only: 2.5M routes, 1000 paths 100 prefix per update.

You can see here that when there is no other neighbors and thus no
redistribution, it takes about 3 seconds to send the 2.5M prefix this is
above 500k (app 853k updates/s)

#+begin_src bash

Sat Jul 28 12:09:29 EDT 2018: Routes: 0 imported, 0 exported, 0 preferred Sat Jul 28 12:09:30 EDT 2018: Routes: 923288 imported, 0 exported, 923288 preferred Sat Jul 28 12:09:32 EDT 2018: Routes: 2171152 imported, 0 exported, 2171152 preferred Sat Jul 28 12:09:34 EDT 2018: Routes: 2621440 imported, 0 exported, 2621440 preferred

(default) 06:00 [12:08:23 tops:~/w/bgptest]$ ./bgp-inject.py -i data/mpath-1000-100.raw fc20::1,30 2018-07-28 12:09:31,033: INFO: CONNECT (IPv6Address('fc20::1'), 179) 2018-07-28 12:09:31,034: INFO: SENT OPEN (IPv6Address('fc20::1'), 179) 2018-07-28 12:09:31,034: INFO: OPENCONFRIM (IPv6Address('fc20::1'), 179) 2018-07-28 12:09:31,034: INFO: ESTABLISHED (IPv6Address('fc20::1'), 179) (assuming 4-octet AS numbers) 2018-07-28 12:09:31,034: INFO: Loading raw data from data/mpath-1000-100.raw 2018-07-28 12:09:31,052: INFO: Sending 19529767 bytes of raw data to (IPv6Address('fc20::1'), 179) 2018-07-28 12:09:34,119: INFO: Done sending after 3.085615873336792 to (IPv6Address('fc20::1'), 179) 2018-07-28 12:09:34,120: INFO: Sent KeepAlive to (IPv6Address('fc20::1'), 179) sleeping 30s

#+end_src

*** FT+BGPUT only: 2.5M routes, all unique paths 1 prefix per update.

If we send *all* unique paths and only a single prefix per update message
(the most stressful and a somewhat unrealistic test) the time jumps to ~8s
slightly (~300k updates/s)

#+begin_src bash

Sat Jul 28 12:13:56 EDT 2018: Routes: 0 imported, 0 exported, 0 preferred Sat Jul 28 12:13:58 EDT 2018: Routes: 416730 imported, 0 exported, 416730 preferred Sat Jul 28 12:14:00 EDT 2018: Routes: 1271654 imported, 0 exported, 1271654 preferred Sat Jul 28 12:14:01 EDT 2018: Routes: 1829414 imported, 0 exported, 1829414 preferred Sat Jul 28 12:14:03 EDT 2018: Routes: 2196810 imported, 0 exported, 2196810 preferred Sat Jul 28 12:14:05 EDT 2018: Routes: 2561020 imported, 0 exported, 2561020 preferred Sat Jul 28 12:14:07 EDT 2018: Routes: 2621440 imported, 0 exported, 2621440 preferred

(default) 00:01 [12:13:09 tops:~/w/bgptest]$ ./bgp-inject.py -i data/mpath-0-1.raw fc20::1,30 2018-07-28 12:13:58,370: INFO: CONNECT (IPv6Address('fc20::1'), 179) 2018-07-28 12:13:58,370: INFO: SENT OPEN (IPv6Address('fc20::1'), 179) 2018-07-28 12:13:58,370: INFO: OPENCONFRIM (IPv6Address('fc20::1'), 179) 2018-07-28 12:13:58,370: INFO: ESTABLISHED (IPv6Address('fc20::1'), 179) (assuming 4-octet AS numbers) 2018-07-28 12:13:58,370: INFO: Loading raw data from data/mpath-0-1.raw 2018-07-28 12:13:58,526: INFO: Sending 188219392 bytes of raw data to (IPv6Address('fc20::1'), 179) 2018-07-28 12:14:06,354: INFO: Done sending after 7.983449935913086 to (IPv6Address('fc20::1'), 179) 2018-07-28 12:14:06,354: INFO: Sent KeepAlive to (IPv6Address('fc20::1'), 179) sleeping 30s #+end_src

** Not used, too slow *** Go BGP **** Using GoBGP #+begin_src bash

⁠list neighbor

docker-compose exec bgput gobgp neighbor

⁠neighbor info

docker-compose exec bgput gobgp neighbor 10.0.0.20 docker-compose exec bgput gobgp neighbor 10.0.1.31

⁠Global RIB

gobgp global rib

⁠RIB-IN

gobgp neighbor 10.0.0.20 adj-in gobgp neighbor 10.0.1.31 adj-in

⁠RIB-OUT

gobgp neighbor 10.0.0.20 adj-out gobgp neighbor 10.0.1.31 adj-out #+end_src *** FRR - BGPD