Sign inSign up

laoluade/run-fibonacci

By laoluade

•Updated 11 months ago

Image for running a network of servers sending Fibonacci numbers to each other in numerous ways.

Image
Networking
0

1.0K

laoluade/run-fibonacci repository overview

⁠Table of Contents

  1. Versioning⁠
    1. Changelog⁠
  2. Required Software⁠
  3. Explanation of Image⁠
    1. Files Used in Image⁠
    2. Other Image Details⁠
  4. Other Files in Directory⁠
  5. Software Bill of Materials⁠
  6. Configurable Server Environmental Settings⁠
    1. Settings List⁠

⁠Versioning

Image Repository: https://hub.docker.com/r/laoluade/run-fibonacci⁠

Latest Image Version: docker.io/laoluade/run-fibonacci:2.3.0

⁠Changelog

Note: Still haven’t added full TLS for 3rd-party datastores yet, that’s a thing for later. It’s not that I can’t, I want a way to test it from pure python without having to change file permissions on my Windows laptop.

Version 2.3.0 (Available tags are latest, 2.3.0, 2.3.0-alpine, 2.3.0-alma)

  • Updated Python Cryptography library from 46.0.2 to 46.0.3.

  • Added the Graphene and GraphQL-Server libraries to requirements.txt.

  • Finished graphql.py and made it ready for Gunicorn serving.

  • Added GraphQL-compatible communication to datastore_utils.py and send_healthcheck.py

  • Reworked TestVersion.py to be smarter with looping through possible container configurations.

  • Renamed different APIs using app_{API} scheme.

  • Changed the log processing function for server app modules to return the hash of the log rather than a server state ID.

Version 2.2.0 (Available tags are 2.2.0, 2.2.0-alpine, 2.2.0-alma)

  • Added placeholder files for other API options.

  • Got a working Elasticstack setup during testing and uncommented the logic for it.

  • Added back a test logstash configuration YAML and a pipeline to testing folder.

  • Found bug where the Elasticstack log sending utility in datastore_utils is expecting a json response and converted it to outputting plain text.

  • Added TLS utility to server so it can generate its own server stuff.

  • Changed configuration scheme from an environmental scheme to a JSON file scheme that is supported by optional environmental configuration. Updated all relevant python files.

  • Modified SERVER_IDENTIFIER to be more concise and more descriptive of the server worker reporting logs.

  • Added environmental variables SERVER_CONFIG_FILEPATH and DEFAULT_SERVER_CONFIG_FILEPATH to Dockerfiles.

  • Updated Configurable Server Environmental Settings section in image README to reflect new changes.

Version 2.1.0 (Available tags are 2.1.0, 2.1.0-alpine, 2.1.0-alma)

  • Added new environmental variables SECRET_PEM_TARGET, DATASTORE_USER, DATASTORE_PASSWORD to support datastore authentication and tls setup.

  • Added pymongo and psycopg2 Python libraries to requirements.txt.

  • Changed constant SERVER_CONFIG to SERVER_IDENTIFIER and simplified SERVER_IDENTIFIER to take up less space when saved to datastores in rest.py.

  • Restructured log schema to create fields time, server, type, kinds, details, and hash as the information to work with for datastores in datastore_utils.py.

  • Added logic for creating PostgreSQL and MongoDB connections in datastore_utils.py. For now, they are unencrypted communications.

  • Completed logic for contacting available datastores in datastore_utils.py.

  • Updated TestVersion.py and TestGenTLS.py to reflect updates to version.

  • Reworked Dockerfiles into multi-build stages to prepare images to properly run psycopg2 functions.

  • Note: Elasticstack logic exists within codebase, but I could not get setup to work in a dynamic matter, so I greyed it out for now.

Version 2.0.0 (Available tags are 2.0.0, 2.0.0-alpine, 2.0.0-alma)

  • Added new datastore_utils.py for handling log definitions, log output, and log saving. Updated rest.py to reflect changes.

  • Added new environmental variables SERVER_DATASTORE, DATASTORE_ADDRESS, DATASTORE_PORT, DATASTORE_FILEPATH, DATASTORE_DEFAULT_FILEPATH, and DATASTORE_OPERATIONS_FILEPATH to support multiple kinds of datastores.

  • Updated Dockerfiles to copy entire components folder rather than individual files, and to install Python libraries using a requirements file.

  • Added new log saving API to rest.py.

  • Converted shell scripts for healthcheck and start API to Python programs. Adjusted rest.py accordingly.

  • Servers now run python from a virtual environment instead of a system environment.

  • Warning: Only rest API and none/file datastores are available

Version 1.2.0 (Available tags are 1.2.0, 1.2.0-alpine, 1.2.0-alma)

  • Added new environmental variable SERVER_API To support multiple kinds of APIs.

  • Revised Dockerfile to use a base Alpine layer with Python manually installed rather than a Python base layer that is Alpine specific.

  • Added Alma linux version of server.

  • Risk: Python version is now 3.12 instead of 3.13 for consistency across platforms. Is looking into ways to get Alma back up a version up without too much added image size.

  • Tests are now run using Python instead of Powershell/Bash to automate testing different platforms, apis, and storage combinations in the future.

Version 1.1.0 (Available tags are 1.1.0, 1.1.0-alpine)

  • Split SELF_ADDRESS into SELF_LISTENING_ADDRESS and SELF_HEALTHCHECK_ADDRESS for better Pod capabilities.

  • Updated test scripts to reflect changes. Changed container testing scripts to first check what container engine is running.

Version 1.0.1 (Available tags are 1.0.1, 1.0.1-alpine)

  • Changed SERVER_STAGE_INDEX check to allow equality to SERVER_STAGE_COUNT

  • Added test scripts for both Windows and Linux systems

Version 1.0.0 (Available tags are 1.0.0, 1.0.0-alpine)

  • Initial image

  • generalized image components

  • Added README

⁠Required Software

  1. Docker or Podman

  2. OpenSSL

  3. PowerShell or POSIX-compliant Shell

  4. Curl

  5. Python (version 3.12 or higher is recommended, code is written in 3.13)

⁠Explanation of Image

This container image is a generic image for running a small fibonacci number passing server. This image is made to work in a number of different contexts, such as running as a standalone container or as part of a larger kubernetes deployment.

The following diagram displays the components of the server at a high level.

Diagram of the componets of the generic fibonacci server. The biggest components are colored in dark blue boxes. They are the Gunicorn server, the send number daemon, and the health check daemon. There is a dark purple component called the TLS Materials as well. The gunicorn server contains a light blue box labeled Flask API, which represents the software REST endpoints for the server. Inside the Flask API box are three green boxes named STDOUT Logger, Send Threader, and REST APIs respectively. Inside the REST API box are three orange boxes that represent the default, healthcheck, and start endpoints of the server. Five directional arrows connect boxes, with the direction indicating that the first box utilizes another box. The REST API is connected to the STDOUT Logger. The default API Endpoint is connected to the Send Threader. The Send Threader is connected to the Send Number Daemon. The Health Check Daemon is connected to the healthcheck API Endpoint. The TLS Materials is connected to the Gunicorn Server.

In this diagram, we can see what components comprise the server image. The majority of the server is contained within the Flask API, which in turn is managed by the Gunicorn Server. TLS materials are given to the Gunicorn Server to enable encrypted communication. Two daemons (which are actually shells scripts) are connected to the Flask API through different functions and endpoints. Lastly, the STDOUT Logger is there to record any activity that takes place.

The container image exposes three REST API points. The first is the Default route (/) that is used to pass along fibonacci numbers. The second is the healthcheck route (/healthcheck) that is used to perform health checks on the server. The third is the start route (/start) that is used to start the fibonacci number passing chain.

All important information about the server is printed to STDOUT using the Python print command’s flush argument.

⁠Files Used in Image

The four files used in this image are gunicorn.conf.py, rest.py, send_healthcheck.sh, and send_next_fib.sh. These files are located in the components folder.

gunicorn.conf.py is the configuration file used by Gunicorn to tailor the server. This file specifies the Flask server to use, the network socket that the server listens on, and the TLS files necessary for encrypted communication among other things.

rest.py is a Flask definition file that is served by Gunicorn. This file specifies REST logic for the API endpoints and instantiates the constants that are used during the server’s runtime.

send_healthcheck.sh is the shell script that is called by an external management process to check the server’s health. It utilizes Curl to send an HTTPS message to the running server. It is represented by the Health Check Daemon box in the diagram above.

send_next_fib.sh is the shell script that is called by the healthcheck API to send a pair of fibonacci numbers to another server stage. It utilizes Curl to send an HTTPS message to the destination endpoint. It is represented by the Send Number Daemon box in the diagram above.

⁠Other Image Details

Here is a list of hardcoded details in the Dockerfile for the image. Feel free to change any of these values on your own system.

  1. The image defines two platforms as options to run the server.

    1. Alpine Linux is the default platform used to run the server due to its lightweight nature. The image defines docker.io/library/alpine:3.22 as the base layer image.

    2. Alma Linux is an additional platform used for cases where a RHEL variant is preferred. The image defines docker.io/library/almalinux:10-minimal as the base layer image.

  2. The image defines the non-root user app to run the server.

  3. The image defines server contents to be kept in the /usr/src/app directory. The user app is given ownership of this directory.

  4. The image defines a healthcheck to monitor the state of the server.

    1. The healthcheck is given a grace period of 10 seconds in the beginning.

    2. The healthcheck is set to run every 10 seconds.

    3. The healthcheck times out after 5 seconds if not complete.

    4. The healthcheck has a maximum of 3 attempts to succeed before the server is deemed unhealthy.

    5. The healthcheck runs the /usr/src/app/send_healthcheck.sh script to check server health.

  5. The image defines the gunicorn command to be the entrypoint of the container built by the image.

⁠Other Files in Directory

README.adoc is the file you are reading right now that explains everything about the image.

changelog.adoc is the secondary AsciiDoc file used to hold changelog information about the image.

latest_image.adoc is an AsciiDoc file used to hold the fully qualified name of the latest image.

alpine.Dockerfile is the file that defines the Alpine image.

alma.Dockerfile is the file that defines the Alma image.

CreateImages.py is the Python file used to create and push images and tags in one go.

testing/TestGenTLS.py is a Python script used to create the TLS materials for the testing images.

testing/TestVersion.py is a Python script used to test settings and platforms of the fibonacci server.

Note: For the test scripts, you will be on your own for scaling down the test. All that’s created is a container and some TLS credential stuff though, so it should be easy.

⁠Software Bill of Materials

  1. Docker Official Images

    1. This image uses the Alpine 3.22 and Alma 10 Minimal base layers.
  2. Python

    1. This image uses Python 3.12 by Default.
  3. Python Flask

    1. This image uses Flask version 3.1.2 by Default.
  4. Python Gunicorn

    1. This image uses Gunicorn version 23.0.0 by Default.
  5. Curl

    1. This image uses Curl version 8.16.0-r1 by Default.

⁠Configurable Server Environmental Settings

This section discusses the current server environmental settings available.

Firstly, the configuration file is JSON, and the settings are formatted in a way where a dot would correspond to them being nested inside the key before the dot. For example, network.datastore.address is equal to the following:

{
    "network": {
        "datastore": {
            "address": "sampleAddress"
        }
    }
}

Secondly, it is possible to use environmental variables to pass in settings. The format for environmental variable uppercases all letters and replaces . with _. For example, to alter the network.datastore.address setting, pass in the environmental variable NETWORK_DATASTORE_ADDRESS.

Lastly, to set what JSON file the server uses to configure itself to check, set the SERVER_CONFIG_FILEPATH environmental variable. By default, it is set to /usr/src/app/server_config.json at build time. If you submit a custom JSON that does not have all settings defined or only configure a few settings by environmental variables, then the default JSON configuration (through the DEFAULT_SERVER_CONFIG_FILEPATH backup environmental variable) will be used to configure the rest of the settings. Please do not replace that file, and the server WILL throw a fit and crash itself through assertions if it can’t find a setting.

The following precedent is set for configured settings:

  • Highest Priority: Environmental variable counterparts

  • Medium Priority: SERVER_CONFIG_FILEPATH configuration file

  • Lowest Priority: DEFAULT_SERVER_CONFIG_FILEPATH configuration file

⁠Settings List

  1. api

    1. Definition → The type of Server Web API to use.

    2. Schema → Must be one of a set of constants defined for the server.api key in the project README.

    3. Default → "rest"

  2. datastore.auth.username

    1. Definition → The username of the account for accessing the datastore.

    2. Schema → Must be a string with alphanumerical characters with no spaces. Optional.

    3. Default → ""

  3. datastore.auth.password

    1. Definition → The password of the account for accessing the datastore.

    2. Schema → Must be a complex, decently long string. Optional.

    3. Default → ""

  4. datastore.logs.defaultPath

    1. Definition → The default filesystem location where the datastore should save server logs.

    2. Schema → Must be a UNIX absolute filepath.

    3. Default → "/tmp/default.csv"

  5. datastore.logs.operationPath

    1. Definition → The filesystem location where the datastore operations that can’t be sent should be saved.

    2. Schema → Must be a UNIX filepath.

    3. Default → "/tmp/operations.csv"

  6. datastore.logs.serverPath

    1. Definition → The primary filesystem location where the datastore should save server logs.

    2. Schema → Must be a UNIX filepath.

    3. Default → "/tmp/datastore.csv"

  7. datastore.type

    1. Definition → The type of Datastore to use.

    2. Schema → Must be one of a set of constants defined for the server.datastore key in the project README.

    3. Default → "none"

  8. network.datastore.address

    1. Definition → The network address of the datastore that the server should contact in the test network.

    2. Schema → Must be either a IPv4 address or a FQDN.

    3. Default → "127.0.0.1"

  9. network.datastore.port

    1. Definition → The network port of the datastore that the server should contact in the test network.

    2. Schema → Must be non-privileged port number.

    3. Default → 8080

  10. network.dest.address

    1. Definition → The network address of the server stage that the server should contact in the test network.

    2. Schema → Must be either a IPv4 address or a FQDN.

    3. Default → "127.0.0.1"

  11. network.dest.port

    1. Definition → The network port of the server stage that the server should contact in the test network.

    2. Schema → Must be non-privileged port number.

    3. Default → 8080

  12. network.self.address.healthcheck

    1. Definition → The network address the server uses to call itself for a healthcheck in the test network.

    2. Schema → Must be either a IPv4 address or a FQDN.

    3. Default → "127.0.0.1"

  13. network.self.address.listening

    1. Definition → The network address the server binds to in the test network.

    2. Schema → Must be either a IPv4 address or a FQDN.

    3. Default → "0.0.0.0"

  14. network.self.port

    1. Definition → The network port of the server in the test network.

    2. Schema → Must be non-privileged port number.

    3. Default → 8080

  15. stage.count

    1. Definition → The number of server stages in the test network.

    2. Schema → Must be a number that can be turned into a Python integer.

    3. Default → 1

  16. stage.index

    1. Definition → The server stage designator of the server in the test network.

    2. Schema → Must be a number that can be turned into a Python integer. Must be between 0 and SERVER_STAGE_COUNT or equal to SERVER_STAGE_COUNT.

    3. Default → 1

  17. throttleSecs

    1. Definition → The interval that the server should wait in seconds between receiving fibonacci numbers and sending fibonacci numbers.

    2. Schema → Must be a number that can be turned into a Python integer.

    3. Default → 5

  18. tls.ca.keyPath

    1. Definition → The location of the certificate authority TLS key of the server in the test network.

    2. Schema → Must be a UNIX filepath.

    3. Default → "./ca.key"

  19. tls.ca.certPath

    1. Definition → The location of the certificate authority TLS certificate of the server in the test network.

    2. Schema → Must be a UNIX filepath.

    3. Default → "./ca.crt"

  20. tls.gen.caSuffix

    1. Definition → The suffix to append to generated PEM files containing both the server’s and CA’s certificate.

    2. Schema → Must be a string with letters, dashes, and/or hyphens.

    3. Default → "ca"

  21. tls.gen.certDays

    1. Definition → The length that the server’s created certificate should be active.

    2. Schema → Must be a number that can be turned into a Python integer.

    3. Default → 1

  22. tls.gen.ext.cert

    1. Definition → The file extension to use for TLS certificates.

    2. Schema → Must be a valid extension.

    3. Default → "crt"

  23. tls.gen.ext.key

    1. Definition → The file extension to use for TLS keys.

    2. Schema → Must be a valid extension.

    3. Default → "key"

  24. tls.gen.ext.pem

    1. Definition → The file extension to use for TLS certificate-key combinations.

    2. Schema → Must be a valid extension.

    3. Default → "pem"

  25. tls.gen.keyLength

    1. Definition → The bit length of the RSA key.

    2. Schema → Must be a number that can be turned into a Python integer.

    3. Default → 4096

  26. tls.gen.pubExponent

    1. Definition → The public exponent of the RSA key.

    2. Schema → Must be a number that can be turned into a Python integer.

    3. Default → 65537

  27. tls.gen.secretTarget

    1. Definition → The filepath (minus the file extension) that should be used to name created server TLS materials.

    2. Schema → Must be a UNIX filepath.

    3. Default → "./self"

  28. tls.san.ips

    1. Definition → The list of IP addresses to use as SANs.

    2. Schema → Must be an IP address or a string formatted as IP addresses separated with a comma.

    3. Default → "127.0.0.1"

  29. tls.san.names

    1. Definition → The list of FDQNs to use as SANs.

    2. Schema → Must be an FQDN or a string formatted as FQDNs separated with a comma.

    3. Default → "localhost"

  30. upperBound

    1. Definition → The number that the server must stop sending new fibonacci numbers if the last number the server received is larger than.

    2. Schema → Must be a number that can be turned into a Python integer.

    3. Default → 4000000000

  31. workers

    1. Definition → The number of server workers to create.

    2. Schema → Must be a number that can be turned into a Python integer.

    3. Default → 3

Tag summary

Content type

Image

Digest

sha256:979b2ac89…

Size

47.6 MB

Last updated

11 months ago

docker pull laoluade/run-fibonacci