msusta/elder-members repository overview

⁠Brief

Original repository: https://github.com/elderstudios/infrastructure-tech-test⁠

Original requirements moved to docs/README.md⁠

⁠Architecture

⁠Inputs

Product team developed a simple container-based application backed by PostgreSQL database and requires a production deployment. There are no specific CIA and performance requirements.

⁠Assumptions

Quick look at the data stored by the application suggest that this will be handling PII. This will require data in-transit and at-rest encryption.

Due to missing performance requirements I'm going to assume the worst scenarion which is a wide-scalability requirement. This means that all components should handle a significant load while also being capable of scaling in to save money. This will help also with future development environment deployment where the same template can be used without incuring significant charges but also supporting even load-testing scenarios.

⁠Design

⁠Application

Application is written in Go and product team already provided a well written Dockerfile. There's a build stage and final container is build on distroless.

Logs are outputed by the program to stdout and captured by the ECS into Cloudwatch Logs.

Application container is build by Docker Hub automatically from the repository. Docker tags are published from repository tags prefixed with "docker-". So a docker-1.0 Git repository tag would lead to a docker.io/msusta/elder-members:1.0 container tag.

msusta/elder-members repository⁠

⁠Network

This deployment requires existing VPC with set of public and private subnets. VPC information is supplied as variables to the Terraform template.

Whole solution is fronted by Application Load Balancer. It provides scalable ingress point into the private network from the Internet. The TLS connection from clients is terminated here. This removes the need to handle the certificate within the application container and lowers compute requirements too.

⁠Compute

• compute capacity provided by ECS/Fargate
• task defined in repository link
• both ECS as container orchestrator and Fargate as capacity provider are completely managed
• Fargate is a virtual capacity provider
  • no servers to deploy, manage, update
  • supports both on-demand and spot capacity

⁠Database

DB is deployed on top of Amazon Aurora with PostgreSQL compatibility with the Serverless capacity provider.

• capacity is automatically scaled without Ops interaction
• DB can be set to "pause" after certain time of idling - saves money for compute capacity
• backups are transparently handled at storage layer without impacting performance
• possibility to use Point-in-Time recovery
• testing environments can build databases upon production data snapshots easily and quickly

⁠Backup

Application is stateless. Only DB backups are required. Backup retention policy and offsite backups to be agreed upon with product team.

⁠Monitoring

• Load balancer
  • performance metrics are available as part of the service
  • can stream logs into Cloudwatch Logs and/or S3
• Compute
  • ECS & Container(s):
    • both logs and metrics are available
    • Container Insights (enhanced montioring) is available
  • Container: logs are available in CW Logs
• Database - both DB metrics & logs are available

⁠Future development

• redirect or remove port 80 on ALB
• service authentication
  • there are many options for solving this - collaborate with product team
• AWS service endpoints in VPC -> limit the outgoing connections from containers
• application development suggestion
  • adapt the container to execute in Lambda and put it behind API GW
    • can provide significant long-term savings
    • handles more load if needed
• if not adapted to Lambda:
  • consider standard RDS for stable loads
  • ECS service scaling configuration