I think we need to distinguish between two cases:
For a hobby project, using Docker Compose or Podman combined with systemd and some shell scripts is perfectly fine. You’re the only one responsible, and you have the freedom to choose whatever works best for you.
However, in a company setting, things are quite different. Your boss may assign you new tasks that could require writing a lot of custom scripts. This can become a problem for other team members and contractors, as such scripts are often undocumented and don’t follow industry standards.
In this case, I would recommend using Kubernetes (k8s), but only if the company has a dedicated Kubernetes team with an established on-call rotation. Alternatively, I suggest leveraging a managed cloud service like ECS Fargate to handle container orchestration.
There’s also strong competition in the "Container as a Service" (CaaS) space, with smaller and more cost-effective options available if you prefer to avoid the major cloud providers. Overall, these CaaS solutions require far less maintenance compared to managing your own cluster.
> dedicated Kubernetes team with an established on-call rotation.
Using EKS or GKS is basically this. K8s is much nicer than ECS in terms of development and packaging your own apps.
Up until a few thousand instances, a well designed setup should be a part time job for a couple of people.
To that scale you can write a custom orchestrator that is likely to be smaller and simpler than the equivalent K8S setup. Been there, done that.
How would you feel if bash scripts were replaced with Ansible playbooks?
At a previous job at a teeny startup, each instance of the environment is a docker-compose instance on a VPS. It works great, but they’re starting to get a bunch of new clients, and some of them need fully independent instances of the app.
Deployment gets harder with every instance because it’s just a pile of bash scripts on each server. My old coworkers have to run a build for each instance for every deploy.
None of us had used ansible, which seems like it could be a solution. It would be a new headache to learn, but it seems like less of a headache than kubernetes!
Ansible is better than Bash if your goals include:
* Automating repetitive tasks across many servers.
* Ensuring idempotent configurations (e.g., setting up web servers, installing packages consistently).
* Managing infrastructure as code for better version control and collaboration.
* Orchestrating complex workflows that involve multiple steps or dependencies.
However, Ansible is not a container orchestrator.
Kubernetes (K8s) provides capabilities that Ansible or Docker-Compose cannot match. While Docker-Compose only supports a basic subset, Kubernetes offers:
* Advanced orchestration features, such as rolling updates, health checks, scaling, and self-healing.
* Automatic maintenance of the desired state for running workloads.
* Restarting failed containers, rescheduling pods, and replacing unhealthy nodes.
* Horizontal pod auto-scaling based on metrics (e.g., CPU, memory, or custom metrics).
* Continuous monitoring and reconciliation of the actual state with the desired state.
* Immediate application of changes to bring resources to the desired configuration.
* Service discovery via DNS and automatic load balancing across pods.
* Native support for Persistent Volumes (PVs) and Persistent Volume Claims (PVCs) for storage management.
* Abstraction of storage providers, supporting local, cloud, and network storage.
If you need these features but are concerned about the complexity of Kubernetes, consider using a managed Kubernetes service like GKE or EKS to simplify deployment and management. Alternatively, and this is my prefered option, combining Terraform with a Container-as-a-Service (CaaS) platform allows the provider to handle most of the operational complexity for you.
Ansible ultimately runs scripts, in parallel, in a defined order across machines. It can help a lot, but it's subject to a lot of the same state bitrot issues as a pole of shell scripts.