reissbaker 3 days ago

Even without needing to spawn additional Docker containers, I think people are more afraid of Kubernetes than is warranted. If you use a managed K8s service like Azure, AWS, GCP, and tons of others provide, it's... Pretty simple and pretty bulletproof, assuming you're doing simple stuff with it (i.e. running a standard web app).

The docs for K8s are incredibly bad for solo devs or small teams, and introduce you to a lot of unnecessary complexity upfront that you just don't need: the docs seem to be written with megacorps in mind who have teams managing large infrastructure migrations with existing, complex needs. To get started on a new project with K8s, you just need a pretty simple set of YAML files:

1. An "ingress" YAML file that defines the ports you listen to for the outside world (typically port 80), and how you listen to them. Using Helm, the K8s package manager, you can install a simple default Nginx-based ingress with minimal config. You probably were going to put Nginx/Caddy/etc in front of your app anyway, so why not do it this way?

2. A "service" YAML file that allocates some internal port mapping used for your web application (i.e. what port do you listen on within the cluster's network, and what port should that map to for the container).

3. A "deployment" YAML file that sets up some number of containers inside your service.

And that's it. As necessary you can start opting into more features; for example, you can add health checks to your deployment file, so that K8s auto-restarts your containers when they die, and you can add deployment strategies there as well, such as rolling deployments and limits on how many new containers can be started before old ones are killed during the deploy, etc. You can add resource requests and limits e.g. make sure my app has at least 500MB RAM, and kill+restart it if it cross 1GB. But it's actually really simple to get started! I think it compares pretty well even to the modern Heroku-replacements like Fly.io... It's just that the docs are bad and the reputation is that it's complicated — and a large part of that reputation is from existing teams who try to do a large migration, and who have very complex needs that have evolved over time. K8s generally is flexible enough to support even those complex needs, but... It's gonna be complex if you have them. For new projects, it really isn't. Part of the reason other platforms are viewed as simpler IMO is just that they lack so many features that teams with complex needs don't bother trying to migrate (and thus never complain about how complicated it is to do complicated things with it).

You can have Claude or ChatGPT walk you through a lot of this stuff though, and thereby get an easier introduction than having to pore through the pretty corporate official docs. And since K8s supports both YAML and JSON, in my opinion it's worth just generating JSON using whatever programming language you already use for your app; it'll help reduce some of the verbosity of YAML.

1
Agingcoder 3 days ago

What you’re saying is that starting a service in kubernetes as a dev is ok, what other people say is that operating a k8s cluster is hard.

Unless I’m mistaken the managed kubernetes instances were introduced by cloud vendors because regular people couldn’t run kubernetes clusters reliably, and when they went wrong they couldn’t fix them.

Where I am, since cloud is not an option ( large mega corp with regulatory constraints ) they’ve decided to run their own k8s cluster. It doesn’t work well, it’s hard to debug, and they don’t know why it doesn’t work.

Now if you have the right people or can have your cluster managed for you, I guess it’s a different story.

reissbaker 3 days ago

Most megacorps use AWS. It's regrettable that your company can't, but that's pretty atypical. Using AWS Kubernetes is easy and simple.

Not sure why you think this is just "as a dev" rather than operating in production — K8s is much more battle-hardened than someone's random shell scripts.

Personally, I've run K8s clusters for a Very Large tech megacorp (not using managed clusters; we ran the clusters ourselves). It was honestly pretty easy, but we were very experienced infra engineers, and I wouldn't recommend doing it for startups or new projects. However, most startups and new projects will be running in the cloud, and you might as well use managed K8s: it's simple.

Agingcoder 3 days ago

Sorry I wasn’t clear.

By ´As a dev´ I meant that listing three files was not going to help you when things went wrong in production, and you had bizarre network problems because of problems with your cni, kernel vs kubernetes issues, etc. I understand fully it can be made to work, but not by everyone and not everyone has the time to do so.

So I take from your explanations that as a k8s user I have two options : have very good infra engineers, or rent k8s from a cloud provider.

( side note : I work primarily in hpc, and have found that having strong infra engineers makes a world of difference , and many software engineers don’t appreciate it enough )

anentropic 3 days ago

> Most megacorps use AWS. It's regrettable that your company can't, but that's pretty atypical.

Even then, it seems like you can run EKS yourself:

https://github.com/aws/eks-anywhere

"EKS Anywhere is free, open source software that you can download, install on your existing hardware, and run in your own data centers."

(Never done it myself, no idea if it's a good option)