Ah, interesting. What kind of customization are you using CoreDNS for? If you don’t have Ingress/Gateway API for your HTTP traffic, Traefik is likely a good option for adopting it.

All containers in a pod share an IP, so you can just use localhost: https://www.baeldung.com/ops/kubernetes-pods-sidecar-containers

Between pods, the universal pattern is to add a Service for your pod(s), and just use the name of the service to connect to the pods the Service is tracking. Internally, the Service is a load-balancer, running on top of Kube-Proxy, or Cilium eBPF, and it tracks all the pods that match the correct labels. It also takes advantage of the Kubelet’s health checks to connect/disconnect dying pods. Kubedns/coredns resolves DNS names for all of the Services in the cluster, so you never have to use raw IP addresses in Kubernetes.

Go all out and register your container IPs on your router with BGP 😁

https://codeberg.org/jlh/h5b/src/branch/main/argo/custom_applications/kube-system/cilium-bgp-policy.yaml

(This comment was sent over a route my automation created with BGP)

I’ve inherited it on production systems before, automated service discovery and certificate renewal is definitely what admins should have in 2025. I thought the label/annotation system it used on Docker had some ergonomics/documentation issues, but nothing serious.

It feels like it’s more meant for Docker/Podman though. On Kubernetes I use cert-manager and Gateway API+Project Contour. It does seem like Traefik has support for Gateway API too, so it’s probably a good choice for Kubernetes too?

Good debugging!

I’m thinking that it’s best for production to use dynamic IP addresses, to avoid this kind of conflict. In the kubernetes space, all containers must have dynamic IP addresses, which are then tracked by a ebpf load balancer with a (somewhat) static IP.

“Hey guys what’s the name of the children’s book character that’s a bear with a red shirt in the woods”

“Winnie the poo-” *banned*

Europe is increasingly looking like the battlefield between China and the US

The nextcloud helm chart is nice

Maybe crowdsec could add a list for blocking scraping for LLMs

https://app.crowdsec.net/blocklists/search?page=1

Exciting!

Ah, ok I see.

only need dedup if your data is duplicated

Nope, you don’t need any VPS to use it, it comes with an SFTP interface.

https://www.hetzner.com/storage/storage-box/

offsite backup for $2/TB and no download fees, 1/3rd the price of B2.

Very cool! Fun fact, this is basically a simplified version of bitcoin mining.

In order to mine a reward of 3 bitcoin, you just need to find a sha-256 hash that is lower than 2.7×10⁶⁷

https://bitcoin.stackexchange.com/questions/85896/mining-difficulty-and-leading-zeros

The world currently calculates around 780 bitcoin exahashes/s, so one of these $300k hashes is found about every 10 minutes on average.

https://www.blockchain.com/explorer/charts/hash-rate https://www.coinwarz.com/mining/bitcoin/difficulty-chart https://www.bitcoinblockhalf.com/

Hetzner storage box is super cheap and works with rclone. They have a web interface for configuring regular zfs snapshots too so you don’t have to worry about accidental deletions/ransomware.

Hardware-wise:

• Reorganize my networking closet and rack up my switches
• Replace my core switch with 10 gbit, connect up 10Gbit fiber to my laptop dock and one of my nodes still on copper
• Add 3 more nodes to my cluster with nvme storage so that I can start an erasure-coding pool in ceph.

Software wise, too many projects to count lol

Renovate is a very useful tool for automatically updating containers. It just watches a git repo and automatically updates stuff.

I have it configured to automatically deploy minor updates, and for bigger updates, it opens a pull request and sends me an email.

https://docs.renovatebot.com/

https://codeberg.org/jlh/h5b/pulls/53

Yeah full VMs are pretty old school, there are a lot more management options and automation available with containers. Not to mention the compute overhead.

Red Hat doesn’t even recommend businesses to use VMs anymore, and they offer a virtualization tool that runs the VMs inside a container for legacy apps. Its called Openshift Virtualization.

Yeah unraid is the same, it just adds a Gui to make it easier to learn. The downside is that unraid is very non-standard and is basically impossible to back up or manage in source control like vanilla docker or kubernetes

You should keep your docker/kubernetes configuration saved in git, and then have something like rclone take daily backups of all your data to something like a hetzner storage box. That is the setup I have.

My entire kubernetes configuration: https://codeberg.org/jlh/h5b/src/branch/main/argo/custom_applications

My backup cronjob: https://codeberg.org/jlh/h5b/src/branch/main/argo/custom_applications/backups/rclone-velero.yaml

With something like this, your entire setup could crash and burn, and you would still have everything you need to restore safely stored offsite.