undefined | Better HN

0 pointswhalesalad23h ago0 comments

I haven't abandoned Proxmox yet, but the take here resonates with me. I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc. I have had pretty good luck managing Proxmox clusters with the Proxmox API (https://pve.proxmox.com/pve-docs/api-viewer/index.html) or just letting the agent shell in as root (lol). I built a very simple provisioning tool called vmfactory that takes some really somple config on disk, bakes a fresh qcow image, pushes it to proxmox and then configures networking and boots it. It's extremely rudimentary but has been working well for me.

I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS. I have a domain-joined Ubuntu/ZFS box that inherits a lot of policy from FreeIPA and/or Ansible config that is all backed by files on disk. It's been really easy to orchestrate what many would consider overkill in my homelab because literally everything is represented in a single Github repo.

I yanked vmfactory out and into a standalone repo if anyone is interested: https://github.com/whalesalad/vmfactory

0 comments

6 comments · 2 top-level

mikestorrent22h ago· 4 in thread

https://github.com/Telmate/terraform-provider-proxmox ...

I run Ansible against Proxmox hosts to ensure the config on them is what I want, and then Terraform all VMs into place on them. It's not too far off from having your own mini-EC2, minus some of the nice trappings like load balancers.

wingmanjd19h ago

At $DAYJOB, we use the Telmate terraform proxmox provider, too. Upgrades over the years have had a few papercuts (mainly when values are shifted around), but it has overall been an immense timesaver. A terraform repo contains the 2 colocation datacenter proxmox environments plus a local lab edition of the same builds (locallab uses containers vs full VM's on the "real" hardware so that I can run almost all of the configuration on my laptop). Saltstack gets deployed during the buildouts, which picks up the OS/ application configuration after the OS is deployed.

No agentic stuff on our stack, as our security posture can't afford that currently.

As for the load-balancing, I think the later versions have supported targeting proxmox clusters vs a single node, and the newest Proxmox can do resource auto-balancing. That might get you what you need

mikestorrent16h ago

Ah, I mean load balancing like an AWS ALB; obviously one can use an Nginx or HAproxy for that, but it's not a primitive, you need to deploy a machine and run it.

They have auto-balancing now? Damn. I wrote a balancer using the Google OR-SAT solver because there was no VMWare DRS equiv.

catdog18h ago

Switched to that one some time ago, more features and adapts faster to changes in newer proxmox versions https://github.com/bpg/terraform-provider-proxmox

mikestorrent16h ago

I will consider switching, but with like 400 VMs now it'd be a lot of work. Do you happen to know if it lets you deal with disks separately from the VM host? Used to use that flow on VMware, taint a machine but not its stateful secondary disk, and bring it back up with a fresh image and re-attach a database disk or whatever.

h4kunamata19h ago

>I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc.

I hardle ever access Proxmox GUI, everything is fully managed via Ansible playbooks, from deploying the LXC containers to updating them, from taking snapshot to syncing stuff to my TrueNAS is done via batching.

>I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS

The most common mistake people seem to make when it comes to NAS, TrueNAS is a NAS not a Proxmox server.

I see people running dozens if not hundreds of services on UnRaid, one fart and the whole infrastructure, if I can call it that, goes to sh1t.

1 more reply

j / k navigate · click thread line to collapse

0 comments

6 comments · 2 top-level

mikestorrent22h ago· 4 in thread

https://github.com/Telmate/terraform-provider-proxmox ...

wingmanjd19h ago

No agentic stuff on our stack, as our security posture can't afford that currently.

mikestorrent16h ago

Ah, I mean load balancing like an AWS ALB; obviously one can use an Nginx or HAproxy for that, but it's not a primitive, you need to deploy a machine and run it.

They have auto-balancing now? Damn. I wrote a balancer using the Google OR-SAT solver because there was no VMWare DRS equiv.

catdog18h ago

Switched to that one some time ago, more features and adapts faster to changes in newer proxmox versions https://github.com/bpg/terraform-provider-proxmox

mikestorrent16h ago

h4kunamata19h ago

>I do not like configuring appliances. I prefer defining infra as code, having that diffable, assertable, etc.

>I did abandon TrueNAS, however. It really is a locked-down appliance. Good luck installing custom software on the base OS

The most common mistake people seem to make when it comes to NAS, TrueNAS is a NAS not a Proxmox server.

I see people running dozens if not hundreds of services on UnRaid, one fart and the whole infrastructure, if I can call it that, goes to sh1t.

1 more reply

j / k navigate · click thread line to collapse