The following explanation from Strimzi about the difficulty of implementing autoscaling in Kubernetes can also be applied to answer this question. Using spot instances requires the computing layer to be stateless.
ref:https://github.com/orgs/strimzi/discussions/6635
For Kafka brokers, auto-scaling is complicated because of their architecture. Adding or removing brokers is simple. But directing some load to them is complicated because they are in a way form of data storage. And moving the whole partitions between brokers is expensive. The partitions often contain huge amounts of data which need to be shifted from one broker to another - that will take time, it will have a performance penalty on the other traffic and possibly cost even real money for the data transfers.
