AutoMQ

AutoMQ is a fork of Apache Kafka with the storage engine replaced by one built on object storage, turning the classic shared-nothing design into shared storage. It targets the two things that make Kafka painful in the cloud: brokers hold state, so scaling means moving data, and block storage plus cross-zone traffic plus over-provisioning make the bill grow faster than the workload.

Because brokers keep no durable state, they scale in and out within seconds, and a built-in auto-balancer moves partitions and network traffic between them with no manual reassignment. Data lives on S3 or any compatible store, MinIO included.

The properties that follow from that design are concrete.

• Zero cross-zone traffic: object storage carries replication, removing the transfer costs that dominate a typical cloud Kafka bill.
• Durability: zero RPO, recovery measured in seconds and eleven nines of durability, inherited from the object store.
• Auto scaling: cluster metrics drive capacity up and down, so you pay for the workload rather than the annual peak.
• Kafka compatible: existing clients, tools and protocols work unchanged, since only the storage layer differs.
• Metrics export: Prometheus and OpenTelemetry natively, by push or pull, instead of JMX.

A single-node Docker Compose file with MinIO gives you an evaluation cluster in minutes, and a three-node file exercises clustering. Production deployment is documented for Linux and Kubernetes, and the maintainers say plainly that the quick start is not production shaped.