Devii · Data & analytics · 2026-03-27 · 9 min read
Apache Kafka: Topics, Partitions, Offsets, And Consumer Groups
Distributed commit log basics from the Kafka documentation, not stream hype.
**Apache Kafka** is a distributed event streaming platform. Producers append records to **topics** split into **partitions** ordered by offset. Brokers replicate partitions for fault tolerance.
**Consumer groups** divide partitions among members; each partition is consumed by one member in the group at a time. Offsets track progress; commits can be automatic or manual after processing.
Retention is time or size based. Kafka is not a task queue for every job: keys route related events to the same partition for ordering; poison messages need DLQ topics and operational playbooks.
Read `kafka.apache.org/documentation` for your broker version. Test rebalance behavior during deploys and scale events.