Kafka’s Operational Architecture — Part -2

Kafka is written in Scala and Java programming languages. It runs on Java Virtual Machine (JVM), the wonderful thing about JVM is that it allows Kafka to run anywhere independently. As it is on JVM, it helped the developer to create tools and libraries easy for Kafka. This made Kafka pervasive in any data related works.

Kafka cluster has one or more brokers using ZooKeeper Kafka configures and identifies the stack

Key terms which I will be using in this blog:

• Broker- A member server of the Kafka cluster.
• Cluster- A group of Kafka brokers working together to manage production and consumption calls.
• Zookeeper- Used by Kafka brokers to determine and stack a partition and topic, as well as Kafka configuration.

All Kafka broker belongs to a Kafka cluster, Kafka cluster has one more broker in it. Kafka cluster can have one or 1000s of Kafka brokers, in order to keep track of all the brokers. Apache created ZooKeeper, this used for few major things in Kafka :

1. Zookeeper used to determine which brokers belong to which Kafka cluster. It also uses zookeepers to understand which broker is the leader of the cluster.
2. Zookeeper stores user permission and roles. Popularly known as Access Control Lists (ACLs)
3. Using Zookeeper new brokers can join and leave seamless. Allowing the Kafka cluster to grow and…