Getting Started with Apache Kafka and Spring Boot (Using KRaft Mode)

 Apache Kafka is one of the most widely used distributed streaming platforms in modern applications. It allows you to publish, subscribe, store, and process streams of records in real-time. When combined with Spring Boot, Kafka becomes a powerful tool for building event-driven microservices that can handle massive data flows reliably and efficiently.

---

🔹 What is Apache Kafka?

Kafka is an open-source distributed event streaming platform used for high-performance data pipelines, streaming analytics, and event-driven architectures. Unlike traditional message brokers, Kafka is designed for horizontal scalability, fault tolerance, and high throughput.

At its core, Kafka works with:

• Producers → applications that publish (write) data to topics.

• Consumers → applications that subscribe to (read) data from topics.

• Topics → categories or feeds to which records are published.

• Brokers → servers that store and serve Kafka data.

---

🔹 Why Use Kafka with Spring Boot?

Spring Boot provides seamless integration with Kafka via Spring for Apache Kafka (spring-kafka dependency). Together, they offer:

1. Event-Driven Microservices – services communicate via Kafka topics instead of REST, reducing tight coupling.

2. Scalability & Resilience – Kafka can handle millions of events per second, and Spring Boot apps can consume/produce at scale.

3. Asynchronous Communication – services don’t block each other; messages are delivered reliably.

4. Integration Flexibility – Kafka integrates easily with databases, monitoring tools, and external systems.

---

🔹 Installing and Running Kafka in KRaft Mode (No ZooKeeper)

Since Kafka 3.3+, you can run Kafka without ZooKeeper using KRaft mode. Below are the steps to set up and run Kafka on Windows.

1. Download and Extract Kafka

• Download from Apache Kafka Downloads.

• Extract it to C:\kafka.

---

2. First-Time Setup

Open a terminal in C:\kafka and run:

C:\kafka> .\bin\windows\kafka-storage.bat random-uuid

Copy the UUID, then format the storage directory:

C:\kafka> .\bin\windows\kafka-storage.bat format -t <YOUR-UUID-HERE> -c .\config\server.properties

3. Start Kafka Broker

C:\kafka> .\bin\windows\kafka-server-start.bat .\config\server.properties

 Leave this window running – it’s your Kafka broker.

4. Create a Topic

In a new terminal: 

C:\kafka> .\bin\windows\kafka-topics.bat --create --topic test-topic --partitions 1 --replication-factor 1 --bootstrap-server localhost:9092

5. List Topic

C:\kafka> .\bin\windows\kafka-topics.bat --list --bootstrap-server localhost:9092

6 Start a Producer

C:\kafka> .\bin\windows\kafka-console-producer.bat --topic test-topic --bootstrap-server localhost:9092

Now type a few messages (press Enter after each).

7 Start a Consumer

In another terminal: 

C:\kafka> .\bin\windows\kafka-console-consumer.bat --topic test-topic --from-beginning --bootstrap-server localhost:9092

You should see the producer messages flow into the consumer: 

Hello Kafka

This is my first message

another message

🔹 Conclusion

Kafka is essential for building event-driven, scalable microservices with Spring Boot. By running Kafka in KRaft mode, you simplify setup (no ZooKeeper needed) while keeping the full power of Kafka. With a few commands, you can start publishing and consuming messages locally, then integrate them into your Spring Boot applications for real-world use.