Produce and consume messages from a Kafka topic using docker
Hello Sudo, docker is an easy way to run applications in a containerized manner, and let us run the Apache Kafka server using docker in this article. Let us also consume and produce messages inside a docker container.
Copy the following and save it as “docker-compose.yml” file in your local folder.
version: '3'services: zookeeper: image: wurstmeister/zookeeper container_name: zookeeper ports: - "2181:2181" kafka: image: wurstmeister/kafka container_name: kafka ports: - "9092:9092" environment: KAFKA_ADVERTISED_HOST_NAME: localhost KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
After saving the file navigate to the folder, then we can run the Kafka inside the docker container using the following command.
If you want to run Kafka server as daemon docker container run
docker-compose up -d
In order to create and delete topics, we need to get into the docker container. Once after running the Kafka server through docker open a new terminal window and we need to get into the Kafka’s docker container using the following command.
docker exec -it kafka /bin/sh
then navigate to the bin folder after getting into the docker container by,
You will get the current working path like the below image.
If you list the directory you can see files that are related to Kafka.
To create a topic execute
kafka-topics.sh --bootstrap-server localhost:9092 --topic myFirstTopic --create
To list all topics execute
kafka-topics.sh --list --bootstrap-server localhost:9092
To delete a topic execute
kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic myFirstTopic
In order to produce messages on a particular topic, execute
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic myFirstTopic
then you will get like the following image
In order to consume a message from a particular topic, execute
(Open a new terminal and get into the docker container, change the directory to the bin folder and execute the following command)
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic myFirstTopic --from-beginning
Now you can type messages in the producer and you will get those messages in the consumer
Thanks for reading my first medium post, your feedback or most welcomed. Stay tuned for the next medium post on “Understanding topics, partitions and group-ids in Kafka”