Install Apache Kafka
Copy and paste this to Centos8 terminal (just change some commands if your OS is different):
java -version
dnf install java-1.8.0-openjdk-devel -y
tee /etc/systemd/system/zookeeper.service <<EOF
[Unit]
# bellow config for Rocky Linux or Centos Stream
# Requires=network-online.target
# After=network-online.target
Requires=network.target remote-fs.target
After=network.target remote-fs.target
[Service]
Type=simple
User=kafka
ExecStart=/bin/sh -c '/home/kafka/kafka/bin/zookeeper-server-start.sh /home/kafka/kafka/config/zookeeper.properties > /home/kafka/zookeeper.log 2>&1'
ExecStop=/bin/sh -c '/home/kafka/kafka/bin/zookeeper-server-stop.sh'
Restart=on-abnormal
[Install]
WantedBy=multi-user.target
EOF
tee /etc/systemd/system/kafka.service <<EOF
[Unit]
Requires=zookeeper.service
After=zookeeper.service
[Service]
Type=simple
User=kafka
ExecStart=/bin/sh -c '/home/kafka/kafka/bin/kafka-server-start.sh /home/kafka/kafka/config/server.properties > /home/kafka/kafka/kafka.log 2>&1'
ExecStop=/bin/sh -c '/home/kafka/kafka/bin/kafka-server-stop.sh'
Restart=on-abnormal
[Install]
WantedBy=multi-user.target
EOF
killall -u kafka
userdel -r kafka
adduser kafka
usermod -aG wheel kafka
echo '%wheel ALL=(ALL) NOPASSWD: ALL' &>>/etc/sudoers
su -l kafka
mkdir /home/kafka/kafka
curl "https://dlcdn.apache.org/kafka/3.1.0/kafka_2.13-3.1.0.tgz" -o /home/kafka/kafka/kafka.tgz
cd /home/kafka/kafka
tar -xzf kafka.tgz --strip 1
sed -i 's/\/tmp\/kafka-logs/\/home\/kafka\/logs\ndelete.topic.enable = true/g' ~/kafka/config/server.properties
sed -i 's/\/tmp\/zookeeper/\/home\/kafka\/zookeeper/g' ~/kafka/config/zookeeper.properties
sudo systemctl enable zookeeper
sudo systemctl enable kafka
sudo systemctl start kafka
sudo systemctl start zookeeper
sudo systemctl status kafka
You can run Kafka test command after “cd /home/kafka/kafka “
Create Kafka Cluster
Read more: https://quicktricks.net/create-kafka-cluster-with-3-nodes/
Install and config Kafka Connector
Read more: https://quicktricks.net/download-and-config-kafka-mqtt-connector/
