Kafka搭建(单机版)

部署前提

VMware环境 : 两台centos系统

Jdk包:jdk-8u202-linux-x64.tar.gz
Kafka包:kafka_2.12-3.5.0.tgz
Zookeeper包:apache-zookeeper-3.7.2-bin.tar.gz

百度网盘自取: 链接: https://pan.baidu.com/s/11EWuhBoSmH3musd_3Rgodw?pwd=e32t 提取码: e32t

Kafka搭建（单机版）

Centos搭建JDK8

mkdir /usr/local/env/java
cd /usr/local/env/java
wget https://repo.huaweicloud.com/java/jdk/8u202-b08/jdk-8u202-linux-x64.tar.gz
#解压java包到usr/local/env下
tar -zxvf jdk-8u202-linux-x64.tar.gz -C /usr/local/env/java
mv jdk1.8.0_411/* ./
#配置环境变量
vim /etc/profile

#将如下命令添加到文件末尾
export JAVA_HOME=/usr/local/java/jdk1.8.0_202
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH

#ESC退出并输入:wq保存

#使配置生效
source /etc/profile

#检查是否配置成功
java -version

Centos搭建Kafka

解压Kafka安装包

mkdir /usr/local/env/kafka
cd /user/local/env/kafka
tar -zxvf /usr/local/env/kafka_2.12-3.5.0.tgz -C /usr/local/env/kafka
mv kafka_2.12-3.5.0/* ./

配置Zookeeper

#创建zookeeper对应的数据地址、log地址
mkdir -p /usr/local/env/zookeeper/data
mkdir -p /usr/local/env/zookeeper/logs
#修改zookper配置
vim /usr/local/env/kafka/config/zookeeper.properties
#添加如下配置
dataDir=/usr/local/env/zookeeper/data
dataLogDir=/usr/local/env/zookeeper/logs

#设置zookeeperserver-service
#创建zookeeper自启动service
vim /etc/systemd/system/zookeeper.service

#添加如下配置
[Unit]
Description=zookeeper   #描述
Requires=network.target #需要的服务
After=network.target #在什么服务之后启动
[Service]
Type=forking
Environment=JAVA_HOME=/usr/local/env/java/jdk1.8.0_202  #依赖环境
WorkingDirectory=/usr/local/env/zookeeper
ExecStart=/usr/local/env/kafka/bin/zookeeper-server-start.sh -daemon /usr/local/env/kafka/config/zookeeper.properties 
ExecStop=/usr/local/env/kafka/bin/zookeeper-server-start.sh stop
Restart=always
RestartSec=10
TimeoutSec=360
[Install]
WantedBy=multi-user.target


#系统启动命令
systemctl daemon-reload   #重新加载service
systemctl enable zookeeper #开机自启动
systemctl start zookeeper
systemctl status zookeeper
systemctl stop zookeeper

配置Kafka

#修改Kafka配置
vim /usr/local/env/kafka/config/server.properties

firewall-cmd --zone=public --add-port=9092/tcp --permanent 
# 重启防火墙
firewall-cmd --reload

 #添加如下配置
listeners=PLAINTEXT://0.0.0.0:9092
advertised.listeners=PLAINTEXT://127.0.0.1:9092 #如果需要开启外网的情况下 比如外网IP为127.0.0.1
zookeeper.connect=:2181
delete.topic.enable=true
#advertised.listeners=PLAINTEXT://0.0.0.0:9092

#配置Kafka服务
vim /etc/systemd/system/kafka.service

[Unit]
Description=kafka
Requires=network.target
After=network.target zookeeper.service
[Service]
Type=forking
Environment=JAVA_HOME=/usr/local/env/java/jdk1.8.0_202
ExecStart=/usr/local/env/kafka/bin/kafka-server-start.sh -daemon /usr/local/env/kafka/config/server.properties
ExecStop=/usr/local/env/kafka/bin/kafka-server-stop.sh
Restart=always
RestartSec=10
TimeoutSec=360
[Install]
WantedBy=multi-user.target

#启动命令
systemctl daemon-reload
systemctl enable kafka
systemctl start kafka
systemctl status kafka
systemctl restart kafka
systemctl stop kafka


#测试命令
# 生产消息（创建名为testTopic的主题）
./bin/kafka-console-producer.sh --topic testTopic --bootstrap-server 172.16.2.25:9092
# 监听消息（重开一个终端监听testTopic主题的消息，在生产消息的终端发消息，此终端收消息）
./bin/kafka-console-consumer.sh --topic testTopic --from-beginning --bootstrap-server 172.16.2.25:9092

基本命令

Kafka基本命令

生产者启动

./bin/kafka-console-producer.sh --topic myTestTopic --bootstrap-server 101.126.112.112:9092
消费者启动

./bin/kafka-console-consumer.sh --topic myTestTopic --from-beginning --bootstrap-server 101.126.112.112:9092
创建topic

./bin/kafka-topics.sh --create --replication-factor 1 --partitions 1 --topic myTestTopic --bootstrap-server 101.126.112.112:9092
查看全部Topic

./kafka-topics.sh --bootstrap-server 101.126.112.112:9092 --list

部署参考文档

Kafka

WARN AdminClient clientId=adminclient-1 Connection to node -1 (localhost/127.0.0.1:9092) could not_2023-10-24 12:42:45,766warn adminclient client-CSDN博客

CentOS7安装配置Kafka3.2.0（含SpringBoot连接测试）_centos下载kafka-CSDN博客

SpringBoot3.1.7集成Kafka和Kafka安装_springboot3集成kafka-CSDN博客