Tips to improve Kafka performance
Create Kafka Cluster with 3 nodes
Edit file “server.properties” with different broker.id and advertised.listeners: Delete folder “logs” which defined in config (“/home/kafka/logs”) in secondary nodes and folder “zookeeper”. Stop all nodes then restart all node: Important! In node 01: Open meta.properties file in /home/kafka/logs. Copy cluster.id value. Paste cluster.id value to /home/kafka/logs/meta.properties in other node, for the same cluster. For multi…
[Full install] Streaming HLS with Nginx & FFMPEG
This guide for Centos/ RHEL/ Rocky Linux,… So, you can use for other Linux. Install some required packages Install NGINX from source Download some required packages: ZLIB: Find lastest version here: https://zlib.net/ My version is https://zlib.net/zlib-1.2.12.tar.gz PCRE Find lastest version here: https://www.pcre.org/ or https://sourceforge.net/projects/pcre/files/pcre2/ My version is pcre2-10.37.tar.gz OPEN SSL Find lastest version here: https://www.openssl.org/source/…
Kafka CLI library- most Apache Kafka & Zookeeper commands in Linux OS
Make sure that you are in Kafka folder: Bellow example is Localhost, but you can do with any server if the true port is opened. Default Kafka port is 9092. You can change port number in config/server.properties. 1- Topics Kafka command 1.1- Create new topic more options: If replication-factor > 1 and partitions > 1,…
Send (publish, produce) a KAFKA message from a server to another server
Send (publish, produce) a message from a server to another server. In destination Kafka server, add this config to server.properties (my file is: /home/kafka/kafka/config/server.properties): Your IP is exactly which you set in producer… Or replace with domain. If you want to delete a KAFKA topic: So, it’s easy to send a KAFKA message to another…
NestJS (NodeJS) sending message to Kafka topic is easy
NestJS (NodeJS) send message to Kafka topic easily in this quick trick: Create a new NestJS test project: Paste this code in main.ts: Install package: Make sure topic: ‘test-server’ was created on target Kafka server, and you are subscribing on that topic to watch the result. Run script: Note: Your script only runs if you…
Download and config Kafka MQTT Connector
After Install Kafka… Download Kafka MQTT Connector Besure that java and maven were installed. Download: If not working, download from this source: https://drive.google.com/file/d/10Gw-NGxdz48eg7LpJ1klIpVjv2XxlolB/view Note: Kafka Connector default port is TCP 8083. It’s conflict Mosquitto default port. We need to change config of Mosquitto (or your broker). Example, I change websocket port from 8083 to 9093:…
Kafka quick install in ONE paste
Install Apache Kafka Copy and paste this to Centos8 terminal (just change some commands if your OS is different): You can run Kafka test command after “cd /home/kafka/kafka “ Create Kafka Cluster Read more: https://quicktricks.net/create-kafka-cluster-with-3-nodes/ Install and config Kafka Connector Read more: https://quicktricks.net/download-and-config-kafka-mqtt-connector/
Install and Config Mosquitto (MQTT broker server)
MQTT is a super lightweight IoT protocol on TCP/IP with …8 bits header. Send and receive messages with others in “topic” like as a chatroom. Install Mosquitto (MQTT Broker Server) Install MQTT broker on server Create user and password (exp: admin): Sercurity config: Open file /etc/mosquitto/mosquitto.conf to add this config to require password when connect:…