Zystkmt

I have a confluent kafka setup and docker-compose.yml file

https://github.com/confluentinc/cp-docker-images/blob/5.0.0-post/examples/kafka-cluster/docker-compose.yml

I have edited the file and added the ports option to the docker compose file so that I can access them outside of the host.

---

version: '2' 

services: 

  zookeeper-1: 

    image: confluentinc/cp-zookeeper:latest 

    environment: 

      ZOOKEEPER_SERVER_ID: 1 

      ZOOKEEPER_CLIENT_PORT: 22181 

      ZOOKEEPER_TICK_TIME: 2000 

      ZOOKEEPER_INIT_LIMIT: 5 

      ZOOKEEPER_SYNC_LIMIT: 2 

      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888 

    network_mode: "host" 



  zookeeper-2: 

    image: confluentinc/cp-zookeeper:latest 

    environment: 

      ZOOKEEPER_SERVER_ID: 2 

      ZOOKEEPER_CLIENT_PORT: 32181 

      ZOOKEEPER_TICK_TIME: 2000 

      ZOOKEEPER_INIT_LIMIT: 5 

      ZOOKEEPER_SYNC_LIMIT: 2 

      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888 

    network_mode: "host" 



  zookeeper-3: 

    image: confluentinc/cp-zookeeper:latest 

    environment: 

      ZOOKEEPER_SERVER_ID: 3 

      ZOOKEEPER_CLIENT_PORT: 42181 

      ZOOKEEPER_TICK_TIME: 2000 

      ZOOKEEPER_INIT_LIMIT: 5 

      ZOOKEEPER_SYNC_LIMIT: 2 

      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888 

    network_mode: "host" 



  kafka-1: 

    image: confluentinc/cp-kafka:latest 

    depends_on: 

      - zookeeper-1 

      - zookeeper-2 

      - zookeeper-3 

    ports: 

      - "19092" 

    environment: 

      KAFKA_BROKER_ID: 1 

      KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181 

      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092 

    network_mode: "host" 



  kafka-2: 

    image: confluentinc/cp-kafka:latest 

    depends_on: 

      - zookeeper-1 

      - zookeeper-2 

      - zookeeper-3 

    ports: 

      - "29092" 

    environment: 

      KAFKA_BROKER_ID: 2 

      KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181 

      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:29092 

    network_mode: "host" 



  kafka-3: 

    image: confluentinc/cp-kafka:latest 

    depends_on: 

      - zookeeper-1 

      - zookeeper-2 

      - zookeeper-3 

    ports: 

      - "39092" 

    environment: 

      KAFKA_BROKER_ID: 3 

      KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181 

      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:39092 

    network_mode: "host" 

I have run the following commands:

docker-compose up --no-start

docker-compose start zookeeper-1

docker-compose start zookeeper-2

docker-compose start zookeeper-3



docker-compose run -d --service-ports kafka-1

docker-compose run -d --service-ports kafka-2

docker-compose run -d --service-ports kafka-3

The zookeeper-1,2,3 started but kafka-1 failed with exit code 0.
When I do docker-compose start kafka-1, the ports are not exposed but the service is started.

The output of the above commands:

docker ps

CONTAINER ID        IMAGE                              COMMAND                  CREATED             STATUS              PORTS               NAMES

7abec60edd7a        confluentinc/cp-zookeeper:latest   "/etc/confluent/dock…"   3 minutes ago       Up 3 minutes                            kafka-cluster_zookeeper-2_1_c9f58ba3fbc8

68ec403740d6        confluentinc/cp-zookeeper:latest   "/etc/confluent/dock…"   3 minutes ago       Up 3 minutes                            kafka-cluster_zookeeper-3_1_31e4762a61bb

69d6645487aa        confluentinc/cp-zookeeper:latest   "/etc/confluent/dock…"   3 minutes ago       Up 3 minutes                            kafka-cluster_zookeeper-1_1_8bbd729b09d8

docker logs of kafka-1, kafka-2, kafka-3 (all are same)

[main-SendThread(localhost:42181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:42181. Will not attempt to authenticate using SASL (unknown error)

[main-SendThread(localhost:42181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:42181, initiating session

[main-SendThread(localhost:42181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect

[main-SendThread(localhost:32181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:32181. Will not attempt to authenticate using SASL (unknown error)

[main-SendThread(localhost:32181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/0:0:0:0:0:0:0:1:32181, initiating session

[main-SendThread(localhost:32181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect

[main-SendThread(localhost:22181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:22181. Will not attempt to authenticate using SASL (unknown error)

[main-SendThread(localhost:22181)] INFO org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/0:0:0:0:0:0:0:1:22181, initiating session

[main-SendThread(localhost:22181)] INFO org.apache.zookeeper.ClientCnxn - Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect

[main] ERROR io.confluent.admin.utils.ClusterStatus - Timed out waiting for connection to Zookeeper server [localhost:22181,localhost:32181,localhost:42181].

[main] INFO org.apache.zookeeper.ZooKeeper - Session: 0x0 closed

docker-compose ps

                  Name                              Command            State    Ports

-------------------------------------------------------------------------------------

kafka-cluster_kafka-1_1_c79e5ef5d397       /etc/confluent/docker/run   Exit 0        

kafka-cluster_kafka-2_1_d4399ed0a670       /etc/confluent/docker/run   Exit 0        

kafka-cluster_kafka-3_1_2df6f47759c0       /etc/confluent/docker/run   Exit 0        

kafka-cluster_zookeeper-1_1_8bbd729b09d8   /etc/confluent/docker/run   Up            

kafka-cluster_zookeeper-2_1_c9f58ba3fbc8   /etc/confluent/docker/run   Up            

kafka-cluster_zookeeper-3_1_31e4762a61bb   /etc/confluent/docker/run   Up            

The problem here is that in KAFKA_ADVERTISED_LISTENERS must be set to the IP address of the host. So that when you run it on another machine, that machine will be able to access the kafka using the IP address.

      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://10.1.1.1:39092 

Next point, pointed out by @Ntwobike is to remove the -p option. Not that it is not needed, but it is redundant when the --net=host option is set.

Most important, is to check the firewall. Check the iptables rules and change them to FORWARD ACCEPT (in my case I have done so,since mine is a development host).

If you are using CentOS or RHEL, you can try stopping the firewalld and check to see if it is the firewall that is creating the problem.

systemctl stop firewalld

For others, it is ufw (Ubuntu, Mint)

systemctl stop ufw

Alternatively, you can also do an iptables -F but before that make sure that you backup your iptables rules using

iptables-save > /home/iptables_rules_bak

and then after flushing the rules, you can do

iptables -P FORWARD ACCEPT

If you look at the all-in-one Compose example by Confluent, then everything is setup correctly for broker access in all directions, and not using network: host "hacks" (which only would work on Linux)

Refer

KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT

  KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:9092,PLAINTEXT_HOST://localhost:29092

Outside of the Docker network, you can connect via port 29092, but within, it's 9092

More importantly, the Zookeeper connection strings should actually point at one another, not localhost

I would also like to point out that multiple brokers on one machine doesn't have many benefits, and if you want persistent data (don't lose everything if you restart your computer, or Docker), you'll want volume mounts