Procedure

  1. Create a file /var/zookeeper/myid on each cluster node with unique value (for example 1,2,3) on each node.
    mkdir -p /var/zookeeper
    echo "1" > /var/zookeeper/myid
  2. Edit zookeeper.properties file under <KAFKA_HOME>/config directory.
    dataDir=/var/zookeeper
    clientPort=2181
    maxClientCnxns=0
    tickTime=2000
    initLimit=5
    syncLimit=10
    dataLogDir=/var/log/zookeeper
    server.1=<KAFKA_CLUSTER_HOST1_IPADDRESS>:3181:4181
    server.2=<KAFKA_CLUSTER_HOST2_IPADDRESS>:3181:4181
    server.3=<KAFKA_CLUSTER_HOST3_IPADDRESS>:3181:4181
    authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
    authProvider.2=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
    authProvider.3=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
    requireClientAuthScheme=sasl
    Note: 2181 is the port on which Zookeeper will listen for connections. Instead of 3181 and 4181, any port number can be used, but ensure same ports are used across all 3 nodes.
  3. Create a new zookeeper_jaas.conf file under <KAFKA_HOME>/config directory with below content:
    Server {
    org.apache.zookeeper.server.auth.DigestLoginModule required
    user_super="zookeeper"
    user_kafka="zoo-pwd";
    };
    Note: The file zookeeper_jaas.conf is used for authentication.
    Note: user_super: superuser will have automatically administrator privileges.
    Note: user_kafka: kafka is the username and password is zoo-pwd. The user kafka and password can be anything.
  4. Start Zookeeper server by running below command:
    KAFKA_OPTS="-Djava.security.auth.login.config <KAFKA_HOME>/config/zookeeper_jaas.conf" <KAFKA_HOME>/bin/zookeeper-server-start.sh -daemon <KAFKA_HOME>/config/zookeeper.properties
  5. Check the status of Zookeeper service by running any the below command:
    a. ps -aef | grep zookeeper or ps -aef | grep zookeeper.properties 
    b. lsof -i:2181 
    c. netstat -tnlup | grep 2181