Zookeeper

Sources:https://zookeeper.apache.org/

Prerequisite:

Supported Platforms

    GNU/Linux is supported as a development and production platform for both server and client.

    Sun Solaris is supported as a development and production platform for both server and client.

    FreeBSD is supported as a development and production platform for clients only. Java NIO selector support in the FreeBSD JVM is broken.

    Win32 is supported as a development platform only for both server and client.

    MacOSX is supported as a development platform only for both server and client.

Required Software

    ZooKeeper runs in Java, release 1.6 or greater (JDK 6 or greater). It runs as an ensemble of ZooKeeper servers. Three ZooKeeper servers is the minimum recommended size for an ensemble, and we also recommend that they run on separate machines. At Yahoo!, ZooKeeper is usually deployed on dedicated RHEL boxes, with dual-core processors, 2GB of RAM, and 80GB IDE hard drives.

Downloads:

   We can get it here: http://archive.apache.org/dist/zookeeper/

Configure:

   After extraction of downloaded zookeeper file, we need to look into conf directory.

   Copy zoo_sample.cfg to zoo.cfg within conf dir.

   Update value of data dir property in zoo.cfg. 

   If configuring for server mode then zoo.cfg needs following:

        tickTime=2000

        dataDir=/var/zookeeper/

        clientPort=2181

        initLimit=5

        syncLimit=2

        server.1=zoo1:2888:3888

        server.2=zoo2:2888:3888

        server.3=zoo3:2888:3888

   Set <ZOOKEEPER DIR>/bin to system path

   Set <JDK DIR>/bin to system path

Start/Stop:

   zkServer.sh start    #starts zookeeper 

   zkServer.sh stop    #stops zookeeper

How to connect different zkServer using zkCli.sh?

zkCli.sh -server 127.0.0.1:2183

Create a ZNode via CLI.

create /NODENAME -s #This creates a persistent node in zookeeper. '-e' (Ephemeral ZNode) switch can be used for session-scoped nodes and '-s' (Sequential ZNode) switch for znode with unique path.

Get and set Znode.

get /nodeName

set /nodeName {"key1":["val1"],"key2":["val1","val2"]}

Problem-1: Zookeeper error: Cannot open channel to X at election address

Soln : iptables service was running and that was the culprit. ran "service iptables stop" and woooooo!.

Problem-2: 'zkServer.sh start' fails with error process already running. But actually its not running.

Soln : Zookeeper maintains its process id in <ZOOKEEPER_DATAFOLDER>/zookeeper_server.pid file. which is not deleted in case of improper shutdown or termination and hence if another process starts at that PID, zookeeper feels that its process is already running. We need to remove that pid file and re-attempt to start zkServer.sh

Running zookeeper server in win-10

1. Downloaded - https://archive.apache.org/dist/kafka/2.3.1/kafka_2.12-2.3.1.tgz

2. Extracted to - E:\OtherInstallations\kafka_2.12-2.3.1

3. Added Path to evnironment variable PATH - "E:\OtherInstallations\kafka_2.12-2.3.1\bin\windows"

4. Created directory - E:\ProgramData\zookeeper\data and E:\ProgramData\zookeeper\config

5. Created property file - E:\ProgramData\zookeeper\config\zookeeper.properties and added following - 

tickTime=2000

dataDir=E:\\ProgramData\\zookeeper\\data

clientPort=2181

initLimit=5

syncLimit=2

server.1=localhost:2888:3888

6. In cmd, ran - E:\>zookeeper-server-start.bat E:\ProgramData\zookeeper\config\zookeeper.properties

7. Next, checking if its usable from kafka.