Recently there have been conversations online regarding the proper configuration of a SolrCloud/ZooKeeper with Sitecore setup. This post will walk through the diagram above which illustrates a possible configuration of the ensemble.
The purpose of ZooKeeper is to maintain, in memory, the current state of each Solr server, the index configuration information stored in ‘z-nodes’, and also ensure that there is always a ZooKeeper leader via a leader election process. Using in-memory to store this information allows for low latency and high throughput when accessing.
In the diagram above, there are 3 ZooKeepers to maintain server requests. A primary principle of ZooKeeper is to maintain a majority, or quorum. That is the recommended minimum as 3 zk servers tolerates 1 zk server being down; 5 zk servers tolerates 2 zk servers being down at the same time. An odd number of servers (greater than 1) is recommended.
All ZooKeeper servers are known each Solr node and stored in the solr.xml file in the zkhost node, comma-separated. Example of 3 zk nodes:
A good detailed reference on ZooKeeper (not in conjunction with Solr/Sitecore) is: https://www.hakkalabs.co/articles/apache-zookeeper-introduction.
The number of SolrCloud servers (aka nodes) in the example of above (2) is demonstrating two replica servers. In SolrCloud, there is no master or slave(s). Instead, a server is a replica and/or shard, which means the data on the server is duplicated on another node (replica) and/or the entire index is divided amongst the servers and a server contains a portion, or shard, of the data. Typically sharding is reserved for indexes that are too large to fit on a single server, however, sharding without necessarily maxing out the capacity of the servers may be a performance strategy as well.
Contrary to some information out there, a load balance between SolrCloud/Zookeeper and Sitecore would not ‘load balance’ the ZooKeeper nodes. The load balancer address would be in Sitecore’s configuration file, however, the load balancer is balancing the index and query requests into the Solr nodes.
In comments below, please feel free to put in additional relevant high-level information and I can add it to this post.