Downscaling the ES cluster

Downscaling the DataNodes in Elastic Search Cluster

We will use the following steps to remove the datanodes:

1. Remove elasticsearch node from the cluster just run the following command

curl -XPUT P.P.P.P:9200/_cluster/settings -d '{
 "transient" :{
 "cluster.routing.allocation.exclude._ip" : "X.X.X.X"
 }
}';echo

Here P.P.P.P is the private IP of the master node.This command will give acknowledgement true if the node is accepted to be removed and the data relocation will start.

2. Check if the data relocation is over and the node doesn't have any shards left on it using the following command:

curl -XGET 'localhost:9200/_cat/allocation?v&pretty'

 When the shards and disk.indices on the node gets 0 that means all data is being transferred to other nodes

3. Stop the elastic search service on the data node and terminate the instances.

Downscaling the master nodes

1. Identify the actual master of the cluster by using the following command.

curl -XGET 'localhost:9200/_cat/master?v&pretty'

2. Remove the all candidate masters from the elastic load balancer one at a time and stop the elasticsearch service using the following command.

service elasticsearch stop

3. Update the elasticsearch.yml on the remaining masters with the IP address of the remaining master nodes by removing one master node at a time from ELB and restart the elasticsearch service.

    discovery.zen.ping.unicast.hosts: ["host1", "host2"]

4. Add the machine behind the ELB. 

5. Repeat the steps from 1 to 4 for all the candidate masters.

6. While updating the configuration on the actual master please make sure that you perform the above steps as soon as possible.

7. While removing the masters monitor the server stats for the cluster.