Manually allocating shards when Elasticsearch cluster is red

If you have large number of shards  with replica sets with huge amount of data its possible that you  get the ES cluster as red. The ES cluster goes red due to the issues with the primary shards which gets unassigned now depending on the situation you can take number of steps to resolve this issue.

However as the last resort you might have to allocate the shard manually but its last recommendation best way it to figure out whats the issue with the cluster i.e. why its not assigning the shards.

As a pre step you need to set the replication off otherwise you would have comparatively higher number of unassigned shards and that might take lot of time so if you want to quickly recover its better to set the replicas to 0 and than you can allow them back at a later point in time

curl -XPUT 'localhost:9200/_settings' -d ' { "index" : { "number_of_replicas" : 0 } }'

You can assign the shards using the following command

 curl -XPOST 'localhost:9200/_cluster/reroute' -d '{  

 "commands": [

   {

    "allocate": {

    "index": "<index-name>",

    "shard": '2',

    "node": "<esdatanode01>",

    "allow_primary": true

    }

}

]

}'

The major problem comes when you have large number of unallocated shards lets say about 150 than you won't be running this command manually moreover you will face issue when you to pass multiple values together. This  can be done as follows

 curl -s localhost:9200/_cat/shards | grep UNASS | head -20 | while read line ; do   read -a fields <<<"$line" ;   curl -XPOST -d '{

  "commands" : [

  {

  "allocate" : {

  "index" : "'${fields[0]}'",

  "shard" : '${fields[1]}',

  "node" : "datanode02",

  "allow_primary": "true"

  }

  }

  ]

  }' http://localhost:9200/_cluster/reroute?pretty ; done

The above commands finds the UNASSIGNED shards and limits them to 20 on 1 node which you can adjust as per the number of shards that you want to allocate. Moreover you can control this way how many shards should to allocated  and to which node.

Once all the shards has been assigned than you can start the replica's again as follows, change the value to how many replica shards that you require

curl -XPUT 'localhost:9200/_settings' -d ' { "index" : { "number_of_replicas" : 1 } }'