面临多集群的企业级庞大架构,准备一套备份集群显得非常重要,备份集群在平时不介入读写操作,然则始终包罗营业集群的所有数据,在有营业集群不能使用的时刻,可以直接通过切换Elasticsearch来完成灾备效果,这篇文章写的是跨集群实现数据迁徙和数据灾备


一.Snapshot And Restore说明

  快照是从正在运行的Elasticsearch集群中获取的备份。您可以获取单个索引或整个集群的快照,并将其存储在共享文件系统的存储库中

  快照可以通过restore API恢复到正在运行的集群中


二.首先,你需要一个堆栈

vim conf/elasticsearch.yml
path.repo /data/backup

注重:这个目录必须是一个网络共享盘,如Nfs,Samba等


三.告诉Elasticsearch堆栈在那里

PUT /_snapshot/my_backup #使用_snapshot API指定堆栈名称
{
  "type": "fs",
  "settings": {
        "location": "my_backup_location" #指定堆栈位置
        "compress": true #压缩Mappings和settings,不能压缩数据
  }
}
GET /_snapshot/my_backup #获取堆栈位置


四.准备好了就最先备份

PUT /_snapshot/my_backup/kuaizhao1?wait_for_completion=true

#wait_for_completion守候快照完成后再返回

1.返回如下字符串为乐成

{"snapshot":{"snapshot":"kuaizhao1,"uuid":"x3AXRHavTySEe2BpBhU_FQ","version_id":5060199,"version":"5.6.1","indices":["test-aop-2014.04","test-xtx-2020.05","test2-newlis-2020.05","test-xtx-2014.04"],"state":"
SUCCESS","start_time":"2020-05-28T06:58:09.818Z","start_time_in_millis":1590649089818,"end_time":"2020-05-28T06:58:10.258Z",
"end_time_in_millis":1590649090258,"duration_in_millis":440,"failures":[],"shards":{"total":8,"failed":0,"successful":8}}}


2.在备份堆栈/data/backup中天生如下文件

-rw-rw-r-- 1 elk  elk   29 May 28 11:09 incompatible-snapshots
-rw-rw-r-- 1 elk  elk   275 May 28 11:03 index-0
-rw-rw-r-- 1 elk  elk   8 May 28 11:03 index.latest
drwxrwxr-x 6 elk  elk    4096 May 28 11:03 indices
-rw-rw-r-- 1 elk  elk   698 May 28 11:03 meta-RUiiWfeISgWFNrVfQ80sGw.dat
-rw-rw-r-- 1 elk  elk   232 May 28 11:03 snap-RUiiWfeISgWFNrVfQ80sGw.dat


五.增量备份,实在就是在堆栈里再建一个快照

PUT /_snapshot/my_backup/kuaizhao2?wait_for_completion=true

#自动识别堆栈中其他的快照信息,只对新增或更新的索引举行快照


六.把包罗快照数据的网盘挂到另一套集群的任一节点

最先恢复快照

1.这是一套新的集群,上面没有其他索引,没有重名索引

POST _snapshot/my_backup/kuaizhao1/_restore

2.之前恢复过快照,或者上面有重名索引

POST test-xtx-2020.05/_close #先把要还原的索引关闭
POST _snapshot/my_backup/kuaizhao2/_restore
GET _cat/shards #还原乐成后自动打开索引


七. 错误排查

RepositoryVerificationException[[backup5.28] a file written by master to the store [/data/backup] cannot be accessed on the node [{test1-2}{OZKGLppLRgy1cFdhiVrVGw}{5CgnmLyaS_yGLnxFx1Wrxg}{10.1.5.13}{10.1.5.13:9300}]. This might indicate that the store [/data/backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node]

快照堆栈不能是内陆目录,只能是网络共享盘

location [/data/backup] doesn't match any of the locations specified by path.repo because this setting is empty

要先在elasticsearch.yml中设置堆栈重启后才气继续