Elasticsearch 学习笔记

Elasticsearch 可以用于快速地储存、搜索和分析海量数据。这里会将其简称为 ES。大部分DSL在 Kibana Dev Tools 中执行。

👉文章列表
Elasticsearch 7 学习笔记 ⏬ 快速上手 Index、Type、Document 安装与启动 Kibana 的使用 下载历史版本 文档唯一性 默认端口与端口设置 创建和删除索引 自定义 mapping 和 settings 设置索引副本数量和分片数量 查看所有索引 数据类型 字符串类型 keyword 、text 数组 添加和更新文档 通过 _bulk 批量添加文档 使用 from 、size 进行分页查询 查询中使用 sort 进行排序 查询结果只展示部分字段 查询结果中展示 _version 字段 使用 ignore_above 限制字符串长度 动态映射 精确搜索/全文搜索示例
客户端 ⏬ Python 客户端 Java TransportClient API 客户端 Java REST Client API 客户端
其他 ⏬ ES 6 快速上手 ES 5 快速上手

Elasticsearch 6 快速上手


截止2019-06-28,ES最新版是 7.2.0 ,官方网站中默认的安装包也是 7.2.0 。

可以进入 https://www.elastic.co/cn/downloads/past-releases 如果要下载旧版本的安装包。

安装

前置条件是安装 Java。

然后去 ES 官网下载 ES。下载地址。截止2019-06-28,ES 的最新版本是 7.2.0。

下载后,解压。

启动

$ cd elasticsearch-6.6.0
$ ./bin/elasticsearch

Index、Type、Document

  • Index 对应 MySQL 中的 Database;
  • Type 对应 MySQL 中的 Table;
  • Document 对应 MySQL 中表的记录。

6.0 之后 Index 下只允许有一个Type。下面的示例会验证。

7.0 之后会废弃 Type 。

Index 的复数是 Indices。

使用 curl 操作与 ES 交互

查看信息

请求:

$ curl http://localhost:9200

# 响应如下
{
  "name" : "s5yIQVI",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "t3TCQygNRHCmjUlmWw6YLw",
  "version" : {
    "number" : "6.6.0",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "a9861f4",
    "build_date" : "2019-01-24T11:27:09.439740Z",
    "build_snapshot" : false,
    "lucene_version" : "7.6.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

查看所有的 Index

可以用过查看映射(映射可以理解为文档中各个字段及其类型之间的映射)的方式查看索引:

$ curl 'localhost:9200/_mapping?pretty=true'

# 响应如下
{ }

pretty=true 用来让响应数据格式化输出。也可以简写成pretty

$ curl 'localhost:9200/_mapping?pretty'

# 响应如下
{ }

创建 Index

使用 HTTP PUT 请求创建 Index:

$ curl -X PUT 'localhost:9200/school?pretty'

# 响应如下
{
    "acknowledged":true,
    "shards_acknowledged":true,
    "index":"school"
}

再次查询所有 Index:

$ curl 'localhost:9200/_mapping?pretty'

# 响应如下
{
  "school" : {
    "mappings" : { }
  }
}

删除 Index

使用 HTTP DELETE 请求删除 Index:

$ curl -X DELETE 'localhost:9200/school?pretty'

# 响应如下
{
  "acknowledged" : true
}

新增文档

文档要建立在索引的类型中,索引和类型不必提前创建。

例如在 school 索引的 student 类型中创建id为1的记录:

$ curl -X POST -H "Content-Type: application/json" 'localhost:9200/school/student/1?pretty=true' -d '
{
  "name": "张三"
}' 

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "1",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

可以看到响应中_id为 1。

如果不指定 id,那么 ES 会自动生成ID:

$ curl -X POST -H "Content-Type: application/json" 'localhost:9200/school/student/?pretty=true' -d '
{
  "name": "李四"
}'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "63pXuGsB3CQLwapBja05",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 2
}

再次查看 mapping:

$ curl 'localhost:9200/_mapping?pretty=true'

# 响应如下
{
  "school" : {
    "mappings" : {
      "properties" : {
        "name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    }
  }
}

上文说到 6.0 之后一个Index下只允许一个Type。我们尝试在 school 索引下创建新的 teacher 类型,会报错:

$ curl -X POST -H "Content-Type: application/json" 'localhost:9200/school/teacher/2?pretty=true' -d '
{
  "name": "张老师"
}'

# 响应如下
{
  "error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "Rejecting mapping update to [school] as the final mapping would have more than 1 type: [teacher, student]"
      }
    ],
    "type" : "illegal_argument_exception",
    "reason" : "Rejecting mapping update to [school] as the final mapping would have more than 1 type: [teacher, student]"
  },
  "status" : 400
}

查询记录

$ curl 'localhost:9200/school/student/1?pretty=true'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "张三"
  }
}
$ curl 'localhost:9200/school/student/63pXuGsB3CQLwapBja05?pretty=true'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "63pXuGsB3CQLwapBja05",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "李四"
  }
}

查找不存在的记录:

$ curl 'localhost:9200/school/student/123?pretty=true'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "123",
  "found" : false
}

查看所有记录:

$ curl -X GET -H "Content-Type: application/json" 'localhost:9200/school/student/_search?size=50&pretty=true' -d '
{
    "query" : {
        "match_all" : {}
    }
}'

# 响应如下
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "school",
        "_type" : "student",
        "_id" : "63pXuGsB3CQLwapBja05",
        "_score" : 1.0,
        "_source" : {
          "name" : "李四"
        }
      },
      {
        "_index" : "school",
        "_type" : "student",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "张三"
        }
      }
    ]
  }
}

更新记录

$ curl -X POST -H "Content-Type: application/json" 'localhost:9200/school/student/1?pretty=true' -d '
{
  "name": "张三换名字了"
}'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "1",
  "_version" : 2,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 2
}
$ curl 'localhost:9200/school/student/1?pretty=true'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "1",
  "_version" : 2,
  "_seq_no" : 1,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "张三换名字了"
  }
}

删除记录

先创建一条记录:

$ curl -X POST -H "Content-Type: application/json" 'localhost:9200/school/student/?pretty=true' -d '
{
  "name": "测试用户"
}' 

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "7HpauGsB3CQLwapBJa1g",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 2
}

查询:

$curl 'localhost:9200/school/student/7HpauGsB3CQLwapBJa1g?pretty=true'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "7HpauGsB3CQLwapBJa1g",
  "_version" : 1,
  "_seq_no" : 1,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "测试用户"
  }
}

删除:

$ curl -X DELETE 'localhost:9200/school/student/7HpauGsB3CQLwapBJa1g?pretty=true'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "7HpauGsB3CQLwapBJa1g",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 2,
  "_primary_term" : 2
}

再次查询:

$ curl 'localhost:9200/school/student/7HpauGsB3CQLwapBJa1g?pretty=true'

# 响应如下
{
  "_index" : "school",
  "_type" : "student",
  "_id" : "7HpauGsB3CQLwapBJa1g",
  "found" : false
}

kibana 简单入门

安装

到官网下载 kibana,解压。

启动

$ cd kibana-6.6.0
$ ./bin/kibana

使用 Dev Tools 查询

进入 http://localhost:5601 ,选择 Dev tools。或者直接进入 http://localhost:5601/app/kibana#/dev_tools 。

运行:

GET /school/_mapping

输出:

{
  "school" : {
    "mappings" : {
      "student" : {
        "properties" : {
          "name" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          }
        }
      }
    }
  }
}

运行:

GET /school/student/_search
{
  "query": {
    "match_all": {}
  }
}

输出:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 2,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "school",
        "_type" : "student",
        "_id" : "63pXuGsB3CQLwapBja05",
        "_score" : 1.0,
        "_source" : {
          "name" : "李四"
        }
      },
      {
        "_index" : "school",
        "_type" : "student",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "张三换名字了"
        }
      }
    ]
  }
}

运行:

GET /school/student/1

输出:

{
  "_index" : "school",
  "_type" : "student",
  "_id" : "1",
  "_version" : 2,
  "_seq_no" : 1,
  "_primary_term" : 2,
  "found" : true,
  "_source" : {
    "name" : "张三换名字了"
  }
}

查询 name 中含有的记录:

GET /school/student/_search
{
  "query": {
    "match": {"name": "李"}
  }
}

输出:

{
  "took" : 8,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.2876821,
    "hits" : [
      {
        "_index" : "school",
        "_type" : "student",
        "_id" : "63pXuGsB3CQLwapBja05",
        "_score" : 0.2876821,
        "_source" : {
          "name" : "李四"
        }
      }
    ]
  }
}

使用 Dev tools 添加记录

运行:

POST /school/student
{
  "name": "王五"
}

输出:

{
  "_index" : "school",
  "_type" : "student",
  "_id" : "7XpduGsB3CQLwapBFa0E",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 2
}

参考链接


( 本文完 )

文章目录