生活_傻大方

  1. 首页 > 潮·科技 > >

在kubernetes中部署企业级ELK并使用其APM

一、架构为了增加es的扩展性 ， 按角色功能分为master节点、data数据节点、client客户端节点 。 其整体架构如下：
在kubernetes中部署企业级ELK并使用其APM文章插图
其中：

  • Elasticsearch数据节点Pods被部署为一个有状态集（StatefulSet）
  • Elasticsearch master节点Pods被部署为一个Deployment
  • Elasticsearch客户端节点Pods是以Deployment的形式部署的 ， 其内部服务将允许访问R/W请求的数据节点
  • Kibana和APMServer部署为Deployment ， 其服务可在Kubernetes集群外部访问
1.1、版本说明软件 版本 Kibana 7.8.0 Elasticsearch 7.8.0 Filebeat 7.8.0 Kubernetes 1.17.2 APM-Server 7.8.0
二、部署ES先创建estatic的命名空间（es-ns.yaml）：
apiVersion: v1kind: Namespacemetadata:name: elastic执行kubectl apply -f es-ns.yaml
2.1、生成证书启动es的xpack功能 ， 传输需要加密传输 。 脚本如下（es-create-ca.sh）：
#!/bin/bash# 指定 elasticsearch 版本RELEASE=7.8.0# 运行容器生成证书docker run --name elastic-charts-certs -i -w /app \elasticsearch:${RELEASE} \/bin/sh -c " \elasticsearch-certutil ca --out /app/elastic-stack-ca.p12 --pass ''--tt-darkmode-color: #35B378;">2.1、部署es master节点配置清单如下（es-master.yaml）：
---apiVersion: v1kind: ConfigMapmetadata:namespace: elasticname: elasticsearch-master-configlabels:app: elasticsearchrole: masterdata:elasticsearch.yml: |-cluster.name: ${CLUSTER_NAME}node.name: ${NODE_NAME}discovery.seed_hosts: ${NODE_LIST}cluster.initial_master_nodes: ${MASTER_NODES}network.host: 0.0.0.0node:master: truedata: falseingest: falsexpack.security.enabled: truexpack.monitoring.collection.enabled: truexpack.security.transport.ssl.enabled: truexpack.security.transport.ssl.verification_mode: certificatexpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12xpack.ml.enabled: truexpack.license.self_generated.type: basicxpack.monitoring.exporters.my_local:type: localuse_ingest: false---apiVersion: v1kind: Servicemetadata:namespace: elasticname: elasticsearch-masterlabels:app: elasticsearchrole: masterspec:ports:- port: 9300name: transportselector:app: elasticsearchrole: master---apiVersion: apps/v1kind: Deploymentmetadata:namespace: elasticname: elasticsearch-masterlabels:app: elasticsearchrole: masterspec:replicas: 1selector:matchLabels:app: elasticsearchrole: mastertemplate:metadata:labels:app: elasticsearchrole: masterspec:initContainers:- name: init-sysctlimage: busybox:1.27.2command:- sysctl- -w- vm.max_map_count=262144securityContext:privileged: truecontainers:- name: elasticsearch-masterimage: docker.elastic.co/elasticsearch/elasticsearch:7.8.0env:- name: CLUSTER_NAMEvalue: elasticsearch- name: NODE_NAMEvalue: elasticsearch-master- name: NODE_LISTvalue: elasticsearch-master,elasticsearch-data,elasticsearch-client- name: MASTER_NODESvalue: elasticsearch-master- name: "ES_JAVA_OPTS"value: "-Xms512m -Xmx512m"ports:- containerPort: 9300name: transportvolumeMounts:- name: configmountPath: /usr/share/elasticsearch/config/elasticsearch.ymlreadOnly: truesubPath: elasticsearch.yml- name: storagemountPath: /data- name: localtimemountPath: /etc/localtime- name: keystoremountPath: /usr/share/elasticsearch/config/certs/elastic-certificates.p12readOnly: truesubPath: elastic-certificates.p12volumes:- name: configconfigMap:name: elasticsearch-master-config- name: "storage"emptyDir:medium: ""- name: localtimehostPath:path: /etc/localtime- name: keystoresecret:secretName: elastic-certificatesdefaultMode: 044然后执行kubectl apply -f ``es-master.yaml创建配置清单 ， 然后pod变为running状态即为部署成功 。
# kubectl get pod -n elasticNAMEREADYSTATUSRESTARTSAGEelasticsearch-master-77d5d6c9db-xt5kq1/1Running067s2.2、部署es data节点配置清单如下（es-data.yaml）：
---apiVersion: v1kind: ConfigMapmetadata:namespace: elasticname: elasticsearch-data-configlabels:app: elasticsearchrole: datadata:elasticsearch.yml: |-cluster.name: ${CLUSTER_NAME}node.name: ${NODE_NAME}discovery.seed_hosts: ${NODE_LIST}cluster.initial_master_nodes: ${MASTER_NODES}network.host: 0.0.0.0node:master: falsedata: trueingest: falsexpack.security.enabled: truexpack.monitoring.collection.enabled: truexpack.security.transport.ssl.enabled: truexpack.security.transport.ssl.verification_mode: certificatexpack.security.transport.ssl.keystore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12xpack.security.transport.ssl.truststore.path: /usr/share/elasticsearch/config/certs/elastic-certificates.p12xpack.ml.enabled: truexpack.license.self_generated.type: basicxpack.monitoring.exporters.my_local:type: localuse_ingest: false---apiVersion: v1kind: Servicemetadata:namespace: elasticname: elasticsearch-datalabels:app: elasticsearchrole: dataspec:ports:- port: 9300name: transportselector:app: elasticsearchrole: data---apiVersion: apps/v1kind: StatefulSetmetadata:namespace: elasticname: elasticsearch-datalabels:app: elasticsearchrole: dataspec:serviceName: "elasticsearch-data"selector:matchLabels:app: elasticsearchrole: datatemplate:metadata:labels:app: elasticsearchrole: dataspec:initContainers:- name: init-sysctlimage: busybox:1.27.2command:- sysctl- -w- vm.max_map_count=262144securityContext:privileged: truecontainers:- name: elasticsearch-dataimage: docker.elastic.co/elasticsearch/elasticsearch:7.8.0env:- name: CLUSTER_NAMEvalue: elasticsearch- name: NODE_NAMEvalue: elasticsearch-data- name: NODE_LISTvalue: elasticsearch-master,elasticsearch-data,elasticsearch-client- name: MASTER_NODESvalue: elasticsearch-master- name: "ES_JAVA_OPTS"value: "-Xms1024m -Xmx1024m"ports:- containerPort: 9300name: transportvolumeMounts:- name: configmountPath: /usr/share/elasticsearch/config/elasticsearch.ymlreadOnly: truesubPath: elasticsearch.yml- name: elasticsearch-data-persistent-storagemountPath: /data/db- name: keystoremountPath: /usr/share/elasticsearch/config/certs/elastic-certificates.p12readOnly: truesubPath: elastic-certificates.p12volumes:- name: configconfigMap:name: elasticsearch-data-config- name: keystoresecret:secretName: elastic-certificatesdefaultMode: 044volumeClaimTemplates:- metadata:name: elasticsearch-data-persistent-storagespec:accessModes: [ "ReadWriteOnce" ]storageClassName: managed-nfs-storageresources:requests:storage: 20Gi---



上一篇：手机电量耗完需充电，为何不设计成可拆卸电池，提供满格电量？

下一篇：使用KPI仪表板，创建完整的信息数据生态系统