1、安装elasticsearch

1.)关闭防火墙及SELinuxservice iptables stopchkconfig iptables offchkconfig iptables --listvim /etc/sysconfig/selinuxSELinux=disabledsetenforce 0

2.)配置jdk环境

vim /etc/profile.d/java.shexport JAVA_HOME=/home/admin/jdk1.8.0_172/export CLASSPATH=.:$JAVA_HOME/lib.tools.jarexport PATH=$JAVA_HOME/bin:$PATHsource /etc/profile.d/java.sh

3.)安装ElasticSearch6.x

wget tar -zxvf elasticsearch-6.2.4.tar.gz -C /home/admin/project/elkcd /home/admin/project/elkelasticsearch-6.2.4vim config/elasticsearch.ymlcluster.name: elasticsearchnode.name: node-1network.host: 10.2.151.203http.port: 9200bootstrap.memory_lock: falsebootstrap.system_call_filter: falsehttp.cors.enabled: truehttp.cors.allow-origin: "*"

4.)启动elasticsearch

useradd elkchown –R elk.elk /home/admin/project/elk/elasticsearch-6.2.4./bin/elasticsearch –dnetstat –luntp #查看监听端口9200 9300curl 10.2.151.203:9200

5.)启动常见错误

uncaught exception in thread [main]org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root问题原因：不能使用root用户启动 解决方法：切换要其他用户启动

unable to install syscall filter:

java.lang.UnsupportedOperationException: seccomp unavailable:问题原因：其实只是一个警告，主要是因为你Linux版本过低造成的解决方法：警告不影响使用，可以忽略

ERROR: bootstrap checks failed

memory locking requested for elasticsearch process but memory is not locked问题原因：锁定内存失败解决方法：切换到root用户，编辑limits.conf配置文件vim /etc/security/limits.conf

• hard nproc 65536
• soft nproc 65536
• hard nofile 65536
• soft nofile 65536

max number of threads [1024] for user [es] is too low, increase to at least [2048]

原因：无法创建本地线程问题,用户最大可创建线程数太小 解决方案：切换到root用户，进入limits.d目录下，修改90-nproc.conf 配置文件vim /etc/security/limits.d/90-nproc.conf

• soft nofile 65536
• soft nproc 65536
• soft nproc 2048

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

原因：最大虚拟内存太小 解决方案：切换到root用户下，修改配置文件sysctl.confvim /etc/sysctl.confvm.max_map_count=655360 sysctl -p

system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk

问题原因：因为Centos6不支持SecComp解决方法：在elasticsearch.yml中配置bootstrap.system_call_filter为false，注意要在Memory下面: bootstrap.memory_lock: false bootstrap.system_call_filter: false

2、elasticsearch-head插件安装

通过web界面来查看elasticsearch集群状态信息

1.)下载安装nodejs

wget tar -zxvf node-v8.11.3-linux-x64.tar.gz -C /home/admin/project/elk/cd /home/admin/project/elk/mv node-v8.11.3-linux-x64/ node-v8.11.3#配置nodejs环境变量vim /etc/profile.d/node.shexport NODE_HOME=/home/admin/project/elk/node-v8.11.3export PATH=$NODE_HOME/bin:$PATHexport NODE_PATH=$NODE_HOME/lib/node_modulessource /etc/profile.d/node.sh#查看nodejs是否生效[admin@localhost node-v8.11.3]$ node -vv8.11.3[admin@localhost node-v8.11.3]$ npm -v5.6.0

2.)安装grunt

npm config set registry vim ~/.npmrcregistry=strict-ssl = falsenpm install -g grunt-cli#将grunt加入系统文件ln -s /home/admin/project/elk/node-v8.11.3/lib/node_modules/grunt-cli/bin/grunt /usr/bin/grunt

3.)下载head二进制包

wget unzip elasticsearch-head-master.zipcd elasticsearch-head-masternpm install#如果速度较慢或安装失败，建议使用国内镜像 npm install --ignore-scripts -g cnpm --registry=

4.)修改elasticserach配置文件

vi ./config/elasticsearch.yml#增加新的参数，这样head插件可以访问eshttp.cors.enabled: truehttp.cors.allow-origin: "*"

5.)修改Gruntfile.js配置

vim Gruntfile.js#port: 9100上面增加hostname地址hostname: "0.0.0.0",

6.)修改_site/app.js配置

vim _site/app.js#localhost替换为IP地址this.base_uri = this.config.base_uri || this.prefs.get("app-base_uri") || "";

7.) 启动grunt

grunt server#如果启动成功，则可以直接使用后台运行，命令行可继续输入（但是如果想退出，则需要自己kill进程）grunt server &nohup grunt server & exit #后台启动

#启动提示模块未找到

Local Npm module "grunt-contrib-jasmine" not found. Is it installed?

npm install grunt-contrib-jasmine #安装模块

3、安装kibana

1.)下载安装wget tar -zxvf kibana-6.2.4-linux-x86_64.tar.gz -C /home/admin/project/elk/ cd /ho me/admin/project/elk/ kibana-6.2.4-linux-x86_642.)修改配置vim config/kibana.ymlserver.port: 5601server.host: “IP"elasticsearch.url: 3.)启动kibana./bin/kibana

4、安装logstash

1.)下载安装wget tar -zxvf logstash-6.2.4.tar.gz -C /home/admin/project/elk/cd /home/admin/project/elk/logstash-6.2.42.)新建模板vim config/test.confinput { kafka { bootstrap_servers => "10.7.1.112:9092" topics => "nethospital_2"codec => "json"}}

output

{ if [fields][tag] == "nethospital_2"{ elasticsearch { hosts => ["10.7.1.111:9200"]index => "nethospital_2-%{+YYYY-MM-dd}"codec => "json"}} }

3.)启动logstash

nohup ./bin/logstash –f config/test.conf & # -f 指定配置文件5、安装kafka1.)下载安装wget wget tar -zxvf kafka_2.11-1.0.0.tgz -C /home/admin/project/elk/tar -zxvf zookeeper-3.4.12.tar.gz -C /home/admin/project/elk/cd /home/admin/project/elk/kafka_2.11-1.0.0/

2.)修改kafka参数及启动

vim config/zookeeper.propertiesdataDir=/tmp/zookeeper/data # 数据持久化路径clientPort=2181 # 连接端口maxClientCnxns=100 # 最大连接数dataLogDir=/tmp/zookeeper/logs #日志存放路径tickTime=2000 # Zookeeper服务器心跳时间，单位毫秒initLimit=10 # 投票选举新leader的初始化时间。#启动zookeeper./bin/zookeeper-server-start.sh config/zookeeper.properties#后台启动nohup ./bin/zookeeper-server-start.sh config/zookeeper.properties &

3.)修改kafka参数及启动

vim config/server.propertiesbroker.id=0port=9092host.name=10.2.151.203num.network.threads=3num.io.threads=8socket.send.buffer.bytes=102400socket.receive.buffer.bytes=102400socket.request.max.bytes=104857600log.dirs=/data/logs/kafkanum.partitions=2num.recovery.threads.per.data.dir=1log.retention.check.interval.ms=300000zookeeper.connect=localhost:2181zookeeper.connection.timeout.ms=6000

#启动zookeeper

./bin/kafka-server-start.sh config/server.properties#后台启动nohup bin/kafka-server-start.sh config/server.properties &

4.)测试kafka

#创建topic (test)bin/kafka-topics.sh --create --zookeeper 10.2.151.203:2181 --replication-factor 1 --partitions 1 --topic test#查看topicbin/kafka-topics.sh --list --zookeeper 10.2.151.203:2181#启动生产进程测试bin/kafka-console-producer.sh --broker-list 10.2.151.203:9092 --topic test

#启动启动消费者进程

bin/kafka-console-consumer.sh --zookeeper 10.2.151.203:2181 --topic test --from-beginning

6、安装filebeat

1.)下载安装wget tar –zxvf filebeat-6.2.4-linux-x86_64.tar.gz –C /home/admin/project/elkcd /home/admin/project/elk/ filebeat-6.2.4-linux-x86_64

2.)配置filebeat

vim filebeat.yml

• input_type: log
  #Paths that should be crawled and fetched. Glob based paths.
  paths:
• /home/admin/project/other_project/nh-interface/nh-interface.log
  fields:
  tag: nh-interface
  multiline:
  pattern: '^[0-9]{4}-[0-9]{2}.*'
  negate: true
  match: after

output.kafka:

enabled: truehosts: ["AppElk1:9092","AppElk2:9092","AppElk3:9092"]topic: 'hospital'compression: gzipmax_message_bytes: 100000000

3）启动filebeat

nohup ./filebeat -e -c filebeat.yml &

查看集群状态

curl -XGET ''curl -XGET ''curl -XGET ''

查看集群master

curl -XGET ''或curl -XGET ''

查询集群的健康状态

curl -XGET ''或curl -XGET ''

7、安装cerebro插件

cerebo是kopf在es5上的替代者，通过web界面来管理和监控elasticsearch集群状态信息

1.)下载安装

#wget #tar –zxvf cerebro-0.8.1.tgz /home/admin/project/elk#cd /home/admin/project/elk/cerebro-0.8.1##vim conf/application.conf#hosts = [****{ host = ""name = "my-elk"},]2.）启动/访问nohup ./bin/cerebro & #后台运行

8、安装bigdesk插件

bigdesk 统计分析和图表化elasticsearch集群状态信息1.）下载安装#wget unzip bigdesk-master.zip#mv bigdesk-master /home/admin/project/elk/elasticsearch-6.2.4/plugins/#cd /home/admin/project/elk/elasticsearch-6.2.4/plugins/bigdesk-master/_site**

2．）使用 python -m SimpleHTTPServer 快速搭建http服务

指定端口8000nohup python -m SimpleHTTPServer 8000 & #后台运行