-
Notifications
You must be signed in to change notification settings - Fork 0
/
search.xml
129 lines (129 loc) · 72.2 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title><![CDATA[安装部署 Zabbix]]></title>
<url>%2F2017%2F11%2F17%2Finstall-zabbix%2F</url>
<content type="text"><![CDATA[--- title: 安装部署 Zabbix date: 2017-11-17 20:19:34 tags: mariadb layout: post categories: 监控系统 --- # Zabbix 安装教程 ## 安装说明 本文档介绍的安装步骤是基于 rpm 包方式安装 Zabbix 的，不涉及源码编译安装。 ### 软件版本说明 1. Redhat 版本：`rhel-server-6.4 x86_64` 2. Zabbix 版本：`3.2.1`，并且是 MySQL 版本 ### 依赖说明 Zabbix 的安装运行需要依赖于第三方的数据存储（MySQL or PgSQL）和服务容器（Apache），需要先部署好并确认其可以正常使用，以下列表为需要用到的依赖软件： 1. **MySQL**：MySQL 是安装 Zabbix 的前置依赖，需要先安装好 MySQL 并确认起可以正常运行； 2. **PHP**：Zabbix web 采用的是 PHP 语言开发，并且版本要求不低于 5.4，且不支持 PHP7； 3. **Apache**：Zabbix web 运行容器，整个 Zabbix 采用的是 LAMP 这一套； ### 其他 1. Redhat 系统下使用 **root** 用户进行命令操作，否则需要使用 sudo 命令操作。 ## 配置系统 ### 关闭系统 iptables 和 selinux **关闭 selinux** 1. 永久有效 ``` vi /etc/selinux/config ``` 注释掉原来 SELINUX= 开头的一行，重新加入以下内容： `SELINUX=disabled` 保存，重启服务器。 2. 即时生效（重启后失效） 如果需要在不重启的情况下，可以临时关闭 selinux，则执行以下命令 ``` setenforce 0 ``` **关闭防火墙 iptables**： 1. 永久性生效 开启：`chkconfig iptables on` 关闭：`chkconfig iptables off` 2. 即时生效（重启后失效） 开启：`service iptables start` 关闭：`service iptables stop` ### 挂载 iso 为系统源 上传 `rhel-server-6.0-x86_64-dvd.iso` 到 `/repo/iso` 目录下，执行： ``` mount -t iso9660 -o loop /repo/iso/rhel-server-6.0-x86_64-dvd.iso /mnt ``` 增加源文件配置： ``` vi /etc/yum.repos.d/rhel6.repo [Server] name=rhel6server baseurl=file:https:///mnt enable=1 gpcheck=0 ``` 取消验证： ``` vi /etc/yum.conf ``` 把里面的 `gpgcheck=1` 改为 `gpgcheck=0`，即跳过验证。 ## 安装 Zabbix server ### 0. 创建用户账号 ``` # groupadd zabbix # useradd -g zabbix zabbix # usermod -s /sbin/nologin zabbix ``` ### 1. 安装 MySQL 此处安装以 5.7 版本为例子。如果本机有 MySQL/MariaDB 并且可以运行，则本章节的步骤可以忽略。 1. **下载 rpm 包** 下载安装 MySQL 需要的 rpm 包及其依赖包。 ``` # wget https://repo.mysql.com/yum/mysql-5.7-community/el/6/x86_64/mysql-community-devel-5.7.16-1.el6.x86_64.rpm # wget https://repo.mysql.com/yum/mysql-5.7-community/el/6/x86_64/mysql-community-libs-compat-5.7.16-1.el6.x86_64.rpm # wget https://repo.mysql.com/yum/mysql-5.7-community/el/6/x86_64/mysql-community-common-5.7.16-1.el6.x86_64.rpm # wget https://repo.mysql.com/yum/mysql-5.7-community/el/6/x86_64/mysql-community-libs-5.7.16-1.el6.x86_64.rpm # wget https://repo.mysql.com/yum/mysql-5.7-community/el/6/x86_64/mysql-community-client-5.7.16-1.el6.x86_64.rpm # wget https://repo.mysql.com/yum/mysql-5.7-community/el/6/x86_64/mysql-community-server-5.7.16-1.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/numactl-2.0.9-2.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/perl-DBI-1.609-4.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/openssl-1.0.1e-48.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/e2fsprogs-libs-1.41.12-22.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libss-1.41.12-22.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libcom_err-1.41.12-22.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/e2fsprogs-1.41.12-22.el6.x86_64.rpm ``` `numactl`、`perl-DBI`、`openssl` 等为安装 MySQL 所以依赖的包。 2. **安装 rpm 包** ``` # yum install mysql-community-server-5.7.16-1.el6.x86_64.rpm mysql-community-client-5.7.16-1.el6.x86_64.rpm mysql-community-common-5.7.16-1.el6.x86_64.rpm mysql-community-devel-5.7.16-1.el6.x86_64.rpm mysql-community-libs-5.7.16-1.el6.x86_64.rpm mysql-community-libs-compat-5.7.16-1.el6.x86_64.rpm numactl-2.0.9-2.el6.x86_64.rpm perl-DBI-1.609-4.el6.x86_64.rpm openssl-1.0.1e-48.el6.x86_64.rpm e2fsprogs-libs-1.41.12-22.el6.x86_64.rpm libss-1.41.12-22.el6.x86_64.rpm libcom_err-1.41.12-22.el6.x86_64.rpm e2fsprogs-1.41.12-22.el6.x86_64.rpm ``` 3. **启动 MySQL 服务** ``` # service mysqld start ``` 4. **验证 MySQL 是否能正常使用** ``` # ps -ef|grep mysql # mysql -uroot ``` 如果能正常进入 MySQL 命令行，则表示 MySQL 服务已经正常运行。 5. **TODO: 设置 MySQL root 用户密码安全** **TODO**： ### 2. 安装 Zabbix server 下载 Zabbix server 的 rpm 包及其依赖包。 ``` # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/OpenIPMI-libs-2.0.16-14.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libtool-ltdl-2.2.6-15.5.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/lm_sensors-libs-3.1.1-17.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/net-snmp-libs-5.5-57.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/unixODBC-2.2.14-14.el6.x86_64.rpm # wget ftp:https://mirror.switch.ch/pool/4/mirror/epel/6/x86_64/iksemel-1.4-2.el6.x86_64.rpm # wget ftp:https://mirror.switch.ch/pool/4/mirror/epel/6/x86_64/fping-2.4b2-10.el6.x86_64.rpm # wget http:https://repo.zabbix.com/zabbix/3.2/rhel/6/x86_64/deprecated/zabbix-server-mysql-3.2.1-1.el6.x86_64.rpm # wget http:https://repo.zabbix.com/zabbix/3.2/rhel/6/x86_64/zabbix-get-3.2.1-1.el6.x86_64.rpm # wget http:https://repo.zabbix.com/zabbix/3.2/rhel/6/x86_64/zabbix-proxy-mysql-3.2.1-1.el6.x86_64.rpm ``` 安装已经下载好的 rpm 包。 ``` # yum install OpenIPMI-libs-2.0.16-14.el6.x86_64.rpm libtool-ltdl-2.2.6-15.5.el6.x86_64.rpm lm_sensors-libs-3.1.1-17.el6.x86_64.rpm net-snmp-libs-5.5-57.el6.x86_64.rpm unixODBC-2.2.14-14.el6.x86_64.rpm iksemel-1.4-2.el6.x86_64.rpm fping-2.4b2-10.el6.x86_64.rpm # yum install zabbix-server-mysql-3.2.1-1.el6.x86_64.rpm zabbix-get-3.2.1-1.el6.x86_64.rpm zabbix-proxy-mysql-3.2.1-1.el6.x86_64.rpm ``` 确认安装过程中没有出错及少依赖包。 ### 3. 配置 Zabbix server 的数据库配置 需要先安装好 Zabbix server 后才能找到 `/usr/share/doc/zabbix-server-mysql-3.2.1/create.sql.gz` 文件。 ``` # mysql -uroot mysql> create database zabbix character set utf8 collate utf8_bin; mysql> GRANT ALL privileges ON zabbix.* TO 'zabbix'@'localhost' IDENTIFIED BY 'zabbix'; mysql> GRANT ALL privileges ON zabbix.* TO 'zabbix'@'%' IDENTIFIED BY 'zabbix'; mysql> flush privileges; mysql> quit; # cd /usr/share/doc/zabbix-server-mysql-3.2.1 # zcat create.sql.gz | mysql -uzabbix -Pzabbix zabbix ``` `zcat create.sql.gz | mysql -uzabbix -Pzabbix zabbix` 是把 Zabbix server 的初始化脚本写入 MySQL 数据库，这里执行完可以进入 MySQL 看下数据有没有写成功。 ### 4. 配置 Zabbix server 参数 修改配置文件，使其使用安装好的 MySQL。 ``` # vi /etc/zabbix/zabbix_server.conf LogFile=/var/log/zabbix/zabbix_server.log DBHost=localhost DBName=zabbix DBUser=zabbix DBPassword=zabbix ``` ### 5. 启动 Zabbix server 安装完成后既可启动 Zabbix server 服务了。 ``` # service zabbix-server start ``` 日志目录：`LogFile=/var/log/zabbix/zabbix_server.log` 检查进程和日志是否正常启动成功： ``` # ps -ef|grep zabbix_server # netstat -lntup | grep 10051 # tail -f /var/log/zabbix/zabbix_server.log ``` Zabbix server 启动端口为 10051。 若服务器打开 iptables 服务，则需添加该两个端口通行。 ``` # iptables -A INPUT -p tcp --dport 10050 -j ACCEPT # iptables -A INPUT -p tcp --dport 10051 -j ACCEPT ``` ## 安装 Zabbix agent ### 1. 安装 rpm 包 下载需要的 rpm 包 ``` # wget http:https://repo.zabbix.com/zabbix/3.2/rhel/6/x86_64/zabbix-agent-3.2.1-1.el6.x86_64.rpm ``` ``` # yum install zabbix-agent-3.2.1-1.el6.x86_64.rpm ``` ### 2. 配置 Zabbix agent ``` # vi /etc/zabbix/zabbix_agentd.conf LogFile=/var/log/zabbix/zabbix_agentd.log Server=127.0.0.1 UnsafeUserParameters=1 ``` 上述配置中的 Server 指向 Zabbix Server 的地址，如果是本机的话则采用 `127.0.0.1` ### 3. 启动 Zabbix agent ``` # service zabbix-agent start ``` 日志目录：`LogFile=/var/log/zabbix/zabbix_agentd.log` 检查进程和日志是否正常启动成功 ``` # ps -ef|grep zabbix_agentd # netstat -lntup | grep 10050 # tail -f /var/log/zabbix/zabbix_agentd.log ``` Zabbix agentd 启动端口为 10050。 ## 安装 Zabbix web Zabbix 3.2 对 PHP 的要求最低为 5.4，而 CentOS6 默认为 5.3.3，完全不满足要求，故需要利用第三方源将 PHP 升级到 5.4 以上。(注意：同时不支持 PHP7) ### 1. 卸载本地 PHP 版本 检查本机的 PHP 版本，如果 < 5.4 需要卸载重新安装 ``` # yum erase php php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc php-mcrypt php-mbstring php-devel php-pecl-memcached php-pecl-memcache php-common php-pdo php-cli php-fpm libmemcached ``` ### 2. 下载 rpm 包 ``` # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-mysql-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-gd-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-imap-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-ldap-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-odbc-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-xml-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-xmlrpc-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-mcrypt-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-mbstring-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-devel-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-pecl-memcached-2.2.0-2.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-common-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-pdo-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-cli-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-pecl-memcache-3.0.8-2.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-bcmath-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-fpm-5.6.28-1.w6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/libmemcached10-1.0.16-1.ius.el6.x86_64.rpm # wget https://repo.webtatic.com/yum/el6/x86_64/php56w-pecl-igbinary-1.2.1-2.w6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/apr-util-ldap-1.3.9-3.el6_0.1.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/freetype-2.3.11-17.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libX11-1.6.3-2.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libX11-common-1.6.3-2.el6.noarch.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libXau-1.0.6-4.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libXpm-3.5.10-2.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libc-client-2007e-11.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libjpeg-turbo-1.2.1-3.el6_5.x86_64.rpm # wget ftp:https://mirror.switch.ch/pool/4/mirror/epel/6/x86_64/libmcrypt-2.5.8-9.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libpng-1.2.49-2.el6_7.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libxcb-1.11-2.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/libxslt-1.1.26-2.el6_3.1.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/mailcap-2.1.31-2.el6.noarch.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/pcre-devel-7.8-7.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/t1lib-5.1.2-6.el6_2.1.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/pcre-7.8-7.el6.x86_64.rpm # wget https://raw.githubusercontent.com/zabbixcn/curl-rpm/master/RPMS/libcurl-devel-7.29.0-25.el6.x86_64.rpm # wget https://raw.githubusercontent.com/zabbixcn/curl-rpm/master/RPMS/libcurl-7.29.0-25.el6.x86_64.rpm # wget https://raw.githubusercontent.com/zabbixcn/curl-rpm/master/RPMS/curl-7.29.0-25.el6.x86_64.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/dejavu-fonts-common-2.33-1.el6.noarch.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/dejavu-sans-fonts-2.33-1.el6.noarch.rpm # wget http:https://mirror.centos.org/centos/6/os/x86_64/Packages/fontpackages-filesystem-1.41-1.1.el6.noarch.rpm # wget http:https://repo.zabbix.com/zabbix/3.2/rhel/6/x86_64/deprecated/zabbix-web-mysql-3.2.1-1.el6.noarch.rpm # wget http:https://repo.zabbix.com/zabbix/3.2/rhel/6/x86_64/deprecated/zabbix-web-3.2.1-1.el6.noarch.rpm ``` ### 3. 安装 PHP & Apache & CURL 先安装依赖包 ``` # yum install apr-util-ldap-1.3.9-3.el6_0.1.x86_64.rpm freetype-2.3.11-17.el6.x86_64.rpm libX11-1.6.3-2.el6.x86_64.rpm libX11-common-1.6.3-2.el6.noarch.rpm libXau-1.0.6-4.el6.x86_64.rpm libXpm-3.5.10-2.el6.x86_64.rpm libc-client-2007e-11.el6.x86_64.rpm libjpeg-turbo-1.2.1-3.el6_5.x86_64.rpm libmcrypt-2.5.8-9.el6.x86_64.rpm libpng-1.2.49-2.el6_7.x86_64.rpm libxcb-1.11-2.el6.x86_64.rpm libxslt-1.1.26-2.el6_3.1.x86_64.rpm mailcap-2.1.31-2.el6.noarch.rpm pcre-devel-7.8-7.el6.x86_64.rpm t1lib-5.1.2-6.el6_2.1.x86_64.rpm pcre-7.8-7.el6.x86_64.rpm dejavu-fonts-common-2.33-1.el6.noarch.rpm dejavu-sans-fonts-2.33-1.el6.noarch.rpm fontpackages-filesystem-1.41-1.1.el6.noarch.rpm ``` 安装 5.6 版本的 PHP ``` # yum install php56w-5.6.28-1.w6.x86_64.rpm php56w-mysql-5.6.28-1.w6.x86_64.rpm php56w-gd-5.6.28-1.w6.x86_64.rpm php56w-imap-5.6.28-1.w6.x86_64.rpm php56w-ldap-5.6.28-1.w6.x86_64.rpm php56w-odbc-5.6.28-1.w6.x86_64.rpm php56w-xml-5.6.28-1.w6.x86_64.rpm php56w-xmlrpc-5.6.28-1.w6.x86_64.rpm php56w-mcrypt-5.6.28-1.w6.x86_64.rpm php56w-mbstring-5.6.28-1.w6.x86_64.rpm php56w-devel-5.6.28-1.w6.x86_64.rpm php56w-pecl-memcached-2.2.0-2.w6.x86_64.rpm php56w-common-5.6.28-1.w6.x86_64.rpm php56w-pdo-5.6.28-1.w6.x86_64.rpm php56w-cli-5.6.28-1.w6.x86_64.rpm php56w-pecl-memcache-3.0.8-2.w6.x86_64.rpm php56w-bcmath-5.6.28-1.w6.x86_64.rpm php56w-fpm-5.6.28-1.w6.x86_64.rpm libmemcached10-1.0.16-1.ius.el6.x86_64.rpm php56w-pecl-igbinary-1.2.1-2.w6.x86_64.rpm ``` curl >= 7.20 (如需支持 SMTP 认证)，安装较高版本的 CURL ``` # yum install libcurl-devel-7.29.0-25.el6.x86_64.rpm libcurl-7.29.0-25.el6.x86_64.rpm curl-7.29.0-25.el6.x86_64.rpm ``` ### 4. 安装 Zabbix web ``` # yum install zabbix-web-3.2.1-1.el6.noarch.rpm zabbix-web-mysql-3.2.1-1.el6.noarch.rpm ``` 将 Zabbix web 发布到 Apache web 容器里面。 在 CentOS 中，Apache web 的默认路径是：`/var/www/html` ``` # mkdir /var/www/html/zabbix # cd /usr/share/zabbix/ # cp -a . /var/www/html/zabbix/ ``` ### 5. 配置 php.init PHP 中默认的一些参数配置不适用与 Zabbix，需要调整。 ``` # vi /etc/php.ini post_max_size=16M max_execution_time=300 max_input_time=300 date.timezone = Asia/Shanghai always_populate_raw_post_data = -1 ``` ### 6. 启动 Apache ``` # service httpd start ``` 验证 ``` # ps -ef|grep httpd # tail -f /var/log/httpd/error_log ``` ### 7. 登录 Zabbix web 进行配置 访问：[http:https://127.0.0.1/zabbix](http:https://127.0.0.1/zabbix) 按一下步骤，执行下去即可完成配置  步骤 2 这里需要确认 PHP 的参数是正确的，如果有错误提示，请参照 **5. 配置 php.ini** 章节修改配置。  步骤 3 这里配置的即是 MySQL 数据库的链接信息，默认可以和 Zabbix server 放在一个库上，若参照上述文档描述，数据库用户密码为 `zabbix/zabbix`。     登录用户/密码：`Admin/zabbix`  ## 参考资料 0. Zabbix rpm 包下载地址：[Index of /zabbix](http:https://repo.zabbix.com/zabbix/) 1. Zabbix 源码包下载地址：[Zabbix Sources](http:https://www.zabbix.com/download) - ([zabbix-3.2.1.tar.gz](http:https://sourceforge.net/projects/zabbix/files/ZABBIX%20Latest%20Stable/3.2.1/zabbix-3.2.1.tar.gz/download)) 2. Zabbix 官方源码编译安装文档：[4 Installation from sources - Zabbix Documentation 3.2](https://www.zabbix.com/documentation/3.2/manual/installation/install) 3. 源码编译安装过程中需要配置数据库，参照：[1 Database creation scripts](https://www.zabbix.com/documentation/3.2/manual/appendix/install/db_scripts) 4. 别人组织的一个 CentOS 6.x 版本的 curl 7.29.0 版本的 rpm 源放在 GitHub 上：[zabbixcn/curl-rpm - GitHub](https://github.com/zabbixcn/curl-rpm) 5. 别人组织的一个 CentOS 6.x 版本的 Zabbix 3.0 版本的 rpm 源放在 GitHub 上：[zabbixcn/zabbix3.0-rpm](https://github.com/zabbixcn/zabbix3.0-rpm) 6. 安装过程中一些依赖包的源： - [http:https://mirror.centos.org/centos/6/os/x86_64/Packages/](http:https://mirror.centos.org/centos/6/os/x86_64/Packages/) - [ftp:https://mirror.switch.ch/pool/4/mirror/epel/6/x86_64/](ftp:https://mirror.switch.ch/pool/4/mirror/epel/6/x86_64/) - MySQL rpm 源：[https://repo.mysql.com/yum/mysql-5.6-community/el/6/x86_64/](https://repo.mysql.com/yum/mysql-5.6-community/el/6/x86_64/) - MySQL 5.7 rpm 源：[https://repo.mysql.com/yum/mysql-5.7-community/el/6/x86_64](https://repo.mysql.com/yum/mysql-5.7-community/el/6/x86_64) - PHP 5.6 源：[https://repo.webtatic.com/yum/el6/x86_64/](https://repo.webtatic.com/yum/el6/x86_64/) 7. 一些相关的安装教程： - [CENTOS6.3 下 zabbix 安装部署](http:https://showerlee.blog.51cto.com/2047005/1189484) - [CentOS6.5 上 Zabbix3.0 的 RPM 安装【一】-安装并配置 Server](https://www.cnblogs.com/linprogram/p/5730639.html) - [CentOS 6 下安装 Zabbix 3.0](http:https://www.hi-linux.com/2016/03/29/CentOS%206%E4%B8%8B%E5%AE%89%E8%A3%85Zabbix%203.0/) - [Zabbix 3.2.0 yum 或编译安装 on CentOS 7](http:https://www.jianshu.com/p/fa81ede09711) 8. **更方便的方式**：请 **Google**：`一键安装 zabbix` ## 扩展阅读 ### Zabbix 架构  ### 阅读 0. [Zabbix 学习笔记](http:https://www.linux78.com/zabbix-system-study-notes)]]></content>
<categories>
<category>监控系统</category>
</categories>
<tags>
<tag>mariadb</tag>
</tags>
</entry>
<entry>
<title><![CDATA[MariaDB 的安装部署]]></title>
<url>%2F2017%2F11%2F17%2F%E5%AE%89%E8%A3%85%20MariaDB%2F</url>
<content type="text">< 下载 MariaDB 的安装介质，选择版本为 10.1.x 的，x 为小版本号，建议选择较高的。请勿选择 10.2.x 或者 10.0.x 版本，这个是大版本的变更，其功能和安装、配置等可能存在差异。 以下安装说明均以下载的 mariadb-10.1.19-linux-x86\_64.tar.gz 包为例，安装过程中，请自行根据具体的包名和版本做适当调整。 ### 安装 MariaDB 服务 `mariadb-10.1.19-linux-x86_64.tar.gz` 包为 MariaDB 的二进制包，基本是解压即可使用，以下内容主要是配置一些环境信息和开机启动脚本。 将 .tar.gz 包上传至需要安装的服务器，并解压（建议将其解压到 `/opt` 目录下，并命名为 `mariadb`） ``` # cp ./mariadb-10.1.19-linux-x86_64.tar.gz /opt/mariadb-10.1.19-linux-x86_64.tar.gz # cd /opt # tar -zxvf mariadb-10.1.19-linux-x86_64.tar.gz # mv mariadb-10.1.19-linux-x86_64 mariadb ``` **注：** `/opt` 目录下操作需要 root 权限。 **创建用户组及用户** 运行 MySQL 服务时，建议其数据权限指定为 mysql 用户。 ``` # groupadd mysql # useradd -g mysql mysql # passwd mysql New password: # 输入 mysql 用户密码 Retype new password: # 再次输入 mysql 用户密码 ``` **将 MariaDB 的运行脚本添加到环境变量** 修改 mysql 用户的 `~/.bash_profile`，添加 `/opt/mariadb/bin` 到 $PATH 变量里面。 ``` $ su - mysql $ vim ~/.bash_profile # 在最后添加一下内容 PATH=$PATH:/opt/mariadb/bin:/opt/mariadb/scripts export PATH ``` 保存退出后执行 `source ~/.bash_profile` 重新加载当前 shell 变量。 ### 配置数据库实例参数 安装完成 MariaDB 服务后，需要初始化数据库实例才能正常运行，以下内容仅以在一台服务器上安装一个 MySQL 实例为例子编写，对于需要在一台服务器上安装多个 MySQL 实例请参考相关内容。 把 /opt/mariadb/support-files 下的 my-large.cnf 复制到 /etc 目录下，并改为 my.cnf。 ``` # cp /opt/mariadb/support-files/my-large.cnf /etc/my.cnf ``` 修改 /etc/my.cnf 文件内容为以下内容： ``` # vim /etc/my.cnf [client] default-character-set=utf8 [mysqld_multi] mysqld=/opt/mariadb/bin/mysqld_safe mysqladmin=/opt/mariadb/bin/mysqladmin user=mysql log=/var/log/mysql/multi.log [mysqld] # tmp for mysql load data infile,load finish ,need to set to 1 #innodb_doublewrite=0 innodb_doublewrite #**********************Server************************** #******server start related optimizer_switch='index_condition_pushdown=on' optimizer_switch='mrr=on' optimizer_switch='mrr_sort_keys=on' optimizer_switch='mrr_cost_based=off' mrr_buffer_size=64M optimizer_switch='join_cache_incremental=on' optimizer_switch='join_cache_hashed=on' optimizer_switch='join_cache_bka=on' join_cache_level=2 #4 for hash join join_buffer_size=64M join_buffer_space_limit=64M userstat = 1 user=mysql bind-address=0.0.0.0 port=3306 #port-open-timeout= server-id=90743306 gtid-domain-id=7406 #chroot #init-file=file_name #core-file=OFF #skip-grant-tables #******location basedir=/opt/mariadb/ plugin-dir= /opt/mariadb/lib/plugin datadir=/home/mysql/mysql3306/data pid-file=/home/mysql/mysql3306/mysql.pid socket=/tmp/mysql3306.sock tmpdir=/home/mysql/mysql3306/tmp #******security #secure-auth safe-user-create #skip-show-database max_user_connections=2980 max_connect_errors=100000 #secure-file-priv=path #max_prepared_stmt_count= #skip-ssl #ssl-ca=file_name #ssl-capath=directory_name #ssl-cert=file_name #ssl-cipher=cipher_list #ssl-key=file_name # extra-max-connections=10 #******features default-storage-engine=InnoDB #ansi #sql-mode= #auto_increment_increment=1 #auto_increment_offset=1 #div_precision_increment=4 event-scheduler=on #skip-event-scheduler #flush #flush_time= #old #old-alter-table #old-style-user-limits partition #skip-partition #plugin_dir= #plugin-load=plugin_list #symbolic-links #skip-symbolic-links lock_wait_timeout=600 #sync_frm #temp-pool #updatable_views_with_limit #******function #allow-suspicious-udfs #des-key-file=file_name group_concat_max_len=10240 #max_long_data_size= sysdate-is-now #default_week_format= #******character set&time zone etc.. character-set-server=utf8 collation-server=utf8_bin #character-set-client-handshake #skip-character-set-client-handshake character-set-filesystem=utf8 #character-sets-dir= #lower_case_file_system lower_case_table_names=1 #lc-messages= #lc-messages-dir= default-time-zone='+8:00' #******buffer&cache #memlock #large-pages join_buffer_size=128K sort_buffer_size=8m table_open_cache=1024 table_definition_cache=1024 #range_alloc_block_size= #query_prealloc_size= #query_alloc_block_size= #stored_program_cache= #*****query cache query_cache_type=0 query_cache_size=0 #query_cache_min_res_unit= query_cache_limit=0 #query_cache_wlock_invalidate #******thread&connection thread_handling=pool-of-threads thread_pool_stall_limit=200 thread_pool_size=16 # sqlasyn=1 # extra-max-connection=20 #slow_launch_time= #init_connect= back_log=300 thread_cache_size=512 max_connections=3000 #******temptable #big-tables #tmp_table_size=64m tmp_table_size=128m #for mairadb #max_heap_table_size=64m max_heap_table_size=128m #for mairadb #******network #skip-networking skip-name-resolve #skip-host-cache net_buffer_length=8k max_allowed_packet=64m connect_timeout=10 wait_timeout=1800 interactive_timeout=1800 net_read_timeout=3 #set to 10 if across IDC net_write_timeout=6 #set to 10 if across IDC net_retry_count=2 #set to 5 if across IDC #*****profile&optimizer #profiling profiling_history_size=5 #optimizer_prune_level= #optimizer_search_depth= #optimizer_switch= #max_seeks_for_key=1000 max_length_for_sort_data=4096 #******limitation #max_error_count= #max_join_size= #max_sort_length= #max_sp_recursion_depth= open-files-limit=8192 #thread_stack=512k #**********************Logs**************************** log-output=FILE #*****error log log-error=/home/mysql/mysql3306/logs/error3306.log #log-warnings skip-log-warnings #*****slow log slow-query-log slow_query_log_file=/home/mysql/mysql3306/logs/slow3306.log long_query_time=2 #log-queries-not-using-indexes log-slow-admin-statements log-slow-slave-statements #min-examined-row-limit=10 log_slow_verbosity='Query_plan,Innodb' #for mariadb #*****general log #general-log #general_log_file=/home/mysql/mysql3306/logs/query.log #**********************Replication********************* #skip-slave-start #read_only slave_parallel_threads=16 #init_slave= master-info-file=master.info #sync_master_info=1000 #slave_type_conversions= #slave_transaction_retries= #slave_exec_mode= #slave-skip-errors=1062 1064 1146 #only for 216 slave-load-tmpdir=/home/mysql/mysql3306/tmp #******network #slave_compressed_protocol #for RBR #slave-max-allowed-packet= slave-net-timeout=30 #master-retry-count= #******report #show-slave-auth-info #report-host= #report-password= #report-port= #report-user= #*****binlog log-bin=/home/mysql/mysql3306/binlog/mysql-bin log-bin-index=/home/mysql/mysql3306/binlog/mysql-bin.index sync_binlog=1 #close for sas #binlog-format=MIXED binlog-format=row max_binlog_size=512m expire_logs_days=7 binlog_cache_size=256K #max_binlog_cache_size=256m #increase this for RBR max_binlog_cache_size=1024m binlog_stmt_cache_size=32k max_binlog_stmt_cache_size=256m binlog-row-event-max-size=256m #log-short-format #log_slave_updates log-bin-trust-function-creators #binlog_direct_non_transactional_updates #*****relaylog relay-log=relay-bin relay-log-index=relay-bin.index relay-log-info-file=relay-log.info #sync_relay_log=1000 #sync_relay_log_info=1000 #max_relay_log_size= relay_log_space_limit=100G relay_log_purge=1 #relay_log_recovery #******filter #binlog-do-db= #binlog-ignore-db= #replicate-do-db= #replicate-ignore-db= #replicate-do-table= #replicate-ignore-table= #replicate-wild-do-table= #replicate-wild-ignore-table= #replicate-same-server-id #replicate-rewrite-db= #**********************InnoDB************************** innodb_data_home_dir=/home/mysql/mysql3306/data innodb_data_file_path=ibdata1:1000M:autoextend innodb_log_group_home_dir=/home/mysql/mysql3306/data innodb_log_files_in_group=4 innodb_log_file_size=1000M #*****feature innodb_open_files=4096 innodb_change_buffering=inserts innodb_adaptive_hash_index=ON innodb_autoinc_lock_mode=1 #innodb_large_prefix #innodb_strict_mode #innodb_use_sys_malloc= #******buffer&cache innodb_buffer_pool_size=100G #for mariadb innodb_buffer_pool_instances=4 innodb_max_dirty_pages_pct=50 innodb_old_blocks_pct=25 innodb_old_blocks_time=3000 innodb_additional_mem_pool_size=32m innodb_log_buffer_size=8m #******IO innodb_flush_method=O_DIRECT innodb_use_native_aio innodb_adaptive_flushing innodb_flush_log_at_trx_commit=1 innodb_io_capacity=1200 innodb_read_io_threads=4 innodb_write_io_threads=4 innodb_read_ahead_threshold=56 #innodb_doublewrite innodb_purge_threads=1 innodb_purge_batch_size=20 #innodb_max_purge_lag= #*****fileformat innodb_file_per_table innodb_autoextend_increment=32 innodb_file_format=Barracuda #innodb_file_format_check= #innodb_file_format_max= #******static&status #innodb_stats_on_metadata innodb_stats_on_metadata=0 #for mariadb innodb_stats_sample_pages=32 innodb_stats_method=nulls_unequal #timed_mutexes #innodb-status-file #******recovery&related innodb_fast_shutdown=1 #innodb_force_load_corrupted #innodb_force_recovery= innodb_checksums #******transaction,lock,concurrency,rollback autocommit=1 #transaction-isolation=REPEATABLE-READ transaction-isolation=READ-COMMITTED #for mariadb #transaction_prealloc_size=64k #transaction_alloc_block_size=64k completion_type=NO_CHAIN innodb_support_xa innodb_table_locks #innodb_lock_wait_timeout=30 innodb_lock_wait_timeout=45 # for mariadb setup #innodb_locks_unsafe_for_binlog innodb_spin_wait_delay=12 innodb_sync_spin_loops=90 innodb_commit_concurrency=0 innodb_thread_concurrency=32 #at least equal cpu nums #innodb_concurrency_tickets=500 #innodb_replication_delay=0 #innodb_thread_sleep_delay= #innodb_rollback_on_timeout #innodb_rollback_segments=128 #**********************MyISAM************************** #******feature #myisam_data_pointer_size= #myisam_use_mmap #keep_files_on_create myisam-block-size=4096 delay-key-write=on #preload_buffer_size myisam_stats_method=nulls_unequal #myisam-recover-options=OFF myisam_repair_threads=1 #******buffer&cache key_buffer_size=2048M key_cache_block_size=4096 key_cache_age_threshold=300 key_cache_division_limit=20 #read_buffer_size=1m #read_rnd_buffer_size=2m read_buffer_size=4m #for mariadb read_rnd_buffer_size=8m #for mariadb myisam_sort_buffer_size=32m #*******delayed insert #delayed_queue_size= #max_delayed_threads= #delayed_insert_limit= #delayed_insert_timeout= #******fulltext #ft_boolean_syntax= #ft_max_word_len= #ft_min_word_len= #ft_query_expansion_limit= #ft_stopword_file=file_name #******limitation bulk_insert_buffer_size=8m myisam_max_sort_file_size=10G #myisam_mmap_size= #******lock&concurrency #external-locking skip-external-locking concurrent_insert=AUTO #skip-concurrent-insert max_write_lock_count=10000 #low-priority-updates #********************performance_schema**************** performance_schema=1 #performance_schema_events_waits_history_long_size= #performance_schema_events_waits_history_size= #performance_schema_max_cond_classes= #performance_schema_max_cond_instances= #performance_schema_max_file_classes= #performance_schema_max_file_handles= #performance_schema_max_file_instances= #performance_schema_max_mutex_classes= #performance_schema_max_mutex_instances= #performance_schema_max_rwlock_classes= #performance_schema_max_rwlock_instances= #performance_schema_max_table_handles= #performance_schema_max_table_instances= #performance_schema_max_thread_classes= #performance_schema_max_thread_instances= #**************************federated******************** # federated [mysqldump] quick max_allowed_packet = 2G log-error=/home/mysql/mysql3306/logs/dump3306.log net_buffer_length=8k [mysql] no-auto-rehash no-beep default-character-set = utf8 socket=/tmp/mysql3306.sock # prompt="\\U : \\d \\R:\\m:\\s> " # tee="/home/mysql/mysql3306/logs/audit.log" # pager="less -i -n -S" net_buffer_length=64K unbuffered [mysqladmin] default-character-set = utf8 socket=/tmp/mysql3306.sock [myisamchk] key_buffer = 512M sort_buffer_size = 512M read_buffer = 8M write_buffer = 8M [mysqlhotcopy] interactive-timeout [mysqld_safe] open-files-limit = 8192 ledir=/opt/mariadb/bin ``` 以上配置将 MySQL 实例初始化到 `/home/mysql/mysql3306` 目录下，请安装时根据实际的路径修改上面配置文件中的路径，涉及配置项如下： ``` datadir=/home/mysql/mysql3306/data pid-file=/home/mysql/mysql3306/mysql.pid tmpdir=/home/mysql/mysql3306/tmp log-error=/home/mysql/mysql3306/logs/error3306.log slow_query_log_file=/home/mysql/mysql3306/logs/slow3306.log #general_log_file=/home/mysql/mysql3306/logs/query.log slave-load-tmpdir=/home/mysql/mysql3306/tmp log-bin=/home/mysql/mysql3306/binlog/mysql-bin log-bin-index=/home/mysql/mysql3306/binlog/mysql-bin.index innodb_data_home_dir=/home/mysql/mysql3306/data innodb_log_group_home_dir=/home/mysql/mysql3306/data log-error=/home/mysql/mysql3306/logs/dump3306.log ledir=/opt/mariadb/bin ``` 其他参数的修改： * `server-id=90883306` 命名规则：机器 ip 后两个 + 端口 * `gtid-domain-id=8890926` 一主两从 ip 的后一位 + 端口后一位 * `innodb_buffer_pool_size=100G` 单实例不超过机器内存的 80%，双实例加起来不超过机器内存的 80% * `innodb_thread_concurrency=32` 小于等于 cpu 线程数 * `log_slave_updates` 一主一从关闭 log\_slave\_updates（即注释掉），一主两从开启 `log_slave_updates` **初始化实例** 切换用户到 mysql ``` $ su - mysql Password: ``` 创建需要的目录 ``` $ mkdir -p /home/mysql/mysql3306/ $ cd /home/mysql/mysql3306/ && mkdir {data,logs,binlog,tmp,bin} ``` 运行一下命令进行初始化： ``` $ mysql_install_db --basedir=/opt/mariadb/ --defaults-file=/etc/my.cnf --user=mysql Installing MariaDB/MySQL system tables in '/home/mysql/mysql3306' ... 2016-12-12 9:12:48 139952692410208 [Note] /opt/mariadb/bin/mysqld (mysqld 10.1.19-MariaDB) starting as process 14731 ... OK Filling help tables... 2016-12-12 9:12:54 140080526587744 [Note] /opt/mariadb/bin/mysqld (mysqld 10.1.19-MariaDB) starting as process 14763 ... OK Creating OpenGIS required SP-s... 2016-12-12 9:12:59 140681527736160 [Note] /opt/mariadb/bin/mysqld (mysqld 10.1.19-MariaDB) starting as process 14795 ... OK To start mysqld at boot time you have to copy support-files/mysql.server to the right place for your system PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER ! To do so, start the server, then issue the following commands: '/opt/mariadb/bin/mysqladmin' -u root password 'new-password' '/opt/mariadb/bin/mysqladmin' -u root -h 127.0.0.1 password 'new-password' Alternatively you can run: '/opt/mariadb/bin/mysql_secure_installation' which will also give you the option of removing the test databases and anonymous user created by default. This is strongly recommended for production servers. See the MariaDB Knowledgebase at http:https://mariadb.com/kb or the MySQL manual for more instructions. You can start the MariaDB daemon with: cd '/opt/mariadb/' ; /opt/mariadb/bin/mysqld_safe --datadir='/home/mysql/mysql3306/data' You can test the MariaDB daemon with mysql-test-run.pl cd '/opt/mariadb/mysql-test' ; perl mysql-test-run.pl Please report any problems at http:https://mariadb.org/jira The latest information about MariaDB is available at http:https://mariadb.org/. You can find additional information about the MySQL part at: http:https://dev.mysql.com Support MariaDB development by buying support/new features from MariaDB Corporation Ab. You can contact us about this at [email protected]. Alternatively consider joining our community based development effort: http:https://mariadb.com/kb/en/contributing-to-the-mariadb-project/ ``` 可以通过上面信息看到 OK。  查询 `cat /home/mysql/mysql3306/logs/error3306.log` 日志，没有 `[ERROR]` 日志标识安装成功。 ### 启动 MariaDB 实例 **方式一：本文采用服务方式启动 MariaDB 服务。** 首先，需要拷贝配置脚本到服务启动目录： ``` # cp /opt/mariab/support-files/mysql.server /etc/rc.d/init.d/mysqld # chmod +x /etc/rc.d/init.d/mysqld ``` 修改启动脚本内容： ``` # vi /etc/rc.d/init.d/mysqld # 修改 basedir= 及 datadir= 两个参数 basedir=/opt/mariadb datadir=/home/mysql/mysql3306/data ``` 通过 service 启动服务 ``` $ service mysqld start Starting MySQL.161212 10:02:09 mysqld_safe Logging to '/home/mysql/mysql3306/logs/mysql_3306.err'. ... $ ps -ef|grep mysql mysql 24488 1 0 10:02 pts/1 00:00:00 /bin/sh /opt/mariadb/bin/mysqld_safe --datadir=/home/mysql/mysql3306/data --pid-file=/home/mysql/mysql3306/mysqld3306.pid mysql 25032 24488 0 10:02 pts/1 00:00:04 /opt/mariadb/bin/mysqld --basedir=/opt/mariadb/ --datadir=/home/mysql/mysql3306/data --plugin-dir=/opt/mariadb/lib/plugin --log-err or=/home/mysql/mysql3306/logs/mysql_3306.err --open-files-limit=8192 --pid-file=/home/mysql/mysql3306/mysqld3306.pid --socket=/tmp/mysql3306.sock --po rt=3306 ``` **方式二：采用 mysqld\_safe 启动** ``` $ mysqld_safe --defaults-file=/etc/my.cnf & $ ps -ef|grep mysql mysql 24488 1 0 10:02 pts/1 00:00:00 /bin/sh /opt/mariadb/bin/mysqld_safe --datadir=/home/mysql/mysql3306/data --pid-file=/home/mysql/mysql3306/mysqld3306.pid mysql 25032 24488 0 10:02 pts/1 00:00:04 /opt/mariadb/bin/mysqld --basedir=/opt/mariadb/ --datadir=/home/mysql/mysql3306/data --plugin-dir=/opt/mariadb/lib/plugin --log-err or=/home/mysql/mysql3306/logs/mysql_3306.err --open-files-limit=8192 --pid-file=/home/mysql/mysql3306/mysqld3306.pid --socket=/tmp/mysql3306.sock --po rt=3306 ``` 至此，MariaDB 的安装部署已经完成。 ### 配置 root 用户密码 启动服务成功后，MySQL 实例 root 用户的默认密码为空。 可以通过一下命令登录： ``` $ mysql -uroot -p -S /tmp/mysql3306.sock Enter password: ``` `Enter password:` 处回车即可。 然后通过一下命令修改密码： ``` $ mysqladmin -u root password 'new_password' ``` `new_password` 处即为新密码，请修改后保存改密码，后续登录 root 用户需要输入此密码。 如果需要修改新密码，可以先登录到 mysql 中，执行以下命令进行修改（新密码为：`Aa123456`）： ``` MariaDB [(none)]> update mysql.user set password=password('Aa123456') where user='root'; MariaDB [(none)]> flush privileges; ``` ### 配置 root 用户远程访问 默认安装的 MySQL 实例 root 用户只有本地访问权限，需要通过修改 `mysql.user` 表，给 root 用户授权。 执行以下命令： ``` MariaDB [(none)]> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'root123' WITH GRANT OPTION; MariaDB [(none)]> flush privileges; ``` ### 遇到问题 1： ``` $ mysql_install_db --basedir=/opt/mariadb/ --defaults-file=/etc/my.cnf [95/19157] Installing MariaDB/MySQL system tables in '/home/mysql/mysql3306' ... 2016-12-12 8:59:36 140187907016544 [Note] Using unique option prefix 'extra-max-connection' is error-prone and can break in the future. Please use the full name 'extra_max_conne ctions' instead. 2016-12-12 8:59:36 140187907016544 [Note] /opt/mariadb//bin/mysqld (mysqld 10.1.19-MariaDB) starting as process 14447 ... Installation of system tables failed! Examine the logs in /home/mysql/mysql3306 for more information. The problem could be conflicting information in an external my.cnf files. You can ignore these by doing: shell> /opt/mariadb//scripts/scripts/mysql_install_db --defaults-file=~/.my.cnf You can also try to start the mysqld daemon with: shell> /opt/mariadb//bin/mysqld --skip-grant --general-log & and use the command line tool /opt/mariadb//bin/mysql to connect to the mysql database and look at the grant tables: shell> /opt/mariadb//bin/mysql -u root mysql mysql> show tables; Try 'mysqld --help' if you have problems with paths. Using --general-log gives you a log in /home/mysql/mysql3306 that may be helpful. The latest information about mysql_install_db is available at https://mariadb.com/kb/en/installing-system-tables-mysql_install_db MariaDB is hosted on launchpad; You can find the latest source and email lists at http:https://launchpad.net/maria Please check all of the above before submitting a bug report at http:https://mariadb.org/jira $ cat /home/mysql/mysql3306/logs/error3306.log 2016-12-12 08:59:36 7f800a664760 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option in$ odb_use_sys_malloc and with the InnoDB's internal memory allocator. innodb_open_files should not be greater than the open_files_limit. 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Using mutexes to ref count buffer pool pages 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: The InnoDB memory heap is disabled 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: GCC builtin __sync_synchronize() is used for memory barrier 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Compressed tables use zlib 1.2.3 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Using Linux native AIO 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Using SSE crc32 instructions 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Initializing buffer pool, size = 512.0M 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Completed initialization of buffer pool 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: The first specified data file /home/mysql/mysql3306/ibdata1 did not exist: a new database to be created! 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Setting file /home/mysql/mysql3306/ibdata1 size to 128 MB 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Database physically writes the file full: wait... InnoDB: Progress in MB: 100 2016-12-12 8:59:36 140187907016544 [Note] InnoDB: Setting log file /home/mysql/mysql3306/ib_logfile101 size to 1000 MB InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000 2016-12-12 8:59:38 140187907016544 [Note] InnoDB: Setting log file /home/mysql/mysql3306/ib_logfile1 size to 1000 MB InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000 2016-12-12 8:59:39 140187907016544 [Note] InnoDB: Setting log file /home/mysql/mysql3306/ib_logfile2 size to 1000 MB InnoDB: Progress in MB: 100 200 300 400 500 600 700 800 900 1000 2016-12-12 8:59:40 140187907016544 [Note] InnoDB: Setting log file /home/mysql/mysql3306/ib_logfile3 size to 1000 MB InnoDB: Progress in MB: 100 200 300 400 500 600 700 8002016-12-12 08:59:41 7f800a664760 InnoDB: Error: Write to file /home/mysql/mysql3306/ib_logfile3 failed at offset 894435 328. InnoDB: 1048576 bytes should have been written, only 126976 were written. InnoDB: Operating system error number 28. InnoDB: Check that your OS and file system support files of this size. InnoDB: Check also that the disk is not full or a disk quota exceeded. InnoDB: Error number 28 means 'No space left on device'. InnoDB: Some operating system error numbers are described at InnoDB: http:https://dev.mysql.com/doc/refman/5.6/en/operating-system-error-codes.html 2016-12-12 8:59:41 140187907016544 [ERROR] InnoDB: Cannot set log file /home/mysql/mysql3306/ib_logfile3 to size 1000 MB 2016-12-12 8:59:41 140187907016544 [ERROR] Plugin 'InnoDB' init function returned error. 2016-12-12 8:59:41 140187907016544 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2016-12-12 8:59:41 140187907016544 [ERROR] Unknown/unsupported storage engine: InnoDB 2016-12-12 8:59:41 140187907016544 [ERROR] Aborting 2016-12-12 8:59:41 140186063206144 [Warning] mysqld: Disk is full writing '/home/mysql/mysql3306/aria_log.00000001' (Errcode: 28 "No space left on device"). Waiting for some one to free space... (Expect up to 60 secs delay for server to continue after freeing disk space) 2016-12-12 8:59:41 140186063206144 [Warning] mysqld: Retry in 60 secs. Message reprinted in 600 secs ``` 修改 /etc/my.cnf 改小：innodb\_log\_file\_size=128M 删除 /home/myql/mysql3306 目录后重新初始化目录，并重新执行 mysql\_install\_db 命令 ### 遇到问题 2： ``` mysql_install_db --basedir=/opt/mariadb/ --defaults-file=/etc/my.cnf WARNING: The host 'vagrant-centos64.vagrantup.com' could not be looked up with resolveip. This probably means that your libc libraries are not 100 % compatible with this binary MariaDB version. The MariaDB daemon, mysqld, should work normally with the exception that host name resolving will not work. This means that you should use IP addresses instead of hostnames when specifying MariaDB privileges ! ``` `vagrant-centos64.vagrantup.com` 是本机的 hostname，可以通过 `hostname` 命令查看当前主机名 修改 /etc/hosts ``` # vi /etc/hosts # 添加以下内容 127.0.0.1 vagrant-centos64.vagrantup.com ``` ### 遇到问题 3： 使用服务方式和 mysqld\_safe 方式启动，均不成功。并且没有错误日志： ``` $ /opt/mariadb/bin/mysqld_safe --datadir='/home/mysql/mysql3306' --default-files='/etc/my.cnf' 161213 02:40:33 mysqld_safe Logging to '/home/mysql/mysql3306/mysql3306.err'. $ service mysqld start Starting MySQL.161213 02:52:34 mysqld_safe Logging to '/home/mysql/mysql3306/error3306.log'. ERROR! ``` 因为这两个都是 mysqld\_safe 方式启动的，怀疑是 my.cnf 里面 `[mysqld_safe]` 节点配置错误，遂检查 my.cnf 配置文件，发现 `[mysqld_safe]` 节点下的 `ledir` 配置的是 `ledir=/home/mysql/mysql3306`，查看 MySQL 官方文档：[[[5.3.2 mysqld\_safe — MySQL Server Startup Script](http:https://dev.mysql.com/doc/refman/5.7/en/mysqld-safe.html)\]，此参数需要配置的是指向 mysqld 程序的目录路径，使用该选项来显式表示服务器位置。故此，修改为 `ledir=/opt/mariadb/bin`。重启服务成功。]]></content>
<categories>
<category>数据库</category>
</categories>
<tags>
<tag>mariadb</tag>
</tags>
</entry>
<entry>
<title><![CDATA[MySQL 安装部署]]></title>
<url>%2F2017%2F11%2F17%2Finstall-mysql%2F</url>
<content type="text"><。 我们这里选择下载版本为：5.7.19 的程序包：`mysql-5.7.19-linux-glibc2.12-x86_64.tar.gz`。 ## 部署程序 **上传程序包到服务器** 这里假设上传程序包到 `~/apps` 目录下，`~/apps` 需要自己先手工创建。 上传完成后，查看上传的结果： ``` shell $ cd ~/apps $ ls -ahl total 611M drwxrwxr-x. 4 user9 user9 129 Sep 5 07:04 . drwx------. 4 user9 user9 143 Sep 5 07:07 .. -rw-r--r--. 1 user9 user9 611M Sep 5 07:03 mysql-5.7.19-linux-glibc2.12-x86_64.tar.gz ``` 这里可以看到我们上传的程序包。 **解压程序包** 把程序包解压到 `~/apps` 目录，后续程序运行目录和数据目录都在这个根目录下。 ``` shell $ cd ~/apps $ tar -zxvf mysql-5.7.19-linux-glibc2.12-x86_64.tar.gz $ ln -s /home/user9/apps/mysql-5.7.19-linux-glibc2.12-x86_64 /home/user9/apps/mysql # 这里创建一个软链接，方便后续配置或升级版本 ``` ## 配置及初始化 ### 初始化配置文件 我们将 MySQL 的配置文件放在 `~` 目录下，文件名为 `.my.cnf`。配置内容如下： ``` shell $ vi ~/.my.cnf # For advice on how to change settings please see # http:https://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html [mysqld_multi] mysqld = /home/user9/apps/mysql/bin/mysqld_safe mysqladmin = /home/user9/apps/mysql/bin/mysqladmin [mysqld1] # These are commonly set, remove the # and set as required. basedir = /home/user9/apps/mysql datadir = /home/user9/apps/mysqldata/mysql3309 port = 3309 server_id = 1 socket = /home/user9/apps/mysqldata/mysql3309/mysql3309.sock pid_file = /home/user9/apps/mysqldata/mysql3309/mysql3309.pid log_error = /home/user9/apps/mysqldata/mysql3309/mysqld3309.log sync_binlog = 1 log_bin = mysql-bin binlog_format = ROW binlog_ignore_db = mysql log_slave_updates skip_name_resolve explicit_defaults_for_timestamp = true sql_mode = NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES default_storage_engine = InnoDB transaction_isolation = REPEATABLE-READ lower_case_table_names = 1 ``` 这里你也可以用其他工具编辑好，将 `.my.cnf` 文件放到 `~` 目录下即可。 ### 初始化环境配置 我们需要将 MySQL 的二进制脚本加入到环境变量里面，这里我们配置到 `~/.bash_profile` 文件中： ```shell $ vi ~/.bash_profile PATH=$PATH:$HOME/apps/mysql/bin export PATH ``` 修改完后 source 一下：`source ~/.bash_profile`。 ### 初始化数据目录 ```shell $ mkdir -p /home/user9/apps/mysqldata $ mysql_install_db --defaults-file=/home/user9/.my.cnf --user=user9 --basedir=/home/user9/apps/mysql --datadir=/home/user9/apps/mysqldata/mysql3309 ``` 日志输出如下： ``` shell $ mysql_install_db --defaults-file=/home/user9/.my.cnf --user=user9 --basedir=/home/user9/apps/mysql --datadir=/home/user9/apps/mysqldata/mysql3309 2017-09-05 07:07:48 [WARNING] mysql_install_db is deprecated. Please consider switching to mysqld --initialize 2017-09-05 07:07:51 [WARNING] The bootstrap log isn't empty: 2017-09-05 07:07:51 [WARNING] 2017-09-05T07:07:49.033474Z 0 [Warning] --bootstrap is deprecated. Please consider using --initialize instead 2017-09-05T07:07:49.035706Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000) 2017-09-05T07:07:49.035716Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 2000) ``` 这里提示我们 `mysql_install_db` 脚本已经 `deprecated` 了，不用管。 - `--datadir` 指定了 MySQL 数据存放的路径。 ## 启动 ``` shell $ mysqld_multi --defaults-extra-file=/home/user9/.my.cnf --user=user9 start 1 ``` 日志输出如下： ``` shell $ mysqld_multi --defaults-extra-file=/home/user9/.my.cnf --user=user9 start 1 WARNING: Log file disabled. Maybe directory or file isn't writable? mysqld_multi log file version 2.16; run: Tue Sep 5 07:08:17 2017 Starting MySQL servers ``` 可以看到，MySQL 实例已经启动成功了！ ## 用户密码及远程访问 ### 初始化 root 用户密码 启动服务成功后，MySQL 实例默认会在当前用户的 $HOME 目录下生成一个 `~/.mysql_secret` 文件，里面记录了 MySQL 默认 root 用户的初始化密码。 可以通过一下命令登录： ```shell $ mysql -uroot -p -S /home/user9/apps/mysqldata/mysql3309/mysql3309.sock Enter password: ``` `Enter password:` 处输入我们在 `~/.mysql_secret` 文件中看到的默认密码。 登录后，我们需要修改新密码，执行以下命令进行修改（新密码为：`Aa123456`）： ```mysql mysql> SET PASSWORD = PASSWORD('Aa123456'); mysql> flush privileges; ``` ### 配置 root 用户远程访问 默认安装的 MySQL 实例 root 用户只有本地访问权限，需要通过修改 `mysql.user` 表，给 root 用户授权。 执行以下命令： ```mysql mysql> GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'Aa123456' WITH GRANT OPTION; mysql> flush privileges; ``` **==至此，MySQL 的安装及配置全部完成！==** ## 访问 MySQL 安装完成后，我们就可以正常访问该 MySQL 实例了： ``` shell $ mysql -h127.0.0.1 -P3309 -uroot -p Password: ``` ## 数据导入 ### 进入 MySQL ``` shell $ mysql -h127.0.0.1 -P3309 -uroot -p Password: ... ``` ### 切换到我们要的 Database 上 这里假设我们要用到的数据库名叫：itrace\_startup ``` mysql mysql> use itrace_startup; ### 如果没有这个库我们要先创建： mysql> create database itrace_startup; mysql> use itrace_startup; ``` ### 导入数据 这里假设我们要导入的SQL文件名叫：`cust.sql`，且放在 `/home/vagrant` 目录下。 ``` mysql mysql> source /home/vagrant/cust.sql ``` **==导入数据文件 `cust.sql` 成功！==**]]></content>
<categories>
<category>数据库</category>
</categories>
<tags>
<tag>mysql</tag>
</tags>
</entry>
<entry>
<title><![CDATA[MySQL Streaming 的使用和约束]]></title>
<url>%2F2017%2F11%2F16%2Fmysql-streaming%2F</url>
<content type="text"><![CDATA[--- title: MySQL Streaming 的使用和约束 date: 2017-11-16 17:39:34 options: toc:nil tags: [mysql, java] layout: post categories: 数据库 --- 在 Oracle 下，开发人员开发 Java 连接数据库进行超大结果集的查询后处理的程序时，可以通过设置游标进行小批量、迭代多次获取数据记录，避免程序出现 OOM 异常。 而 MySQL 本身并不提供游标功能，而是通过 Connector/J 客户端阻塞式逐行读取数据记录交给业务进行处理。这种“流式（Streaming）”读取看似是很快很便利方式，实际是对应用场景提出了很高的约束，使用不当容易对照成数据库链接长期占用，链接超时，消耗数据库资源等问题。 在 MySQL 中查询超大结果集时，由于 MySQL JDBC 的 ResultSet 默认是接收完全部数据后再交由应用程序进行处理的。对于数据量很大的查询（如报表统计）、有限的 JVM 内存等程序，很容易就照成应用程序 OOM 异常（`java.lang.OutOfMemoryError: Java heap space`）。对于这个错误解决起来也很简单。 MySQL JDBC 支持客户端的流方式（Streaming）读取数据，也叫 Client Side Cursor。即应用程序一边从数据库拉取数据，每获取到一条数据后就可以立即丢给业务逻辑程序处理。代码实现也很简单： 只要在初始化 Statement 的时候，设置这几个参数即可：`ResultSet.TYPE_FORWARD_ONLY`、`ResultSet.CONCUR_READ_ONLY`、`Integer.MIN_VALUE`。 ``` java preparedStatement = connection.prepareStatement(formatSql, ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY); preparedStatement.setFetchSize(Integer.MIN_VALUE); ``` `setFetchSize=Integer.MIN_VALUE` 这是一个 magic number，而不是指定每次读取多少数据。不能改成其他值。具体参考[官方文档](https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-implementation-notes.html#ResultSet)。 而实际上，在高版本的 JDBC 驱动，`ResultSet.TYPE_FORWARD_ONLY`、`ResultSet.CONCUR_READ_ONLY` 这两个参数是 Statement 里面默认设置的（这个看 JDBC Driver 的源码可以了解到，文档上是没写的，这里是个坑），所以实际我们只需要设置 `preparedStatement.setFetchSize(Integer.MIN_VALUE);` 这一行代码也是可以的。 下面是一个简单的使用示例： ``` java import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.Statement; public class Streaming { public static void main(String[] args) throws Exception { Class.forName("com.mysql.jdbc.Driver"); String url = "jdbc:mysql:https://127.0.0.1:3306/test"; String username = "root"; String password = "Aa123456"; Connection conn = DriverManager.getConnection(url, username, password); Statement statement = conn.createStatement(ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY); statement.setFetchSize(Integer.MIN_VALUE); // MySQL connector 的 StatementImpl 的 createStreamingResultSet 方法中明确指出： // We only stream result sets when they are forward-only, read-only, and the fetch size has been set to Integer.MIN_VALUE // 也就是说必须设置这三个属性才能做到流式读取 int batchSize = 2000; ResultSet rs = statement.executeQuery("select * from steaming_test");//一般的数据读取 while (true) { for (int rowIndex = 0; rowIndex < batchSize; rowIndex++) { if(!rs.next()){ return; } System.out.println("one line"); } System.out.println("one batch"); } } } ``` 指定完 Statement 的这几个参数后，执行 SQL 获取到的 ResultSet 就会在调用 `rs.next()` 方法时从 MySQL 服务端获取一条数据记录，然后执行后续的业务逻辑程序，依次迭代完成所有的数据记录获取和处理。而不是等 ResultSet 接收 MySQL 端查询到的所有数据后才业务逻辑程序。如果是大结果集查询，采用 Streaming 方式，程序很快就有查询记录可以进行处理。 通过简单的测试，如果是单纯的获取数据记录，采用 Streaming 方式是更快的。 ## 为什么 MySQL 没有默认就采用 Streaming 方式？ Streaming 方式看似很好，而且测试显示获取数据更快，那为什么 MySQL 没有默认就采用 Streaming 方式？ 其实很简单，因为采用 Streaming 方式，获取数据库记录于业务逻辑处理就变得耦合了。整个数据结果集的获取时间等于数据库查询结果获取时间 + 业务逻辑程序执行时间。这导致了当前查询需要长时间占用一个链接；MySQL 当前查询结果集需要长时间缓存；如果业务逻辑程序中还有 SQL 语句执行，那当前链接无法复用。 ### 总结 Streaming 的主要问题 - 长时间占用数据库链接 - 单个链接占用时间长 - 同个事务内链接无法复用 - 数据库查询结果集需要缓存的时间大大延长 - 数据库当前锁 - 不同链接无法获取到事务锁 这里有两篇很好地讨论了 Streaming 流式读取记录的代价，分别：[The Cost of Streaming Data from MySQL](https://www.wix.engineering/single-post/2017/09/18/The-Cost-of-Streaming-Data-from-MySQL)、[How does MySQL result set streaming perform vs fetching the whole JDBC ResultSet at once](https://vladmihalcea.com/2016/11/23/how-does-mysql-result-set-streaming-perform-vs-fetching-the-whole-jdbc-resultset-at-once/) ## 采用 Streaming 方式在 MySQL 协议和数据报文上有没什么影响？ 并没有，Streaming 方式有客户端模式（Client Side Cursor）和服务端模式（Server Side Cursor），上面程序例子采用的是客户端模式，MySQL 数据库的数据报文没任何差异。 ## 应用场景 ### Streaming 的应用场景 目前只有这一种场景建议采用 Streaming 方式读取数据：**需要大批量获取数据库记录用于产生其他存储的数据的业务场景**。 具体的使用例子有： - 报表数据统计； - 用户查询大批量数据后用于导出； - 获取整表数据用于全量更新缓存/消息中间件数据等； 而且在这种场景下，**同时需要将查询结果集获取和处理程序异步化，避免由于处理程序执行慢导致整个数据库查询和获取结果集时间被拉得很长。** ### 什么场景下不适合使用 **有一些场景看似可以用 Streaming 方式处理但不应该用的**，比如： - **场景 1：数据批量处理** 典型业务场景如：查询一批数据进行业务处理，然后将数据改成下一个流转状态的。 这种场景可以采用小批量+数据业务状态查询，处理完后再获取下一批数据进行处理，依次遍历完所有待处理的记录。 ## 参考材料 - [JDBC API Implementation Notes](https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-implementation-notes.html#ResultSet) - [MySQL 加锁处理分析](http:https://hedengcheng.com/?p=771) - [MySQL 拉取海量数据报 OutOfMemoryError](http:https://codepub.cn/2017/05/12/MySQL-pull-the-massive-data-out-of-memory-error/) - [The Cost of Streaming Data from MySQL](https://www.wix.engineering/single-post/2017/09/18/The-Cost-of-Streaming-Data-from-MySQL) - [How does MySQL result set streaming perform vs fetching the whole JDBC ResultSet at once](https://vladmihalcea.com/2016/11/23/how-does-mysql-result-set-streaming-perform-vs-fetching-the-whole-jdbc-resultset-at-once/)]]></content>
<categories>
<category>数据库</category>
</categories>
<tags>
<tag>mysql</tag>
<tag>java</tag>
</tags>
</entry>
<entry>
<title><![CDATA[Elasticsearch 的一些使用经验]]></title>
<url>%2F2017%2F07%2F03%2Felasticsearch-best-practices%2F</url>
<content type="text"><![CDATA[#+TITLE: Elasticsearch 的一些使用经验 #+DATE: #+OPTIONS: toc:nil #+TAGS: elasticsearch #+LAYOUT: post #+CATEGORIES: Elastic Stack Elasticsearch 上手使用比较容易，但是想真正用好，却要深入研究其相关设置。以下内容是自己整理的一些实践建议： #+BEGIN_HTML #+END_HTML * es 配置相关 ** 配置 memory_lock ** master 节点数 * 索引相关 ** 拆分索引 索引需要按不同业务维度、运维维度拆分。 - 方便业务查询使用，使每次查询能落在数据比较少的索引，避免在一个索引上查询所有内容； - 按时间或者业务域维度拆分索引，方便数据冷热处理；方便运维定时归档超过一定期限的索引； 尽量避免都放在一个索引上。 ** 对不需要分词检索字段设置 not_analayzed 对一些每次查询都是按内容全文匹配的字段，设置 type="not_analayzed"，减少 es 分词，同时也是避免查询结果不准确。甚至有些字段压根就不会用来做查询用的，设置为"not_indexed"。 * jvm 相关 ** 最好内存能分配到 < 32GB * 大规模集群的一些规划 ** 大集群下规划好 Data node、Master node、Client node ** 大规模环境下，拆分多个集群]]></content>
<categories>
<category>Elastic Stack</category>
</categories>
<tags>
<tag>elasticsearch</tag>
</tags>
</entry>
<entry>
<title><![CDATA[Elasticsearch 日常运维操作]]></title>
<url>%2F2017%2F07%2F03%2Fmaintain-elasticsearch%2F</url>
<content type="text"><![CDATA[#+TITLE: Elasticsearch 日常运维操作 #+DATE: #+OPTIONS: toc:nil #+TAGS: elasticsearch #+LAYOUT: post #+CATEGORIES: Elastic Stack]]></content>
<categories>
<category>Elastic Stack</category>
</categories>
<tags>
<tag>elasticsearch</tag>
</tags>
</entry>
<entry>
<title><![CDATA[Elastic Stack 的相关监控]]></title>
<url>%2F2017%2F07%2F03%2Fmonitoring-elastic-stack%2F</url>
<content type="text"><![CDATA[#+TITLE: Elastic Stack 的相关监控 #+DATE: #+OPTIONS: toc:nil #+TAGS: elasticsearch, logstash, filebeat #+LAYOUT: post #+CATEGORIES: Elastic Stack * 使用 X-Pack 进行监控 不差钱 * 插件 elasticsearch-head * 监控 Elasticsearch 的状态 *Green* *Yellow* *Red* * 基于 Monitoring API 自定义监控]]></content>
<categories>
<category>Elastic Stack</category>
</categories>
<tags>
<tag>elasticsearch</tag>
<tag>logstash</tag>
<tag>filebeat</tag>
</tags>
</entry>
<entry>
<title><![CDATA[通过 Zabbix 实现 Logstash 的监控]]></title>
<url>%2F2017%2F06%2F27%2Fuse-metrics-filter-monitoring-logstash-tps-with-zabbix%2F</url>
<content type="text"><![CDATA[#+TITLE: 通过 Zabbix 实现 Logstash 的监控 #+DATE: #+OPTIONS: toc:nil #+TAGS: zabbix, logstash #+LAYOUT: post #+CATEGORIES: Elastic Stack 参考：[[https://kibana.logstash.es/content/logstash/performance/monitor/heartbeat.html][监控方案]] 由于 Logstash 的 Pipeline 模式，通过 filter metrics 插件和 output zabbix 插件即可实现对 Logstash 处理 TPS 的监控。 #+BEGIN_HTML #+END_HTML *Logstash Pipeline* 模型 [[./logstash-pipeline.png]] 由于 logstash-output-zabbix 插件不是 Logstash 的官方插件，因此使用前需要手动先安装该插件，安装方式非常简单[参考：[[https://www.elastic.co/guide/en/logstash/current/plugins-outputs-zabbix.html#plugins-outputs-zabbix-zabbix_server_host][zabbix|Logstash]] ]： #+BEGIN_SRC sh $ bin/logstash-plugin install logstash-output-zabbix #+END_SRC 配置 Zabbix 的数据采集： 1. 配置新的主机名为 “logstash-indexer-1” [[./zabbix-host.png]] 2. 配置采集项 [[./zabbix-logstash-item.png]] 修改 logstash 的配置文件，增加 metrics filter 和 zabbix output： #+BEGIN_SRC sh input { ... } filter { ... metrics { meter => "events" # 统计每一个经过 filter 的事件 add_tag => "metric" add_field => {"[@metadata][zabbix_key]" => "logstash_events"} # zabbix item 的 key，这个要和 zabbix 的配置一致 add_field => { "[@metadata][zabbix_host]" => "logstash-indexer-1" } # zabbix 写入的主机 flush_interval => 10 # 每隔 10s 写一次 } } output { ... if "metric" in [tags] { zabbix { zabbix_server_host => "192.168.56.113" zabbix_host => "[@metadata][zabbix_host]" zabbix_key => "[@metadata][zabbix_key]" zabbix_value => "[events][count]" # [events][count] 是事件统计数据，还有其他字段内容，具体参考官网文档 } } } #+END_SRC 配置好 logstash 后，reload 配置或重启服务即可。 具体配置参考官方文档说明。 至此，查看 Zabbix web 界面的 item 项即可看到监控 TPS 的图表了： [[./logstash-tps-graph.png]] ** 参考内容 - [[https://www.elastic.co/guide/en/logstash/current/plugins-filters-metrics.html][metrics]] - [[https://www.elastic.co/guide/en/logstash/current/plugins-outputs-zabbix.html#plugins-outputs-zabbix-zabbix_server_host][zabbix]]]]></content>
<categories>
<category>Elastic Stack</category>
</categories>
<tags>
<tag>logstash</tag>
<tag>zabbix</tag>
</tags>
</entry>
<entry>
<title></title>
<url>%2F2017%2F06%2F22%2Fdistributed-transactions-introduce%2F</url>
<content type="text"><![CDATA[#+TITLE: 分布式事务知识整理 #+DATE: #+OPTIONS: toc:nil #+TAGS: #+LAYOUT: post #+CATEGORIES: 分布式架构]]></content>
</entry>
<entry>
<title></title>
<url>%2F2017%2F06%2F05%2Fusing-plantuml-in-emacs%2F</url>
<content type="text"><![CDATA[#+TITLE: 在 Emacs 中使用 PlantUML #+DATE: #+OPTIONS: toc:nil #+TAGS: orgmode, plantuml #+LAYOUT: post #+CATEGORIES: Living in Emacs PlantUML 的使用依赖与 Java 环境，因此需要在系统上先安装好 Java 环境。 - 下载 PlantUML jar：[[http:https://plantuml.com/download][PlantUML compiled Jar]]。 - plantuml-mode：[[https://github.com/skuro/plantuml-mode][skuro/plantuml-mode - GitHub]] #+BEGIN_HTML #+END_HTML * Emacs 配置 需要依赖 Java 环境及 plantuml.jar（[[http:https://plantuml.com/download][下载]]）。 #+BEGIN_SRC emacs-lisp (setq plantuml-java-args (expand-file-name "~/.emacs.d/bin/plantuml.jar")) (use-package plantuml-mode :ensure t :config ;; Enable plantuml-mode for PlantUML files (add-to-list 'auto-mode-alist '("\\.plantuml\\'" . plantuml-mode)) ;; Enable plantuml-mode within an org-mode document (add-to-list 'org-src-lang-modes '("plantuml" . plantuml)) ;; Use fundamental mode when editing plantuml blocks with C-c ' (add-to-list 'org-src-lang-modes (quote ("plantuml" . fundamental))) ;; active Org-babel languages (org-babel-do-load-languages 'org-babel-load-languages '(;; other Babel languages (plantuml . t))) ) #+END_SRC 当执行 org code block 后，显示图片 #+BEGIN_SRC emacs-lisp (add-hook 'org-babel-after-execute-hook 'itsyc/display-inline-images 'append) (defun itsyc/display-inline-images () (condition-case nil (org-display-inline-images) (error nil))) #+END_SRC PlantUML 语法参考：[[http:https://emacsist.com/10250][使用 Emacs 敲出 UML，PlantUML 快速指南]]（官方：[[http:https://plantuml.com/sequence-diagram][时序图（Sequence Diagram）]]）。 使用 PlantUML 的主要问题：缺乏可视化，不直观，但编辑相对简单。 * 中文乱码问题 *解决：* 在 code block 的 option 中添加 =:cmdline -charset utf-8= （参见：[[https://www.cnblogs.com/yangwen0228/p/6825554.html][Windows 平台下在 Emacs 中使用 plantuml 中文乱码问题（已解决）]]） * 参考阅读 - [[http:https://wiki.manan.org/tools/org_mode_drawing.html][使用 Org mode 画图]]]]></content>
</entry>
<entry>
<title><![CDATA[使用 Hexo + Orgmode 构建博客]]></title>
<url>%2F2017%2F06%2F04%2Fblogging-with-hexo-orgmode%2F</url>
<content type="text"><![CDATA[--- title: 使用 Hexo + Orgmode 构建博客 date: 2017-06-04 08:36:32 options: toc:nil tags: [emacs, orgmode, hexo] layout: post categories: hexo --- 使用 Hexo 搭建一个个人博客的过程非常简单，参考官方文档即可 [[GitHub](https://github.com/hexojs/hexo)]。 # 基本设置 ## 使用 orgmode 来编辑文档 感谢 CodeFalling 插件：[CodeFalling/hexo-renderer-org - GitHub](https://github.com/CodeFalling/hexo-renderer-org) ### 禁止 hexo-renderer-org 生成 toc **方法一：** 在每篇文章前面加上 =#+OPTIONS: toc:nil= 。 **方法二：** 统一设置。 参考：[CodeFalling/hexo-renderer-org - GitHub](https://github.com/CodeFalling/hexo-renderer-org)。 ## hexo-asset-image 配合这个插件使用： https://github.com/CodeFalling/hexo-asset-image 这个插件值得使用 - [在 hexo 中无痛使用本地图片](http:https://www.tuicool.com/articles/umEBVfI) - [hexo 中完美插入本地图片](http:https://etrd.org/2017/01/23/hexo%E4%B8%AD%E5%AE%8C%E7%BE%8E%E6%8F%92%E5%85%A5%E6%9C%AC%E5%9C%B0%E5%9B%BE%E7%89%87/) ### 其他解决方案 图床 ## 使用 Emacs 来生成博客静态页面及启停 Hexo 参考：[使用 Hexo 和 emacs-Org 来写博客](http:https://summerisgreen.com/blog/2016-09-30-2016-09-30-%E4%BD%BF%E7%94%A8hexo%E5%92%8Cemacs-org%E6%9D%A5%E5%86%99%E5%8D%9A%E5%AE%A2.html) ## RSS 插件 支持 RSS： ``` sh $ npm install hexo-generator-feed --save ``` 启动服务器，用浏览器打开 http:https://localhost:4000/atom.xml，就可以看到 RSS 已经生效了。 关于 RSS 的设置。 # 主题 Hexo 有丰富的主题资源可以使用，其中比较知名的的是 NexT 主题。 不同的主题配置会很不一样。 **其他一些好看的主题：** - [7ye/maupassant-hexo - GitHub](https://github.com/7ye/maupassant-hexo) ## Wiki Theme For Hexo 同样也有用于构建 Wiki 的主题，参考：[Hexo 搭建 WiKi](http:https://www.jianshu.com/p/e7413116e9d4)([wzpan/hexo-theme-wixo - GitHub](https://github.com/wzpan/hexo-theme-wixo)) # disqus 评论 $ 常用命令 ## Quick Start ### Create a new post ``` bash $ hexo new "My New Post" ``` More info: [Writing](https://hexo.io/docs/writing.html) ### Run server ``` bash $ hexo server ``` More info: [Server](https://hexo.io/docs/server.html) ### Generate static files ``` bash $ hexo generate ``` More info: [Generating](https://hexo.io/docs/generating.html) ### Deploy to remote sites ``` bash $ hexo deploy ``` More info: [Deployment](https://hexo.io/docs/deployment.html) # 参考材料 - [https://lengyueyang.github.io/](https://github.com/lengyueyang/lengyueyang.github.io) - [使用 Hexo 和 emacs-Org 来写博客](http:https://summerisgreen.com/blog/2016-09-30-2016-09-30-%E4%BD%BF%E7%94%A8hexo%E5%92%8Cemacs-org%E6%9D%A5%E5%86%99%E5%8D%9A%E5%AE%A2.html) - [set-up-blog-with-hexo+org-mode](https://weychen.github.io/2016/09/28/set-up-blog-with-hexo-org-mode/) - [Hexo 3.1.1 静态博客搭建指南](https://lovenight.github.io/2015/11/10/Hexo-3-1-1-%E9%9D%99%E6%80%81%E5%8D%9A%E5%AE%A2%E6%90%AD%E5%BB%BA%E6%8C%87%E5%8D%97/) # 解决 NexT 主题 Tags/Categories "Cannot GET /tags/" 错误 [Next 主题 Tags/Categories Cannot GET /tags/](https://beeant0512.github.io/2016/03/20/2016-03-20-21-50-10/) # 一些比较不错的 Hexo 博客 - [Grok](http:https://lowrank.science/) # 目前问题 1. 使用 hexo-renderer-org，如果内容有 _ 的内容，下划线后面的文字都会转换成下标，目前还不知道如何处理。]]></content>
<categories>
<category>hexo</category>
</categories>
<tags>
<tag>emacs</tag>
<tag>orgmode</tag>
<tag>hexo</tag>
</tags>
</entry>
<entry>
<title><![CDATA[Elasticsearch 核心概念]]></title>
<url>%2F2017%2F06%2F04%2Felasticsearch-introduction%2F</url>
<content type="text"><![CDATA[#+TITLE: Elasticsearch 核心概念 #+DATE: #+OPTIONS: toc:nil #+TAGS: elasticsearch #+LAYOUT: post #+CATEGORIES: Elastic Stack 在深入学习使用 Elasticsearch（以下简称：`es`）之前，有必要对其相关的概念进行了解。核心概念如下： #+BEGIN_HTML #+END_HTML * 集群(Cluster) 集群是 es 的一个分布式运作环境，有一个统一的集群名字（`cluster.name`）。由多个节点构成，其中有且只有一个为主节点，这个主节点是可以通过选举产生的，主从节点是对于集群内部来说的。es 的一个概念就是去中心化，字面上理解就是无中心节点，这是对于集群外部来说的，因为从外部来看 es 集群，在逻辑上是个整体，你与任何一个节点的通信和与整个 es 集群通信是等价的。 * 节点(Node) * 索引(index) * 类型(Type) * 文档(Document) * 分片(Shard) * 副本(Replica)]]></content>
<categories>
<category>Elastic Stack</category>
</categories>
<tags>
<tag>elasticsearch</tag>
</tags>
</entry>
</search>