Nginx+Keepalived实现nginx高可用详细步骤.docx

Nginx+Keepalived实现nginx高可用详细步骤一.方案规划VIPIP主机名Nginx 端 口默认主从Nginxl88BACKUP俳抢占模式）Nginx288BACKUP（非抢占模式）二 .操作系统与安装软件如下：Redhat Linux 7.9 x64.安装Nginx(两个节点都是同样操作)1 .安装依赖环境rootZZZZ-EBS-SLB02 yum -y install wget gcc-c+ ncurses ncurses-devel cmake make perl bison openssl openssl-devel gcc* Iibxml2 Iibxml2-devel curl-devel libjpeg* libpng* freetype* autoconf automake zlib* fiex* libxml* libmcrypt* libtool-ltdl-devel* libaio libaio-devel bzr libtool2 .安装 opensslrootZZZZ-EBS-SLB02 local)# cd /usr/local/srcrootZZZZ-EBS-SLB02 src# wget rootZZZZ-EBS-SLB02 src# IsrootZZZZ-EBS-SLB02 openssl-1.0.2s# makerootZZZZ-EBS-SLB02 openssl-1.0.2s# make install3 .安装pcrerootZZZZ-EBS-SLB02 local# cd /usr/local/srcrootZZZZ-EBS-SLB02 src# wget s:rootZZZZ-EBS-SLB02 src# IsrootZZZZ-EBS-SLB02 src# cd pcre-8.43rootZZZZ-EBS-SLB02 pcre-8.43# ./configure -prefix=/usr/local/pcre-8.43rootZZZZ-EBS-SLB02 pcre-8.43# makerootZZZZ-EBS-SLB02 pcre-8.43# make install4 .安装zlibglobal_defs router_id zzzz-nginx-slb02vrrp_version 3)vrrp_script chk_nginx script /etc/keepalived/nginx_check.shinterval 2weight -20)vrrpjnstance Vl_l state BACKUPinterface ensl92virtual_router_id 51priority 90nopreemptadvertjnt 0.5authentication auth_type PASSauth_pass 1111)track_script chk_nginx)virtualjpaddress )5 .编写Nginx状态检测脚本编写 Nginx 状态检测脚本 /etc/keepalived/nginx_check.sh (已在 keepalived.conf 中配置)脚 本。脚本功能要求：如果nginx停止运行，尝试启动，如果无法启动那么杀死本机的keepalived进 程，keepalied将虚拟ip绑定到BACKUP机器上。内容如下。rootzzzz-nginx-slb02 keepalived# cd /etc/keepalived/rootzzzz-nginx-slb02 keepalived# cat nginx_check.sh#!/bin/bashA=ps -C nginx o-header |wc -Iif $A -eq 0 ;thensleep 2if ps -C nginx -no-header |wc -I -eq 0 ;then killall keepalivedfi firootzzzz-nginx-slb02 keepalived# chmod +x /etc/keepalived/nginx_check.sh rootzzzz-nginx-slb02 keepalived# service keepalived start Redirecting to /bin/systemctl start keepalived.service rootzzzz-nginx-slb02 keepalived#6 .开放端口rootzzzz-nginx-slb02 firewall-cmd -permanent -add-port=8088/tcp rootzzzz-nginx-slb02 firewall-cmd -permanent -add-port=6681/tcp测试VIP漂移rootzzzz-nginx-slb01 /# ip a1: Io: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00validjft forever preferred_lft foreverinet6 :1/128 scope hostvalidjft forever preferred_lft foreverensl92: mtu 1500 qdisc mq state UP group default qlen 1000link/ether 00:50:56:be:ea:2c brdinet scope global ensl92 validjft forever preferred_lft foreverinet6 fe80:8fa6:26b9:cl8c:fd60/64 scope link noprefixroutevalid_lft forever preferred_lft forever关闭zzzz-nginx-sIbOl后登陆zzzz-nginx-slb02使用ip a命令查看发现这个虚拟IP 已经飘逸到了 zzzz-nginx-slb02 rootZZZZ-EBS-SLB02 pcre-8.43# wget(rootZZZZ-EBS-SLB02 zlib-1.2.11# makerootZZZZ-EBS-SLB02 zlib-1.2.11# make install.下载 nginx-rtmp-modulerootZZZZ-EBS-SLB02 zlib-1.2.11# cd /usr/local/src/rootZZZZ-EBS-SLB02 zlib-1.2.11# git clone s:/github /arut/nginx-rtmp-module.git5 .安装 NginxrootZZZZ-EBS-SLB02 zlib-1.2.11# cd /usr/local/src/rootZZZZ-EBS-SLB02 src# wget -with-openssl=/usr/local/src/openssl-1.0.2s -with-pcre=/usr/local/src/pcre-8.43 -with-zlib=/usr/local/src/zlib-1.2.11 -add-module=/usr/local/src/nginx-rtmp-module -with- _ssl_modulerootZZZZ-EBS-SLB02 nginx-1.19.1# makerootZZZZ-EBS-SLB02 nginx-1.19.1# make install6 .创立启动Nginx服务器的账号rootZZZZ-EBS-SLB02 nginx-1.19.1# /usr/sbin/groupadd -f wwwrootZZZZ-EBS-SLB02 nginx-1.19.1# /usr/sbin/useradd -g www 配置 NginxrootZZZZ-EBS-SLB02 nginx-1.19.1# vi /usr/local/nginx-1.19.1/conf/nginx.conf user www www;配置8个并行进程处理，根据实际服务器CPU个数和性能配置 worker_processes 8;worker_cpu_affinity 00000001 00000010 00000100 00001000 00010000 00100000 01000000 10000000;#error_log logs/error.log;tterrorjog logs/error.log notice;tterrorjog logs/error.log info;pidlogs/nginx.pid;eventsaccept_mutex on;multi_accept on;use epoll;worker_connections 65536;) (include mime.types;default_type application/octet-stream;log_format main $remote_addr - $remote_user $time_local $request$status $body_bytes_sent $ _referer$ _user_agent $ _x_forwarded_forSupstream-cache-status ,$upstream_addr ,$upstream_status $request_time $upstream_response_time;#access_log logs/access.log main;sendfiletcp_nopush on;tcp_nodelay on;client_header_timeout 60;# send_timeout 60;fastcgi_intercept_errors on;proxy_intercept_errors on;server_tokens off;# keepalive_timeout 300;keepalive_timeout 65;proxy_headers_hash_max_size 51200;p roxy_hea d e rs_hash_bucket_size 6400;server_names_hash_max_size 512;server_names_hash_bucket_size 128;client_max_body_size 200m;# client_header_buffer_size 32k;large_client_header_buffers 4 128k;# proxy_connect_timeout 1800;proxy_send_timeout 1800;# proxy_read_timeout 1800;proxy_buffer_size 16k;# proxy_buffers 4 64k;proxy_busy_buffers_size 128k;# P roxy_te m p_f i I e_write_si ze 128k;client_max_body_size 2048m;client_body_buffer_size 256k;proxy_connect_timeout 1;# proxy_send_timeout 30;# proxy_read_timeout 60;proxy_connect_timeout15s;proxy_read_timeout600s;proxy_send_timeout600s;proxy_buffer_size 256k;proxy_buffers 4 256k;proxy_busy_buffers_size 256k;proxy_tem p_fi Ie_write_size 256k;proxy_next_upstream error timeout invalid_header _500 _503 _404;proxy_max_temp_file_size 128m;gzip on;gzip_min_length lk;gzip_buffers 4 16k;gzip_ _version 1.1;gzip_comp_level 2;# gzip_types text/plain application/x-javascript text/css application/xml;gzip_types text/plain application/x-javascript text/css application/xml text/javascript application/x- d-php image/jpeg image/gif image/png;# gzip_vary on;gzip_vary off;gzip_disable MSIE l-6.;serverlisten 88;#server_name为你实际的服务器名server_name zzzz-nginx-slb02;#charset koi8-r;#access_log logs/host.access.log main;location / root html; index index.html index.htm;)#error_page 404 /4O4.html;# redirect server error pages to the static page /50x.html error_page 500 502 503 504 /50x.html;location = /50x.html root html;)upstream erp_8088 #erp系统我们指定负载均衡模式为IP hash模式，默认为随机 ip_hash;和为ERP系统的两台应用服务器，服务端口为8088server 192.168.0.5:8088 weight=l max_fails=2 fail_timeout=2s;server 192.168.0.6:8088 weight=l max_fails=2 fail_timeout=2s;)Upstream mes_6681 #mes系统负载均衡模式为默认为随机和为MES系统的两台应用服务器,服务端口为6681server 192.168.0.7:6681 weight=l max_fails=2 fail_timeout=2s;server 192.168.0.8:6681 weight=l max_fails=2 fail_timeout=2s;)#具体的配置信息我们存放在./conf.d这个目录中Include conf.d/*.conf;9.针对每个upstream在目录conf.d中创立对应的配置文件 #如果conf目录下没有conf.d子目录，那么创立它rootzzzz-nginx-slb02 conf# mkdir conf.drootzzzz-nginx-slb02 conf# cd conf.d#创立配置文件，这个配置文件名没有具体限制，只需要扩展名为conf即可server listen 8088; #就是 nginx.conf 中 upstream erp_8088 段中指定的服务端口 server_name erp_8088 ; #一,般与文件名相同(去掉.conf)indexindex.html index.htm;access_log logs/erp_8088 _access.log;error_log logs/erp_8088 _error.log;location / proxy_pass :/erp_8088/;proxy_set_header Host $host;proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_ignore_client_abort on;proxy_connect_timeout 2;proxy_send_timeout 2;proxy_read_timeout 2;location /favicon.icoreturn 200; accessjog off;)#保存退出serverlisten 6681;server_name mes_6681 ;indexindex.html index.htm;accessjog logs/mes_6681 _access.log main;errorjog logs/mes_6681 _error.log error;location / proxy_pass :/mes_6681/;proxy_set_header Host $host;proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-For $ p roxy_a d d_x_f o r wa rd e d_f o r; proxy_ignore_client_abort on;)location /favicon.icoreturn 200;accessjog off;)#保存退出.修改Nginx欢迎首页内容(用于后面测试，用于区分两个节点的Nginx) 在文件title节点中修改为如卜代码。Welcome to nginx! 2备注:在zzzz-nginx-sIbOl上同样执行第10步，只是在文件title节点中修改为如卜代码:Welcome to nginx! l.开放Nginx服务端口 88rootzzzz-nginx-slb02 sysconfig# firewall-cmd -permanent -add-port=88/tcprootzzzz-nginx-slb02 sysconfig# systemctl restart firewalld10 .测试 Nginx11 .启动 Nginx12 .设置Nginx开机自启动rootlocalhost conf.d# vi /etc/rc.d/init.d/nginx# ! /bin/bashchkconfig: 35 85 15# description: Nginx is an (S) server, (S) reverseset -ePATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/binDESC=nginx daemonNAME=nginxDAEMON=/usr/local/nginx/sbin/$NAMESCRIPTNAME=/etc/init.d/$NAMEtest -x $DAEMON 11 exit 0d_start()$DAEMON 11 echo -n already running)d_stop() SDAEMON -s quit 11 echo -n not running)d_reload() $DAEMON -s reload 11 echo -n counld not reload)case $1 instart)echo -n Starting $DESC:$NAMEd_startechostop)echo -n Stopping $DESC:$NAMEd_stopechoreload)echo -n Reloading $DESC configuration. d_reload echo reloaded./restart)echo -n Restarting $DESC: $NAMEd_stopsleep 2 d_start echo*)echo Usage: $SCRIPTNAME start | stop | restart | reload &2exit 3esacexit 0rootzzzz-nginx-slb02 conf# chmod +x /etc/rc.d/init.d/nginxrootlocalhost conf.d# chkconfig -add nginxrootlocalhost conf.d# chkconfig -level 35 nginx on四.安装Keepalived (在两个节点上执行同样操作).下载 Keepalived官方卜载链接为： :.解压安装Keepalivedrootzzzz-nginx-slb02 src# cd /usr/local/srcrootzzzz-nginx-slb02 keepalived-2.2.2# ./configure -prefix=/usr/local/keepalivedrootzzzz-nginx-slb02 keepalived-2.2.2# makerootzzzz-nginx-slb02 keepalived-2.2.2# make install1 .将keepalived安装成Linux系统服务rootzzzz-nginx-slb02 /# mkdir /etc/keepalivedrootzzzz-nginx-slb02 /# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/rootzzzz-nginx-slb02 /# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ rootzzzz-nginx-slb02 /# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/rootzzzz-nginx-slb02 /# chkconfig keepalived onNote: Forwarding request to systemctl enable keepalived.service,./usr/lib/systemd/system/keepalived.service. rootzzzz-nginx-slb02 /#2 .修改Keepalived配置文件BACKUP1节点配置文件rootzzzz-nginx-slb01 keepalived# cd /etc/keepalived/ rootzzzz-nginx-slb01 keepalived# vi keepalived.conf ! Configuration File for keepalivedglobal_defs router_id zzzz-nginx-sIbOl vrrp_version 3)vrrp_script chk_nginx script /etc/keepalived/nginx_check.sh interval 2weight -20)vrrpjnstance Vl_l state BACKUP interface ensl92 virtual_router_id 51 mcast_src_ip 192.168.0.2 priority 100 nopreempt advert_int 0.5 authentication auth_type PASS auth_pass 1111 ) track_script chk_nginx)BACKUP2节点配置文件rootzzzz-nginx-slb02 keepalived# cd /etc/keepalived/ rootzzzz-nginx-slb02 keepalived# vi keepalived.conf ! Configuration File for keepalived