php-fpm资源占用高问题处理

43次阅读
没有评论

2024年6月10日,正值端午节,用户反应网站有问题,并且发了截图,从截图中看到,前端界面静态界面是正常加载了,只是数据没有展示出来,应该是接口出现了问题。

通过浏览器的开发者工具查看网络请求,看到接口全部超时,登录宝塔查看,发现CPU负载拉满了。然后通过top命令看到有大量的php-fpm进程。

但是由于服务器上装了多个版本的PHP,先确认下是哪个版本的PHP。

ps -ef | grep php-fpm

root@iZuf67mcw0tro6podzt9fcZ:~# ps -ef | grep php-fpm
www        967  1465  0 Mar07 ?        00:00:24 php-fpm: pool www
root      1465     1  0 Mar04 ?        00:03:39 php-fpm: master process (/www/server/php/56/etc/php-fpm.conf)
www       1466  1465  0 Mar04 ?        00:00:24 php-fpm: pool www
www       1467  1465  0 Mar04 ?        00:00:24 php-fpm: pool www
www       1468  1465  0 Mar04 ?        00:00:24 php-fpm: pool www
www       1469  1465  0 Mar04 ?        00:00:24 php-fpm: pool www
www       1470  1465  0 Mar04 ?        00:00:25 php-fpm: pool www
root      1487     1  0 Mar04 ?        00:03:47 php-fpm: master process (/www/server/php/82/etc/php-fpm.conf)
www       1690  1487  0 Jun07 ?        00:03:02 php-fpm: pool www
www       1721  1487  0 Jun01 ?        00:11:19 php-fpm: pool www
www       1750  1487  0 Jun01 ?        00:11:20 php-fpm: pool www
www       1808  1487  0 Jun01 ?        00:11:11 php-fpm: pool www
www       2063  5186  0 Jun01 ?        00:00:09 php-fpm: pool www
www       2067  5186  0 Jun01 ?        00:00:10 php-fpm: pool www
www       2068  5186  0 Jun01 ?        00:00:09 php-fpm: pool www
www       2085  5186  0 Jun01 ?        00:00:09 php-fpm: pool www
www       2086  5186  0 Jun01 ?        00:00:09 php-fpm: pool www
www       2087  5186  0 Jun01 ?        00:00:09 php-fpm: pool www
www       2088  5186  0 Jun01 ?        00:00:09 php-fpm: pool www
www       2123  5186  0 Jun01 ?        00:00:09 php-fpm: pool www
www       2140  5186  0 Jun01 ?        00:00:08 php-fpm: pool www
www       2141  5186  0 Jun01 ?        00:00:08 php-fpm: pool www
root      2760     1  0 May12 ?        00:01:10 php-fpm: master process (/www/server/php/81/etc/php-fpm.conf)
root      5186     1  0 May12 ?        00:01:10 php-fpm: master process (/www/server/php/80/etc/php-fpm.conf)
www       7809 27780  3 19:15 ?        00:08:48 php-fpm: pool www
www       7810 27780  3 19:15 ?        00:08:48 php-fpm: pool www
www       7811 27780  3 19:15 ?        00:08:46 php-fpm: pool www
www       7812 27780  3 19:15 ?        00:08:45 php-fpm: pool www
www       7813 27780  3 19:15 ?        00:08:43 php-fpm: pool www
www       7814 27780  3 19:15 ?        00:08:47 php-fpm: pool www
www       7815 27780  3 19:15 ?        00:08:46 php-fpm: pool www
www       7816 27780  3 19:15 ?        00:08:47 php-fpm: pool www
www       7817 27780  3 19:15 ?        00:08:47 php-fpm: pool www
www       7818 27780  3 19:15 ?        00:08:47 php-fpm: pool www
www       8184 27780  3 20:05 ?        00:07:23 php-fpm: pool www
www       8185 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8186 27780  3 20:05 ?        00:07:26 php-fpm: pool www
www       8187 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8188 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8189 27780  3 20:05 ?        00:07:27 php-fpm: pool www
www       8190 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8191 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8192 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8193 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8194 27780  3 20:05 ?        00:07:23 php-fpm: pool www
www       8195 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8196 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8197 27780  3 20:05 ?        00:07:26 php-fpm: pool www
www       8198 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8199 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8200 27780  3 20:05 ?        00:07:26 php-fpm: pool www
www       8201 27780  3 20:05 ?        00:07:28 php-fpm: pool www
www       8202 27780  3 20:05 ?        00:07:26 php-fpm: pool www
www       8203 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8204 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8205 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8206 27780  3 20:05 ?        00:07:26 php-fpm: pool www
www       8207 27780  3 20:05 ?        00:07:27 php-fpm: pool www
www       8208 27780  3 20:05 ?        00:07:27 php-fpm: pool www
www       8209 27780  3 20:05 ?        00:07:23 php-fpm: pool www
www       8210 27780  3 20:05 ?        00:07:27 php-fpm: pool www
www       8211 27780  3 20:05 ?        00:07:26 php-fpm: pool www
www       8212 27780  3 20:05 ?        00:07:26 php-fpm: pool www
www       8213 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8214 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8215 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8216 27780  3 20:05 ?        00:07:22 php-fpm: pool www
www       8217 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8218 27780  3 20:05 ?        00:07:21 php-fpm: pool www
www       8219 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8220 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       8221 27780  3 20:05 ?        00:07:25 php-fpm: pool www
www       8222 27780  3 20:05 ?        00:07:27 php-fpm: pool www
www       8223 27780  3 20:05 ?        00:07:24 php-fpm: pool www
www       9782  1487  0 23:24 ?        00:00:01 php-fpm: pool www
root     10165 10137  0 23:59 pts/0    00:00:00 grep php-fpm
www      10448  1487  0 Jun08 ?        00:02:38 php-fpm: pool www
www      10485  1487  0 Jun08 ?        00:02:37 php-fpm: pool www
www      10491  1487  0 Jun08 ?        00:02:38 php-fpm: pool www
www      13673  1465  0 Apr19 ?        00:00:22 php-fpm: pool www
www      15393  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15406  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15407  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15434  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15438  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15439  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15454  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15455  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15456  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      15457  2760  0 May14 ?        00:00:00 php-fpm: pool www
www      17426  1487  0 Jun08 ?        00:01:39 php-fpm: pool www
www      22364  1465  0 May28 ?        00:00:06 php-fpm: pool www
www      22365  1465  0 May28 ?        00:00:06 php-fpm: pool www
root     22813     1  0 May12 ?        00:01:08 php-fpm: master process (/www/server/php/83/etc/php-fpm.conf)
www      22814 22813  0 May12 ?        00:00:02 php-fpm: pool www
www      22815 22813  0 May12 ?        00:00:02 php-fpm: pool www
www      22816 22813  0 May12 ?        00:00:01 php-fpm: pool www
www      22817 22813  0 May12 ?        00:00:01 php-fpm: pool www
www      22818 22813  0 May12 ?        00:00:01 php-fpm: pool www
www      23018 22813  0 May12 ?        00:00:01 php-fpm: pool www
www      23047 22813  0 May12 ?        00:00:01 php-fpm: pool www
root     27780     1  0 Jun09 ?        00:00:03 php-fpm: master process (/www/server/php/74/etc/php-fpm.conf)
www      28974  1487  0 Jun04 ?        00:08:17 php-fpm: pool www

从上面的结果可以看到28行的php-fpm的资源占用比较高,由于他是work进程没有显示出是哪个版本的

从上面的结果可以看到从28行开始,父进程pid为27780的php-fpm占用资源比较高,看下是27780是哪个版本的PHP,

ls -l /proc/27780/exe
lrwxrwxrwx 1 root root 0 Jun 11 18:21 /proc/227780/exe -> /www/server/php/74/sbin/php-fpm

可以看到是php7.4的这个版本的占用资源高,到宝塔上去找下php7.4的日志。通过翻查日志,发现不对劲的日志:

[11-Jun-2024 00:11:19] WARNING: [pool www] child 8218, script '/www/wwwroot/tools.wujingquan.com/public/index.php' (request: "GET /index.php?s=/refresh/&url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1") executing too slow (30.216330 sec), logging
[11-Jun-2024 00:11:19] WARNING: [pool www] child 8211, script '/www/wwwroot/tools.wujingquan.com/public/index.php' (request: "GET /index.php?s=/refresh/&url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1") executing too slow (30.138807 sec), logging
[11-Jun-2024 00:11:19] WARNING: [pool www] child 8208, script '/www/wwwroot/tools.wujingquan.com/public/index.php' (request: "GET /index.php?s=/refresh/&url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1") executing too slow (30.908649 sec), logging
[11-Jun-2024 00:11:19] WARNING: [pool www] child 8199, script '/www/wwwroot/tools.wujingquan.com/public/index.php' (request: "GET /index.php?s=/refresh/&url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1") executing too slow (30.513303 sec), logging

看到是这个tools.wujingquan.com这个站点的问题,把该站点关闭,观察服务器状态,负载将下来了,然后重新打开该站点,再观察,发现负载又上来了,可以确认是该站点出现问题了。

然后再去翻查该站点的日志,发现有大量这样的日志:

223.150.255.228 - - [11/Jun/2024:01:00:56 +0800] "GET /refresh/?url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1 HTTP/1.1" 404 146 "https://tools.wujingquan.com/refresh/" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko"
223.150.255.228 - - [11/Jun/2024:01:00:56 +0800] "GET /refresh/?url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1 HTTP/1.1" 404 146 "https://tools.wujingquan.com/refresh/" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko"
223.150.255.228 - - [11/Jun/2024:01:00:56 +0800] "GET /refresh/?url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1 HTTP/1.1" 404 146 "https://tools.wujingquan.com/refresh/" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko"
223.150.255.228 - - [11/Jun/2024:01:00:56 +0800] "GET /refresh/?url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1 HTTP/1.1" 404 146 "https://tools.wujingquan.com/refresh/" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko"
223.150.255.228 - - [11/Jun/2024:01:00:56 +0800] "GET /refresh/?url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1 HTTP/1.1" 404 146 "https://tools.wujingquan.com/refresh/" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko"
223.150.255.228 - - [11/Jun/2024:01:00:56 +0800] "GET /refresh/?url=https%3A%2F%2Fwww.oneptp.com%2Fax%2Fpop2.php%3Fuid%3D507297%26ad%3D1 HTTP/1.1" 404 146 "https://tools.wujingquan.com/refresh/" "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko"

发现这个223.150.255.228这个IP发送了大量请求,导致服务器卡死,最后使用fail2ban禁掉该IP,问题解决。

正文完
 
wujingquan
版权声明:本站原创文章,由 wujingquan 2024-06-11发表,共计7775字。
转载说明:Unless otherwise specified, all articles are published by cc-4.0 protocol. Please indicate the source of reprint.
评论(没有评论)