prometheus-nginxlog-exporter构建Nginx日志监控
时间:2022-07-22
本文章向大家介绍prometheus-nginxlog-exporter构建Nginx日志监控,主要内容包括其使用实例、应用技巧、基本知识点总结和需要注意事项,具有一定的参考价值,需要的朋友可以参考一下。
lunimous 可以开启 dashborad
rgw 通过 prometheus-nginxlog-exporter 可以解析 Nginx 日志,还有一些正则可以去处理,然后通过 relabel
的配置,让其在指标数据上打上标签。
civetweb 没有在日志记录请求返回的时间,所以考虑用 Nginx 来替换,为了能给 Ceph RGW 加一个请求相关的监控,所以需要去解析日志,在 prometheus-nginxlog-exporter 中,通过配置文件,可以把标签和端口等设置好,然后让 prometheus 加上这个 endpoint 就可以拉出来数据了。
listen {
port = 4040
}
enable_experimental = true
namespace "nginx" {
source = {
files = [
"/var/log/nginx/access.log"
]
}
format = "$remote_addr - $remote_user [$time_local] "$request" $status $body_bytes_sent "$request_time""
labels {
app = "default"
}
#relabel "request" {
# from = "request"
#}
relabel "bucket" {
from = "request"
split = 2
match "^/general__lingqu/.*" {
replacement = "general__lingqu"
}
}
}
Nginx 日志。
xx.xx.xxx.x - - [26/Jun/2020:01:23:37 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/_SUCCESS HTTP/1.1" 200 0 "0.001"
xx.xx.xxx.x - - [26/Jun/2020:01:23:43 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00000 HTTP/1.1" 200 1165212977 "6.006"
xx.xx.xxx.x - - [26/Jun/2020:01:23:49 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00001 HTTP/1.1" 200 1180678766 "6.130"
xx.xx.xxx.x - - [26/Jun/2020:01:23:55 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00002 HTTP/1.1" 200 1181307932 "6.049"
xx.xx.xxx.x - - [26/Jun/2020:01:24:01 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00003 HTTP/1.1" 200 1207342395 "6.138"
xx.xx.xxx.x - - [26/Jun/2020:01:24:08 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00004 HTTP/1.1" 200 1198727695 "6.080"
xx.xx.xxx.x - - [26/Jun/2020:01:24:14 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00005 HTTP/1.1" 200 1206446826 "6.137"
xx.xx.xxx.x - - [26/Jun/2020:01:24:20 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00006 HTTP/1.1" 200 1204772146 "6.144"
xx.xx.xxx.x - - [26/Jun/2020:01:24:26 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00007 HTTP/1.1" 200 1165911854 "5.985"
xx.xx.xxx.x - - [26/Jun/2020:01:24:32 +0800] "GET /general__lingqu/kp/out/390280/n1590907415729/107916/20200601220441/part-00008 HTTP/1.1" 200 1179420268 "6.077"
prometheus-nginxlog-exporter 采集到的指标。
# HELP nginx_http_response_count_total Amount of processed HTTP requests
# TYPE nginx_http_response_count_total counter
nginx_http_response_count_total{app="default",bucket="",method="GET",status="200"} 1
nginx_http_response_count_total{app="default",bucket="general__lingqu",method="DELETE",status="204"} 2
nginx_http_response_count_total{app="default",bucket="general__lingqu",method="GET",status="200"} 214
nginx_http_response_count_total{app="default",bucket="general__lingqu",method="HEAD",status="200"} 8474
nginx_http_response_count_total{app="default",bucket="general__lingqu",method="PUT",status="200"} 5
# HELP nginx_http_response_size_bytes Total amount of transferred bytes
# TYPE nginx_http_response_size_bytes counter
nginx_http_response_size_bytes{app="default",bucket="",method="GET",status="200"} 338
nginx_http_response_size_bytes{app="default",bucket="general__lingqu",method="DELETE",status="204"} 0
nginx_http_response_size_bytes{app="default",bucket="general__lingqu",method="GET",status="200"} 2.1549611919e+10
nginx_http_response_size_bytes{app="default",bucket="general__lingqu",method="HEAD",status="200"} 0
nginx_http_response_size_bytes{app="default",bucket="general__lingqu",method="PUT",status="200"} 0
# HELP nginx_http_response_time_seconds Time needed by NGINX to handle requests
# TYPE nginx_http_response_time_seconds summary
nginx_http_response_time_seconds{app="default",bucket="",method="GET",status="200",quantile="0.5"} NaN
nginx_http_response_time_seconds{app="default",bucket="",method="GET",status="200",quantile="0.9"} NaN
nginx_http_response_time_seconds{app="default",bucket="",method="GET",status="200",quantile="0.99"} NaN
nginx_http_response_time_seconds_sum{app="default",bucket="",method="GET",status="200"} 0.002
nginx_http_response_time_seconds_count{app="default",bucket="",method="GET",status="200"} 1
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="DELETE",status="204",quantile="0.5"} 0.002
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="DELETE",status="204",quantile="0.9"} 0.009
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="DELETE",status="204",quantile="0.99"} 0.009
nginx_http_response_time_seconds_sum{app="default",bucket="general__lingqu",method="DELETE",status="204"} 0.011
nginx_http_response_time_seconds_count{app="default",bucket="general__lingqu",method="DELETE",status="204"} 2
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="GET",status="200",quantile="0.5"} 0.014
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="GET",status="200",quantile="0.9"} 0.112
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="GET",status="200",quantile="0.99"} 7.037
nginx_http_response_time_seconds_sum{app="default",bucket="general__lingqu",method="GET",status="200"} 115.89000000000006
nginx_http_response_time_seconds_count{app="default",bucket="general__lingqu",method="GET",status="200"} 214
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="HEAD",status="200",quantile="0.5"} 0.002
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="HEAD",status="200",quantile="0.9"} 0.002
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="HEAD",status="200",quantile="0.99"} 0.024
nginx_http_response_time_seconds_sum{app="default",bucket="general__lingqu",method="HEAD",status="200"} 22.99799999999861
nginx_http_response_time_seconds_count{app="default",bucket="general__lingqu",method="HEAD",status="200"} 8474
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="PUT",status="200",quantile="0.5"} NaN
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="PUT",status="200",quantile="0.9"} NaN
nginx_http_response_time_seconds{app="default",bucket="general__lingqu",method="PUT",status="200",quantile="0.99"} NaN
nginx_http_response_time_seconds_sum{app="default",bucket="general__lingqu",method="PUT",status="200"} 0.839
nginx_http_response_time_seconds_count{app="default",bucket="general__lingqu",method="PUT",status="200"} 5
Reference
- https://github.com/martin-helmich/prometheus-nginxlog-exporter
- https://www.martin-helmich.de/en/blog/monitoring-nginx.html
- PHP高级编程之守护进程
- Spring boot with Docker
- Spring boot with Service
- Spring boot with PostgreSQL
- Struts2 S2-046, S2-045 Firewall(漏洞防火墙)
- 应用程序的通信成本
- Spring cloud 之 Feign Client
- Spring Cloud Netflix
- 重新整理AUTO_INCREMENT字段
- Spring Cloud Config
- Spring boot with Schedule (启用/禁用)
- DevOps Tools
- Docker 虚拟机之 Redis
- Spring boot with HTTPS SSL
- JavaScript 教程
- JavaScript 编辑工具
- JavaScript 与HTML
- JavaScript 与Java
- JavaScript 数据结构
- JavaScript 基本数据类型
- JavaScript 特殊数据类型
- JavaScript 运算符
- JavaScript typeof 运算符
- JavaScript 表达式
- JavaScript 类型转换
- JavaScript 基本语法
- JavaScript 注释
- Javascript 基本处理流程
- Javascript 选择结构
- Javascript if 语句
- Javascript if 语句的嵌套
- Javascript switch 语句
- Javascript 循环结构
- Javascript 循环结构实例
- Javascript 跳转语句
- Javascript 控制语句总结
- Javascript 函数介绍
- Javascript 函数的定义
- Javascript 函数调用
- Javascript 几种特殊的函数
- JavaScript 内置函数简介
- Javascript eval() 函数
- Javascript isFinite() 函数
- Javascript isNaN() 函数
- parseInt() 与 parseFloat()
- escape() 与 unescape()
- Javascript 字符串介绍
- Javascript length属性
- javascript 字符串函数
- Javascript 日期对象简介
- Javascript 日期对象用途
- Date 对象属性和方法
- Javascript 数组是什么
- Javascript 创建数组
- Javascript 数组赋值与取值
- Javascript 数组属性和方法
- PHP PDOStatement::bindValue讲解
- PDO::lastInsertId讲解
- thinkPHP框架RBAC实现原理分析
- PHP PDOStatement::debugDumpParams讲解
- PDO::prepare讲解
- sklearn的predict_proba使用说明
- 如何使用python记录室友的抖音在线时间
- 使用keras实现BiLSTM+CNN+CRF文字标记NER
- 图解MyBatis的SQL执行流程
- Keras: model实现固定部分layer,训练部分layer操作
- 在Keras中CNN联合LSTM进行分类实例
- DeepWalk:图网络与NLP的巧妙融合
- PHP扩展mcrypt实现的AES加密功能示例
- PHP PDOStatement::bindColumn讲解
- PHP PDOStatement::fetchAll讲解