This is a OpenResty Lua and Redis powered logger. We use it at The Movie Database (TMDb) to keep track of some basic metrics:
- unique api keys per day
- unique ips per day
- unique ips per api key per day
- total requests per day
- total requests per api key per day
We take advantage of Redis' HyperLogLog to store our unique counts. This makes it incredibly trivial to merge our daily totals and rollup uniques across multiple days, weeks and months.
It might seem odd to put this in a access_by_lua block but we need access to the cosocket API and log_by_lua doesn't provide this. Nginx also discourages putting a content_by_lua block inside a location where you use a proxy.
You have to compile OpenResty with the --with-http_realip_module option.
http {
# http:https://serverfault.com/questions/331531/nginx-set-real-ip-from-aws-elb-load-balancer-address
# http:https://serverfault.com/questions/331697/ip-range-for-internal-private-ip-of-amazon-elb
set_real_ip_from 127.0.0.1;
set_real_ip_from 10.0.0.0/8;
set_real_ip_from 172.16.0.0/12;
set_real_ip_from 192.168.0.0/16;
real_ip_header X-Forwarded-For;
real_ip_recursive on;
}
# Location of this Lua package
lua_package_path "/opt/lua-resty-api-metrics/lib/?.lua;;";
upstream api {
server unix:/run/api.sock;
}
server {
listen 80;
server_name api.dev;
access_log /var/log/openresty/api_access.log;
error_log /var/log/openresty/api_error.log;
location / {
access_by_lua '
local api_metrics = require "resty.api.metrics"
api_metrics.log { ip = ngx.var.remote_addr,
log_level = ngx.NOTICE,
redis_config = { host = "127.0.0.1", port = 6379, timeout = 1, pool_size = 100 } }
';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_connect_timeout 1s;
proxy_read_timeout 30s;
proxy_pass http:https://api;
}
}
You can customize some of the info we send back to collect by changing the following values:
- ip: The IP value to use as a identifier in Redis.
- log_level: Set an Nginx log level. All errors from this plugin will be dumped here
- redis_config: The Redis host, port, timeout and pool size