Metrics, Graphite, Log Files, Some Linux, Some Python
You want to make pretty graphs that let you measure things to do with third party applications.
You want to measure the response codes nginx is firing out
Fork the nginx repo and fire up gcc
Logster - https://github.com/etsy/logster
Reads log files and turns them into metrics
Written in Python with metric parsers also written in Python
Intended to be invoked by cron or similar
10.41.78.12 - - [28/Aug/2015:08:02:22 +0000] "GET /healthcheck HTTP/1.1" 200 6 "-" "ELB-HealthChecker/1.0" 0.000 - .
class DemoLogster(LogsterParser):
def __init__(self, option_string=None):
self.http_404 = 0
self.reg = re.compile('.*HTTP/1.\d\" (?P<http_status_code>\d{3}) .*')
def parse_line(self, line):
try:
regMatch = self.reg.match(line)
if regMatch:
linebits = regMatch.groupdict()
status = int(linebits['http_status_code'])
if status == "404":
self.404 = self.404 + 1
else:
raise LogsterParsingException("regmatch failed to match")
except Exception as e:
raise LogsterParsingException("regmatch or contents failed with %s" % e)
def get_state(self, duration):
self.duration = float(duration)
return [
MetricObject("http_404",
(self.http_404 / self.duration),
"Responses per sec")]
Install Logster + Custom Scrapers
Setup a cron task to run once a minute and fire up Logster
Yes! We're using it for our brand new in production Lomo image resizing service
This can work with any log file you know the format of!
I'm Chris McCluskey from Travel Content Platform Squad