Reliable Insights

A blog on monitoring, scale and operational sanity

Writing JSON Exporters in Python

A common question is is there a way to ingest JSON metrics from a random system into Prometheus? It’s not possible to extract useful metrics from an arbitrary JSON blob, so that’s not something the can be offered out of the box. However it’s easy to write an exporter in Python to produce meaningful metrics.

Update: The Python client has a new API that’s easier to use for custom collectors, see Writing a Jenkins exporter in Python for and example of how to use it.


Let’s say you have a service called “SVC” that produces JSON output over HTTP.

  "requests_handled": 14324,
  "requests_duration_milliseconds": 1235257,
  "request_failures": 7,
  "documents_loaded": {
    "fast": 7,
    "slow": 60

To start, let’s install the Prometheus Python client, and the Requests library..

pip install prometheus_client requests

Then we can write a simple exporter. Put the following in a file called

from prometheus_client import start_http_server, Metric, REGISTRY
import json
import requests
import sys
import time

class JsonCollector(object):
  def __init__(self, endpoint):
    self._endpoint = endpoint
  def collect(self):
    # Fetch the JSON
    response = json.loads(requests.get(self._endpoint).content.decode('UTF-8'))

    # Convert requests and duration to a summary in seconds
    metric = Metric('svc_requests_duration_seconds',
        'Requests time taken in seconds', 'summary')
        value=response['requests_handled'], labels={})
        value=response['requests_duration_milliseconds'] / 1000.0, labels={})
    yield metric

    # Counter for the failures
    metric = Metric('svc_requests_failed_total',
       'Requests failed', 'summary')
       value=response['request_failures'], labels={})
    yield metric

    # Metrics with labels for the documents loaded
    metric = Metric('svc_documents_loaded', 'Requests failed', 'gauge')
    for k, v in response['documents_loaded'].items():
      metric.add_sample('svc_documentes_loaded', value=v, labels={'repository': k})
    yield metric

if __name__ == '__main__':
  # Usage: port endpoint

  while True: time.sleep(1)

This can now be run as

python 1234 http://host/path/to/metrics.json

And if you visit http://localhost:1234/metrics you’ll see your metrics!

Brian BrazilWriting JSON Exporters in Python
Share this post

Related Posts