Reliable Insights

A blog on monitoring, scale and operational sanity

How to unit test Prometheus instrumentation

If you’ve determined a metric should be tested, how do you go about that?

While you shouldn’t automatically instrument metrics, Prometheus client libraries offer facilities to make it as easy as possible to do so.

 

Taking Python as an example, let’s say we had a counter that we wanted to check was incremented:

from prometheus_client import Counter
my_counter = Counter('my_counter_total', 'A useful help string.')

def my_function():
    my_counter.inc()

To test this we compare the before and after values:

import unittest
from prometheus_client import REGISTRY

class TestMyFunction(unittest.TestCase):
    def test_metric_incremented(self):
       before =  REGISTRY.get_sample_value('my_counter_total')
       my_function()
       after = REGISTRY.get_sample_value('my_counter_total')
       self.assertEqual(1, after - before)

For gauges that are incremented/decremented and histograms you can use the same technique. For gauges that are set, you can simply compare the values.

The same approach works for Java using the getSampleValue method of CollectorRegistry.defaultRegistry.

 

Prometheus metrics are usually file-level global variables, registered with a global registry. While you could plumb around the registry to use, this would cause friction and decrease the number of useful metrics that’d end up being added. The global nature of metrics does not make them harder to unit test. In fact it reduces the chances that you’ll miss a bit of plumbing, resulting in the metric and tests passing perfectly but not being exposed.

 

Need help testing your metrics? Contact us.

Brian BrazilHow to unit test Prometheus instrumentation
Share this post

Related Posts