It's a frequently asked question as to how to do various security related features with Prometheus. Let's take a deeper look at why we chose the approach we did.

 

The Prometheus project takes the stance that server side security features are outside its scope. This is one of our most questioned product decisions. To see why we made this choice, let's consider a world where we did handle security in Prometheus and accepted some of the PRs we regularly receive to add it.

Starting off someone sends a PR to add basic authentication to Prometheus. It's just a username and password coming from flags, so that's only a few lines of code.

That's got a few security problems though. Firstly we're putting a password on the command line, and secondly we're sending a password in the clear over the network.  We'll need to at least store a hashed and salted version of the password, and keep up to date as hash best practices changed.

We also need to add encryption of data over the wire, which means TLS. There are at present 18 different configuration options for TLS in Go. Of these there are 13 that an end user could plausibly want to configure today, and potentially more in the future as the protocol evolves and new security threats emerge.

So that's basic auth covered. Then there's also Digest access, bearer tokens, KerberosGSSAPI, OpenID, SSL client certificates, Radius and PAM - each with their own nuances and evolution over time.

 

Then there's authorization. Authentication tells you who someone is, but we also need to know if they're allowed to access a given endpoint. Maybe there's different basic auth users for different levels of access, or talk to OpenLDAP, or ActiveDirectory, or Kerberos, or MySQL, or PostgresQL - the configuration and settings of each are likely to be organisation specific.

Going a step deeper, should we prohibit certain types of query? Challenging given PromQL is Turing Complete. Maybe time series based access control then?

 

Security is a fractally complicated space, and on top of that it is ever changing. Prometheus as a project could decide to take this support into our core and provide a coherent and secure system - however that would be a substantial and Sisyphean task.

Prometheus is a monitoring system, not a security framework. Accordingly given the massive work that'd be involved in creating and maintaining a security framework, we've decided to instead leave the task up to 3rd party systems such as Nginx and Apache. These already implement the security feature that our users ask for, are well maintained, and allow the Prometheus project to focus on its core goal of monitoring.