Monitoring container performance in 30 (or so) lines of code

There are many tools that can be used to monitor system performance. In today’s containerized DevOps environment, it is also necessary to see how individual containers are performing. Luckily, there are some tools available that will let you monitor the performance of individual containers. In this blog post, we will setup a system that allows you to bootstrap a system for monitoring container performance in style. The following steps assume you have docker and docker-compose installed.

First, let’s add cAdvisor. cAdvisor is an open-source container monitoring tool built by Google. cAdvisor exposes container and hardware statistics by collecting and aggregating various metrics on running containers. We’ll start with a docker-compose.yaml file and add the following to it:

version: "3.8"
services:
  cadvisor:
    container_name: cadvisor
    image: gcr.io/cadvisor/cadvisor:latest
    restart: unless-stopped
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:rw
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
    ports:
      - 8080:8080

Next, we’ll add Prometheus. Prometheus is a toolkit that allows us to scrape and store the data exposed by cAdvisor as time-series data.

  prometheus:
    container_name: prometheus
    image: prom/prometheus:latest
    restart: unless-stopped
    command:
      - --config.file=/etc/prometheus/prometheus.yaml
    volumes:
      - ./prometheus.yaml:/etc/prometheus/prometheus.yaml:ro
    ports:
      - 9090:9090
    depends_on:
      - cadvisor

We also need to specify some settings for Prometheus by using a config file. In prometheus.yaml, include

scrape_configs:
  - job_name: cadvisor
    scrape_interval: 5s
    static_configs:
      - targets:
          - cadvisor:8080

Lastly, we will add some data visualization by using Grafana. Grafana allows us to create beautiful-looking graphs and dashboards based on the data scraped by prometheus.

  grafana:
    container_name: grafana
    image: grafana/grafana:latest
    restart: unless-stopped
    volumes:
      - grafana_storage:/var/lib/grafana
    ports:
      - 3000:3000
volumes:
  grafana_storage:

Now run these images using docker-compose. If you go to localhost:3000 you should see a login prompt from Grafana. The default credentials are admin/admin. Once you’re logged in, add Prometheus as a datasource. Since we are running Grafana and Prometheus as containers themselves, you will need to use http://prometheus:9090 as the url instead of localhost:9090 because that’s what we set the container name as.

Finally, you can start building a dashboard by creating a new panel, adding a query, and choosing one of the metrics that you are interested in.