Camel K Integration Monitoring

The Camel K monitoring architecture relies on Prometheus and the eponymous operator. Make sure you’ve checked the Camel K monitoring prerequisites.

Instrumentation

The Prometheus trait automates the configuration of integration pods to expose a metrics endpoint, that can be discovered and scraped by a Prometheus server.

The Prometheus trait can be enabled when running an integration, e.g.:

$ kamel run -t prometheus.enabled=true

Alternatively, the Prometheus trait can be enabled globally once, by updating the integration platform, e.g.:

$ kubectl patch ip camel-k --type=merge -p '{"spec":{"traits":{"prometheus":{"configuration":{"enabled":true}}}}}'

Or by directly editing the IntegrationPlatform resource, e.g.:

apiVersion: camel.apache.org/v1
kind: IntegrationPlatform
metadata:
  name: camel-k
spec:
  traits:
    prometheus:
      configuration:
        enabled: true (1)
1 Activates the Prometheus trait at the platform level

The Camel Quarkus Micrometer Metrics extension is responsible for collecting and exposing metrics in the OpenMetrics text format.

The Micrometer Metrics extension registers and exposes the following metrics out-of-the-box:

It is possible to extend this set of metrics by using either, or both:

Discovery

The Prometheus trait automatically configures the resources necessary for the Prometheus Operator to reconcile, so that the managed Prometheus instance can scrape the integration metrics endpoint.

By default, the Prometheus trait creates a PodMonitor resource, with the camel.apache.org/integration label, which must match the podMonitorSelector field from the Prometheus resource. Additional labels can be specified with the pod-monitor-labels parameter from the Prometheus trait, e.g.:

$ kamel run -t prometheus.pod-monitor-labels="label_to_be_match_by=prometheus_selector" ...

The creation of the PodMonitor resource can be disabled using the pod-monitor parameter, e.g.:

$ kamel run -t prometheus.pod-monitor=false ...

More information can be found in the Prometheus trait documentation.

The Prometheus Operator getting started guide documents the discovery mechanism, as well as the relationship between the operator resources.

In case your integration metrics are not discovered, you may want to rely on Troubleshooting ServiceMonitor changes, which also applies to PodMonitor resources troubleshooting.

Alerting

The Prometheus Operator declares the AlertManager resource that can be used to configure AlertManager instances, along with Prometheus instances.

Assuming an AlertManager resource already exists in your cluster, you can register a PrometheusRule resource that is used by Prometheus to trigger alerts, e.g.:

$ cat <<EOF | kubectl apply -f -
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  labels:
    prometheus: example
    role: alert-rules
  name: prometheus-rules
spec:
  groups:
  - name: camel-k.rules
    rules:
    - alert: CamelKAlert
      expr: application_camel_context_exchanges_failed_total > 0
EOF

More information can be found in the Prometheus Operator Alerting user guide. You can also find more details in Creating alerting rules from the OpenShift documentation.