Camel K Pipe Scaling

Manual Scaling

A Pipe can be scaled using the kubectl scale command, e.g.:

$ kubectl scale klb <kamelet_binding_name> --replicas <number_of_replicas>

This can also be achieved by editing the Pipe resource directly, e.g.:

$ kubectl patch klb <kamelet_binding_name> -p '{"spec":{"replicas":<number_of_replicas>}}'

The Pipe also reports its number of replicas in the .status.replicas field, e.g.:

$ kubectl get klb <kamelet_binding_name> -o jsonpath='{.status.replicas}'

Autoscaling with Knative

A Pipe that binds an HTTP-based source Kamelet can automatically scale based on incoming traffic when installed on a cluster with Knative enabled, including scaling to zero.

The incoming traffic measures either as:

  • The number of simultaneous requests, that are processed by each replica at any given time;

  • Or the number of requests that are processed per second, per replica.

The webhook-source Kamelet is one of the sources that enables auto-scaling when used in a Pipe:

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: webhook-binding
spec:
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: webhook-source
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: log-sink

The Knative Autoscaler can be configured using the Knative Service trait, e.g., to set the scaling upper bound (the maximum number of replicas):

apiVersion: camel.apache.org/v1
kind: Pipe
metadata:
  name: webhook-binding
spec:
  integration:
      traits:
        knative-service:
          configuration:
            maxScale: 10
  source:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: webhook-source
  sink:
    ref:
      kind: Kamelet
      apiVersion: camel.apache.org/v1
      name: log-sink

More information can be found in the Knative Autoscaling documentation.

When manually scaling a Pipe that deploys as a Knative Service, both scale bounds, i.e., minScale and maxScale, are set to the specified number of replicas. Scale bounds can be reset by removing the .spec.replicas field from the Pipe, e.g., with:

$ kubectl patch klb <kamelet_binding_name> --type=json -p='[{"op": "remove", "path": "/spec/replicas"}]'

Autoscaling with HPA

A Pipe can automatically scale based on its CPU utilization and custom metrics using horizontal pod autoscaling (HPA).

For example, executing the following command creates an autoscaler for the Pipe, with target CPU utilization set to 80%, and the number of replicas between 2 and 5:

the HPA can work when the Pipe replica field needs to be specified. You need to scale the Pipe via kubectl scale pipe my-pipe --replicas 1 or edit the .spec.replicas field of your Pipe to 1. This is due to a Kubernetes behavior which does not allow an empty value on the resource to scale. 
$ kubectl autoscale klb <kamelet_binding_name> --min=2 --max=5 --cpu-percent=80

Refer to the Integration scaling guide for information about using custom metrics.

HPA can also be used with Knative, by installing the HPA autoscaling Serving extension.
Back to top