Setting up Load Balancing

Load balancing in KraftCloud is very easy: as soon as you attach more than one instance to a service group, KraftCloud will automatically start balancing traffic between the multiple instances. The load balancing is done based on the number of connections (in TCP mode) or requests (in HTTP mode); more on that below. This diagram illustrates the overall mechanism:

Because of load balancing, instances in a service group must provide the same service. You can remove instances from a service group at any time, and, when you do, KraftCloud will immediately take the instance out of the load balancing service group.

To set load balancing up, first use kraft cloud deploy with the -p flag so that a service group is created as part of the instance creation. For example, let’s use NGINX as the app:

git clone https://github.com/kraftcloud/examples
cd examples/nginx/
kraft cloud deploy -p 443:8080 .

This single command will (a) create a service group via the -p flag and (b) start an NGINX instance:

[●] Deployed successfully!
 │
 ├────────── name: nginx-8ujeu
 ├────────── uuid: d6238ac6-27d2-47b3-8a45-c6cac99fb4ef
 ├───────── state: running
 ├─────────── url: https://wandering-shape-n6mhimgn.fra0.kraft.host
 ├───────── image: nginx@sha256:269192f523dca7498423bc54676ab08e415e9c7442d1bd3d65f07ab5e50a43d
 ├───── boot time: 20.18 ms
 ├──────── memory: 128 Mi
 ├─ service group: wandering-shape-n6mhimgn
 ├── private fqdn: nginx-8ujeu.internal
 ├──── private ip: 172.16.6.7
 └────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

With this in place, it’s now time to start a second instance and attach it to the service group that was just created (named, in this case, wandering-shape-n6mhimgn):

cd examples/nginx/
kraft cloud deploy --service-group wandering-shape-n6mhimgn .

Tip

If you get a prompt saying “deployment already exists: what would you like to do with the 1 existing instance(s)?”, simply choose keep.

The command’s output should be similar to this one:

[●] Deployed successfully!
 │
 ├────────── name: nginx-djta3
 ├────────── uuid: 06c972a6-a117-4b07-8eba-9389b4cccb42
 ├───────── state: running
 ├─────────── url: https://wandering-shape-n6mhimgn.fra0.kraft.host
 ├───────── image: nginx@sha256:c00c11a5cbd6a3020dd4d9703fbeb2a2f2aab37f18f7a0ba9c66db5a71897c3a
 ├───── boot time: 20.46 ms
 ├──────── memory: 128 MiB
 ├─ service group: wandering-shape-n6mhimgn
 ├── private fqdn: nginx-djta3.internal
 ├──── private ip: 172.16.6.3
 └────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

Notice that both the url and service group fields in the 2 instances are the same, as they should be. To check that it worked, run the following command:

kraft cloud service list -o list

You should see output such as:

       uuid: 954b4a52-fc51-4eac-b5c9-1fc2a368a237
       name: wandering-shape-n6mhimgn
       fqdn: wandering-shape-n6mhimgn.fra0.kraft.host
   services: 443:8080/tls+http 80:443/http+redirect
  instances: 06c972a6-a117-4b07-8eba-9389b4cccb42 d6238ac6-27d2-47b3-8a45-c6cac99fb4ef
 created at: 14 minutes ago
 persistent: true

Note the two instances (their UUIDs) under the instances field. You’re now load balancing across 2 NGINX instances!

Tip

Currently it’s not possible to set the service group’s name via kraft cloud deploy. If you’d like to set the service group’s name, first use the kraft cloud service command to create the service group, and then use the --service-group flag of kraft cloud deploy to attach the instance being created to the service group, as outlined in this guide

Tip

If you want to make sure that no existing connections are dropped when stopping an instance, use KraftCloud’s draining feature by stopping an instance through kraft cloud instance stop -w TIMEOUT_MS

Load Balancing Algorithm

The load balancing algorithm is a variant of least_conn. For every instance, we track the number of current in-flight TCP connections (if in tcp mode) or requests (if in http mode).

To select an instance, we go over all instances in the service group and find the set of instances that have the least amount of in-flight requests/connections, and pick randomly from that set.

To illustrate, imagine we had the following scenario:

Instance ID | # in-flight conns
i-0         |         4
i-1         |         1
i-2         |         2
i-3         |         1

In this case, the algorithm would first choose instances i-1 and i-3, since they both have the least number of connections at the moment (only 1 each). After that, the algorithm would choose randomly between these 2 instances and assign the new connection to it. For example, if it chose i-1, the next new connection would go to i-3 since it’d be now the only instance with only 1 connection (assuming none of the connections that the other instances are handling are closed).

Learn More

• The kraft cloud CLI reference, and in particular the services sub-command.
• KraftCloud’s REST API reference, and in particular the section on service groups.