Skip to content

Setting up Load Balancing

Load balancing in KraftCloud is very easy: as soon as you attach more than one instance to a service group, KraftCloud will automatically start balancing traffic between the multiple instances. The load balancing is done based on the number of connections (in TCP mode) or requests (in HTTP mode); more on that below. This diagram illustrates the overall mechanism:

Service groups in KraftCloud

Because of load balancing, instances in a service group must provide the same service. You can remove instances from a service group at any time, and, when you do, KraftCloud will immediately take the instance out of the load balancing service group.

To set load balancing up, first use kraft cloud deploy with the -p flag so that a service group is created as part of the instance creation. For example, let’s use NGINX as the app:

Terminal window
git clone https://github.com/kraftcloud/examples
cd examples/nginx/
kraft cloud deploy -p 443:8080 .

This single command will (a) create a service group via the -p flag and (b) start an NGINX instance:

Terminal window
[] Deployed successfully!
────────── name: nginx-8ujeu
────────── uuid: d6238ac6-27d2-47b3-8a45-c6cac99fb4ef
───────── state: running
─────────── url: https://wandering-shape-n6mhimgn.fra0.kraft.host
───────── image: nginx@sha256:269192f523dca7498423bc54676ab08e415e9c7442d1bd3d65f07ab5e50a43d
───── boot time: 20.18 ms
──────── memory: 128 Mi
service group: wandering-shape-n6mhimgn
── private fqdn: nginx-8ujeu.internal
──── private ip: 172.16.6.7
────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

With this in place, it’s now time to start a second instance and attach it to the service group that was just created (named, in this case, wandering-shape-n6mhimgn):

Terminal window
cd examples/nginx/
kraft cloud deploy --service-group wandering-shape-n6mhimgn .

And the command’s output:

Terminal window
[] Deployed successfully!
────────── name: nginx-djta3
────────── uuid: 06c972a6-a117-4b07-8eba-9389b4cccb42
───────── state: running
─────────── url: https://wandering-shape-n6mhimgn.fra0.kraft.host
───────── image: nginx@sha256:c00c11a5cbd6a3020dd4d9703fbeb2a2f2aab37f18f7a0ba9c66db5a71897c3a
───── boot time: 20.46 ms
──────── memory: 128 MiB
service group: wandering-shape-n6mhimgn
── private fqdn: nginx-djta3.internal
──── private ip: 172.16.6.3
────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

Notice that both the url and service group fields in the 2 instances are the same, as they should be. To check that it worked, run the following command:

Terminal window
kraft cloud service list -o list

You should see output such as:

Terminal window
uuid: 954b4a52-fc51-4eac-b5c9-1fc2a368a237
name: wandering-shape-n6mhimgn
fqdn: wandering-shape-n6mhimgn.fra0.kraft.host
services: 443:8080/tls+http 80:443/http+redirect
instances: 06c972a6-a117-4b07-8eba-9389b4cccb42 d6238ac6-27d2-47b3-8a45-c6cac99fb4ef
created at: 14 minutes ago
persistent: true

Note the two instances (their UUIDs) under the instances field. You’re now load balancing across 2 NGINX instances!

Load Balancing Algorithm

The load balancing algorithm is a variant of least_conn. For every instance, we track the number of current in-flight TCP connections (if in tcp mode) or requests (if in http mode).

To select an instance, we go over all instances in the service group and find the set of instances that have the least amount of in-flight requests/connections, and pick randomly from that set.

To illustrate, imagine we had the following scenario:

Terminal window
Instance ID | # in-flight conns
i-0 | 4
i-1 | 1
i-2 | 2
i-3 | 1

In this case, the algorithm would first choose instances i-1 and i-3, since they both have the least number of connections at the moment (only 1 each). After that, the algorithm would choose randomly between these 2 instances and assign the new connection to it. For example, if it chose i-1, the next new connection would go to i-3 since it’d be now the only instance with only 1 connection (assuming none of the connections that the other instances are handling are closed).

Learn More

  • The kraft cloud CLI reference, and in particular the services sub-command.
  • KraftCloud’s REST API reference, and in particular the section on service groups.